Data verification method and device for redundant disk array, equipment and medium

Abstract

The invention discloses a data verification method and device for a redundant disk array, equipment and a medium, and relates to the technical field of data verification, when a data verification value of target data is generated, the target data is firstly divided into a plurality of data blocks, and correspondingly reference data is also divided into one-to-one corresponding reference blocks; then, the unit verification value of each data block is determined by taking the data block as a unit, and the data verification value of the target data is obtained through block-by-block calculation, so that the data verification of the target data is realized according to the data verification value, the memory occupation in the data verification value generation process is reduced, the storage requirement of temporary data is reduced, and the data verification efficiency is improved. The high memory overhead of processing the whole target data at a time is avoided, and the calculation efficiency of the data verification value is improved. The size of the data block can be flexibly adjusted according to requirements, the application range is wide, the flexibility is high, the check and problem positioning of the stored data are facilitated, and the troubleshooting of abnormal conditions such as data damage of the stored data is facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of data verification, and particularly to a data verification method, apparatus, device and medium for a redundant disk array. Background Art

[0002] With the continuous growth of data volume and the increasing requirement for data reliability, data storage technology has been constantly evolving. RAID (Redundant Array of Independent Disks) technology, as an effective means to improve data storage reliability, availability and performance, has become an indispensable part of modern data centers. RAID combines multiple disks into a logical unit and uses redundant storage of data to improve data reliability and read / write performance. The core idea of RAID technology is to combine multiple physical disks into a logical disk and improve data access speed by dispersing data storage and read requests among multiple disks. To improve data reliability, it is necessary to regularly check the stored data in the RAID to avoid potential data corruption.

[0003] It can be seen that how to generate corresponding verification information to implement the check of stored data is a problem that needs to be solved by those skilled in the art. Summary of the Invention

[0004] The purpose of the embodiments of the present invention is to provide a data verification method, apparatus, device and medium for a redundant disk array, which can solve the problem of how to generate verification information for stored data in a redundant disk array.

[0005] To solve the above technical problems, the embodiments of the present invention provide a data verification method for a redundant disk array, including:

[0006] Dividing the target data in the redundant disk array into a plurality of data blocks;

[0007] Determining a plurality of reference blocks corresponding one-to-one to the data blocks from the reference data; wherein, the data length of the reference block is equal to the data length of the corresponding data block;

[0008] Determining the unit verification value of each data block based on the one-to-one corresponding data block and reference block;

[0009] Determining the data verification value of the target data as the unit verification values of all data blocks corresponding to the target data;

[0010] Performing data verification on the target data based on the data verification value of the target data.

[0011] In some embodiments, performing data verification on the target data based on the data verification value of the target data includes:

[0012] If the target data has a data verification requirement, load each data block corresponding to the target data in a preset order;

[0013] For any data block, after any data block is loaded, load the reference block corresponding to any data block;

[0014] Calculate the logical operation value of the loaded data block and the corresponding reference block based on a preset operation method;

[0015] Compare the logical operation value with the unit verification value corresponding to any data block to implement data verification of the target data.

[0016] In some embodiments, after comparing the logical operation value with the unit verification value corresponding to any data block, it further includes:

[0017] If there is a data block in the target data whose corresponding logical operation value is inconsistent with the corresponding unit verification value, perform data recovery of the target data based on the unit verification value and reference data.

[0018] In some embodiments, comparing the logical operation value with the unit verification value corresponding to any data block includes:

[0019] Starting from the lowest bit of the logical operation value, compare the bit values at the corresponding positions of the logical operation value and the unit verification value in sequence until there is a situation where the bit values are different or all bits of the logical operation value have been compared;

[0020] Before performing data recovery of the target data based on the unit verification value and reference data, it further includes:

[0021] If the comparison results of the bit values of all bits of the logical operation value all indicate the same, it is determined that the data block passes the data verification;

[0022] If there is a situation where the bit values are different, it is determined that the logical operation value is inconsistent with the unit verification value, and the data block corresponding to the inconsistent logical operation value and the unit verification value is marked as an abnormal block;

[0023] Performing data recovery of the target data based on the unit verification value and reference data includes:

[0024] Determine the reference block and unit verification value corresponding to the abnormal block;

[0025] Perform the inverse operation of the preset operation method based on the reference block and unit verification value corresponding to the abnormal block to obtain the normal block corresponding to the abnormal block;

[0026] Restore the abnormal block to the normal block to implement data recovery of the target data.

[0027] In some embodiments, it further includes:

[0028] Store the unit check values of all data blocks corresponding to the target data at corresponding preset positions respectively;

[0029] Or,

[0030] Merge the unit check values of all data blocks corresponding to the target data according to the division status of the data blocks to obtain the data check value of the target data, and store the data check value of the target data at the preset position.

[0031] In some embodiments, it further includes:

[0032] If there is a situation of secondary writing of the target data, then when the target data is written for the second time, jump back to the step of dividing the target data in the redundant disk array into several data blocks to update the data check value of the target data.

[0033] In some embodiments, determining the unit check value of each data block based on the corresponding data block and reference block includes:

[0034] Allocate the calculation operations of the unit check values of several data blocks to several processing cores respectively, so that each processing core determines the unit check value of the data block based on the corresponding data block and reference block.

[0035] To solve the above technical problems, an embodiment of the present invention further provides a data check device for a redundant disk array, including:

[0036] A division unit, configured to divide the target data in the redundant disk array into several data blocks;

[0037] A reference acquisition unit, configured to determine several reference blocks corresponding one-to-one to the data blocks from the reference data; wherein, the data length of the reference block is equal to the data length of the corresponding data block;

[0038] A check value determination unit, configured to determine the unit check value of each data block based on the corresponding data block and reference block;

[0039] A check value generation unit, configured to determine the unit check values of all data blocks corresponding to the target data as the data check value of the target data;

[0040] A check unit, configured to perform data check on the target data based on the data check value of the target data.

[0041] To solve the above technical problems, an embodiment of the present invention further provides an electronic device, including:

[0042] A memory, configured to store a computer program;

[0043] A processor, configured to execute the computer program to implement the steps of the data check method for the redundant disk array as described above.

[0044] To solve the above technical problems, an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data verification method for a redundant disk array as described above are implemented.

[0045] As can be seen from the above technical solution, when generating the data verification value of the target data, first divide the target data into several data blocks, and correspondingly divide the reference data into corresponding reference blocks. Then, determine the unit verification value of each data block with the data block as the unit, and calculate block by block to obtain the data verification value of the target data, so as to implement the data verification of the target data according to this data verification value. The beneficial effect of the present invention is that it reduces the memory occupation in the process of generating the data verification value, reduces the storage requirement of temporary data, avoids the high memory overhead of processing the entire target data at one time, and improves the calculation efficiency of the data verification value. The size of the data block can be flexibly adjusted according to requirements, with a wide application range and high flexibility, which helps to implement the inspection of stored data and problem location, and helps to troubleshoot abnormal situations such as data corruption of stored data. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the embodiments of the present invention, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained according to these drawings.

[0047] Figure 1 It is a schematic flowchart of a data verification method for a redundant disk array provided by an embodiment of the present invention;

[0048] Figure 2 It is a schematic diagram of a block division method of target data provided by an embodiment of the present invention;

[0049] Figure 3 It is a schematic diagram of a data verification process of target data provided by an embodiment of the present invention;

[0050] Figure 4 It is a schematic diagram of a data recovery method of target data provided by an embodiment of the present invention;

[0051] Figure 5 It is a schematic structural diagram of a data verification device for a redundant disk array provided by an embodiment of the present invention;

[0052] Figure 6 It is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0054] The terms "including" and "having" in the specification of the present invention and the accompanying drawings above, and any variations related to "including" and "having", are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may include steps or units not listed.

[0055] In order to enable those skilled in the art of this technology to better understand the solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0056] Next, a data verification method for a redundant disk array provided by an embodiment of the present invention will be introduced in detail. Refer to Figure 1 as shown Figure 1 is a schematic flowchart of a data verification method for a redundant disk array provided by an embodiment of the present invention. The data verification method for the redundant disk array includes:

[0057] S11: Divide the target data in the redundant disk array into several data blocks.

[0058] It is not difficult to understand that in order to improve the accuracy and reliability of the stored data in the redundant disk array, when storing data in the redundant disk array, the verification information of the stored data will be generated and saved at the same time. The verification information is stored in the redundant disk array in the form of a verification value, so as to perform data check operations such as integrity verification of the stored data based on the stored verification value in the future. The present invention provides a data verification method for a redundant disk array, which is used to generate the data verification value of the stored data in the redundant disk array to realize the data check and verification of the stored data.

[0059] It is understandable that any stored data in the redundant disk array can be used as the target data to generate the corresponding data check value. To reduce the memory occupation during the generation of the data check value and improve the calculation efficiency, when generating the data check value of the target data, the target data is divided into multiple data blocks. The present application does not make specific limitations on the specific division method, the specific number and data length of the obtained data blocks. The data lengths of each data block can be the same or different, and the size of the data block loaded block by block is dynamically adjusted according to the system memory and disk I / O (Input / Output) performance. For the convenience of subsequent accurate verification, generally, the target data is divided starting from the starting bit of the target data. When dividing, the data length being a preset value can be used as the division basis. Taking the preset value as 4 bits as an example, starting from the starting bit of the target data, the 0th bit to the 3rd bit of the target data are divided into the first 4-bit data block, and the 4th bit to the 7th bit of the target data are divided into the second 4-bit data block until the entire target data is divided, that is, until the last bit of the target data is also divided into the corresponding data block.

[0060] S12: Determine a number of reference blocks corresponding one by one to the data blocks from the reference data; wherein, the data length of the reference block is equal to the data length of the corresponding data block.

[0061] Furthermore, to improve the accuracy of data verification, in the present invention, when generating the data check value of the target data, a reference data is used to cooperate with the target data to generate the data check value of the target data. Therefore, to calculate the unit check value corresponding to each data block, a number of reference blocks corresponding one by one to the data blocks need to be determined from the reference data. The data blocks and the reference blocks correspond one by one. At the same time, to ensure the comprehensiveness of verification, the data lengths of the corresponding data blocks and reference blocks are kept the same, so that each byte / each bit (bit) in the data block can generate a corresponding check value. The present application does not make specific limitations on the division method of the reference data, etc. A reference data with the same length as the target data can be selected, and the same division method as when generating the data blocks is adopted, so as to ensure the bit-by-bit correspondence between the target data and the reference data and the bit-by-bit correspondence between the data blocks and the reference blocks.

[0062] It should be noted that the reference data can be another stored data of the non-target data in the redundant disk array, or a preset fixed reference data. The specific implementation manner of the reference data and the like are not particularly limited in this application. The total number of all stored data in the redundant disk array can be determined, and all stored data are combined in pairs without repetition or omission of any stored data. The two stored data after combination are respectively used as the target data and the reference data to generate a data check value, so as to generate the data check value of the stored data in the entire redundant disk array. The stored data in the redundant disk array is directly reused as the reference data, avoiding additional occupation and being easy to implement. It is also possible to pre-determine the data length of the stored data in the redundant disk array, and then preset the corresponding fixed reference data as the reference data. The fixed reference data can provide a unified standard and facilitate subsequent data recovery, and can effectively ensure the feasibility and reliability of data recovery; for example, when the redundant disk array includes 5-bit stored data and 4-bit stored data, a 5-bit reference data and a 4-bit reference data are respectively preset.

[0063] S13: Determine the unit check value of each data block based on the one-to-one corresponding data block and reference block.

[0064] S14: Determine the data check value of the target data as the unit check values of all data blocks corresponding to the target data.

[0065] It can be understood that after obtaining the one-to-one corresponding data blocks and reference blocks, the unit check values corresponding to each data block can be calculated one by one. Compared with the entire target data, the data length and the number of occupied bytes of each data block after partitioning are greatly reduced. Therefore, the calculation efficiency when generating the unit check value in units of data blocks will be greatly improved. At the same time, since the data blocks are obtained by partitioning the target data, all data blocks combined are equivalent to the complete target data after combination. Therefore, the unit check values corresponding to all data blocks can be used as the data check value of the entire target data after combination. At this time, the combination method of the unit check values needs to be determined according to the partitioning method when partitioning the data blocks. The generation method of the unit check value and the like are not particularly limited in this application. In order to maintain the check consistency and improve the accuracy of data check, the unit check values corresponding to each data block can all be obtained by the same calculation method.

[0066] S15: Perform data check on the target data based on the data check value of the target data.

[0067] It is not difficult to understand that the data verification value of the target data includes several unit verification values, and each unit verification value corresponds to each data block one by one. When performing data verification, it is not necessary to reorganize the target data and its data verification value. For example, when verifying the target data, the unit verification value can be used to verify each data block of the target data respectively. If all the data blocks corresponding to the target data pass the data verification, it means that the target data has passed the data verification; if there is a data block that fails to pass the verification, the abnormal data part can also be effectively located, which is convenient for subsequent data recovery and exception elimination. Data verification includes data integrity verification and whether there are inconsistencies in the data, etc., and realizes the inspection and verification of abnormal situations such as data errors, data loss, and data invalidity.

[0068] It should be noted that the specific type and implementation method of the redundant disk array are not particularly limited in this application. According to different data storage methods and redundant information, the RAID technology is divided into multiple levels, including RAID 0, RAID 1, RAID 2, RAID 3, RAID 4, RAID 5, RAID 6, RAID10, RAID50, etc. Each level has its specific application scenarios, advantages and disadvantages. The data verification method of the redundant disk array provided by the present invention can be applied to various types of RAID, and has a wide range of applications. Especially in RAID 5, the data is divided into multiple stripes, and each stripe contains a certain number of data blocks. These data blocks are alternately stored on different disks. The data lengths of each data stripe are the same, so any data stripe can be used as the target data, and another data stripe can be selected as the reference data to generate the data verification value of the target data.

[0069] Furthermore, the data verification method of the redundant disk array provided by the present invention is mainly applied to software RAID. Software RAID is a technology that manages a hard disk array through an operating system or specific software, which has high cost-effectiveness, is easy to configure, and has strong flexibility. Software RAID can use the algorithms provided by the operating system to manage the data distribution and parity check calculation on the hard disk, so as to achieve data redundancy and performance optimization. Therefore, the data verification method of the redundant disk array provided by the present invention can be implemented by CPU (Central Processing Unit) software in software RAID, with stronger flexibility. For the number of data blocks, the length of each data block, and the calculation process of the unit verification value, they can all be dynamically configured and adjusted according to the actual application. It does not rely on a dedicated RAID controller, but is completely implemented at the operating system level, which is more flexible and easier to upgrade.

[0070] As a specific embodiment, see Figure 2 as shown Figure 2Schematic diagram of a method for partitioning target data provided by an embodiment of the present invention; taking the generation of a parity check value of target data through exclusive OR calculation as an example, Figure 2 As shown, data blocks and reference blocks are divided in units of 5 bits, and exclusive OR calculation is performed in units of 5 bits to generate respective unit check values. Among them, target data D1 is divided into four data blocks ①②③④ in units of 5 bits, and at the same time, reference data D2 is also divided into corresponding four reference blocks in units of 5 bits. Exclusive OR calculations are performed on the four data blocks and the corresponding reference blocks respectively to generate a data check value P1 including four unit check values. In actual applications, the number of bits of each data block can be flexibly adjusted.

[0071] The present invention provides a data check method for a redundant disk array, which generates a data check value of stored data in the redundant disk array through a block calculation method. Especially for very large data in a cloud storage or distributed storage system, the method of calculating block by block effectively reduces the memory overhead of one-time loading. The number of blocks calculated block by block can be flexibly configured. According to the application scenario, the granularity of unit check value calculation can be flexibly determined, and the data block division can be implemented bit by bit, byte by byte, or block by block. At the same time, by storing redundant copies or parity check information of data on different disks, the reliability of data is improved, and parallel reading and writing of data and fault tolerance capabilities are realized, improving the storage performance and data redundancy capabilities of the redundant disk array. In the read operation of data in the redundant disk array, it is possible to quickly verify data integrity block by block based on the unit check value and locate data anomaly points. In the write operation of data, the check block is updated bit by bit block by block, reducing the data loading amount and performing parallel processing to improve the write performance. In the data recovery process, the data to be recovered is calculated bit by bit block by block and checked in a timely manner.

[0072] In some embodiments, data check of target data is performed based on the data check value of the target data, including:

[0073] If there is a data check requirement for the target data, each data block corresponding to the target data is loaded in a preset order;

[0074] For any data block, after any data block is loaded, the reference block corresponding to any data block is loaded;

[0075] Based on a preset operation method, the logical operation value of the loaded data block and the corresponding reference block is calculated;

[0076] The logical operation value is compared with the unit check value corresponding to any data block to implement data check of the target data.

[0077] It can be understood that after determining the unit check value corresponding to each data block in the target data in units of data blocks, if data verification of the target data is required, data verification of the target data can be performed in units of data blocks. For any data block of the target data, load the data block and the corresponding reference block, recalculate and generate the corresponding logical operation value based on the currently loaded data block and reference block, and then implement data verification of the data block by comparing whether the logical operation value is consistent with the corresponding unit check value, thereby implementing data verification of the target data.

[0078] It should be noted that the preset operation method needs to be consistent with the calculation method of the unit check value. For example, if the unit check value is the parity check information of the target data, the unit check value is calculated through an XOR (exclusive OR) operation. At this time, the preset operation method during data verification also needs to be an exclusive OR operation. If there is no data loss or inconsistency in the target data during transmission and storage, the recalculated logical operation value will be consistent with the unit check value, and the corresponding data block can pass the data verification. At the same time, in order to verify the integrity of the target data, all data blocks corresponding to the target data need to be loaded in the preset order. If all data blocks can pass the data verification, the target data passes the data verification. The specific implementation method of the preset order and the like are not particularly limited in this application. Generally speaking, when dividing data blocks, it starts from the starting bit of the target data. Then, when performing data verification, it can also start from the data block corresponding to the starting bit of the target data and load them into a CPU in sequence for data verification. It is also possible to load each data block into different CPUs for data verification. It is also possible to randomly load each data block. Finally, as long as the verification of all data blocks of the target data is achieved.

[0079] As a specific embodiment, software RAID usually runs an integrity verification task in the background to check whether the stored data has errors. Based on the unit check value provided by the present invention, the calculation and comparison of the target data can be performed bit by bit. See Figure 3 shown Figure 3 is a schematic diagram of the data verification process of a target data provided by an embodiment of the present invention; taking the data length of each data block of the target data as 1 bit as an example, that is, dividing the data block in units of 1 bit, as Figure 3 shown, where the grayed-out bit is the error bit. When performing data integrity verification, similarly, after calculating the logical operation value corresponding to each data block by XOR operation in units of 1 bit respectively, it is then compared with the verification data, that is, after sequentially performing XOR calculation bit by bit and then comparing with the verification value, the comparison process is as Figure 3As shown in the figure. Calculate the exclusive OR result of the first bit of the target data D1 and the first bit of the reference data. After the calculation, compare it with the first bit of the data check value P1. If they are the same, calculate the exclusive OR result of the next data block, that is, the next bit, until it is found that the third bit is inconsistent with the third bit of the data check value P1 after the third bit is calculated. At this time, the damaged bit can be marked. When the data is repaired, only the data of the abnormal bit needs to be repaired, reducing the calculation amount during the recovery process. The problem of the data can be known through the third comparison. Only 3-bit calculation amount is required, and the integrity verification of the data can be realized without calculating the entire target data D1.

[0080] Specifically, the data integrity verification is realized by calculating and comparing bit by bit for each data block, reducing the storage requirement of temporary data in software RAID and avoiding the high memory overhead of processing all data at once. By flexibly adjusting the number of calculated bits, the abnormal bit can be quickly located, and the accurate positioning of data anomalies can be quickly realized.

[0081] See Figure 4 As shown in the figure. Figure 4 It is a schematic diagram of a data recovery method for target data provided by an embodiment of the present invention; in some embodiments, after comparing the logical operation value with the unit check value corresponding to any data block, it further includes:

[0082] If there is an inconsistency between the logical operation value corresponding to the data block in the target data and the corresponding unit check value, perform data recovery of the target data based on the unit check value and the reference data.

[0083] It is not difficult to understand that based on the unit check value, not only can the data integrity be more efficiently verified during the read operation of the redundant disk array by calculating block by block, loading the data block and the reference block for verification bit by bit for each block, and quickly verifying whether the read data is consistent with the unit check value by using the method of comparing bit by bit for each block. If the data of a certain bit is inconsistent with the check value, it can also be quickly marked and further trigger data recovery. During the data recovery process, in the same way, recover the data and compare it bit by bit for each data block. If the data of certain bits is inconsistent, the specific block and bit can be quickly located, and the error cause can be further investigated by compilation. As Figure 4 As shown in the figure, the target data, data block, and unit check value correspond byte by byte and bit by bit. Each of the four data blocks corresponding to the target data includes 256 bytes of data. Suppose it is found that the data D2 is abnormal after data verification, the data D1 is normal, and the data D2 is used as the reference data to generate the data check value P1 of the target data D1. Then the inverse operation D2 = D1 XOR P1 can be used to calculate D2 bit by bit for each block according to the data of D1 and P1, so as to realize the data recovery of the data D2.

[0084] Specifically, when generating the data verification value and performing data verification, by loading the data to be calculated block by block bit by bit, the memory occupation of the software RAID is effectively reduced. Through the data reading and writing verification block by block bit by bit, the data anomaly point can be quickly located during the reading process. Correspondingly, when performing data recovery, only the data block corresponding to the data anomaly point needs to be recovered. In large-scale data recovery, especially in the scenarios of reading and writing and multi-disk failure recovery of RAID5 and RAID6, the recovery efficiency of the software RAID can be effectively improved.

[0085] In some embodiments, comparing the logical operation value and the unit check value corresponding to any data block includes:

[0086] Starting from the lowest bit of the logical operation value, compare the bit values at the corresponding positions of the logical operation value and the unit check value in sequence until there is a situation where the bit values are different or all bits of the logical operation value have been compared.

[0087] Before performing data recovery of the target data based on the unit check value and the reference data, it further includes:

[0088] If the comparison results corresponding to the bit values of all bits of the logical operation value all indicate the same, it is determined that the data block passes the data verification.

[0089] If there is a situation where the bit values are different, it is determined that the logical operation value and the unit check value are inconsistent, and the data block corresponding to the inconsistent logical operation value and the unit check value is marked as an abnormal block.

[0090] Performing data recovery of the target data based on the unit check value and the reference data includes:

[0091] Determine the reference block and the unit check value corresponding to the abnormal block;

[0092] Perform the inverse operation of the preset operation method based on the reference block and the unit check value corresponding to the abnormal block to obtain the normal block corresponding to the abnormal block;

[0093] Restore the abnormal block to the normal block to achieve data recovery of the target data.

[0094] It can be understood that, in order to ensure the accuracy and reliability of data verification, when generating data verification values, exclusive OR calculations are performed bit by bit in blocks to generate respective unit verification values. That is, for any data block, the data block of equal length and the corresponding reference block are exclusive OR calculated bit by bit to generate a unit verification value of the same length as the data block. If the data block length is n bits, then the corresponding reference block and unit verification value are also both n bits, and each bit corresponds to each other. Similarly, an n-bit logical operation value is also calculated, and then the bit values are compared bit by bit. Data verification is completed by whether the bit values of the logical operation value and the unit verification value are exactly the same. If the comparison results corresponding to the bit values of all bits of the logical operation value all indicate the same, it means that the data block has no abnormality; if all data blocks have no abnormality, it means that the entire target data passes the data verification. Once there is a situation where a certain bit value is inconsistent during the comparison process, the abnormal block can be located, and then data recovery is triggered. The normal block corresponding to the abnormal block is calculated using the reference block, unit verification value, and the inverse operation of the preset operation method, and the abnormal block is restored to an accurate and effective normal data block using the normal block.

[0095] Specifically, the unit verification value and / or logical operation value are calculated by corresponding each byte or each bit of the data block and the reference block one by one. The verification and recovery of the data block are realized through bit-by-bit calculation, and then the verification and recovery of the entire target data are realized block by block based on this. A calculation optimization method for generating verification values of software RAID is provided with data blocks as units, which can flexibly adjust the size of data blocks and reduce the memory occupancy of software RAID. The problem data location can be quickly located through block calculation, the subsequent error correction ability is improved, and the efficiency of data recovery is increased.

[0096] In some embodiments, it further includes:

[0097] Storing the unit verification values of all data blocks corresponding to the target data to respective preset positions;

[0098] Or,

[0099] Merging the unit verification values of all data blocks corresponding to the target data according to the division situation of the data blocks to obtain the data verification value of the target data, and storing the data verification value of the target data to a preset position.

[0100] It is not difficult to understand that when verifying and restoring target data, each data block can be calculated independently without reorganizing the data. Therefore, the data verification value can be saved in units of unit verification values, that is, each unit verification value is saved independently. For the convenience of data management, all unit verification values can also be merged and saved, but when verifying the data sum and restoring the data, they can still participate in the calculation in units of unit verification values. This application does not make special limitations on the specific type and implementation method of the preset position. A dedicated disk can be reserved in the redundant disk array to store the data verification value, which is convenient for centralized management and query, improving the reliability and security of storage. It can also be stored on the disk where the data is stored, closely associated with the stored data, facilitating data verification.

[0101] Specifically, there are multiple options for the storage method of the data verification value, which can be selected and adjusted according to the actual situation of the redundant disk array and the application requirements of the stored data, so that it can adapt to different application scenarios, improving the flexibility of data management, enhancing the reliability and security of data, and optimizing the system performance of the redundant disk array.

[0102] In some embodiments, it further includes:

[0103] If there is a situation of secondary writing of the target data, then when the target data is written for the second time, it jumps back to the step of dividing the target data in the redundant disk array into several data blocks to update the data verification value of the target data.

[0104] It can be understood that the stored data in the redundant disk array may be accessed and modified. After the stored data is modified, it is rewritten into the redundant disk array to achieve the storage of the latest data. Therefore, if there is a situation of secondary or repeated writing of the target data, the target data may be modified. At this time, it is necessary to recalculate the data verification value operation to obtain the data verification value corresponding to the latest target data to update the data verification value of the target data. Secondary writing refers to the target data being written into the redundant disk array again, not limited to the operation of the second write.

[0105] Specifically, when the redundant disk array performs a write operation, it is necessary to update the verification value of the currently written stored data. Using the method of calculating bit by bit for each block can optimize the verification value update operation in the write operation, reduce the amount of data that needs to be loaded in the write operation, and improve the write performance through parallel processing. By updating the data verification value, the consistency of the data is guaranteed, data synchronization is ensured, situations such as data tampering are prevented, the security and authenticity of the data are guaranteed, and the accuracy and reliability of data recovery are ensured.

[0106] In some embodiments, determining the unit verification value of each data block based on the one-to-one corresponding data block and reference block includes:

[0107] The calculation operations of the unit check values of several data blocks are respectively allocated to several processing cores, so that each processing core determines the unit check value of a data block based on the corresponding data block and reference block.

[0108] It is not difficult to understand that for multiple data blocks in a target data, operations such as the calculation of their corresponding unit check values, the calculation of logical operation values, and comparison can all be processed in parallel in an independent CPU. Therefore, the exclusive OR calculation tasks corresponding to multiple data blocks can be allocated to multiple CPUs for parallel processing to improve the efficiency of data verification and data recovery. At the same time, the comparison operations of the unit check values and logical operation values corresponding to multiple data blocks can be processed in parallel on the comparators set in independent CPUs. Especially when a redundant disk array is applied in the application scenario of a multi-core CPU, when generating unit check values, multiple CPUs are used to generate the unit check values corresponding to each data block in parallel. When performing data verification, multiple CPUs are used to perform the calculation of each logical operation value and the comparison with the corresponding unit check value in parallel. When recovering data, if there are multiple abnormal blocks, parallel recovery processing of multiple data blocks is supported. The specific type and implementation method of each processing core are not particularly limited in this application. In this embodiment, a multi-core CPU is taken as an example, and the tasks corresponding to each data block can be allocated to each CPU. The processing core can also be other processing modules such as a GPU (Graphics Processing Unit).

[0109] Specifically, on the basis of performing data verification processing in units of data blocks, the calculation and comparison processing operations of multiple data blocks corresponding to a target data can be respectively allocated to different processing cores for implementation. Especially in the scenario applicable to a multi-core CPU, it can further improve the data reading, writing and recovery speed, improve the processing efficiency, and at the same time improve the utilization rate of the CPU by parallelly allocating tasks.

[0110] See Figure 5 as shown in Figure 5 is a schematic structural diagram of a data verification device for a redundant disk array provided by an embodiment of the present invention; to solve the above technical problems, an embodiment of the present invention also provides a data verification device for a redundant disk array, including:

[0111] A partitioning unit 11, configured to partition the target data in the redundant disk array into several data blocks;

[0112] A reference obtaining unit 12, configured to determine several reference blocks corresponding one-to-one to the data blocks from the reference data; wherein, the data length of the reference block is equal to the data length of the corresponding data block;

[0113] A check value determination unit 13, which is used to determine the unit check value of each data block based on the data block and the reference block in one-to-one correspondence;

[0114] A check value generation unit 14, which is used to determine the data check value of the target data by taking the unit check values of all data blocks corresponding to the target data;

[0115] A check unit 15, which is used to perform data check on the target data based on the data check value of the target data.

[0116] In some embodiments, the check unit 15 includes:

[0117] A first loading unit, which is used to load each data block corresponding to the target data in a preset order if there is a data check requirement for the target data;

[0118] A second loading unit, which is used to load the reference block corresponding to any data block after any data block is loaded for any data block;

[0119] A calculation unit, which is used to calculate the logical operation value of the loaded data block and the corresponding reference block based on a preset operation method;

[0120] A comparison unit, which is used to compare the logical operation value with the unit check value corresponding to any data block to implement the data check of the target data.

[0121] In some embodiments, the data check device of the redundant disk array further includes:

[0122] A data recovery unit, which is used to perform data recovery of the target data based on the unit check value and the reference data if there is an inconsistency between the logical operation value corresponding to the data block in the target data and the corresponding unit check value.

[0123] In some embodiments, the comparison unit includes:

[0124] A comparison subunit, which is used to start from the lowest bit of the logical operation value and sequentially compare the bit values at the corresponding positions of the logical operation value and the unit check value until there is a situation where the bit values are different or all bits of the logical operation value have been compared;

[0125] The data check device of the redundant disk array further includes:

[0126] A first determination unit, which is used to determine that the data block passes the data check if the comparison results corresponding to the bit values of all bits of the logical operation value all indicate the same;

[0127] A second determination unit, which is used to determine that the logical operation value is inconsistent with the unit check value if there is a situation where the bit values are different, and mark the data block corresponding to the inconsistent logical operation value and the unit check value as an abnormal block;

[0128] The data recovery unit includes:

[0129] An anomaly location unit for determining a reference block and a unit check value corresponding to an anomalous block;

[0130] A normal value determination unit for performing an inverse operation of a preset operation method based on the reference block and the unit check value corresponding to the anomalous block to obtain a normal block corresponding to the anomalous block;

[0131] A data recovery subunit for restoring the anomalous block to a normal block to achieve data recovery of target data.

[0132] In some embodiments, the data verification device of the redundant disk array further includes:

[0133] A first storage unit for storing the unit check values of all data blocks corresponding to the target data to corresponding preset positions respectively;

[0134] Or,

[0135] A second storage unit for merging the unit check values of all data blocks corresponding to the target data according to the division status of the data blocks to obtain the data check value of the target data, and storing the data check value of the target data to a preset position.

[0136] In some embodiments, the data verification device of the redundant disk array further includes:

[0137] An update unit for, if there is a situation of secondary writing of the target data, re-triggering the division unit 11 when the target data is secondarily written to update the data check value of the target data.

[0138] In some embodiments, the check value generation unit 14 includes:

[0139] A parallel processing unit for respectively allocating the calculation operations of the unit check values of several data blocks to several processing cores, so that each processing core determines the unit check value of the data block based on the corresponding data block and reference block.

[0140] For the description of the features in the data verification device of the redundant disk array provided by the embodiments of the present invention, reference can be made to the relevant descriptions in the embodiments of the data verification method of the redundant disk array, which will not be elaborated here one by one.

[0141] See Figure 6 as shown Figure 6 is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. To solve the above technical problems, an embodiment of the present invention further provides an electronic device, including:

[0142] A memory 60 for storing a computer program;

[0143] A processor 61 for executing a computer program to implement the steps of the data verification method for a redundant disk array as described above.

[0144] The electronic device provided in this embodiment may include, but is not limited to, a smart phone, a tablet computer, a laptop computer, a desktop computer, etc.

[0145] Among them, the processor 61 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 61 may be implemented in at least one of the following hardware forms: digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 61 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the CPU; the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 61 may be integrated with a GPU, and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 61 may further include an artificial intelligence (AI) processor for processing computational operations related to machine learning.

[0146] The memory 60 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory 60 may further include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory 60 is at least used to store the following computer program 601. After the computer program is loaded and executed by the processor 61, it can implement the relevant steps of the data verification method for the redundant disk array disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory 60 may further include an operating system 602 and data 603, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 602 may include Windows, Unix, Linux, etc. The data 603 may include, but is not limited to, the data in the data verification method for the redundant disk array.

[0147] In some embodiments, the electronic device may further include a display screen 62, an input / output interface 63, a communication interface 64, a power supply 65, and a communication bus 66.

[0148] Those skilled in the art can understand that Figure 6 the structure shown in does not constitute a limitation on the electronic device, and it may include more or fewer components than those shown in the figure.

[0149] For the description of the features in the electronic device provided by the embodiments of the present invention, reference can be made to the relevant descriptions in the embodiments of the data verification method of the redundant disk array, which will not be elaborated here one by one.

[0150] It can be understood that if the data verification method of the redundant disk array in the above embodiments is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the current technology, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and executes all or part of the steps of the methods in the various embodiments of the present invention. The aforementioned storage media include: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), electrically erasable programmable ROMs, registers, hard disks, removable disks, CD-ROMs, magnetic disks, or optical discs, etc., which can store program codes.

[0151] To solve the above technical problems, an embodiment of the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the data verification method of the redundant disk array as described above are implemented.

[0152] For the description of the features in the computer-readable storage medium provided by the embodiments of the present invention, reference can be made to the relevant descriptions in the embodiments of the data verification method of the redundant disk array, which will not be elaborated here one by one.

[0153] An embodiment of the present invention also provides a computer program product, including a computer program / instructions. When the computer program / instructions are executed by a processor, the steps of the data verification method of the redundant disk array described in the above embodiments are implemented.

[0154] For the description of the features in the computer program product provided by the embodiments of the present invention, reference can be made to the relevant descriptions in the embodiments of the data verification method of the redundant disk array, which will not be elaborated here one by one.

[0155] The above has introduced in detail a data verification method, device, equipment, and medium of a redundant disk array provided by the embodiments of the present invention. The embodiments in the specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The same or similar parts among the embodiments can be referred to each other. For the device disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple. For the relevant parts, reference can be made to the description in the method part.

[0156] Those skilled in the art can further realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described according to functions in the above description. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0157] The above has introduced in detail a data verification method, device, equipment, and medium of a redundant disk array provided by the present invention. Specific examples are used herein to elaborate on the principle and implementation manner of the present invention. The description of the above embodiments is only used to help understand the method and its core idea of the present invention. It should be noted that for those of ordinary skill in the art in the technical field, without departing from the principle of the present invention, several improvements and modifications can be made to the present invention, and these improvements and modifications also fall within the protection scope of the present invention.

Claims

1. A data verification method for a redundant disk array, characterized in that: include: Divide the target data in the redundant disk array into a number of data blocks; Determine a plurality of reference blocks corresponding to the data blocks one by one from the reference data; wherein the data length of the reference blocks is equal to the data length of the corresponding data blocks; Determine a unit check value of each data block based on the one-to-one corresponding data blocks and reference blocks; Determining the unit check values ​​of all data blocks corresponding to the target data as the data check values ​​of the target data; The target data is verified based on the data verification value of the target data.

2. The data verification method of a redundant disk array according to claim 1, characterized in that: Performing data verification on the target data based on the data verification value of the target data includes: If the target data has a data verification requirement, load each data block corresponding to the target data in a preset order; For any data block, after loading of any data block is completed, a reference block corresponding to the any data block is loaded; Calculate the logical operation value of the loaded data block and the corresponding reference block based on a preset operation method; The logic operation value is compared with a unit check value corresponding to any one of the data blocks to implement data check of the target data.

3. The data verification method of a redundant disk array as claimed in claim 2, characterized in that: After comparing the logic operation value with the unit check value corresponding to any one of the data blocks, the method further includes: If there is a logic operation value corresponding to a data block in the target data and a corresponding unit check value that is inconsistent, data recovery of the target data is performed based on the unit check value and the reference data.

4. The data verification method of a redundant disk array as claimed in claim 3, characterized in that: Comparing the logic operation value with a unit check value corresponding to any one of the data blocks includes: Starting from the lowest bit of the logic operation value, the bit values ​​of the corresponding positions of the logic operation value and the unit check value are compared in sequence until there is a difference in bit values ​​or all bits of the logic operation value have been compared; Before performing data recovery of the target data based on the unit check value and the reference data, the method further includes: If the comparison results corresponding to the bit values ​​of all bits of the logic operation value are all characterized as the same, it is determined that the data block passes the data check; If there is a situation where the bit values ​​are different, it is determined that the logic operation value and the unit check value are inconsistent, and the data block corresponding to the inconsistent logic operation value and unit check value is marked as an abnormal block; Performing data recovery of the target data based on the unit check value and the reference data includes: Determine a reference block and a unit check value corresponding to the abnormal block; Performing an inverse operation of the preset operation method based on a reference block corresponding to the abnormal block and a unit check value to obtain a normal block corresponding to the abnormal block; The abnormal block is restored to the normal block to achieve data recovery of the target data.

5. The data verification method of a redundant disk array according to claim 1, characterized in that: Also includes: The unit check values ​​of all data blocks corresponding to the target data are stored in corresponding preset locations respectively; or, According to the division status of the data blocks, the unit check values ​​of all the data blocks corresponding to the target data are combined to obtain the data check value of the target data, and the data check value of the target data is stored in a preset position.

6. The data verification method of a redundant disk array according to claim 1, characterized in that: Also includes: If the target data is written twice, when the target data is written twice, the process jumps to the step of dividing the target data in the redundant disk array into a plurality of data blocks to update the data check value of the target data.

7. The data verification method of a redundant disk array according to any one of claims 1 to 6, characterized in that: Determining a unit check value of each data block based on the one-to-one corresponding data blocks and reference blocks includes: The calculation operations of the unit check values ​​of the plurality of data blocks are respectively distributed to the plurality of processing cores, so that each of the processing cores determines the unit check value of the data block based on the corresponding data block and the reference block.

8. A data verification device for a redundant disk array, characterized in that: include: A partitioning unit, used for partitioning target data in a redundant disk array into a plurality of data blocks; A reference acquisition unit, used to determine a number of reference blocks corresponding to the data blocks one by one from the reference data; wherein the data length of the reference block is equal to the data length of the corresponding data block; A check value determination unit, configured to determine a unit check value of each data block based on the one-to-one corresponding data blocks and reference blocks; A check value generating unit, used to determine the unit check values ​​of all data blocks corresponding to the target data as the data check value of the target data; A verification unit is used to perform data verification on the target data based on the data verification value of the target data.

9. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the data verification method for a redundant disk array according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the data verification method for a redundant disk array according to any one of claims 1 to 7 are implemented.

Images