A data correction and erasure method and system based on FPGA hardware logic

Abstract

The present invention relates to a data erasure correction method and system based on FPGA hardware logic, belonging to the technical field of integrated circuits. The steps are as follows: (1) receiving original data carrying coding redundancy, and performing verification and stripping processing; (2) calculating and outputting a data packet using the verification data; (3) calculating the lost original data using the data packet; (4) writing the coded data obtained in step (3) into the lost original data packet to complete data erasure correction. The present invention realizes a fast and efficient erasure correction code, and quickly restores the lost data packet when receiving the transmission data.

Description

Technical Field

[0001] The present invention relates to a data correction and erasure method and system based on FPGA hardware logic, belonging to the technical field of integrated circuits. Background Art

[0002] The current information age is constantly developing and digital content is growing rapidly. Especially with the rapid development of network technology, extremely large amounts of data are generated in various fields, which makes data management face huge challenges. The explosive growth of data volume has led to the continuous increase in the scale and complexity of storage systems, making data storage and transmission increasingly difficult. Data may encounter various problems during transmission, such as network congestion, transmission errors or malicious attacks, which may lead to data loss or damage. Ensuring the reliability of data during transmission is one of the important issues that need to be solved urgently.

[0003] To prevent data loss, erasure coding technology is usually used to divide the original data into blocks, and the redundant block data is calculated based on the original data blocks using an algorithm. The original data blocks carrying redundancy are transmitted together. When the number of lost packets is less than the number of redundant blocks during the transmission process, the lost data packets can be restored by using an algorithm when the data is received.

[0004] Erasure code technology is mainly divided into Reed-Solomon (RS) code and array code. RS erasure codes usually use polynomials in the Galois field GF (2w) to perform encoding and decoding operations, which will lead to higher encoding and decoding computational complexity and require huge hardware resources for hardware implementation. The encoding and decoding process of array codes usually uses XOR and cyclic shift operations, and their computational complexity and hardware implementation resources are greatly reduced compared to RS codes.

[0005] In the decoding system after receiving the encoded data, a system interrupt is usually required to make the software drive the CPU to participate in the calculation of the matrix, which greatly reduces the decoding speed. For this reason, the present invention is proposed to utilize the existing resources of FPGA hardware, without using any software and CPU to participate in the calculation, to quickly realize decoding and erasure correction, and improve the erasure correction speed. Summary of the invention

[0006] In view of the deficiencies in the prior art, the present invention provides a data erasure correction method and system based on FPGA hardware logic, which realizes fast and efficient erasure correction codes and quickly restores lost data packets when receiving transmitted data.

[0007] The technical solution of the present invention is as follows:

[0008] A data erasure method based on FPGA hardware logic, the steps are as follows:

[0009] (1) Receive the original data with coding redundancy and perform verification and stripping processing;

[0010] (2) Calculate and output the data packet using the verification data;

[0011] (3) Calculate the lost original data using data packets;

[0012] (4) Write the encoded data obtained in step (3) into the lost original data packet to complete data erasure.

[0013] Preferably, according to the present invention, in step (1), the verification step is:

[0014] Perform CRC32 check on the 512-bit original data. The check method is to use combinational logic to check the 512-bit bits in the clock 8 times for 64 bits respectively, and get a 256-bit CRC check code. By performing XOR operation on each 32 bits of the check code, the check code is compressed to 32 bits.

[0015] The 32-bit data of the compression check is compared with the 32-bit CRC of the first end of the data input. If the comparison result is inconsistent, it is regarded as a data error. When an error occurs, the error is counted. The count is divided into the error count when the input data is the original data and the coding redundancy error count under the coding redundancy enable signal.

[0016] According to the preferred embodiment of the present invention, in step (1), the stripping step is:

[0017] The CRC in the original data and the coded redundancy is stripped off, and the remaining original data carrying INFO and the coded redundancy are stored in RAM in sequence.

[0018] According to the preferred embodiment of the present invention, in step (2), specifically:

[0019] According to the two error counts generated by the check, the total amount of error data is obtained by addition operation, and it is determined whether the total amount of error data is zero. If it is zero, it means that there is no packet loss during the transmission process, and the original data stored in the RAM is read and output directly;

[0020] If the total amount of error data exceeds the total amount of coded redundant data, it means that the packet loss or error exceeds the maximum range of erasure correction, an error excess signal is generated, and the data in the RAM is output in sequence;

[0021] If the total amount of erroneous data does not exceed the total amount of coded redundant data, the correct original data and coded redundant data in the RAM are read. The total number of correct original data and coded redundant data read is the total number of original data in the input data. The original data numbers of errors or packet loss are recorded, and then a new coefficient matrix is ​​generated.

[0022] According to the present invention, further preferably, in step (2), the step of generating a new coefficient matrix is:

[0023] ① According to the original data number of the error or packet loss, read the coefficient matrix stored in the coefficient matrix memory during encoding;

[0024] ② Use Gaussian elimination method and other methods to calculate the inverse matrix of the coefficient matrix corresponding to the erroneous or lost original data, XOR the remaining coefficient matrix in the coefficient matrix memory with the inverse matrix, generate the code, and splice the inverse matrix data to the end of the code to generate a new coefficient matrix.

[0025] According to the preferred embodiment of the present invention, in step (3), specifically:

[0026] Combine the data read in step (2) when the total amount of error data does not exceed the total amount of coded redundant data with the new coefficient matrix

[0027] XOR operation, the result of the operation is the lost original data.

[0028] Preferably, in step (4), the encoded data obtained in step (3) is rewritten into the lost original data packet in the RAM according to the recorded address, and all the original data packets in the RAM are output to complete the correction and deletion of lost or erroneous original data.

[0029] A data erasure correction system based on FPGA hardware logic, comprising:

[0030] Verification and stripping module: used to receive the original data carrying coding redundancy and perform verification and stripping processing;

[0031] Verification data calculation module: uses verification data to calculate and output data packets;

[0032] Lost data calculation module: uses data packets to calculate the lost original data;

[0033] Repair module: used to write the encoded data obtained in step (3) into the lost original data packet to complete data correction and erasure.

[0034] The beneficial effects of the present invention are:

[0035] The present invention determines the destination of the original data through verification analysis and judgment of the original data and coding redundancy. When the original data is lost, the lost original data packet is retrieved by calling, splicing, and calculating the stored coefficient matrix, and the correct original data packet is completely output. The existing resources of FPGA hardware are used, and no software and CPU are used to participate in the calculation, so that decoding and erasure correction can be quickly realized, thereby improving the erasure correction speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is the timing diagram for crc calculation, counting and input data bit width interpretation when the encoded data is input;

[0037] Figure 2 This is a correction and erasure flow chart of the present invention;

[0038] Figure 3 It is a schematic diagram of the coefficient matrix during encoding of the present invention. DETAILED DESCRIPTION

[0039] The present invention will be further described below by way of embodiments in conjunction with the accompanying drawings, but is not limited thereto.

[0040] Embodiment 1:

[0041] The application scenario of this embodiment is that after the MDS array code is encoded, the encoding method is an XOR operation of the original data and the coefficient matrix;

[0042] See the timing diagram of crc calculation, counting and input data width interpretation when the encoded data is input. Figure 1 The data transmitted by data_in includes CRC, INFO and original data or coded redundant data, where CRC is the check code for the original data or coded redundant data, INFO is the packet position where the data is recorded, and the position of the coefficient matrix when the coefficient matrix participates in the operation. Figure 1 The signal interpretation is shown in Table 1:

[0043] Table 1: Figure 1 Explanation table of each signal in

[0044]

[0045] The data correction and erasure method based on FPGA hardware logic in this embodiment is as follows: Figure 2 As shown, the steps are as follows:

[0046] A01: After receiving the encoded original data and coding redundancy, perform CRC32 check on the 512-bit data. The check method is to use combinational logic to check the 512-bit bits in the clock for 8 times in 64 bits respectively, and obtain a 256-bit CRC check code. Perform XOR operation on each 32 bits of the check code to compress the check code to 32 bits. Compare the 32-bit data of the check with the 32-bit CRC of the first end of the data input. If an error occurs, count the error. The count is divided into the error count when the input data is the original data, and the coding redundancy error count under the coding redundancy enable signal;

[0047] A02: Strip the carried CRC and store the remaining original data and coded redundancy into RAM;

[0048] A03: Determine whether the original data and coding redundancy error counters are zero?

[0049] A04: If it is zero, it means that there is no packet loss in the original data and coding redundancy, and the original data stored in RAM is read and output directly;

[0050] A05: If it is not zero, it means that there is packet loss or error in the original data and coding redundancy. Determine whether the sum of the original data and coding redundancy error counters exceeds the maximum redundancy number?

[0051] A06: If the sum of the original data and coding redundancy error counter counts exceeds the maximum redundancy number, it means that the packet loss or error exceeds the maximum range of erasure correction, an error excess signal is generated and the data stored in the RAM is read;

[0052] A07: If the sum of the original data and encoding redundancy error counter counts does not exceed the maximum redundancy count, record the positions of the original data and encoding redundancy errors, and read the original data and encoded data of the non-error addresses in the RAM in sequence until the total amount of read data is the total amount of original data input when the system is input;

[0053] B01: Read and store the coefficient matrix of the original data when encoding to generate coding redundancy;

[0054] B02: Read the coefficient matrix when the stored original data is encoded, but do not read the coefficient matrix involved in XOR when the original data is wrong;

[0055] B03: Read the coefficient matrix involved in XOR when the original data is wrong, and use the Gaussian elimination method and other methods to calculate the inverse matrix;

[0056] B04: XOR the coefficient matrix read in step B02 with the inverse matrix calculated in step B03, and concatenate the inverse matrix of step B03 to the end of the matrix generated by XOR as a new coefficient matrix;

[0057] A08: performing an XOR operation on the data packet read in step A07 and the new coefficient matrix generated in step B04 to calculate new coded data, which is the lost original data packet;

[0058] A09: rewrite the newly generated original data packet into the lost original data packet in the RAM according to the recorded address, and output all the original data packets in the RAM, thus completing the correction and deletion of lost or erroneous original data.

[0059] The schematic diagram of the coefficient matrix during encoding is shown in Figure 3 .

[0060] Since the number of packet losses needs to be calculated in the counting register, the loss of the original data packet is an uncertain number, so the inverse matrix of the coefficient matrix corresponding to the packet loss data is calculated for all possibilities of losing the original data. Taking 4 coding redundancies as an example, there are 4 packet loss situations as shown in Table 2:

[0061] Table 2: Redundant packet loss of 4 codes

[0062]

[0063] It is necessary to calculate the inverse matrix of the corresponding coefficient matrix for the four cases, or construct the inverse matrix to solve the IP. In practical applications, the elements of the coefficient matrix may be composed of unit matrices. If the unit matrix is ​​n×n, the inverse matrix that needs to be solved when 2 original data packets are lost is 2n×2n.

[0064] Embodiment 2:

[0065] A data erasure correction system based on FPGA hardware logic for implementing the method described in Example 1, comprising:

[0066] Verification and stripping module: used to receive the original data carrying coding redundancy and perform verification and stripping processing;

[0067] Verification data calculation module: uses verification data to calculate and output data packets;

[0068] Lost data calculation module: uses data packets to calculate the lost original data;

[0069] Repair module: used to write the encoded data obtained in step (3) into the lost original data packet to complete data correction and erasure.

[0070] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A data erasure method based on FPGA hardware logic, characterized in that: Here are the steps: (1) Receive the original data with coding redundancy and perform verification and stripping processing; The verification steps are: Perform CRC32 check on the 512-bit original data. The check method is to use combinational logic to check the 512-bit bits in the clock 8 times in 64 bits respectively, and get a 256-bit CRC check code. By performing XOR operation on each 32 bits of the check code, the check code is compressed to 32 bits. Compare the 32-bit data of the compression check with the 32-bit CRC of the first end of the data input. If the comparison result is inconsistent, it is regarded as a data error. When an error occurs, the error is counted. The count is divided into the error count when the input data is the original data and the coding redundancy error count under the coding redundancy enable signal; The peeling steps are: The CRC in the original data and the coded redundancy is stripped off, and the remaining original data carrying INFO and the coded redundancy are stored in RAM in sequence; (2) Use the verification data to calculate and output the data packet, specifically: According to the two error counts generated by the check, the total amount of error data is obtained by addition operation, and it is determined whether the total amount of error data is zero. If it is zero, it means that there is no packet loss during the transmission process, and the original data stored in the RAM is read and output directly; If the total amount of error data exceeds the total amount of coded redundant data, it means that the packet loss or error exceeds the maximum range of erasure correction, an error excess signal is generated, and the data in the RAM is output in sequence; If the total amount of erroneous data does not exceed the total amount of coded redundant data, the correct original data and coded redundant data in the RAM are read. The total amount of correct original data and coded redundant data read is the total amount of original data in the input data. The number of the erroneous or lost original data is recorded, and then a new coefficient matrix is ​​generated. The steps to generate a new coefficient matrix are: ① According to the original data number of the error or packet loss, read the coefficient matrix stored in the coefficient matrix memory during encoding; ② Calculate the inverse matrix of the coefficient matrix corresponding to the original data with errors or packet loss, XOR the remaining coefficient matrix in the coefficient matrix memory with the inverse matrix, generate the code, and splice the inverse matrix data to the end of the code to generate a new coefficient matrix; (3) Using the data packets to calculate the lost original data, perform an XOR operation on the data read in step (2) when the total amount of error data does not exceed the total amount of coded redundant data and the new coefficient matrix. The result of the operation is the lost original data. (4) Write the encoded data obtained in step (3) into the lost original data packet to complete data erasure.

2. The data erasure method based on FPGA hardware logic according to claim 1, characterized in that: In step (4), specifically, the encoded data obtained in step (3) is rewritten into the lost original data packet in the RAM according to the recorded address, and all the original data packets in the RAM are output, thereby completing the correction and deletion of lost or erroneous original data.

3. A data erasure correction system based on FPGA hardware logic, used to implement the data erasure correction method based on FPGA hardware logic according to claim 1, characterized in that: include: Verification and stripping module: used to receive the original data carrying coding redundancy and perform verification and stripping processing; Verification data calculation module: uses verification data to calculate and output data packets; Lost data calculation module: uses data packets to calculate the lost original data; Repair module: used to write the encoded data obtained in step (3) into the lost original data packet to complete data correction and erasure.

Images