A fast data erasure correction method for MDS code based on real-time decoding

Abstract

The present invention relates to a MDS code fast data correction and erasure method based on real-time decoding, belonging to the field of Internet security technology. By pre-storing the inverse matrix of the decoding coefficient matrix in memory, when encoding is performed at the data transmitting end, the receiving end performs decoding preprocessing in real time, and the correctness of data transmission is detected in real time during the data transmission process. When the number of data losses is equal to 2 or the number of losses is equal to 1 (but all messages are transmitted), the transmitting end is notified to perform redundant matrix transmission and the receiving end retrieves the required data from the memory for the final decoding operation. The present invention adopts a scheme of real-time decoding while receiving data, and provides a method for fast correction and erasure of MDS encoded data that is both efficient and occupies little storage resources.

Description

Technical Field

[0001] The invention relates to an MDS code fast data correction and erasure method based on real-time decoding, belonging to the technical field of Internet security. Background Art

[0002] With the rapid development of communication technology, the Internet has a more and more profound impact on various fields such as finance, manufacturing, and entertainment, and the security and stability of network transmission have received more and more attention.

[0003] FEC (Forward Error Correction) adds redundancy to the transmitted data. When the receiving end receives erroneous data, it calculates and processes the redundancy to achieve correction and recovery of lost data. It is widely used in network transmission to reduce the transmission bit error rate and improve the reliability of network data transmission.

[0004] MDS (Maximum Distance Separable) codes have better fault tolerance and are widely used in coding technology. The widely used Reed-Solomon (RS) codes and MSR codes are both MDS codes.

[0005] When the MDS code is encoded progressively, the encoding calculation formula is:

[0006]

[0007] Among them, D ori is the original data matrix; W code is the matrix that is multiplied with the original data to realize the encoding of the original data, that is, the encoding coefficient matrix; R ori is a redundant matrix, and its decoding formula is:

[0008]

[0009] Among them, D rev To recover the lost data; , is the matrix D after data loss occurs ori-rev A new matrix composed of and redundant matrix R; is the coefficient matrix W code Matrix W after deleting the coefficients corresponding to the missing data code-rev The new matrix composed of the unit matrix E; W rec The inverse matrix obtained by inverting the coefficient matrix corresponding to the missing data is referred to as the inverse matrix in the following text. It can also be called the decoding coefficient matrix.

[0010] In the current decoding scheme, the schematic diagram of the traditional MDS code data erasure method is shown in Figure 1 There are two traditional methods. One is to convert the coefficient matrix All of them are calculated and then put into memory. The corresponding data can be directly retrieved during decoding. Its advantage is that the decoding speed is fast and no software calculation is required, but the required storage space is large. Taking 256*512bit data as an example, considering 4 redundancy, the storage space required for decoding is about 320Mb.

[0011] The second traditional method is to receive the data D after the loss occurs. ori-rev The decoding coefficient matrix is ​​calculated by MCU software after adding the redundant R However, this method relies on software participation, which will cause a large network transmission delay. Therefore, the present invention is proposed. Summary of the invention

[0012] In view of the shortcomings of the prior art, the present invention provides an MDS code fast data correction and erasure method based on real-time decoding, which takes into account both storage space and network delay. The storage space only requires about 260Kb, and no MCU is required to participate in the calculation. The decoding is calculated by the hardware circuit, ensuring low delay in network transmission.

[0013] The technical solution of the present invention is as follows:

[0014] A MDS code fast data erasure method based on real-time decoding, the steps are as follows:

[0015] (1) Coding operation;

[0016] (2) While performing the encoding operation, the receiving end begins to use the array compilation hardware circuit B (used for the first time) to process the original data matrix (i.e., D mentioned in the background technology) ori ) and the pre-calculation of the coding coefficient matrix, that is, the decoding calculation formula mentioned in the background technology Some perform real-time operations;

[0017] (3) When the receiving end receives the network data, it performs CRC check and header comparison, and then the transmitting end sends the redundant matrix to the receiving end;

[0018] (4) After receiving the redundant matrix, the receiving end inputs it into the array compilation hardware circuit B (first use) to complete the final Partial calculation is performed, and the calculation result is placed in the first-in-first-out memory FIFO for temporary storage, and then input into the array encoding hardware circuit B (second use) as input data;

[0019] Some calculations require a relatively long process, with step (2) starting the calculation and step (4) completing the calculation;

[0020] (5) Decoding at the receiving end Finally, the lost message is calculated. , realizing real-time correction and deletion of network data.

[0021] Preferably, according to the present invention, in step (1), the coding operation includes coding coefficient matrix calculation and inverse matrix calculation, specifically:

[0022] Pre-calculate the coding coefficient matrix, that is, W introduced in the background technology code , and stored in the non-volatile memory of the receiving and transmitting ends. Taking a 256-bit GF2 8*8 coding coefficient matrix with 2 redundancy as an example, it requires about 4Kb of storage space;

[0023] Pre-calculate the inverse matrix of the coding coefficient matrix when two sets of data are lost, that is, W mentioned above rec , and stored in the non-volatile memory at the receiving end. Taking the 2-redundant 256-bit GF2 8*8 coding coefficient matrix as an example, the required storage space is approximately .

[0024] The coding coefficient matrix and inverse matrix are generated by C language code. When used, they only need to be pre-calculated and stored in the system;

[0025] Preferably, according to the present invention, in step (3), the CRC check is a check of the correctness of the transmitted data, and the header comparison is a comparison of the original data matrix row number information in the message header with the original data matrix row number received by the receiving end.

[0026] Preferably, in step (3), the specific steps are: when a data error caused by bit jump during transmission is found through CRC check, the block of data is discarded, a message loss is recorded once and the position of the lost message in the entire data is recorded; when data loss is found through header comparison, a message loss is recorded once and the position of the lost message in the entire data is recorded;

[0027] When more than 2 messages are lost, or 1 message is lost but all message transmissions are completed, the receiving end notifies the transmitting end to transmit a redundant matrix R containing 2 redundant vectors obtained by operating all transmitted original data matrices with 2 redundant corresponding coefficient matrices.

[0028] According to the present invention, further preferably, in step (3), when there is a scenario in which one message is lost but all message transmissions are completed, the receiving end discards the last message received and constructs a scenario in which two messages are lost. When this scenario is not in place, this step is skipped.

[0029] According to the preferred embodiment of the present invention, in step (5), specifically:

[0030] The receiving end retrieves the inverse matrix of the coefficient matrix corresponding to the lost data in its non-volatile memory , load the inverse matrix into array compilation hardware circuit B (second use);

[0031] In the array compilation hardware circuit B (second use) for decoding part Finally, the lost message is calculated. , realizing real-time correction and deletion of network data.

[0032] The beneficial effects of the present invention are:

[0033] The present invention takes both storage space and network delay into consideration. The storage space only requires about 260Kb. At the same time, no MCU is required to participate in the calculation. The decoding is calculated by the hardware circuit, which ensures low delay in network transmission. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of traditional MDS code data erasure method;

[0035] Figure 2 It is a brief schematic diagram of the present invention;

[0036] Figure 3 It is a detailed schematic diagram of the present invention. DETAILED DESCRIPTION

[0037] like Figure 2-3 As shown, this embodiment provides a MDS code fast data erasure method based on real-time decoding, the steps are as follows:

[0038] S100: The transmitting device A and the receiving device B construct a GF2 finite field matrix that conforms to the MDS code type according to the mathematical definition (for example, constructing a Cauchy coding coefficient matrix, the constructed matrix conforms to the mathematical definition of the Cauchy matrix, and the specific configuration form can be completed by programming software such as C language, matlab, etc.) as the coding coefficient matrix W code Store in memory (both the transmitting device A and the receiving device B need to store the coding coefficient matrix);

[0039] S200: The inverse matrix of the coding coefficient matrix calculated in advance when two sets of data are lost is stored in the memory of the receiving device B (the inverse matrix W rec Only the receiving device B needs to store the inverse matrix, which is obtained by inverting the coefficient matrix corresponding to the lost data);

[0040] S300: The receiving device B performs pre-calculation of the original data matrix and the coding coefficient matrix in real time through the array compilation hardware circuit B (used for the first time) while the transmitting device A performs coding calculation, and the calculation result is temporarily stored in the FIFO;

[0041] The specific calculation content of pre-operation is: decoding formula The data matrix of the transmitted part (D oir+R ) and the coding coefficient matrix corresponding to the transmitted data (W code+E ) performs real-time multiplication calculation, and performs a synchronous calculation every time data is transmitted to obtain the intermediate result of the multiplication calculation of the two matrices, which is expressed as W code+E D is 16 rows and 256 columns. oir+R Take the intermediate result calculated for 256 rows and 512 columns as an example;

[0042] First real-time pre-calculation:

[0043]

[0044] Second real-time pre-calculation:

[0045]

[0046] …

[0047] 256th real-time pre-calculation:

[0048]

[0049] S400: When receiving device B receives network data, it performs CRC check and header comparison in real time, checks for message loss, and records the loss location;

[0050] S500: When more than 2 messages are lost, or 1 message is lost but all message transmissions are completed, the receiving end device B notifies the transmitting end device A to transmit a redundant matrix R including 2 redundant vectors obtained by operating all the above transmitted original data matrices and 2 redundant corresponding coefficient matrices;

[0051] S600: When the number of message losses is equal to 1 but all message transmissions are completed, after executing step S500, the receiving device B discards the last message received and constructs a scenario where two messages are lost. When this scenario is not in place, this process is skipped;

[0052] S700: After receiving the redundant matrix, the receiving device B performs the final The remaining part is calculated, that is, the redundant matrix R and the unit matrix E complete the calculation of the exemplary method of step S300, and the calculation result thereof is added to the intermediate results obtained in all steps S300, and the calculation result is output to the FIFO for temporary storage, and then input as input data into the array encoding hardware circuit B (second use);

[0053] S800: The receiving device B retrieves the inverse matrix of the coefficient matrix corresponding to the lost data in its memory and loads it into the array compilation hardware circuit B (second use);

[0054] S900: Compile hardware circuit B (second use) at receiving device B array for decoding The final calculation is the matrix D ori+R *W code+E With the matrix W rec The lost message is obtained by performing matrix multiplication operations, thus achieving real-time correction and deletion of network data.

[0055] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A MDS code fast data erasure method based on real-time decoding, characterized in that: Here are the steps: (1) Coding operation, including coding coefficient matrix calculation and inverse matrix calculation, specifically: Pre-calculating the coding coefficient matrix and storing it in non-volatile memory at the receiving end and the transmitting end; Precompute the inverse matrix W rec , and stored in the non-volatile memory at the receiving end, the inverse matrix is ​​obtained by inverting the coefficient matrix corresponding to the lost data; (2) While performing the encoding operation, the receiving end begins to perform pre-operation of the original data matrix and the encoding coefficient matrix through the array compilation hardware circuit B, and the operation results are temporarily stored in the first-in-first-out memory; The specific calculation content of the pre-operation is: decoding formula D ori+R *W code+E *W rec =D rev The data matrix D of the transmitted part oir+R The coding coefficient matrix W corresponding to the transmitted data code+E Perform real-time multiplication calculations; (3) When the receiving end receives network data, it performs CRC check and header comparison. When the CRC check finds data errors caused by bit jumps during transmission, the erroneous data is discarded, a message loss is recorded, and the position of the lost message in the entire data is recorded. When data loss is found through header comparison, a message loss is recorded, and the position of the lost message in the entire data is recorded. When more than 2 messages are lost, or 1 message is lost but all message transmissions are completed, the receiving end notifies the transmitting end to transmit the redundant matrix R containing 2 redundant vectors obtained by operating all the transmitted original data matrices with the 2 redundant corresponding coefficient matrices, and then the transmitting end sends the redundant matrix to the receiving end; (4) After receiving the redundant matrix, the receiving end inputs it into the array compilation hardware circuit B and completes the final D with the unit matrix. ori+R *W code+E Partial calculation is performed, and the calculation result is added to the pre-calculation result obtained in step (2), and the calculation result is temporarily stored in a first-in-first-out memory, and then input into the array encoding hardware circuit B as input data; (5) The receiving end performs decoding part D ori+R *W code+E *W rec Finally, the lost message D is calculated. rev , to achieve real-time correction and deletion of network data, specifically: The receiving end retrieves the inverse matrix W of the coefficient matrix corresponding to the lost data in its non-volatile memory rec , load the inverse matrix into the array compilation hardware circuit B; In the array compilation hardware circuit B, the decoding part D ori+R *W code+E *W rec The final calculation is the matrix D ori+R *W code+E With the matrix W rec Matrix multiplication operation is performed to obtain the lost message D rev , realizing real-time correction and deletion of network data.

2. The MDS code fast data erasure method based on real-time decoding as claimed in claim 1, characterized in that: In step (3), the CRC check is to check the correctness of the transmitted data, and the header comparison is to compare the original data matrix row number information in the message header with the original data matrix row number received by the receiving end.

3. The MDS code fast data erasure method based on real-time decoding as claimed in claim 2, characterized in that: In step (3), when there is a scenario where one message is lost but all message transmissions are completed, the receiving end discards the last message received and constructs a scenario where two messages are lost. If this scenario is not the case, skip this step.

Images