A system and method for improving encoding and decoding hardware based on sparse erasure codes

Abstract

The present invention relates to the field of computer technology, and in particular to a system and method for improving encoding and decoding hardware based on sparse erasure codes. The system includes a data input cache module, a selector module, an input control module, a multiplication calculation module, and an output control module. The present application serves as a universal computing hardware solution for encoding / decoding / check codes of sparse erasure codes, thereby greatly reducing the resource consumption of the system and saving the scheduling complexity of the system engine. By adding check code calculations on the basis of RS erasure code calculations and sharing the same set of hardware engines with the codec, the number of calculations and calculation time of the system's inverse matrix are greatly reduced through different configurations and scheduling, thereby improving the overall performance of the system. The hardware sharing mechanism increases the flexibility of the system calculations, saving hardware resources and overhead. The present application also makes up for some scenarios where RS erasure code calculations cannot be calculated, thereby improving the reliability of the system.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a system and method for improving encoding and decoding hardware based on sparse erasure codes. Background Art

[0002] In the fields of data transmission and storage, erasure codes are highly favored for their lower storage costs. RS (Reed-Solomon) codes are a common erasure code. Their principle is to calculate N check blocks from M data blocks. From a total of M+N data blocks, simply selecting M normal blocks can restore the entire original data. Erasure codes are particularly important in ensuring data reliability in the data storage field.

[0003] The RS erasure coding process is as follows Figure 1 As shown, B is the matrix used for encoding, the gray part such as B11 in the lower half is the Cauchy or Vandermonde matrix, D is the storage data disk that needs to be erased, and the obtained C is the encoded data.

[0004] When some data blocks are lost, a new matrix operation relationship is reorganized and multiplied with its inverse matrix to obtain the original data, that is, the RS erasure decoding process is as follows: Figure 2 As shown in the figure, Survivors are the normal data remaining after the abnormal storage disk occurs, and B' is the matrix formed by the encoding matrix B corresponding to the row where the normal data is located. -1 is the inverse matrix of B′.

[0005] However, the traditional method of scheduling erasure coding matrices is to use DMA to schedule the matrices in memory in real time based on the matrix currently being calculated, transferring them to the hardware for erasure coding and decoding. However, this method has several significant drawbacks: First, the coding and decoding calculations require the preparation of multiple engines, that is, multiple hardware resources perform encoding, decoding, and verification calculations completely independently, consuming a large amount of hardware resources. Second, the RS decoding process requires a large number of inverse matrix calculations, which are computationally expensive and complex, and consumes a large amount of computing power. Summary of the Invention

[0006] In response to the shortcomings of traditional real-time RS erasure coding and decoding calculations, this application starts with the use of sparse erasure coding and decoding calculations, and designs a coding and decoding hardware improvement method and system based on sparse erasure codes. By adding check code calculations on the basis of RS erasure coding and decoding calculations, and sharing the same hardware engine with the coding and decoding, the number of calculations and calculation time of the system's inverse matrix are greatly reduced through different configurations and scheduling, thereby improving the overall performance of the system. In addition, the hardware sharing mechanism increases the flexibility of system calculations, saving hardware resources and overhead. This application also compensates for some scenarios where RS erasure coding and decoding cannot be calculated, thereby improving the reliability of the system.

[0007] To achieve the above objectives, the embodiments of the present invention provide the following technical solutions:

[0008] In a first aspect, in one embodiment provided by the present invention, a system for improving encoding and decoding hardware based on sparse erasure codes is provided, comprising:

[0009] A data input buffer module, which is used for data portion of erasure code calculation;

[0010] A selector module, the selector module is used to connect to the input control module, and the input control module selects a matrix or a fixed value for calculation through the selector module according to the read configuration analysis;

[0011] An input control module, configured to control the data input multiplication calculation module of the received data input buffer module, and further configured to schedule the data to align the data on both sides before entering the Galois field multiplier, select the use of fixed value input or matrix input according to the configured mode, determine the number of reads, and internally count whether the current calculation is completed and when to perform the next calculation;

[0012] A multiplication calculation module, wherein the multiplication calculation module calculates data based on a Galois field multiplier;

[0013] The output control module controls the output of data or the entry of data into the cyclic temporary cache according to the configuration, thereby controlling the output flow rate of the current computing system.

[0014] As a further solution of the present invention, the data input cache module is also used to determine the data flow rate based on the reading speed. The data input cache module is the data part of the erasure code calculation, that is, the data block; wherein, the number of data blocks calculated for verification depends on the grouping configuration in the erasure code encoding and decoding verification code process.

[0015] As a further solution of the present invention, the selector module is connected to a matrix data cache module and a fixed value input module. The matrix data cache module is the matrix part of the erasure code calculation. If the current calculation is encoding, this part is B of the encoding calculation; if the current calculation is decoding, this part is B of the decoding calculation. -1 ;

[0016] When the fixed value input module is currently calculating the check code, the fixed value input module part is a fixed value 1.

[0017] As a further solution of the present invention, the improved encoding and decoding hardware system based on sparse erasure codes also includes a configuration input module and a configuration output module. The configuration input module is connected to the input control module, and the configuration output module is connected to the output control module. The configuration input module and the configuration output module are used to cache the configuration data calculated for the current system, and are read by the calculation conditions of the configuration input module and the configuration output module.

[0018] As a further solution of the present invention, if any one of the configuration input module and the configuration output module is about to be full, the system input cannot enter the module back pressure.

[0019] As a further solution of the present invention, the improved encoding and decoding hardware system based on sparse erasure codes further includes a loop temporary cache module, which is used for caching the current loop calculation and temporarily storing the temporary results of the loop addition calculation.

[0020] As a further solution of the present invention, the size of the partial cache of the circular temporary cache module is determined based on the largest data block in system processing.

[0021] As a further solution of the present invention, the improved encoding and decoding hardware system based on sparse erasure codes performs sparse erasure coding and decoding and check code calculations, including three calculation rules: encoding calculation, decoding calculation, and check calculation, wherein:

[0022] The coding calculation: C n =B n1 *D1+B n2 *D2+…+B nN *D N , where N is the calculated original data block constant value, n is the calculated coding block variable value, n<N and both are positive integers;

[0023] The decoding calculation: Wherein, N is the calculated original data block constant value, n is the calculated decoding block variable value, n<N and both are positive integers;

[0024] The verification calculation: X n =1*D m+1+1*D m+2 +…+1*D m+k , where n is the calculated check block variable value, m is the starting data block value, and k is the number of data blocks involved in the calculation.

[0025] As a further solution of the present invention, the improved encoding and decoding hardware system based on sparse erasure codes also includes an added check code. On the basis of RS erasure code calculation, the added check code calculation shares the same hardware engine with the encoding and decoding.

[0026] In a second aspect, in another embodiment provided by the present invention, a method for improving codec hardware based on sparse erasure codes is provided. The method for improving codec hardware based on sparse erasure codes uses the above-mentioned codec hardware improvement system based on sparse erasure codes to perform sparse erasure code system scheduling operations. The method for improving codec hardware based on sparse erasure codes includes the following steps:

[0027] Get the mode and data of the CPU's calculations;

[0028] If the CPU only sends a single encoding, decoding, or checksum calculation mode, based on the current hardware configuration and the data sent, the checksum calculation is scheduled first, and after completion, one of the data blocks is restored;

[0029] Then, RS correction encoding calculation and RS correction decoding calculation are performed in sequence according to the decoding block to complete the grayscale of the data block.

[0030] In a third aspect, in another embodiment provided by the present invention, a computer device is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the encoding and decoding hardware improvement method based on sparse erasure code when loading and executing the computer program.

[0031] In a fourth aspect, in another embodiment provided by the present invention, a storage medium is provided, which stores a computer program, and when the computer program is loaded and executed by a processor, the steps of the method for improving the encoding and decoding hardware based on sparse erasure codes are implemented.

[0032] The technical solution provided by the present invention has the following beneficial effects:

[0033] The present invention provides a system and method for improving codec hardware based on sparse erasure codes. By adding check code calculations to the RS erasure codec calculations, the system shares the same hardware engine with the codec. Through different configurations and scheduling, the system significantly reduces the number of inverse matrix calculations and the calculation time, thereby improving the overall system performance. Furthermore, the hardware sharing mechanism increases the flexibility of system calculations, saving hardware resources and overhead. This application also addresses some scenarios where RS erasure codecs cannot be calculated, thereby improving system reliability.

[0034] This invention provides a system and method for improving encoding and decoding hardware based on sparse erasure codes. As a universal computing hardware solution for encoding, decoding, and checking sparse erasure codes, it significantly reduces system resource consumption and engine scheduling complexity. It also addresses scenarios where traditional RS erasure codec hardware solutions are unable to compute, improving system reliability.

[0035] These and other aspects of the present invention will become more readily apparent in the following description of the embodiments. It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and are not intended to limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. In the drawings:

[0037] Figure 1 Schematic diagram of the RS erasure coding process.

[0038] Figure 2 Schematic diagram of the RS erasure decoding process.

[0039] Figure 3 A schematic diagram showing an example of grouping in the erasure coding and decoding process.

[0040] Figure 4 A schematic diagram illustrating another type of grouping in the erasure coding and decoding process.

[0041] Figure 5 This is a system block diagram of a system for improving encoding and decoding hardware based on sparse erasure codes according to an embodiment of the present invention.

[0042] Figure 6 The present invention is a flowchart of a method for improving encoding and decoding hardware based on sparse erasure codes according to an embodiment of the present invention.

[0043] Figure 7This is a flowchart of encoding / decoding calculation scheduling in a method for improving encoding / decoding hardware based on sparse erasure codes according to an embodiment of the present invention.

[0044] Figure 8 This is a flowchart of check code calculation scheduling in a method for improving encoding and decoding hardware based on sparse erasure codes according to an embodiment of the present invention. DETAILED DESCRIPTION

[0045] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0046] In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this application or may be executed in parallel. The serial numbers of the operations, such as 101, 102, etc., are only used to distinguish between different operations, and the serial numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this application are used to distinguish different messages, devices, modules, etc., and do not represent the order of precedence, nor do they limit "first" and "second" to be different types.

[0047] The following will clearly and completely describe the technical solutions in the exemplary embodiments of the present invention in conjunction with the accompanying drawings of the exemplary embodiments of the present invention. Obviously, the exemplary embodiments described are only part of the embodiments of the present invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0048] The traditional method of scheduling erasure coding matrices is to use the matrix that needs to be calculated at the moment. The software uses DMA to schedule the matrix in memory in real time and transfer it to the hardware for erasure coding and decoding calculations. However, this method has several major drawbacks:

[0049] 1. Encoding and decoding calculations require the preparation of multiple engines, that is, multiple hardware resources will be used to perform encoding, decoding, and verification calculations completely independently of each other, and the calculations will be performed while consuming more hardware resources.

[0050] Second, during the RS decoding process, a large number of inverse matrix calculations are required, which has high overhead complexity and consumes a lot of computing performance.

[0051] In response to the shortcomings of traditional real-time RS erasure code calculations, the present invention provides a codec hardware improvement system and method based on sparse erasure code. By adding check code calculations on the basis of RS erasure code calculations, the same hardware engine is shared with the codec. Through different configurations and scheduling, the number of calculations and calculation time of the system's inverse matrix are greatly reduced, thereby improving the overall performance of the system. The hardware sharing mechanism increases the flexibility of system calculations, saving hardware resources and overhead. This application also compensates for some scenarios where RS erasure code calculations cannot be calculated, thereby improving the reliability of the system.

[0052] Specifically, the embodiments of the present application are further described below with reference to the accompanying drawings.

[0053] See also Figures 1 to 8 As shown, an embodiment of the present invention provides a sparse erasure code-based encoding and decoding hardware improvement system. When the sparse erasure code-based encoding and decoding hardware improvement system performs sparse erasure code encoding and decoding and check code calculation, it includes three calculation rules: encoding calculation, decoding calculation, and check calculation.

[0054] See also Figures 1 to 4 As shown, the coding calculation is: C n =B n1 *D1+B n2 *D2+…+B nN *D N , where N is the calculated original data block constant value, n is the calculated coding block variable value, n<N and both are positive integers;

[0055] The decoding calculation: Wherein, N is the calculated original data block constant value, n is the calculated decoding block variable value, n<N and both are positive integers;

[0056] The verification calculation: X n =1*D m+1 +1*D m+2 +…+1*D m+k , where n is the calculated check block variable value, m is the starting data block value, and k is the number of data blocks involved in the calculation.

[0057] In an embodiment of the present application, the improved codec hardware system based on sparse erasure code also includes an added check code. On the basis of RS erasure codec calculation, the added check code calculation shares the same hardware engine with the codec.

[0058] Based on the above three calculation rules, the sparse erasure codec hardware improvement solution provided by the embodiment of the present invention is as follows: Figure 5 As shown, a common computing module is designed for the three types of calculations, and calculations are performed by scheduling different data, matrices, and configurations.

[0059] See also Figure 5 As shown, an embodiment of the present invention provides a sparse erasure code-based encoding and decoding hardware improvement system, which includes a data input cache module, a matrix input cache module, a fixed value input module, a configuration input cache module, a selector, an input control module, a multiplication calculation module, an output control module, a configuration output cache module, and a loop temporary cache module.

[0060] The data input buffer module stores the data portion of the erasure code calculation, i.e., the data blocks; it is used to determine the data flow rate based on the reading speed. It should be noted that the number of data blocks used for verification calculation depends on the grouping configuration during the erasure code decoding process.

[0061] See also Figure 3 and Figure 4 As shown in the figure, in the erasure coding and decoding process, since the improved scheme adds a check block in addition to the data block and the coding block, the sparse erasure code is essentially RS coding + parity check block. The encoding steps are as follows:

[0062] a) Use RS encoding on the original data, and the encoding result is data block D m , coding block Cn;

[0063] b) Generate D m Check block X k (The number of k can be configured according to the application requirements);

[0064] c) When one of the data blocks is lost, first consider recovery through the check block, then consider recovery through the coding block.

[0065] like Figure 3 and Figure 4 The figure shows the check code generation process. The system can configure the grouping of data blocks according to the application scenario and actual situation, and then determine the number of check blocks.

[0066] like Figure 3 Shown and Figure 4 As shown in FIG, a schematic diagram of different grouping examples in the erasure coding and decoding process.

[0067] And, by Figure 1 and Figure 2 As can be seen, the computational architecture of erasure coding and decoding is essentially the same, involving matrix multiplication by data. In erasure coding, the matrix is ​​the encoding matrix, and the data is the original data block; in erasure decoding, the matrix is ​​the inverse matrix corresponding to the correct row matrix, and the data is the normal data block. Therefore, when implementing the erasure coding and decoding hardware solution, module reuse is possible, requiring only the proper scheduling of the encoding and decoding matrices and data on the input side.

[0068] In an embodiment of the present application, the selector module is used to connect to the input control module, and the input control module selects a matrix or a fixed value for calculation through the selector module according to the read configuration analysis.

[0069] The selector module connects the matrix data cache module and the fixed value input module. The matrix data cache module is the matrix part of the erasure code calculation. If the current calculation is encoding, this part is B of the encoding calculation; if the current calculation is decoding, this part is B of the decoding calculation. -1 When the fixed value input module is currently calculating the check code, the fixed value input module part is a fixed value 1.

[0070] In an embodiment of the present application, the multiplication calculation module calculates data based on a Galois Field multiplier.

[0071] In an embodiment of the present application, the input control module is used to control the data input multiplication calculation module of the received data input cache module, and is also used to schedule the data to align the data on both sides before entering the Galois field multiplier, select the use of fixed value input or matrix input according to the configured mode, determine the reading quantity, and internally count whether the current calculation is completed and when to perform the next calculation.

[0072] In an embodiment of the present application, the output control module includes controlling the output of data or entering the circular temporary cache according to the configuration, controlling the output flow rate of the current computing system, and the output control module also includes a Galois field adder for the entire calculation. The output control module is also used to control the output flow rate of the current computing system.

[0073] In an embodiment of the present application, the configuration input module is connected to the input control module, the configuration output module is connected to the output control module, and the configuration input module and the configuration output module are used to cache the configuration data calculated for the current system, which is read by the calculation status of the configuration input module and the configuration output module.

[0074] If any one of the configured input module and the configured output module is about to be full, the system input cannot enter the module back pressure.

[0075] In an embodiment of the present application, the loop temporary cache module is used for caching the current loop calculation to temporarily store the temporary result of the loop addition calculation. The size of the partial cache of the loop temporary cache module is determined based on the largest data block in the system processing.

[0076] In an embodiment of the present application, the current loop calculation cache temporarily stores the temporary results of the loop addition calculation to avoid repeated data scheduling. The size of this part of the cache depends on the maximum data block processed by the system. For example, if the maximum size of a single data block is 4MB, the size of this part is 4MB.

[0077] See also Figure 6 As shown, an embodiment of the present invention further provides a method for improving encoding and decoding hardware based on sparse erasure codes, the method comprising:

[0078] Get the mode and data of the CPU's calculations;

[0079] If the CPU only sends a single encoding, decoding, or checksum calculation mode, based on the current hardware configuration and the data sent, the checksum calculation is scheduled first, and after completion, one of the data blocks is restored;

[0080] Then, RS correction encoding calculation and RS correction decoding calculation are performed in sequence according to the decoding block to complete the grayscale of the data block.

[0081] In an embodiment of the present application, the CPU sends calculations in a mode that only includes a single encoding / decoding / check code calculation. If three data blocks are lost in the 4+2 mode, based on the current hardware, the system should first schedule the check code calculation, restore one of the data blocks after completion, and then perform 4+2 recovery based on the decoding block calculation.

[0082] See also Figure 7 As shown, Figure 7 This is a flowchart for encoding / decoding calculation scheduling. The encoding / decoding calculation scheduling process includes sequentially performing encoding / decoding configuration scheduling, encoding / decoding matrix scheduling, and data scheduling. During encoding configuration scheduling, a 4+1 code is calculated, with a single 4KB data block, such as cfg_info. During encoding matrix scheduling, the four codes for this code are scheduled, such as B11 / B12 / B13 / B14. During encoding data scheduling, the four 4KB data blocks for this code are scheduled, such as D1 / D2 / D3 / D4, each 4KB in size. This results in a calculation result of C1 being 4KB.

[0083] See also Figure 8 As shown, Figure 8 This is a check code calculation scheduling flow chart. The check code calculation scheduling process is as follows: check code configuration scheduling and data scheduling are performed in sequence. When scheduling check code configuration, a 2+1 check code is calculated, and the data block size is 4KB, for example, cfg_info. When scheduling coded data, two 4KB data blocks of the check code are scheduled, for example, D1 / D2, each 4KB in size, resulting in a calculation result of X1 of 4KB.

[0084] In this embodiment, a hardware improvement method for encoding and decoding based on sparse erasure codes is implemented. Sparse erasure coding and decoding is based on the RS erasure coding and decoding, with the addition of a portion of check blocks. This addition can compensate for the shortcomings of RS erasure coding and decoding in many scenarios. For example, in most storage systems, the loss of one data block accounts for the vast majority. However, RS erasure coding and decoding both require Galois Field multiplication and matrix inversion calculations. However, the addition of check blocks can solve this problem by only performing an XOR calculation, eliminating the need for complex matrix inversion calculations and the need for inverse matrix scheduling. Furthermore, check blocks can compensate for the uncalculated portions of RS erasure codes in certain scenarios. For example, if three blocks of data are lost in a 4+2 scenario, if appropriate grouping is achieved, one of the data blocks can be recovered using a check code, and the other two blocks can be recovered using RS erasure coding and decoding. This scenario solves the problem of RS erasure coding and decoding being unable to recover data.

[0085] In the embodiments of the present invention, the hardware improvement method for encoding and decoding based on sparse erasure codes, as detailed above, demonstrates the superiority of this hardware implementation. By designing a universal computing hardware solution for encoding, decoding, and checking sparse erasure codes, this solution significantly reduces system resource consumption and reduces the scheduling complexity of the system engine. It also addresses some scenarios that cannot be calculated using traditional RS erasure codec hardware solutions, thereby improving system reliability.

[0086] It should be noted that the method for improving codec hardware based on sparse erasure codes uses a codec hardware improvement system based on sparse erasure codes as described above to perform the above steps. Therefore, the composition of the codec hardware improvement system based on sparse erasure codes will not be introduced in detail in this embodiment.

[0087] In one embodiment, a computer device is further provided in an embodiment of the present invention, comprising at least one processor and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the hardware improvement method for encoding and decoding based on sparse erasure codes, and the processor implements the steps in the above-mentioned method embodiments when executing the instructions.

[0088] like Figure 4 As shown, in an embodiment of the present invention, a computer device is provided, the computer device including a memory and a processor, the memory storing a computer program, and the processor being configured to execute the computer program stored in the memory. The memory is configured to store one or more computer instructions, wherein the one or more computer instructions are executed by the processor to implement the steps in the above method embodiment:

[0089] Get the mode and data of the CPU's calculations;

[0090] If the CPU only sends a single encoding, decoding, or checksum calculation mode, based on the current hardware configuration and the data sent, the checksum calculation is scheduled first, and after completion, one of the data blocks is restored;

[0091] Then, RS correction encoding calculation and RS correction decoding calculation are performed in sequence according to the decoding block to complete the grayscale of the data block.

[0092] In one embodiment of the present invention, a storage medium is further provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented:

[0093] Get the mode and data of the CPU's calculations;

[0094] If the CPU only sends a single encoding, decoding, or checksum calculation mode, based on the current hardware configuration and the data sent, the checksum calculation is scheduled first, and after completion, one of the data blocks is restored;

[0095] Then, RS correction encoding calculation and RS correction decoding calculation are performed in sequence according to the decoding block to complete the grayscale of the data block.

[0096] Those skilled in the art will appreciate that all or part of the processes in the above-described embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above-described methods. Any reference to memory, storage, database, or other media used in the embodiments provided herein may include at least one of non-volatile and volatile memory.

[0097] In summary, the present invention provides a system and method for improving codec hardware based on sparse erasure codes. By adding check code calculations to the RS erasure codec calculations, the codecs share the same hardware engine. Through different configurations and scheduling, the system greatly reduces the number of inverse matrix calculations and the calculation time, thereby improving the overall performance of the system. The hardware sharing mechanism increases the flexibility of system calculations, saving hardware resources and overhead. This application also compensates for some scenarios where RS erasure codecs cannot be calculated, thereby improving the reliability of the system.

[0098] This invention provides a system and method for improving encoding and decoding hardware based on sparse erasure codes. As a universal computing hardware solution for encoding, decoding, and checking sparse erasure codes, it significantly reduces system resource consumption and engine scheduling complexity. It also addresses scenarios where traditional RS erasure codec hardware solutions are unable to compute, improving system reliability.

[0099] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A hardware improvement system for encoding and decoding based on sparse erasure codes, characterized in that: include: A data input buffer module, which is used for data portion of erasure code calculation; A selector module, the selector module is used to connect to the input control module, and the input control module selects a matrix or a fixed value for calculation through the selector module according to the read configuration analysis; An input control module, configured to control the data input multiplication calculation module of the received data input buffer module, and further configured to schedule the data to align the data on both sides before entering the Galois field multiplier, select the use of fixed value input or matrix input according to the configured mode, determine the number of reads, and internally count whether the current calculation is completed and when to perform the next calculation; A multiplication calculation module, wherein the multiplication calculation module calculates data based on a Galois field multiplier; An output control module, wherein the output control module controls data output or entry into a circular temporary cache according to configuration, and controls the output flow rate of the current computing system; It also includes the addition of checksums. Based on the RS erasure codec calculation, the added checksum calculation shares the same hardware engine with the codec. The data input cache module is the data part of the erasure code calculation; The selector module is connected to a matrix data cache module and a fixed value input module. The matrix data cache module is the matrix part of the erasure code calculation. If the current calculation is encoding, this part is B of the encoding calculation; if the current calculation is decoding, this part is ; When the fixed value input module is currently calculating the check code, the fixed value is 1.

2. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 1, characterized in that: The data input buffer module is also used to determine the data flow rate according to the reading speed, and the data input buffer module is a data block; wherein, the number of data blocks calculated for verification depends on the grouping configuration in the erasure coding and decoding check code process.

3. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 1, characterized in that: The improved encoding and decoding hardware system based on sparse erasure codes also includes a configuration input module and a configuration output module. The configuration input module is connected to the input control module, and the configuration output module is connected to the output control module. The configuration input module and the configuration output module are used to cache the configuration data calculated for the current system, and are read by the calculation conditions of the configuration input module and the configuration output module.

4. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 3, characterized in that: If any one of the configured input module and the configured output module is about to be full, the system input cannot enter the module back pressure.

5. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 1, characterized in that: The improved encoding and decoding hardware system based on sparse erasure codes further includes a loop temporary cache module, which is used for caching the current loop calculation and temporarily storing the temporary results of the loop addition calculation.

6. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 5, characterized in that: The size of the partial cache of the circular temporary storage module is determined based on the largest data block processed by the system.

7. The encoding and decoding hardware improvement system based on sparse erasure codes according to claim 6, characterized in that: The improved encoding and decoding hardware system based on sparse erasure code performs sparse erasure encoding and decoding and check code calculation, including three calculation rules of encoding calculation, decoding calculation and check calculation, wherein: The encoding calculation: , where N is the calculated original data block constant value, n is the calculated coding block variable value, n<N and both are positive integers; The decoding calculation: , where N is the calculated original data block constant value, n is the calculated decoding block variable value, n<N and both are positive integers; The verification calculation: , where n is the calculated check block variable value, m is the starting data block value, and k is the number of data blocks involved in the calculation.

8. A method for improving encoding and decoding hardware based on sparse erasure codes, characterized in that: The method for improving codec hardware based on sparse erasure codes adopts the codec hardware improvement system based on sparse erasure codes according to any one of claims 1 to 7 to perform sparse erasure code system scheduling operations; The method for improving encoding and decoding hardware based on sparse erasure codes comprises the following steps: Get the mode and data of the CPU's calculations; If the CPU only sends a single encoding, decoding, or checksum calculation mode, based on the current hardware configuration and the data sent, the checksum calculation is scheduled first, and after completion, one of the data blocks is restored; Then, RS correction encoding calculation and RS correction decoding calculation are performed in sequence according to the decoding block to complete the grayscale of the data block.

Images