A method and apparatus applicable to unified and distributed storage erasure correction

Abstract

The present invention provides a method, system, device, and storage medium applicable to unified and distributed storage erasure correction. The method includes: in response to receiving input data, determining an encoding and decoding method and determining parameter information required for encoding and decoding according to the encoding and decoding method; setting control functions based on hardware requirements to meet performance requirements in the hardware; temporarily storing the parameter information and performing concurrent control on the data; and determining the number and relationship of multiplication and addition modules based on natural number domain operations or Galois field operations based on the relationship between multiples, data volume, and operation performance requirements to complete corresponding encoding and decoding operations. In different algorithm requirement scenarios of unified and distributed storage, the present invention has strong versatility by implementing different algorithms and configuring them into the same output format.

Description

Technical Field

[0001] The present invention relates to the field of distributed systems, and more specifically, particularly to a method, system, device, and storage medium applicable to unified and distributed storage erasure coding. Background Art

[0002] Erasure code belongs to a forward error correction technology in coding theory and was first applied in the communication field to solve problems such as loss and attenuation in data transmission. Since the erasure code technology has achieved good results in preventing data loss, it has been introduced into the storage field. Erasure code can effectively reduce storage overhead while ensuring the same reliability, so the erasure code technology is widely applied in major storage systems and data centers such as Microsoft's Azure, Facebook's F4, etc. The application of erasure is divided into two systems: unified and distributed. Among them, the unified system uses the traditional way of using erasure code. In the distributed system, the RAID code after combining erasure code with requirements such as load balancing is used. RAID (Redundant Arrays of Independent Disks) is a disk array with redundancy capabilities. The disk array is obtained by combining multiple independent disks together to form a disk group with a huge capacity. By adopting the RAID storage technology, the storage capacity can be greatly increased, the request processing ability of the system input and output can be improved, and the reliability of data can be improved through data distributed storage technology, parallel access means, and information redundancy technology.

[0003] RAID mainly utilizes data striping, data verification, and mirroring technologies to obtain strong performance, higher reliability, better fault tolerance, and strong scalability. According to different data application requirements, the strategies and architectures of these three technologies can be used or combined. Therefore, according to different strategies and architectures, RAID can be divided into different levels: RAID 0, 1, 5, 6, 10. Although the algorithms used in unified and distributed storage both belong to erasure algorithms, there are corresponding differences in algorithm details and implementation. Different erasure technologies use different algorithms, so different erasure acceleration cards are required to perform corresponding erasure operation work. Summary of the Invention

[0004] In view of this, the purpose of the embodiments of the present invention is to propose a method, system, computer device, and computer-readable storage medium applicable to unified and distributed storage erasure coding. In the embodiments of the present invention, the corresponding firmware only needs to be adapted to different algorithms to be adapted to the general hardware structure, so as to implement different erasure algorithms in different application scenarios, so as to achieve a general design in a wider range of application scenarios.

[0005] For the above purposes, one aspect of an embodiment of the present invention provides a method applicable to unified and distributed storage erasure correction, including the following steps: in response to receiving input data, determining an encoding and decoding method and determining parameter information required for encoding and decoding according to the encoding and decoding method; setting control functions based on hardware requirements to meet the performance requirements in the hardware; temporarily storing the parameter information and performing concurrent control on the data; and determining the number and relationship of multiplication and addition modules based on natural number domain operations or Galois field operations according to the relationship between multiples, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations.

[0006] In some embodiments, the determining an encoding and decoding method and determining parameter information required for encoding and decoding according to the encoding and decoding method includes: determining the selection of data addresses for encoding and decoding operations based on the requirements of load balancing, and controlling input or output to read data from different addresses according to the selected data addresses.

[0007] In some embodiments, the determining an encoding and decoding method and determining parameter information required for encoding and decoding according to the encoding and decoding method includes: determining parameter information based on the data arrangement method of load balancing.

[0008] In some embodiments, the determining the number and relationship of multiplication and addition modules based on natural number domain operations or Galois field operations according to the relationship between multiples, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations includes: reading parameter information from a matrix structure for operations based on the relationship between data and parameters, and outputting combined data through the multiplication and addition modules of natural number domain operations or Galois field operations.

[0009] Another aspect of an embodiment of the present invention provides a system applicable to unified and distributed storage erasure correction, including: a determination module configured to, in response to receiving input data, determine an encoding and decoding method and determine parameter information required for encoding and decoding according to the encoding and decoding method; a setting module configured to set control functions based on hardware requirements to meet the performance requirements in the hardware; a control module configured to temporarily store the parameter information and perform concurrent control on the data; and an operation module configured to determine the number and relationship of multiplication and addition modules based on natural number domain operations or Galois field operations according to the relationship between multiples, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations.

[0010] In some embodiments, the determination module is configured to: determine the selection of data addresses for encoding and decoding operations based on the requirements of load balancing, and control input or output to read data from different addresses according to the selected data addresses.

[0011] In some embodiments, the determination module is configured to: determine parameter information based on the data arrangement method of load balancing.

[0012] In some embodiments, the operation module is configured to: read parameter information from a matrix structure for operation based on the relationship between data and parameters, and output combined data through the multiplication and addition module for natural number domain operation or Galois field operation.

[0013] In another aspect of the embodiments of the present invention, there is also provided a computer device, including: at least one processor; and a memory, where the memory stores computer instructions that can run on the processor, and when the instructions are executed by the processor, the steps of the above method are implemented.

[0014] In yet another aspect of the embodiments of the present invention, there is also provided a computer-readable storage medium storing a computer program that implements the steps of the above method when executed by a processor.

[0015] The present invention has the following beneficial technical effects: In scenarios with different algorithm requirements for unified and distributed storage, through the implementation of different algorithms, they are configured to have the same output format, and operations are implemented through matrices and PEs (multiplication and addition modules based on natural number domain operation or Galois field operation). The division of operations can determine the specific number of PEs based on performance and area requirements to meet different design needs. Finally, through data combination and disk writing, the corresponding encoding and decoding work is completed, achieving an optimally configurable performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for description in the embodiments or the prior art. Obviously, the drawings in the following description are only some embodiments of the present invention, and those of ordinary skill in the art can obtain other embodiments based on these drawings without creative efforts.

[0017] Figure 1 It is a schematic diagram of an embodiment of a method for unified and distributed storage erasure correction provided by the present invention;

[0018] Figure 2 It is a hardware architecture diagram of an embodiment of a system for unified and distributed storage erasure correction provided by the present invention;

[0019] Figure 3 It is a schematic diagram of the working of the matrix module of the present invention;

[0020] Figure 4 It is a schematic diagram of the working of the encoding and decoding operation PE module of the present invention;

[0021] Figure 5 It is a schematic diagram of an embodiment of a system for unified and distributed storage erasure correction provided by the present invention;

[0022] Figure 6 Schematic diagram of the hardware structure of an embodiment of a computer device applicable to unified and distributed storage erasure correction provided by the present invention;

[0023] Figure 7 Schematic diagram of an embodiment of a computer storage medium applicable to unified and distributed storage erasure correction provided by the present invention. Detailed implementation manners

[0024] To make the objectives, technical solutions, and advantages of the present invention clearer and more understandable, the following further describes the embodiments of the present invention in detail with reference to specific embodiments and the accompanying drawings.

[0025] It should be noted that all the expressions using "first" and "second" in the embodiments of the present invention are used to distinguish two entities or parameters with the same name but different, so it can be seen that "first" and "second" are only for the convenience of expression and should not be construed as a limitation on the embodiments of the present invention. This will not be elaborated in the subsequent embodiments.

[0026] In the first aspect of the embodiments of the present invention, an embodiment of a method applicable to unified and distributed storage erasure correction is proposed. Figure 1 The following shows a schematic diagram of an embodiment of a method applicable to unified and distributed storage erasure correction provided by the present invention. As Figure 1 shown, the embodiments of the present invention include the following steps:

[0027] S1. In response to receiving input data, determine the encoding and decoding method and determine the parameter information required for encoding and decoding according to the encoding and decoding method;

[0028] S2. Set the control function based on the hardware requirements to meet the performance requirements in the hardware;

[0029] S3. Temporarily store the parameter information and perform concurrent control on the data; and

[0030] S4. Determine the number and relationship of multiplication and addition modules based on the natural number domain operation or Galois field operation according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations.

[0031] When the erasure correction algorithm is applied to different operating systems of unified and distributed types, in order to adapt to different scenarios and application requirements, the algorithm has changed. The following first makes a simple introduction separately:

[0032] 1. Unified storage:

[0033] The commonly used erasure algorithms for unified storage include algorithms such as the Vandermonde and Cauchy of RS. Let k represent the amount of data and m represent the amount of parity generated from the data. Unified storage usually supports erasure in scenarios with larger k and larger m. The erasure code encodes k data blocks into r additional parity blocks. The way of encoding the r parity blocks based on the Vandermonde matrix or the Cauchy matrix is called the RS erasure code using the Vandermonde matrix or the Cauchy matrix for encoding. The specific encoding process is as follows.

[0034] RS erasure code based on the Vandermonde matrix:

[0035]

[0036] RS erasure code based on the Cauchy matrix: [[ID=,10]]

[0037]

[0038] Its decoding is to obtain the data of D through inverse operation. Taking the decoding when the data from D1 to Dr is lost as an example, the process is as follows.

[0039] RS erasure code data recovery method:

[0040]

[0041] It can be seen that the core concept of the erasure code is to construct an invertible encoding matrix to generate parity data, and its inverse matrix can be calculated to recover the original data.

[0042] 2. Distributed storage:

[0043] The erasure algorithm used in distributed storage is called RAID. Commonly, there are RAID0, 1, 5, 6. Among them, the algorithms involving multi-error recovery are RAID5 and RAID6. The algorithm principle of traditional RAID5 is:

[0044] d1 + d2 + d3 +... + d m + p = 0 (1)

[0045] The algorithm principle of RAID6 is:

[0046]

[0047] The encoding and decoding algorithms of RAID are to solve the equations with p as the unknown in the above relational expressions. The operations here use Galois field operations in storage. Therefore, it can be known that the relational expressions of p in traditional RAID5 and 6 are respectively:

[0048] RAID5:

[0049]

[0050] RAID6:

[0051]

[0052] In a storage system, to reduce the computational complexity and ensure that data does not overflow, the above-mentioned unified and storage encoding and decoding operations generally use the Galois field for implementation. That is, in the hardware implementation, addition and subtraction are implemented through exclusive OR operations, and multiplication and division are implemented using Galois multiplication and division for different Galois field polynomials. As can be seen above, unified and distributed storage use different algorithms for different application environments, and the hardware to be implemented is different when implementing a hardware accelerator.

[0053] Figure 2 This is a hardware architecture diagram of an embodiment of a system for unified and distributed storage erasure correction provided by the present invention. In the embodiment of the present invention, the hardware architecture is divided into four modules, that is Figure 2 1, 2, 3, and 4 outlined by the dashed line in Figure 2 1 in represents the firmware and input / output control module, 2 represents the hardware control module, 3 represents the matrix module, and 4 represents the encoding and decoding operation PE module.

[0054] In response to receiving input data, determine the encoding and decoding method and determine the parameter information required for encoding and decoding according to the encoding and decoding method.

[0055] The following will respectively give examples of the encoding and decoding operation methods and data methods of the firmware under unified or distributed conditions:

[0056] A. Unified storage encoding

[0057] Most unified storage uses RS codes, and among them, Vandermonde or Cauchy is used more frequently. Its encoding conditions satisfy Figure 1 For the introduction of , the matrix output by the firmware to the logic core 3 here is the parameter information required for encoding. Taking Vandermonde encoding with k = 3 and m = 2 as an example, the output information here is:

[0058]

[0059] B. a. Unified storage decoding

[0060] The architecture of the present invention can also solve the decoding problem. Taking the same situation as in A above that requires decoding as an example, the matrix output by the firmware to the logic core 3 at this time is the inverse matrix of its encoding matrix. Taking the first and second data as an example that needs to be recovered due to errors:

[0061]

[0062] H -1The generation method involves different inversion algorithms, and the inversion algorithms are not within the scope of protection of this invention. After completing the inversion of H -1 After the inversion, the parameter combinations corresponding to the data positions that need to be restored in H -1 are matrices. It can be passed to the matrix part in the logic core 3 of the hardware through firmware. In the example described above, the first and second data that need to be restored due to an error. Therefore, after inversion, the parameter combinations of the first and second rows are combined into a matrix and passed to the hardware.

[0063] C. Distributed storage encoding

[0064] The distributed storage uses the RAID algorithm. For the case where k = 3 and m = 2 need to be encoded in the same environment as above, based on the definition of the RAID algorithm, the RAID6 algorithm is used. Then the generated encoding matrix is based on the RAID6 algorithm in formula (4) as follows:

[0065]

[0066] The H matrix corresponds to the parameter information after determining the encoding positions for the used encoding algorithm. The parameter information can form matrix information similar to unified storage based on the order of the multiplicands (data information).

[0067] D. Distributed storage decoding

[0068] The same RAID decoding algorithm is used during decoding. Taking the above situation as an example, if the first and second data have errors, different position information can be substituted into the same formula (4) to obtain the parameters. Using the same situation above for example, the corresponding information is:

[0069]

[0070] H -1 The method for obtaining H is because of the RAID algorithm, so its method is the same as the encoding algorithm, and the parameters corresponding to different position information can be substituted. In the specific implementation of the above operations, the Galois field is often used for operations. The matrix format of data transmission remains unchanged, and the corresponding relationship remains unchanged. The parameters corresponding to the operations can be changed for different Galois fields.

[0071] In some embodiments, the determination of the encoding / decoding method and the determination of the parameter information required for encoding / decoding according to the encoding / decoding method include: determining the data address selection for performing encoding / decoding operations based on the requirements of load balancing, and controlling the input or output to read data from different addresses according to the selected data address. The firmware and the input / output control module 1 belong to the peripheral functional modules of the hardware structure. As the name implies, the input and output control the input of data and the output after encoding or decoding. The hardware structure of the present invention can use the same set of hardware structure, and through different algorithms of the firmware, different parameter configurations are performed to complete the encoding and decoding work. The data input for the encoding or decoding work is split or combined based on the input part and then input into the hardware structure. The splitting and combination of data are defined based on the requirements of 3 and 4 in the logic core. The data output for the encoding or decoding work is determined based on the specific implementation structure of the hardware and the way of disk writing. The firmware part determines the data address selection for performing encoding / decoding operations and the parameter operations of encoding / decoding based on the requirements of load balancing. After completing the data address selection, it controls the input or output to read data from different addresses. After completing the parameter operations, the corresponding parameters to be operated are input to part 3. For unified or distributed encoding / decoding requirements, the algorithms operated and the obtained parameters are different. The control function is set based on the hardware requirements to meet the performance requirements in the hardware. The hardware control module 2 performs control function settings related to registers and the like based on different hardware requirements to meet the requirements in terms of performance and the like in the hardware.

[0072] In some embodiments, the determination of the encoding / decoding method and the determination of the parameter information required for encoding / decoding according to the encoding / decoding method include: determining the parameter information based on the data arrangement mode of load balancing. Taking left rotation as an example, the parameters corresponding to the positions where different data bits are located are the different parameters determined by the positions corresponding to the first row.

[0073] Temporarily store the parameter information and perform concurrent control on the data.

[0074] To complete the corresponding work of encoding or decoding, the matrix module receives the parameter information that has been operated by the firmware and arranged and combined into a matrix, and stores it in the matrix structure of 3. The parameters in the matrix structure participate in the encoding / decoding operations, and each parameter corresponds to different data to be operated.

[0075] Based on the parameter information passed to the matrix structure, the firmware determines the splitting method of the data information. Taking formula (5) as an example, the data to be operated under the data parallel distribution control at this time is the multiple relationship data distributed based on 1, 1, 1 and 1, 2, 3. The multiple relationship is determined based on the performance requirements. The higher the speed requirement, the higher the multiple relationship can be under the premise of divisible distribution. The higher the area requirement, the lower the multiple relationship. The comprehensive result of the two is the reference for determining the final multiple relationship.

[0076] Figure 3 This is a schematic diagram of the operation of the matrix module of the present invention. As Figure 3 shown, if the multiple to be distributed is 2 at this time, and the data to be calculated is d1, d2, and d3, data distribution is performed. Based on the multiple relationship of 2, the data d1, d2, and d3 are evenly divided into two pieces of data. Among them, d1,0 and d1,1 are combined into d1, and the same applies to other data.

[0077] Determine the number and relationship of multiplication and addition modules based on the natural number domain operation or Galois field operation according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations.

[0078] In some embodiments, determining the number and relationship of multiplication and addition modules based on the natural number domain operation or Galois field operation according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations includes: reading parameter information from the matrix structure based on the relationship between the data and parameters for calculation, and outputting combined data through the multiplication and addition modules of the natural number domain operation or Galois field operation.

[0079] After module 3 completes the temporary storage of parameters and data parallel distribution control, it determines the number and relationship of PEs according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations. Here, PE is a multiplication and addition module based on the natural number domain operation or Galois field operation. The number of PEs is determined based on performance requirements. For example, in the above example, when the data volume is 3 and the multiple relationship is 2, and the number of rows in the matrix structure is 2, if the highest speed is pursued, 2*3*2 = 12 PEs can be divided. Each PE corresponds to a data block. Read parameter information from the matrix structure based on the relationship between the data and parameters for calculation, and then output combined data through the PE to complete the output of encoding and decoding.

[0080] Figure 4 This is a schematic diagram of the operation of the PE module for encoding and decoding of the present invention. The above division of the PE group is determined based on the number of rows of parameters in the matrix structure when pursuing speed. The number of PEs is determined based on the above data volume, multiple relationship, and number of rows in the matrix structure. In the case of pursuing area, the number of PEs can be reduced, and operation multiplexing can be performed based on a similar above relationship. Finally, based on the divided relationship, the results obtained from the operation are combined and output. The purpose of the output is to write the data for encoding and decoding to disk, and the disk write position is determined by the load balancing position information in 1.

[0081] In the scenarios of unified and distributed storage with different algorithm requirements, through the implementation of different algorithms, it is configured into the same output format, and the operations are implemented through matrix and PE operation modules. The division of operations can determine the specific number of PEs based on performance and area requirements to meet different design requirements. Finally, through data combination and disk writing, the corresponding encoding and decoding work is completed, achieving the performance of configurable optimization.

[0082] It should be particularly noted that each step in each embodiment of the above method applicable to unified and distributed storage erasure correction can be mutually crossed, replaced, added, or deleted. Therefore, these reasonable permutation and combination transformations for the method applicable to unified and distributed storage erasure correction should also fall within the protection scope of the present invention, and the protection scope of the present invention should not be limited to the embodiments.

[0083] Based on the above objectives, in the second aspect of the embodiments of the present invention, a system applicable to unified and distributed storage erasure correction is proposed. As Figure 5 shown, the system 200 includes the following modules: a determination module configured to determine the encoding and decoding method and determine the parameter information required for encoding and decoding according to the encoding and decoding method in response to receiving input data; a setting module configured to set the control function based on hardware requirements to meet the performance requirements in the hardware; a control module configured to temporarily store the parameter information and perform concurrent control on the data; and an operation module configured to determine the number and relationship of multiplication and addition modules based on the relationship between multiples, data volume, and operation performance requirements for natural number domain operations or Galois field operations to complete the corresponding encoding and decoding operations.

[0084] In some embodiments, the determination module is configured to: determine the data address selection for encoding and decoding operations based on the load balancing requirement, and control the input or output to read data from different addresses according to the selected data address.

[0085] In some embodiments, the determination module is configured to: determine the parameter information based on the data layout mode of load balancing.

[0086] In some embodiments, the operation module is configured to: read the parameter information from the matrix structure based on the relationship between data and parameters for operations, and output combined data through the multiplication and addition modules for natural number domain operations or Galois field operations.

[0087] For the above purposes, in the third aspect of the embodiments of the present invention, a computer device is proposed, including: at least one processor; and a memory that stores computer instructions executable on the processor, and the instructions are executed by the processor to implement the following steps: S1. In response to receiving input data, determine the encoding and decoding method and determine the parameter information required for encoding and decoding according to the encoding and decoding method; S2. Set the control function based on the hardware requirements to meet the performance requirements in the hardware; S3. Temporarily store the parameter information and perform concurrent control on the data; and S4. Determine the number and relationship of multiplication and addition modules based on the natural number domain operation or Galois field operation according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations.

[0088] In some embodiments, the determining the encoding and decoding method and determining the parameter information required for encoding and decoding according to the encoding and decoding method includes: determining the data address selection for encoding and decoding operations based on the load balancing requirements, and controlling the input or output to read data at different addresses according to the selected data address.

[0089] In some embodiments, the determining the encoding and decoding method and determining the parameter information required for encoding and decoding according to the encoding and decoding method includes: determining the parameter information based on the data layout method of load balancing.

[0090] In some embodiments, the determining the number and relationship of multiplication and addition modules based on the natural number domain operation or Galois field operation according to the relationship between the multiple, data volume, and operation performance requirements to complete the corresponding encoding and decoding operations includes: reading parameter information from the matrix structure for operation based on the relationship between the data and the parameters, and outputting combined data through the multiplication and addition modules of the natural number domain operation or Galois field operation.

[0091] As Figure 6 shown, it is a schematic diagram of the hardware structure of an embodiment of the above computer device for unified and distributed storage erasure correction provided by the present invention.

[0092] Take the device as Figure 6 shown. In this device, there is a processor 301 and a memory 302.

[0093] The processor 301 and the memory 302 can be connected through a bus or other means, Figure 6 and here, taking the connection through the bus as an example.

[0094] The memory 302, as a non-volatile computer-readable storage medium, can be used to store non-volatile software programs, non-volatile computer-executable programs, and modules, such as the program instructions / modules corresponding to the method for applicable unified and distributed storage erasure in the embodiments of the present application. The processor 301 executes various functional applications and data processing of the server by running the non-volatile software programs, instructions, and modules stored in the memory 302, that is, implements the method for applicable unified and distributed storage erasure.

[0095] The memory 302 may include a program storage area and a data storage area. Among them, the program storage area can store an operating system and application programs required for at least one function; the data storage area can store data created according to the use of the method for applicable unified and distributed storage erasure, etc. In addition, the memory 302 may include high-speed random access memory and may also include non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, or other non-volatile solid-state storage devices. In some embodiments, the memory 302 may optionally include a memory remotely set relative to the processor 301, and these remote memories can be connected to the local module through a network. Examples of the above network include but are not limited to the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0096] One or more computer instructions 303 corresponding to the method for applicable unified and distributed storage erasure are stored in the memory 302. When executed by the processor 301, they execute the method for applicable unified and distributed storage erasure in any of the above method embodiments.

[0097] Any embodiment of the computer device that executes the above method for applicable unified and distributed storage erasure can achieve the same or similar effects as any of the foregoing method embodiments corresponding thereto.

[0098] The present invention also provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program that, when executed by a processor, executes the method for applicable unified and distributed storage erasure.

[0099] As Figure 7 shown, it is a schematic diagram of an embodiment of the above computer storage medium for applicable unified and distributed storage erasure provided by the present invention. Taking the computer storage medium as Figure 7 shown as an example, the computer-readable storage medium 401 stores a computer program 402 that, when executed by a processor, executes the above method.

[0100] Finally, it should be noted that those of ordinary skill in the art can understand that all or part of the processes in the above-described method embodiments can be completed by instructing relevant hardware through a computer program. A program applicable to the unified and distributed storage erasure method can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the above-described method embodiments. Among them, the storage medium of the program can be a magnetic disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM), etc. The embodiments of the above computer program can achieve the same or similar effects as the corresponding foregoing method embodiments.

[0101] The above are exemplary embodiments disclosed by the present invention. However, it should be noted that various changes and modifications can be made without departing from the scope of the embodiments disclosed by the present invention as defined by the claims. The functions, steps, and / or actions of the method claims according to the disclosed embodiments herein need not be performed in any specific order. In addition, although the elements disclosed in the embodiments of the present invention can be described or claimed in individual form, they can also be understood as multiple unless explicitly limited to the singular.

[0102] It should be understood that, as used herein, unless the context clearly supports an exception, the singular form "a" is also intended to include the plural form. It should also be understood that the "and / or" used herein refers to any and all possible combinations of one or more of the associated listed items.

[0103] The serial numbers of the above disclosed embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0104] Those of ordinary skill in the art can understand that all or part of the steps of the above-described embodiments can be completed by hardware or by instructing relevant hardware through a program. The program can be stored in a computer-readable storage medium. The above-mentioned storage medium can be a read-only memory, a magnetic disk, an optical disk, etc.

[0105] Those of ordinary skill in the art should understand that: the discussion of any of the above embodiments is only exemplary and is not intended to imply that the scope of the embodiments disclosed by the present invention (including the claims) is limited to these examples; under the concept of the embodiments of the present invention, the technical features between the above embodiments or different embodiments can also be combined, and there are many other variations in different aspects of the above embodiments of the present invention, which are not provided in detail for the sake of brevity. Therefore, any omission, modification, equivalent replacement, improvement, etc. made within the spirit and principle of the embodiments of the present invention shall be included within the protection scope of the embodiments of the present invention.

Claims

1. A method for unified and distributed storage erasure, characterized in that: The steps include: In response to receiving the input data, determining a coding / decoding mode and determining parameter information required for coding / decoding according to the coding / decoding mode; Set the control function based on the hardware requirements to meet the performance requirements in the hardware; Temporarily storing the parameter information and performing concurrent control on the data; as well as Determine the number and relationship of multiplication and addition modules based on natural number field operations or Galois field operations based on the performance requirements of the multiples, data volume, and operations to complete the corresponding encoding and decoding operations; Using the same hardware structure, different algorithms are implemented in the firmware and different parameter configurations are performed to complete encoding and decoding tasks. The determining of the encoding and decoding mode and determining the parameter information required for encoding and decoding according to the encoding and decoding mode includes: Determine the data address selection for encoding and decoding operations based on load balancing requirements, and control input or output to read data at different addresses according to the selected data address; Determine parameter information based on load balancing data arrangement.

2. The method according to claim 1, characterized in that Determining the number and relationship of multiplication and addition modules based on natural number field operations or Galois field operations to complete corresponding encoding and decoding operations based on the relationship between the multiple, the amount of data, and the performance requirements of the operation includes: Based on the relationship between data and parameters, parameter information is read from the matrix structure to perform operations, and combined data is output through the multiplication and addition module of the natural number field operation or the Galois field operation.

3. A system applicable to unified and distributed storage erasure, characterized in that: include: a determination module configured to determine a coding / decoding mode in response to receiving input data and determine parameter information required for coding / decoding according to the coding / decoding mode; A setting module configured to set the control function based on hardware requirements to meet the performance requirements in the hardware; A control module configured to temporarily store the parameter information and perform concurrent control on the data; as well as An operation module configured to determine the number and relationship of multiplication and addition modules based on natural number field operations or Galois field operations based on the relationship between the multiple, the amount of data and the performance requirements of the operation to complete the corresponding encoding and decoding operation; Also included are modules for: Using the same hardware structure, different algorithms are implemented in the firmware and different parameter configurations are performed to complete encoding and decoding tasks. The determination module is configured to: Determine the data address selection for encoding and decoding operations based on load balancing requirements, and control input or output to read data at different addresses according to the selected data address; Determine parameter information based on load balancing data arrangement.

4. The system according to claim 3, characterized in that The operation module is configured to: Based on the relationship between data and parameters, parameter information is read from the matrix structure to perform operations, and combined data is output through the multiplication and addition module of the natural number field operation or the Galois field operation.

5. A computer device, characterized in that: include: at least one processor; as well as A memory storing computer instructions executable on the processor, wherein the instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 2.

6. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images