Data stripe encoding method, system, device and medium

Abstract

The present application discloses a data stripe encoding method, system, device, and medium, which obtains a first stripe of data and a second stripe of data to be encoded; generates an operation value for each of the first stripe of data and the second stripe of data, wherein the number of operation values is equal to the number of redundant check blocks; generates an exclusive OR value based on the operation value of the second stripe of data and the operation value of the first stripe of data; and determines the check blocks for each of the first stripe of data and the second stripe of data based on the operation value and the exclusive OR value. In the present application, during the process of performing Hitchhiker erasure coding, it is necessary to calculate the exclusive OR value between the operation value of the second stripe of data and the first operation value of the first stripe of data, and determine the corresponding check block based on the exclusive OR value. The exclusive OR value reduces the computational overhead of the Hitchhiker erasure coding and improves the computational speed. The data stripe encoding system, device, and computer-readable storage medium provided in the present application also solve corresponding technical problems.

Description

Technical Field

[0001] The present application relates to the field of erasure code technology, and more specifically, to a data stripe encoding method, system, device, and medium. Background Art

[0002] With the advent of the big data era, users need to manage more and more data, and data reliability becomes increasingly important. For example, a multi-copy redundancy scheme can be used to ensure data reliability. That is, a certain number of copies are created for each data block and distributed in the system according to certain rules to achieve the purpose of fault tolerance. However, the multi-copy redundancy scheme has high redundancy and consumes a lot of storage overhead for users. Erasure coding technology has the advantages of low redundancy and high flexibility. Therefore, users are increasingly using erasure coding technology to ensure data reliability, such as using Hitchhiker (HH, hitchhiker coding) erasure code to save data.

[0003] However, the existing Hitchhiker erasure coding method has high computational overhead, complex encoding process, long encoding time and low encoding speed, which affects the user's encoding experience.

[0004] In summary, how to improve the calculation speed of Hitchhiker erasure codes is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this application is to provide a data stripe encoding method that can, to a certain extent, solve the technical problem of how to improve the calculation speed of Hitchhiker erasure codes. This application also provides a data stripe encoding system, device, and computer-readable storage medium.

[0006] In order to achieve the above objectives, this application provides the following technical solutions:

[0007] A data stripe encoding method, comprising:

[0008] Obtaining the first stripe of data and the second stripe of data to be encoded;

[0009] generating operation values for the first stripe of data and the second stripe of data, respectively, wherein the number of the operation values is equal to the number of redundant check blocks;

[0010] generating an exclusive OR value based on the operation value of the second stripe of data and the operation value of the first stripe of data;

[0011] A check block of each of the first stripe data and the second stripe data is determined based on the operation value and the XOR value.

[0012] Preferably, generating an XOR value based on the operation value of the second stripe data and the operation value of the first stripe data includes:

[0013] If the number of redundant check blocks is 2, then calculate the XOR value between the second operation value of the second stripe data and the first half of the second operation value of the first stripe data, and calculate the data value between the second operation value of the second stripe data and the second half of the second operation value of the first stripe data.

[0014] Preferably, when the number of the redundant check blocks is 2, determining the check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value includes:

[0015] using the first operation value of the first stripe of data as a first check block of the first stripe of data;

[0016] taking an exclusive OR value of a second operation value of the second stripe of data and a second half of the second operation value of the first stripe of data as a second check block of the first stripe of data;

[0017] using the first operation value of the second stripe of data as a first check block of the second stripe of data;

[0018] An exclusive OR value of the second operation value of the second stripe data and the first half of the second operation value of the first stripe data is used as a second check block of the second stripe data.

[0019] Preferably, generating an XOR value based on the operation value of the second stripe data and the operation value of the first stripe data includes:

[0020] If the number of the redundancy check blocks is greater than 2, calculating an exclusive OR value between corresponding parts of the operation value of the second stripe of data and the first operation value of the first stripe of data, starting from the second operation value of the second stripe of data and ending with the last operation value of the second stripe of data;

[0021] Among them, the corresponding part of the first operation value of the first stripe data includes the sub-partial data obtained after the first operation value of the first stripe data is evenly divided, the number of equal divisions is the difference between the number of the redundant check blocks and 1, and the i-th operation value of the second stripe data corresponds to the i-1-th sub-partial data.

[0022] Preferably, when the number of the redundant check blocks is greater than 2, determining the check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value includes:

[0023] Using the dth operation value of the first stripe of data as the dth check block of the first stripe of data, where d=1, 2, ..., n, and n represents the number of the redundant check blocks;

[0024] using the first operation value of the second stripe of data as a first check block of the second stripe of data;

[0025] The XOR value between the i-th operation value of the second stripe data and the corresponding part of the first operation value of the first stripe data is used as the i-th check block of the second stripe data, where i=2, 3, . . . , n.

[0026] Preferably, the process of generating the respective operation values of the first stripe of data and the second stripe of data includes:

[0027] generating the operation value based on a generation formula;

[0028] The generation formula includes:

[0029]

[0030] Among them, F d-1 (x) represents the dth operation value in the operation value; x0 represents the preset value, x m represents the mth data in the stripe data, m=1, 2, ..., n, k represents the total amount of data in the stripe data; and n represents the number of the redundant check blocks.

[0031] A data stripe encoding method, comprising:

[0032] Acquire target data to be encoded, where the target data includes first stripe data and second stripe data;

[0033] Get the encoding matrix;

[0034] Splitting the encoding matrix to obtain encoding sub-matrices, wherein at least one matrix value in any encoding sub-matrix is non-zero, and any two encoding sub-matrices are different;

[0035] Splitting the target data into target sub-data corresponding to the encoding sub-matrix;

[0036] Calculate the encoding sub-matrix and the corresponding target sub-data to obtain an intermediate result;

[0037] generating, based on the intermediate result, an operation value and a corresponding exclusive-OR value of each of the first stripe data and the second stripe data, and determining, based on the operation value and the exclusive-OR value, a check block of each of the first stripe data and the second stripe data;

[0038] The number of the operation values is equal to the number of redundant check blocks; and the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data.

[0039] Preferably, splitting the encoding matrix to obtain encoding sub-matrices includes:

[0040] Splitting the encoding matrix to obtain a first sub-matrix;

[0041] Deleting the first submatrix whose matrix values are all 0, and using the undeleted first submatrix as the second submatrix;

[0042] Deduplication is performed on the second sub-matrix to obtain a third sub-matrix;

[0043] Merging the third submatrices that satisfy a merging rule to obtain the coding submatrix, wherein the merging rule includes merging the two matrices if at least one of the matrix values at the same position in the two matrices is 0;

[0044] The third sub-matrix that does not satisfy the merging rule is directly used as the encoding sub-matrix.

[0045] A data stripe encoding system, comprising:

[0046] A first acquisition module, configured to acquire first and second stripe data to be encoded;

[0047] A first generating module is configured to generate operation values for the first stripe of data and the second stripe of data, wherein the number of the operation values is equal to the number of redundant check blocks;

[0048] a second generating module, configured to generate an XOR value based on the operation value of the second stripe of data and the operation value of the first stripe of data;

[0049] The first determining module is configured to determine a check block of each of the first stripe data and the second stripe data based on the operation value and the XOR value.

[0050] A data stripe encoding system, comprising:

[0051] A second acquisition module is used to acquire target data to be encoded, wherein the target data includes the first stripe data and the second stripe data;

[0052] A third acquisition module is used to obtain a coding matrix;

[0053] A first splitting module is configured to split the encoding matrix to obtain encoding sub-matrices, wherein at least one matrix value in any encoding sub-matrix is non-zero, and any two encoding sub-matrices are different;

[0054] A second splitting module, configured to split the target data into target sub-data corresponding to the encoding sub-matrix;

[0055] A first operation module is used to operate the encoding sub-matrix and the corresponding target sub-data to obtain an intermediate result;

[0056] a third generating module, configured to generate, based on the intermediate result, an operation value and a corresponding XOR value of each of the first data stripe and the second data stripe, so as to determine, based on the operation value and the XOR value, a check block of each of the first data stripe and the second data stripe;

[0057] The number of the operation values is equal to the number of redundant check blocks; and the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data.

[0058] An electronic device, comprising:

[0059] Memory for storing computer programs;

[0060] A processor is configured to implement the steps of any of the above methods when executing the computer program.

[0061] A computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of any of the above-mentioned user operation processing methods.

[0062] The present application provides a data stripe encoding method, which includes obtaining a first stripe of data and a second stripe of data to be encoded; generating operation values for each of the first stripe of data and the second stripe of data, wherein the number of operation values is equal to the number of redundant check blocks; generating an exclusive-OR value based on the operation value of the second stripe of data and the operation value of the first stripe of data; and determining the check blocks for each of the first stripe of data and the second stripe of data based on the operation value and the exclusive-OR value. In the present application, during the Hitchhiker erasure code process, it is necessary to calculate the exclusive-OR value between the operation value of the second stripe of data and the first operation value of the first stripe of data, and determine the corresponding check blocks based on the exclusive-OR value. The exclusive-OR value reduces the computational overhead of the Hitchhiker erasure code and improves the computational speed. The present application also provides a data stripe encoding system, device, and computer-readable storage medium that solves corresponding technical problems. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without any creative work.

[0064] Figure 1 A flowchart of a data stripe encoding method provided in an embodiment of the present application;

[0065] Figure 2 Schematic diagram of coding when the redundancy check block is 2;

[0066] Figure 3 Schematic diagram of coding when the redundancy check block is 3;

[0067] Figure 4 Schematic diagram of coding when the number of redundant check blocks is 4;

[0068] Figure 5 Another flow chart of a data stripe encoding method provided in an embodiment of the present application;

[0069] Figure 6 A flowchart for splitting the coding matrix provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of the structure of a data stripe encoding system provided in an embodiment of the present application;

[0071] Figure 8 Another structural diagram of a data stripe encoding system provided in an embodiment of the present application;

[0072] Figure 9 Schematic diagram of the hardware structure of the electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0074] See also Figure 1 , Figure 1 A flowchart of a data stripe encoding method provided in an embodiment of the present application.

[0075] A data stripe encoding method provided in an embodiment of the present application may include the following steps:

[0076] Step S101: Acquire a first stripe of data and a second stripe of data to be encoded.

[0077] In practical applications, since the Hitchhiker erasure code encodes two stripes of data, the first stripe of data and the second stripe of data to be encoded can be obtained first. The type and size of the first stripe of data and the second stripe of data can be determined according to actual needs, and this application does not make specific limitations here.

[0078] It should be noted that the sizes of the first stripe data and the second stripe data are equal, and considering that in actual application scenarios, the data to be encoded input by the user is generally a group of data, then the group of data can be split into two groups, that is, split into the first stripe data and the second stripe data, and in this process, if the group of data is not enough to be split into two stripe data, it can be padded with zeros to obtain the required two stripe data.

[0079] Step S102: generating operation values for the first stripe of data and the second stripe of data, respectively. The number of the operation values is equal to the number of redundant check blocks.

[0080] In practical applications, after obtaining the first stripe data and the second stripe data to be encoded, operation values of the first stripe data and the second stripe data can be generated respectively, and the number of operation values is equal to the number of redundant check blocks, so that the check blocks of the first stripe data and the second stripe data can be determined respectively according to the operation values.

[0081] It should be noted that the method for generating the operation value is the same as the method for generating the check block of RS (Reed-Solomon Codes), both of which are based on polynomials and exclusive-OR operations. That is, in a specific application scenario, when generating the operation values of the first stripe of data and the second stripe of data, the operation values can be generated based on a generation formula.

[0082] The generation formula includes:

[0083]

[0084] Among them, F d-1 (x) represents the dth operation value in the operation value; x0 represents the preset value, x m Represents the mth data in the stripe data, where m = 1, 2, ..., n, k represents the total amount of data in the stripe data, and n is the number of redundant check blocks.

[0085] Step S103: Generate an exclusive OR value based on the calculated value of the second stripe data and the calculated value of the first stripe data.

[0086] In practical applications, if all the calculation values are directly used as all the check blocks, the calculation overhead is high and the calculation speed is slow. In order to improve the calculation speed, the calculation value and the XOR value of the corresponding data can be used instead of the calculation value as the corresponding check block. That is, after generating the calculation value of the first stripe data and the second stripe data respectively, the XOR value can be generated based on the calculation value of the second stripe data and the calculation value of the first stripe data, so that the corresponding check block of the stripe data can be determined based on the XOR value subsequently.

[0087] Step S104: Determine the parity blocks of the first stripe data and the second stripe data based on the operation value and the XOR value.

[0088] In practical applications, after obtaining the operation value and the corresponding XOR value, the check blocks of the first stripe of data and the second stripe of data can be determined based on the operation value and the XOR value.

[0089] This application provides a data stripe encoding method, which includes obtaining a first stripe of data and a second stripe of data to be encoded; generating a calculated value for each of the first and second stripes of data, where the number of calculated values is equal to the number of redundant check blocks; generating an exclusive-or value based on the calculated value of the second stripe of data and the calculated value of the first stripe of data; and determining a check block for each of the first and second stripes of data based on the calculated value and the exclusive-or value. In this application, during the Hitchhiker erasure coding process, an exclusive-or value is calculated between the calculated value of the second stripe of data and the first calculated value of the first stripe of data, and a corresponding check block is determined based on the exclusive-or value. This exclusive-or value reduces the computational overhead of the Hitchhiker erasure coding and improves computational speed.

[0090] In a data stripe encoding method provided by an embodiment of the present application, in the process of generating an exclusive OR value based on the operation value of the second stripe data and the operation value of the first stripe data, if the number of redundant check blocks is 2, the exclusive OR value between the second operation value of the second stripe data and the first half of the second operation value of the first stripe data can be calculated, and the data value between the second operation value of the second stripe data and the second half of the second operation value of the first stripe data can be calculated.

[0091] Accordingly, in the process of determining the respective check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value, the first operation value of the first stripe data can be used as the first check block of the first stripe data; the XOR value between the second operation value of the second stripe data and the second half of the second operation value of the first stripe data can be used as the second check block of the first stripe data; the first operation value of the second stripe data can be used as the first check block of the second stripe data; and the XOR value between the second operation value of the second stripe data and the first half of the second operation value of the first stripe data can be used as the second check block of the second stripe data. The process can be as follows: Figure 2As shown. Among them, a1 to a k Indicates the first strip of data, b1 to b k Indicates the second strip of data,

[0092] In a specific application scenario, because Hitchhiker is based on the extension of RS coding, in the coding process when the number of redundant check blocks is 2, RS coding can be performed first to generate RS-coded F0(a), F0(b), and F1(b); then the first and second halves of F1(a) are calculated, the XOR value of F1(b) and the first half of F1(a) is calculated, and the XOR value of F1(b) and the second half of F1(a) is calculated, or F1(a) is directly calculated, the XOR value of F1(b) and the first half of F1(a) is calculated, and the XOR value of F1(b) and the second half of F1(a) is calculated; finally, F0(a) is used as the first check block of the first stripe data, and As the second check block of the first stripe data, F0(b) is used as the first check block of the second stripe data, and Serves as the second check value for the second stripe of data.

[0093] like Figure 2 As shown, assuming that data a1 and b1 need to be restored due to loss, according to the encoding method shown in this application, k+1+k / 2=1.5k+1 data blocks are required, while the existing technology requires 2k data blocks, so this application can reduce the computing overhead, and the larger k is, the closer the overhead reduction is to 25%.

[0094] In a data stripe encoding method provided by an embodiment of the present application, in a process of generating an exclusive-OR value based on an operation value of second stripe data and an operation value of first stripe data, if the number of redundant check blocks is greater than 2, an exclusive-OR value between the operation value of the second stripe data and corresponding parts of the first operation value of the first stripe data can be calculated, starting from the second operation value of the second stripe data and ending with the last operation value of the second stripe data; wherein the corresponding part of the first operation value of the first stripe data includes sub-portion data obtained by evenly dividing the first operation value of the first stripe data, the number of even divisions being the difference between the number of redundant check blocks and 1, and the i-th operation value of the second stripe data corresponds to the i-1-th sub-portion data.

[0095] Accordingly, in the process of determining the respective check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value, the d-th operation value of the first stripe data can be used as the d-th check block of the first stripe data, d = 1, 2, ..., n, n represents the number of redundant check blocks; the first operation value of the second stripe data can be used as the first check block of the second stripe data; the XOR value between the i-th operation value of the second stripe data and the corresponding part of the first operation value of the first stripe data can be used as the i-th check block of the second stripe data, i = 2, 3, ..., n. The process can be as follows: Figure 3 and Figure 4 As shown, etc., among which, Figure 3 This is a coding diagram when the redundant check block is 3. Figure 4 This is a coding diagram when the number of redundant check blocks is 4.

[0096] In a specific application scenario, during the encoding process when the number of redundant check blocks is 3, it is possible to generate Use F0(a), F1(a), and F2(a) as the first, second, and third check blocks of the first stripe data, and use F0(b) as the first check block of the second stripe data. As the second check value of the second stripe data, Serves as the third check value for the second stripe data.

[0097] In a specific application scenario, during the encoding process when the number of redundant check blocks is 4, it is possible to generate Let F0(a), F1(a), F2(a), and F3(a) be the first check block, second check block, third check block, and fourth check block of the first stripe data, and let F0(b) be the first check block of the second stripe data. As the second check value of the second stripe data, As the third check value of the second stripe data, Serves as the fourth check value for the second stripe data.

[0098] See also Figure 5 , Figure 5 Another flow chart of a data stripe encoding method provided in an embodiment of the present application.

[0099] A data stripe encoding method provided in an embodiment of the present application may include the following steps:

[0100] Step S201: Obtain target data to be encoded, where the target data includes a first stripe of data and a second stripe of data.

[0101] Step S202: Obtain a coding matrix.

[0102] In actual applications, in the Hitchhiker erasure code process, a coding matrix is used to represent the coding requirements, and the coding matrix is used to operate on the target data to obtain the corresponding check block. Therefore, after obtaining the target data to be encoded, the coding matrix needs to be obtained. The information of the coding matrix can be determined according to actual needs, and this application does not make specific limitations here.

[0103] Step S203: split the coding matrix to obtain coding sub-matrices, wherein at least one matrix value in any coding sub-matrix is non-zero, and any two coding sub-matrices are different.

[0104] In actual applications, there are continuous 0 values in the obtained coding matrix, and these 0 values have no computational significance. However, in actual applications, the 0 values will still be calculated, which makes the computational overhead too large. In order to solve this technical problem, after obtaining the coding matrix, this application does not directly apply the coding matrix for calculation, but first splits the coding matrix to obtain a coding sub-matrix, and at least one matrix value in any of the split coding sub-matrices is non-zero, and any two coding sub-matrices are different. In this way, there is no meaningless calculation process, nor is there an identical calculation process, which can reduce the computational overhead.

[0105] Step S204: split the target data into target sub-data corresponding to the coding sub-matrices.

[0106] In practical applications, because the coding matrix operates on the corresponding target data, after splitting the coding matrix to obtain the coding sub-matrix, the target data must be split into target sub-data corresponding to the coding sub-matrix, so that the coding sub-matrix can be subsequently applied to operate on the corresponding target sub-data to obtain the required operation results.

[0107] Step S205: Calculate the encoding sub-matrix and the corresponding target sub-data to obtain an intermediate result.

[0108] In practical applications, the matrix values in the encoding submatrix are the corresponding values in the encoding matrix, and the positions of the matrix values in the encoding submatrix in the encoding matrix are known. Therefore, the required calculation values can be determined based on the intermediate results according to the corresponding relationship between the matrix values in the encoding submatrix and the encoding matrix. For example, the encoding matrix is [1 2 0 1 2] and the target data is [a0 a1 a2a3 a4]. T , the encoding sub-matrix obtained after splitting is [1 2], and the target sub-data corresponding to the encoding sub-matrix is [a0 a1] T and [a3 a4] T , then the corresponding intermediate results obtained after operating the encoding sub-matrix and the corresponding target sub-data are a0+2a1 and a3+2a4.

[0109] Step S206 : generating operation values and corresponding XOR values of the first and second stripe data respectively based on the intermediate results, and determining check blocks of the first and second stripe data respectively based on the operation values and XOR values.

[0110] In practical applications, because the intermediate result is the result of the operation between the coding sub-matrix in the coding matrix and the corresponding target sub-data in the target data, and the relationship between the coding sub-matrix and the coding matrix is known, the relationship between the target data and the target sub-data is known, and the operation value and the XOR value can be obtained from the operation result between the coding sub-matrix and the target data, the operation relationship between the operation value (or XOR value) and the coding sub-matrix, the target data, the relationship between the coding sub-matrix and the coding matrix, and the relationship between the target data and the target sub-data can be used to determine the operation relationship between the operation value and the intermediate result, and the operation value (or XOR value) can be generated based on the intermediate result according to the operation relationship.

[0111] For ease of understanding, we still take the above encoding matrix as [1 2 0 1 2] as an example. Assume that the operation value is the result of the operation between the encoding matrix and the target data. Considering that the encoding sub-matrix is obtained by vertically dividing the encoding matrix, we only need to add the two intermediate results to get the required operation value: a0+2a1+a3+2a4.

[0112] In practical applications, the number of operation values in this application is equal to the number of redundant check blocks; and when the number of redundant check blocks is 2, the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data.

[0113] It should be noted that the Hitchhiker erasure code operation method provided in this application can be applied to servers, clients, and intelligent storage acceleration libraries, such as ISA-L libraries, etc., and this application does not make specific limitations here.

[0114] The embodiment of the present application provides a data strip coding method, which obtains target data to be encoded, the target data including first strip data and second strip data; obtains a coding matrix; splits the coding matrix to obtain coding sub-matrices, wherein at least one matrix value in any coding sub-matrix is non-0, and any two coding sub-matrices are different; splits the target data into target sub-data corresponding to the coding sub-matrix; operates the coding sub-matrix and the corresponding target sub-data to obtain an intermediate result; based on the intermediate result, generates the operation value and corresponding XOR value of each of the first strip data and the second strip data, so as to determine the check block of each of the first strip data and the second strip data based on the operation value and the XOR value. In the present application, after obtaining the coding matrix, the coding matrix is not directly applied for operation, but the coding matrix is first split to obtain coding sub-matrices, and at least one matrix value in any coding sub-matrix obtained by splitting is non-0, and any two coding sub-matrices are different. In this way, there is no meaningless calculation process and no identical calculation process, which can reduce the calculation overhead.

[0115] See also Figure 6 , Figure 6 This is a flowchart for splitting the coding matrix provided in an embodiment of the present application.

[0116] In a Hitchhiker erasure code operation method provided in an embodiment of the present application, the process of splitting a coding matrix to obtain coding sub-matrices may include the following steps:

[0117] Step S301: Split the coding matrix to obtain a first sub-matrix.

[0118] In actual applications, in the process of splitting the coding matrix, the coding matrix can be initially split to obtain a first sub-matrix. At this time, the matrix values in all first matrices are equal to the matrix values in the coding matrix. For ease of understanding, assuming that the matrix is [ABC], the first sub-matrix can be a separate A, B, C, etc. It should be noted that the coding matrix is split and the process of obtaining the first sub-matrix can be determined according to actual needs, and this application does not make specific limitations here.

[0119] Step S302: Delete the first submatrix whose matrix values are all 0, and use the undeleted first submatrix as the second submatrix.

[0120] In practical applications, after the encoding matrix is split to obtain the first submatrix, if the matrix values of the first submatrix are all 0, then the first submatrix has no computational significance. In order to avoid such a first submatrix occupying computing resources, the first submatrix with all matrix values 0 needs to be deleted, and the undeleted first submatrix is used as the second submatrix.

[0121] Step S303: De-duplicate the second sub-matrix to obtain a third sub-matrix.

[0122] In practical applications, the second submatrix may contain identical matrices, and the calculation results of the identical second submatrices are the same. Therefore, in order to avoid multiple identical second submatrices occupying computing resources, the second submatrix needs to be deduplicated to obtain a third submatrix without duplicate matrices.

[0123] Step S304: merging the third sub-matrices that meet the merging rule to obtain a coding sub-matrix. The merging rule includes that if at least one of the matrix values at the same position in the two matrices is 0, the two matrices are merged.

[0124] Step S305: directly use the third sub-matrix that does not meet the merging rule as the coding sub-matrix.

[0125] In practical applications, the resulting third submatrix may contain matrix values with a value of 0. The calculation results of such matrix values are meaningless and occupy the data being calculated, causing the data to be unable to be cached, affecting the calculation process of other matrices on the data, and resulting in excessive computational overhead. To avoid this situation, the third submatrices can be merged. That is, the third submatrices that meet the merging rule are merged to obtain the encoding submatrix. The merging rule includes that if at least one of the matrix values at the same position in the two matrices is 0, the two matrices are merged. For ease of understanding, assume that one matrix is [0 0 4 8] and the other matrix is [1 2 0 0]. At least one of the matrix values at the same position in the two matrices is 0, satisfying the merging rule. In this case, the two matrices can be merged into [1 2 4 8], and the calculation result of the data that originally required two cache hits can be obtained with only one cache hit. It should be noted that if the third submatrix does not meet the merging rule, such a third submatrix can be directly used as the encoding submatrix.

[0126] For ease of understanding, assume that the encoding matrix is:

[0127]

[0128] The encoding sub-matrix obtained after splitting according to the method provided in this application can be:

[0129]

[0130] Assume the target data is [a0 a1 a2 a3 b0 b1 b2 b3] T , then the result after running according to the original coding matrix is:

[0131] [a0+a1+a2+a3 b0+b1+b2+b3 4a2+8a3+b0+2b1+4b2+8b3 a0+2a1+b0+2b1+4b2+8b3] T ;

[0132] 8*4=32 calculations are required, and 32 read operations are required on the target data;

[0133] If the encoding sub-matrix is used for calculation, the intermediate result is:

[0134]

[0135] Only 4*2*2=16 calculations need to be performed, and only 16 reads of the target data are required. By comparison, it can be seen that this application can greatly reduce the computational overhead;

[0136] And according to the relationship between the encoding sub-matrix and the encoding matrix, when determining the corresponding result between [1 1 1 1 0 0 0 0] and the target data, it is only necessary to take out a0+a1+a2+a3 in the intermediate result to obtain the corresponding calculation result; when determining the corresponding result between [00 4 8 1 2 4 8] and the target data, it is only necessary to take out the second half 4a2+8a3 of the intermediate result a0+2a1+4a2+8a3 and b0+2b1+4b2+8b3 in the intermediate result and add them to obtain the corresponding calculation result, and the computational overhead is small.

[0137] See also Figure 7 , Figure 7 A schematic structural diagram of a data stripe encoding system provided in an embodiment of the present application.

[0138] An embodiment of the present application provides a data stripe encoding system, comprising:

[0139] A first acquisition module 101 is used to acquire first and second stripe data to be encoded;

[0140] A first generating module 102 is configured to generate operation values for the first data stripe and the second data stripe, wherein the number of operation values is equal to the number of redundancy check blocks;

[0141] A second generating module 103, configured to generate an XOR value based on the calculated value of the second stripe of data and the calculated value of the first stripe of data;

[0142] The first determining module 104 is configured to determine the parity blocks of the first data stripe and the second data stripe based on the operation value and the XOR value.

[0143] In a data stripe coding system provided by an embodiment of the present application, the second generation module is specifically configured to: if the number of redundant check blocks is 2, calculate an exclusive OR value between a second operation value of the second stripe data and a first half of the second operation value of the first stripe data, and calculate a data value between the second operation value of the second stripe data and a second half of the second operation value of the first stripe data.

[0144] An embodiment of the present application provides a data stripe coding system, in which a first determination module is specifically configured to: when the number of redundant check blocks is 2, use the first operation value of the first stripe data as the first check block of the first stripe data; use the exclusive OR value of the second operation value of the second stripe data and the second half of the first operation value of the first stripe data as the second check block of the first stripe data; use the first operation value of the second stripe data as the first check block of the second stripe data; and use the exclusive OR value of the second operation value of the second stripe data and the first half of the first operation value of the first stripe data as the second check block of the second stripe data.

[0145] An embodiment of the present application provides a data stripe encoding system, in which a second generation module is specifically configured to: if the number of redundant check blocks is greater than 2, calculate an exclusive OR value between the operation value of the second stripe data and the corresponding portion of the first operation value of the first stripe data, starting from the second operation value of the second stripe data and ending with the last operation value of the second stripe data; wherein the corresponding portion of the first operation value of the first stripe data includes sub-portion data obtained by evenly dividing the first operation value of the first stripe data, the number of even divisions being the difference between the number of redundant check blocks and 1, and the i-th operation value of the second stripe data corresponding to the i-1-th sub-portion data.

[0146] An embodiment of the present application provides a data stripe coding system, in which a first determination module is specifically configured to: when the number of redundant check blocks is greater than 2, use the dth operation value of the first stripe data as the dth check block of the first stripe data, where d = 1, 2, ..., n, and n represents the number of redundant check blocks; use the first operation value of the second stripe data as the first check block of the second stripe data; and use the exclusive OR value between the i-th operation value of the second stripe data and the corresponding part of the first operation value of the first stripe data as the i-th check block of the second stripe data, where i = 2, 3, ..., n.

[0147] In a data stripe encoding system provided by an embodiment of the present application, the first generating module may be specifically configured to: generate an operation value based on a generating formula;

[0148] The generation formula includes:

[0149]

[0150] Among them, F d-1(x) represents the dth operation value in the operation value; x0 represents the preset value, x m Indicates the mth data in the stripe data, where m = 1, 2, ..., n, k indicates the total amount of data in the stripe data, and n indicates the number of redundant check blocks.

[0151] See also Figure 8 , Figure 8 Another structural diagram of a data stripe encoding system provided in an embodiment of the present application.

[0152] An embodiment of the present application provides a data stripe coding operation system, which may include:

[0153] The second acquisition module 201 is used to acquire target data to be encoded, where the target data includes first stripe data and second stripe data;

[0154] The third acquisition module 202 is used to obtain a coding matrix;

[0155] A first splitting module 203 is configured to split the coding matrix to obtain coding sub-matrices, wherein at least one matrix value in any coding sub-matrix is non-zero, and any two coding sub-matrices are different;

[0156] A second splitting module 204 is used to split the target data into target sub-data corresponding to the encoding sub-matrix;

[0157] The first operation module 205 is used to operate the coding sub-matrix and the corresponding target sub-data to obtain an intermediate result;

[0158] A third generating module 206 is configured to generate a calculated value and a corresponding XOR value for each of the first data stripe and the second data stripe based on the intermediate result, and determine a check block for each of the first data stripe and the second data stripe based on the calculated value and the XOR value;

[0159] The number of operation values is equal to the number of redundant check blocks; and the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data.

[0160] An embodiment of the present application provides a data stripe coding system, in which a first splitting module can be specifically used to: split a coding matrix to obtain a first sub-matrix; delete the first sub-matrix in which all matrix values are 0, and use the undeleted first sub-matrix as the second sub-matrix; deduplicate the second sub-matrix to obtain a third sub-matrix; merge the third sub-matrix that meets the merging rule to obtain a coding sub-matrix, where the merging rule includes merging the two matrices if at least one of the matrix values at the same position in the two matrices is 0; and directly use the third sub-matrix that does not meet the merging rule as the coding sub-matrix.

[0161] Based on the hardware implementation of the above program modules, and in order to implement the method of the embodiment of the present invention, the embodiment of the present invention further provides an electronic device, Figure 9 FIG. 1 is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Figure 9 As shown, the electronic equipment includes:

[0162] Communication interface 1, capable of exchanging information with other devices such as network devices;

[0163] The processor 2 is connected to the communication interface 1 to implement information exchange with other devices and is used to execute the method provided by one or more of the above technical solutions when running a computer program. The computer program is stored in the memory 3.

[0164] Of course, in actual application, the various components in the electronic device are coupled together through the bus system 4. It can be understood that the bus system 4 is used to realize the connection and communication between these components. In addition to the data bus, the bus system 4 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 9 Various buses are labeled as bus system 4.

[0165] The memory 3 in the embodiment of the present invention is used to store various types of data to support the operation of the electronic device. Examples of such data include: any computer program used to operate on the electronic device.

[0166] It is understood that the memory 3 can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a magnetic random access memory (FRAM), a flash memory, a magnetic surface memory, an optical disc, or a compact disc read-only memory (CD-ROM); the magnetic surface memory can be a magnetic disk memory or a magnetic tape memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), synchronous static random access memory (SSRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct rambus random access memory (DRRAM).The memory 2 described in the embodiments of the present invention is intended to include, but is not limited to, these and any other suitable types of memories.

[0167] The method disclosed in the above embodiment of the present invention can be applied to processor 2 or implemented by processor 2. Processor 2 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by the hardware integrated logic circuit in processor 2 or by instructions in software form. The above processor 2 can be a general-purpose processor, a DSP, or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. Processor 2 can implement or execute the various methods, steps and logic block diagrams disclosed in the embodiment of the present invention. A general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present invention can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium, which is located in memory 3. Processor 2 reads the program in memory 3 and completes the steps of the above method in combination with its hardware.

[0168] When the processor 2 executes the program, the corresponding processes in the various methods of the embodiments of the present invention are implemented, which will not be described here for the sake of brevity.

[0169] In an exemplary embodiment, the present invention further provides a storage medium, namely, a computer storage medium, specifically, a computer-readable storage medium, such as a memory 3 storing a computer program. The computer program can be executed by a processor 2 to perform the steps of the aforementioned method. The computer-readable storage medium can be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface mount storage, optical disk, or CD-ROM.

[0170] In the several embodiments provided in this application, it should be understood that the disclosed devices, terminals and methods can be implemented in other ways. The device embodiments described above are only schematic. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0171] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0172] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0173] Those skilled in the art will understand that all or part of the steps of implementing the above-mentioned method embodiment can be completed by hardware related to program instructions, and the aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps of the above-mentioned method embodiment; and the aforementioned storage medium includes: mobile storage devices, ROM, RAM, disks or optical disks, etc. Various media that can store program codes.

[0174] Alternatively, if the above-mentioned integrated unit of the present invention is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for enabling an electronic device (which can be a personal computer, server, or network device, etc.) to execute all or part of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as mobile storage devices, ROM, RAM, magnetic disks or optical disks.

[0175] For descriptions of the relevant parts of the user operation processing system, electronic device, and computer-readable storage medium provided in the embodiments of the present application, please refer to the detailed description of the corresponding parts in the user operation processing method provided in the embodiments of the present application, and no further description is given here. In addition, the parts of the above-mentioned technical solutions provided in the embodiments of the present application that are consistent with the implementation principles of the corresponding technical solutions in the prior art are not described in detail to avoid excessive elaboration.

[0176] It should also be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article, or device comprising the element.

[0177] The above description of the disclosed embodiments will enable one skilled in the art to implement or use the present application. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application is not limited to the embodiments shown herein, but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data strip encoding method, characterized in that: include: Obtaining the first stripe of data and the second stripe of data to be encoded; generating operation values for the first stripe of data and the second stripe of data, respectively, where the number of the operation values is equal to the number of redundant check blocks, and the number of the redundant check blocks is greater than or equal to 2; generating an exclusive OR value based on the operation value of the second stripe of data and the operation value of the first stripe of data; Determine a check block of each of the first stripe data and the second stripe data based on the operation value and the XOR value; When the number of the redundant check blocks is 2, determining the check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value includes: using the first operation value of the first stripe of data as a first check block of the first stripe of data; taking an exclusive OR value of a second operation value of the second stripe of data and a second half of the second operation value of the first stripe of data as a second check block of the first stripe of data; using the first operation value of the second stripe of data as a first check block of the second stripe of data; An exclusive OR value of the second operation value of the second stripe data and the first half of the second operation value of the first stripe data is used as a second check block of the second stripe data.

2. The method according to claim 1, characterized in that The generating of an exclusive OR value based on the operation value of the second stripe data and the operation value of the first stripe data includes: If the number of redundant check blocks is 2, then calculate the XOR value between the second operation value of the second stripe data and the first half of the second operation value of the first stripe data, and calculate the data value between the second operation value of the second stripe data and the second half of the second operation value of the first stripe data.

3. The method according to claim 1, characterized in that The generating of an exclusive OR value based on the operation value of the second stripe data and the operation value of the first stripe data includes: If the number of the redundancy check blocks is greater than 2, calculating an exclusive OR value between corresponding parts of the operation value of the second stripe of data and the first operation value of the first stripe of data, starting from the second operation value of the second stripe of data and ending with the last operation value of the second stripe of data; Among them, the corresponding part of the first operation value of the first stripe data includes the sub-partial data obtained after the first operation value of the first stripe data is evenly divided, the number of equal divisions is the difference between the number of the redundant check blocks and 1, and the i-th operation value of the second stripe data corresponds to the i-1-th sub-partial data.

4. The method according to claim 1, wherein When the number of the redundant check blocks is greater than 2, determining the check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value includes: Using the dth operation value of the first stripe of data as the dth check block of the first stripe of data, where d=1, 2, ..., n, and n represents the number of the redundant check blocks; using the first operation value of the second stripe of data as a first check block of the second stripe of data; The XOR value between the i-th operation value of the second stripe data and the corresponding part of the first operation value of the first stripe data is used as the i-th check block of the second stripe data, where i=2, 3, . . . , n.

5. The method according to any one of claims 1 to 4, characterized in that The process of generating the operation values of the first stripe of data and the second stripe of data includes: generating the operation value based on a generation formula; The generation formula includes: Among them, F d-1 (x) represents the dth operation value in the operation value; x0 represents the preset value, x m represents the mth data in the stripe data, m=1, 2, ..., n, k represents the total amount of data in the stripe data; and n represents the number of the redundant check blocks.

6. A data strip encoding method, characterized in that: include: Acquire target data to be encoded, where the target data includes first stripe data and second stripe data; Get the encoding matrix; Splitting the encoding matrix to obtain encoding sub-matrices, wherein at least one matrix value in any encoding sub-matrix is non-zero, and any two encoding sub-matrices are different; Splitting the target data into target sub-data corresponding to the encoding sub-matrix; Calculate the encoding sub-matrix and the corresponding target sub-data to obtain an intermediate result; generating, based on the intermediate result, an operation value and a corresponding exclusive-OR value of each of the first stripe data and the second stripe data, and determining, based on the operation value and the exclusive-OR value, a check block of each of the first stripe data and the second stripe data; The number of the operation values is equal to the number of redundant check blocks, and the number of the redundant check blocks is greater than or equal to 2; and the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data; When the number of the redundant check blocks is 2, determining the check blocks of the first stripe data and the second stripe data based on the operation value and the XOR value includes: using the first operation value of the first stripe of data as a first check block of the first stripe of data; taking an exclusive OR value of a second operation value of the second stripe of data and a second half of the second operation value of the first stripe of data as a second check block of the first stripe of data; using the first operation value of the second stripe of data as a first check block of the second stripe of data; An exclusive OR value of the second operation value of the second stripe data and the first half of the second operation value of the first stripe data is used as a second check block of the second stripe data.

7. The method according to claim 6, characterized in that The step of splitting the encoding matrix to obtain encoding sub-matrices includes: Splitting the encoding matrix to obtain a first sub-matrix; Deleting the first submatrix whose matrix values are all 0, and using the undeleted first submatrix as the second submatrix; Deduplication is performed on the second sub-matrix to obtain a third sub-matrix; Merging the third submatrices that satisfy a merging rule to obtain the coding submatrix, wherein the merging rule includes merging the two matrices if at least one of the matrix values at the same position in the two matrices is 0; The third sub-matrix that does not satisfy the merging rule is directly used as the encoding sub-matrix.

8. A data stripe coding system, characterized in that: include: A first acquisition module, configured to acquire first and second stripe data to be encoded; A first generating module is configured to generate operation values for the first stripe of data and the second stripe of data, wherein the number of the operation values is equal to the number of redundant check blocks, and the number of the redundant check blocks is greater than or equal to 2; a second generating module, configured to generate an XOR value based on the operation value of the second stripe of data and the operation value of the first stripe of data; a first determining module, configured to determine a check block of each of the first stripe data and the second stripe data based on the operation value and the XOR value; Wherein, the first determination module is used to: when the number of the redundant check blocks is 2, use the first operation value of the first stripe data as the first check block of the first stripe data; use the exclusive OR value of the second operation value of the second stripe data and the second half of the second operation value of the first stripe data as the second check block of the first stripe data; use the first operation value of the second stripe data as the first check block of the second stripe data; and use the exclusive OR value of the second operation value of the second stripe data and the first half of the second operation value of the first stripe data as the second check block of the second stripe data.

9. A data stripe coding system, characterized in that: include: A second acquisition module is used to acquire target data to be encoded, wherein the target data includes the first stripe data and the second stripe data; A third acquisition module is used to obtain a coding matrix; A first splitting module is configured to split the encoding matrix to obtain encoding sub-matrices, wherein at least one matrix value in any encoding sub-matrix is non-zero, and any two encoding sub-matrices are different; A second splitting module, configured to split the target data into target sub-data corresponding to the encoding sub-matrix; A first operation module is used to operate the encoding sub-matrix and the corresponding target sub-data to obtain an intermediate result; a third generating module, configured to generate, based on the intermediate result, an operation value and a corresponding XOR value of each of the first data stripe and the second data stripe, so as to determine, based on the operation value and the XOR value, a check block of each of the first data stripe and the second data stripe; The number of the operation values is equal to the number of redundant check blocks, and the number of the redundant check blocks is greater than or equal to 2; and the XOR value includes an XOR value generated based on the operation value of the second stripe data and the operation value of the first stripe data; Wherein, the third generation module is used to: when the number of the redundant check blocks is 2, use the first operation value of the first stripe data as the first check block of the first stripe data; use the exclusive OR value between the second operation value of the second stripe data and the second half of the second operation value of the first stripe data as the second check block of the first stripe data; use the first operation value of the second stripe data as the first check block of the second stripe data; and use the exclusive OR value between the second operation value of the second stripe data and the first half of the second operation value of the first stripe data as the second check block of the second stripe data.

10. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the steps of the method according to any one of claims 1 to 7 when executing the computer program.

11. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which implements the steps of the method according to any one of claims 1 to 7 when executed by a processor.

Images