Data processing method, device, storage medium and program product

Abstract

This invention provides a data processing method, apparatus, storage medium, and program product, applicable to the fields of data processing and computer technology. The data processing method includes: in response to receiving a data group to be processed sent by a data sender, performing a correctness check on the data group to be processed to obtain a check result; if the check result indicates that there is an erroneous data block with a transmission error among multiple information data blocks, determining a standard data block for data recovery of the erroneous data block based on the dependency relationship between multiple transmission nodes used to send the multiple data blocks, wherein the standard data block includes other information data blocks besides the erroneous data block; restoring the erroneous data block based on the standard data block to obtain the desired data in the erroneous data block; and determining the recovery result of the data group to be processed based on the data blocks without transmission errors and the desired data in the erroneous data blocks.

Description

Technical Field

[0001] The present invention relates to the field of data processing and computer technology, and in particular to a data processing method, device, medium and program product. Background Art

[0002] With the development of network technology, the communication rate of the Internet is getting higher and higher, so the efficiency of data transmission tasks using the Internet is also improving. However, the failure of Internet hardware, such as network components such as nodes, links and switches, may cause data loss and service interruption during data transmission, which has a significant impact on social production and daily life. Therefore, studying the fault tolerance mechanism of the Internet is crucial to ensure the stability and reliability of data transmission.

[0003] In related technologies, an automatic retransmission request mechanism is usually used to resend data in a timely manner after determining that data packets are lost, so as to ensure that the data receiving end can receive complete data. However, the automatic retransmission request mechanism will cause a large amount of data to be retransmitted, which requires occupying channels, increases data redundancy, and increases communication delays. Summary of the invention

[0004] In view of the above problems, the present invention provides a data processing method, device, medium and program product.

[0005] According to a first aspect of the present invention, there is provided a data processing method, comprising: in response to receiving a data group to be processed sent by a data sending end, performing correctness verification on the data group to be processed to obtain a verification result, wherein the data group to be processed includes a plurality of data to be processed of the same length, each data to be processed includes a plurality of sub-data, a plurality of data in the plurality of data to be processed having the same relative position constitute a data block, and the plurality of data blocks include information data blocks and verification data blocks; in a case where the verification result indicates that there are erroneous data blocks with transmission errors among the plurality of information data blocks, determining a standard data block for data recovery of the erroneous data blocks based on a dependency relationship between a plurality of transmission nodes for sending the plurality of data blocks, wherein the standard data block includes other information data blocks except the erroneous data blocks; based on the standard data block, restoring the erroneous data blocks to obtain expected data in the erroneous data blocks; and determining a recovery result of the data group to be processed based on the data blocks without transmission errors and the expected data in the erroneous data blocks.

[0006] A second aspect of the present invention provides a data processing device applied to a data receiving end, comprising:

[0007] A data verification module, configured to, in response to receiving a data group to be processed sent by a data sending end, perform correctness verification on the data group to be processed to obtain a verification result, wherein the data group to be processed includes a plurality of data to be processed with the same length, each data to be processed includes a plurality of sub-data, a plurality of data with the same relative position in the plurality of data to be processed constitutes a data block, and the plurality of data blocks include an information data block and a verification data block;

[0008] a data determination module, configured to determine, when the verification result indicates that there is an error data block in which a transmission error occurs among the plurality of information data blocks, a standard data block for recovering data from the error data block based on a dependency relationship between a plurality of transmission nodes for transmitting the plurality of data blocks, wherein the error data block includes the information data block;

[0009] A data restoration module, used to restore the erroneous data block based on the standard data block to obtain the expected data in the erroneous data block; and

[0010] The result determination module is used to determine the recovery result of the to-be-processed data group based on the data blocks without transmission errors and the expected data in the error data blocks.

[0011] A third aspect of the present invention provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0012] The fourth aspect of the present invention further provides a computer-readable storage medium on which a computer program or instruction is stored, and the steps of the above method are implemented when the above computer program or instruction is executed by a processor.

[0013] The fifth aspect of the present invention also provides a computer program product, including a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0014] According to an embodiment of the present invention, by setting dependencies, when there is an error data block with a transmission error among multiple information data blocks in the data group to be processed represented by the verification result, based on the above dependencies, a standard data block for data recovery of the error data block is determined, and the error data block is restored using the standard data block to obtain the recovery result of the data group to be processed. Through the dependencies between different data blocks, the amount of data required to read and process in order to recover the error data block can be reduced in different situations, thereby improving the bandwidth and communication delay required for data processing. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above contents and other objects, features and advantages of the present invention will become more apparent through the following description of the embodiments of the present invention with reference to the accompanying drawings, in which:

[0016] Figure 1 An application scenario diagram of a data processing method, device, medium, and program product according to an embodiment of the present invention is shown.

[0017] Figure 2 A flow chart of a data processing method according to an embodiment of the present invention is shown.

[0018] Figure 3 A schematic diagram of the loading relationship of the data processing method according to an embodiment of the present invention is shown.

[0019] Figure 4 A schematic diagram showing data exchange and interference elimination of multiple erroneous data blocks according to an embodiment of the present invention is shown.

[0020] Figure 5 A structural block diagram of a data processing device according to an embodiment of the present invention is shown.

[0021] Figure 6 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0022] Below, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present invention. In the following detailed description, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of embodiments of the present invention. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessary confusion of concepts of the present invention.

[0023] The terms used herein are only for describing specific embodiments and are not intended to limit the present invention. The terms "comprise", "include", etc. used herein indicate the existence of the features, steps, operations and / or components, but do not exclude the existence or addition of one or more other features, steps, operations or components.

[0024] All terms (including technical and scientific terms) used herein have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0025] When using expressions such as "at least one of A, B, and C, etc.", they should generally be interpreted according to the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0026] An embodiment of the present invention provides a data processing method performed by a data receiving end, comprising: in response to receiving a data group to be processed sent by a data sending end, performing correctness verification on the data group to be processed to obtain a verification result, wherein the data group to be processed includes multiple data to be processed with the same length, each data to be processed includes multiple sub-data, multiple data with the same relative position in the multiple data to be processed constitute a data block, and the multiple data blocks include information data blocks and verification data blocks; when the verification result indicates that there are error data blocks with transmission errors in the multiple information data blocks, based on the dependency relationship between multiple transmission nodes used to send the multiple data blocks, determining a standard data block for data recovery of the error data block, wherein the standard data block includes other information data blocks except the error data block; based on the standard data block, restoring the error data block to obtain the expected data in the error data block; and determining the recovery result of the data group to be processed based on the data blocks without transmission errors and the expected data in the error data blocks.

[0027] Figure 1 An application scenario diagram of a data processing method, device, medium, and program product according to an embodiment of the present invention is shown.

[0028] like Figure 1 As shown, the application scenario 100 according to this embodiment may include a data transmitter 101, a network 102, and a data receiver 103. The network 102 is used to provide a medium for a communication link between the data transmitter 101 and the data receiver 103. The network 102 may include various connection types, such as wired, wireless communication links, or optical fiber cables, etc.

[0029] The data sending end 101 is used to determine the data to be transmitted, add a check code for correctness verification to the data to be transmitted, encapsulate the encapsulated data to be transmitted, and then transmit the encapsulated data to be transmitted to the data receiving end 103 through the network 102.

[0030] The data receiving end 103 is used to receive the encapsulated data to be transmitted, determine it as data to be processed, and parse and process the data to be processed. When it is determined that a transmission error occurs in the process of sending the data to be processed through the network 102, the data with transmission errors is corrected using the checksum and other information encapsulated in the data to be processed to obtain correct data.

[0031] The following will be based on Figure 1 The scene described by Figure 2~Figure 4 The data processing method of the embodiment of the invention is described in detail.

[0032] Figure 2 A flow chart of a data processing method according to an embodiment of the present invention is shown.

[0033] like Figure 2 As shown, the data processing method of this embodiment includes operations S210 to S240.

[0034] In operation S210, in response to receiving a data group to be processed sent by a data sending end, a correctness check is performed on the data group to be processed to obtain a check result.

[0035] The data transmitting end can be used to perform pre-processing such as encapsulation on the data to be transmitted, obtain a data group to be processed, and send the data group to be processed to the data receiving end.

[0036] The data group to be processed may include multiple data to be processed of the same length, and each data to be processed may include multiple sub-data, and multiple sub-data with the same relative position in the data to be processed constitute a data block, and the types of the multiple data blocks may include information data blocks and check data blocks, wherein the check data may be determined based on the information data and added to the information data. The check data is used to perform correctness check and error correction on the multiple sub-data in the data to be processed.

[0037] In the case where the correctness check determines that data loss occurs in the data group to be processed, it can be determined that the check result indicates that there is an erroneous data block with a transmission error in the multiple information data blocks.

[0038] In operation S220, when the verification result indicates that there is an erroneous data block with a transmission error among the multiple information data blocks, based on the dependency relationship between multiple transmission nodes used to send the multiple data blocks, a standard data block for data recovery of the erroneous data block is determined, and the standard data block includes other information data blocks except the erroneous data block.

[0039] Since the purpose of adding the check data block after the information data block is to check the correctness of the data to be processed and to perform subsequent error correction, after receiving the data group to be processed, if it is determined that only the check data block is wrong, it will not affect the data information of the data to be processed itself, and no error correction is required. Only when the data in the information data block is wrong, it is necessary to restore the data in the error data block through data processing.

[0040] The dependency relationship between multiple transmission nodes may include a mounting relationship between multiple transmission nodes and / or a cooperation relationship between multiple transmission nodes.

[0041] The piggyback relationship may indicate that one or several information data of the data to be processed are piggybacked onto the verification data of other data to be processed. In this case, based on the piggyback relationship, the data block that carries the data in the error data block may be determined, the data in the data block may be read, and the information data carried by the data block may be restored based on the verification data before the data block carries the information data.

[0042] When the dependency relationship between multiple transmission nodes is a piggyback relationship, it can be determined that the information data in the erroneous data block can be accurately restored through the piggyback relationship and the verification data block, so it can be determined that the standard data block includes other data blocks except the erroneous data block.

[0043] The cooperative relationship can mean that when there are multiple erroneous data blocks, after using other information data blocks to preliminarily restore the information data of each of the multiple erroneous data blocks, due to the mutual influence between the data in the multiple erroneous data blocks, data exchange can be performed by using the data after the preliminary restoration of the multiple erroneous data blocks, thereby eliminating the mutual influence between the erroneous data and obtaining accurate information data.

[0044] When the dependency between multiple transmission nodes is a cooperative relationship, since data exchange is required between multiple error data blocks after initial restoration, it can be determined that the standard data block includes other information data blocks used during initial restoration and multiple error data blocks, that is, all information data blocks.

[0045] In operation S230, the error data block is restored based on the standard data block to obtain expected data in the error data block.

[0046] The check data may be a Reed-Solomon Code (RS code) generated based on the information data, or may be other forward error correction codes (FEC codes). In addition to the information data block, the standard data block may also include one or more check data blocks.

[0047] After determining the standard data block, other information data in each data to be processed except the information data in the error data block and one or more verification data can be determined. Based on the data in the standard data block, the error data block can be restored to obtain the expected data.

[0048] In the case where there are multiple erroneous data blocks, after the erroneous data blocks are initially restored based on the data in the standard data blocks, data can be exchanged between the multiple erroneous data blocks through the above-mentioned cooperative relationship to determine the expected data of each of the multiple erroneous data blocks.

[0049] In operation S240, a recovery result of the to-be-processed data group is determined based on the data blocks without transmission errors and the expected data in the error data blocks.

[0050] After recovering the expected data in the erroneous data block, the data in the data block without transmission error and the expected data are spliced ​​to determine the recovery result of the data group to be processed. The recovery result includes the recovery results of each of the multiple data to be processed. The multiple recovered data to be processed can be obtained by dividing the matrix corresponding to the data group to be processed by columns.

[0051] Since the check data block and the check data included therein are used for data recovery and have no actual meaning, and are not data that needs to be transmitted, after the multiple recovered data to be processed are obtained through the above operation, the check data in the multiple recovered data to be processed can be deleted to obtain multiple information data included in each of the multiple data to be processed. According to the multiple remaining information data in each of the data to be processed, the data that the data sending end wants to transmit to the data receiving end can be determined, so that the data receiving end can perform subsequent data storage or data processing according to the processed data.

[0052] According to an embodiment of the present invention, by setting dependencies, when there is an error data block with a transmission error among multiple information data blocks in the data group to be processed represented by the verification result, based on the above dependencies, a standard data block for data recovery of the error data block is determined, and the error data block is restored using the standard data block to obtain the recovery result of the data group to be processed. Through the dependencies between different data blocks, the amount of data required to read and process in order to recover the error data block can be reduced in different situations, thereby improving the bandwidth and communication delay required for data processing.

[0053] According to an embodiment of the present invention, multiple data blocks include information data blocks and check data blocks, wherein the number of the multiple data blocks is n, the multiple data blocks include k1 information data blocks and p check data blocks, k1+p=n, k1, p and n are all integers greater than 1; the p check data blocks include 1 first check data block that does not carry the information in the information data block and p-1 second check data blocks that carry the information in the information data block; based on the dependency relationship between multiple transmission nodes used to send the multiple data blocks, determine the standard data block for data recovery of the erroneous data block, including: determining k1-1 information data blocks and the first check data block other than the erroneous data block as standard data blocks.

[0054] For ease of understanding, the data group to be processed can be regarded as an n×l two-dimensional matrix, where each column of data corresponds to a data to be processed, each row of data constitutes a data block, the first k1 rows of data are k1 information data blocks, and the last p rows of data are p verification data blocks.

[0055] Each to-be-processed data may include k1 information data and p verification data, and the information data includes a plurality of data obtained by dividing the original information to be transmitted.

[0056] According to the properties of the RS code, it can be determined that at least the same amount of data as the information data and the original information data to be processed is required to use the RS code for forward data recovery. Therefore, when there are erroneous data blocks, the amount of correct information data is insufficient, and the data in the check data block is needed to complete the forward data recovery process.

[0057] In the process of forward data recovery using the data in the verification data block, it is found that the information in the second verification data block includes the verification data and the correct information data in the information data block carried by the data block. Since the information data carried by the verification data block and the verification data belong to different data to be processed, in order to determine the verification data in the second verification data block, it is also necessary to read the information data from other data to be processed, which increases the number of data reading and writing and reduces data processing efficiency.

[0058] A check data block can be selected from the p check data blocks as the first check data block. Since the first check data block does not carry information data and is only used to restore the data to be processed to which the check data belongs, the above problem of reduced data processing efficiency can be avoided. When restoring data, k1-1 information data blocks and the first check data block other than the error data block are determined as standard data blocks. At this time, the number of standard data blocks is k1, which is the same as the amount of information data in the data to be processed. Therefore, k1 standard data blocks can be used to restore the data in the error data block.

[0059] According to an embodiment of the present invention, the check data block is divided into a second check data block that carries the information in the information data block and a first check data block that does not carry the information in the information data block, thereby ensuring that the check data in the first check data block and the information data can be used to recover the data to be processed according to the properties of the RS code.

[0060] According to an embodiment of the present invention, the number of multiple data to be processed is l, k1 information data blocks respectively include l original information data, and p verification data blocks respectively include l original verification data; the data processing method also includes: for the l original information data in the k1 information data blocks, randomly dividing the l original information data into data to be carried and data not to be carried, so that there is and only one data not to be carried in the k1 information data blocks, and there are l-1 data not to be carried in the multiple data to be processed; and for multiple data to be carried, the data to be carried is carried on the original verification data of the data to be processed that belongs to different data to be carried.

[0061] The l original information data in each information data block are divided into one data that does not need to be carried and l-1 data that need to be carried, and the multiple carrying data in all information data blocks are respectively carried on the original verification data in the data to be processed that belongs to different data to be carried.

[0062] Figure 3 A schematic diagram of the loading relationship of the data processing method according to an embodiment of the present invention is shown.

[0063] like Figure 3 As shown, take k1 as 6, p as 3, and l as 3 as an example. Determine that the first check data in each data to be processed does not carry information data, then the check data block composed of the row of data is [p 1,1 ,p 1,2 ,p 1,3 ] is the first verification data block, and the verification data block consisting of the remaining two rows of data is the second verification data block.

[0064] When carrying data, in addition to random division, the carrying relationship between information data and verification data can also be determined in the following way: for each data to be processed, k1 information data are divided into l parts. When it is impossible to divide them evenly, k1=ul+v, 0≤v≤l can be set, and the k1 information data are divided into l parts, where the first v parts each include u+1 information data, and the last lv parts each include u information data, so that the difference in the number of information data in each part does not exceed 1, ensuring that the division is as equal as possible.

[0065] After the information data of the i-th data to be processed is divided, the n-th part is represented as s n,i , 1≤i,n≤l. For the information data in the nth share, divide it into p-1 shares, so that the difference in the number of information data in each share does not exceed 1, and the information data set in each share is expressed as , then .

[0066] Based on the subscripts of the verification data included therein, the information data carried by the verification data is determined, as shown in formula (1):

[0067] (1)

[0068] Among them, q∈[2,p], n∈[1,n], δ is a non-zero element in the same finite field as the data to be processed, is a set of information data, and c is the information data in the data to be processed.

[0069] When selecting δ, it is necessary to ensure that the data to be processed after adding δ satisfies the properties of Maximum Distance Separable Code (MDS code). Figure 3 For example, when any n-k1=9-6=3 sub-data are lost or damaged, the lost or damaged sub-data can be restored through the remaining n-3=9-3=6 sub-data. Computers can be used to search in a finite field so that δ satisfies the above properties.

[0070] According to the above formula, the information data is loaded into p 2,1 ,p 2,2 ,p 2,3 ,p 3,1 ,p 3,2 ,p 3,3 On, we can get Figure 3 The data set to be processed is shown.

[0071] According to an embodiment of the present invention, by dividing the information data and mounting it on the verification data, it can ensure that in the event of a transmission error in any data block, the remaining data can accurately restore the erroneous data based on the properties of the RS code.

[0072] According to an embodiment of the present invention, a plurality of information data blocks respectively include original data that is not carried to the second verification data block and original data that is carried to the second verification data block; based on the standard data block, the error data block is restored to obtain expected data in the error data block, including: determining the first error data that is not carried to the second verification data block and the second error data that is carried to the second verification data block in the error data block; determining the target data to be processed where the first error data is located; based on a plurality of standard data blocks, the first error data is restored to obtain the first expected data; determining the target verification data block for carrying the second error data from the target data to be processed; based on the target verification data block and other original data carried in the target verification data block, the second error data is restored to obtain the second expected data; and the expected data is determined based on the first expected data and the second expected data.

[0073] Among the l sub-data in each information data block, there may be one first error data that is not carried into the second verification data block, and l-1 second error data that are carried into the second verification data block.

[0074] In the process of recovering the first error data, the target data to be processed to which the first error data belongs is determined, wherein, except for the first error data, all other data of the target data to be processed are correct data. Based on multiple standard data blocks, the first error data is restored by using the properties of the RS code to obtain the first expected data.

[0075] Since the target check data block is transmitted without error, the target check data block correctly stores the check data corresponding to the target check data block and the correct information data of the second error data carried by the data block, that is, the second expected data. The second expected data corresponding to the second error data can be determined by simply determining the check data corresponding to the target check data block.

[0076] Using the same recovery method as that for the first erroneous data, the verification data in multiple second verification data blocks in the target data to be processed can also be restored to obtain the verification data corresponding to the target verification data block. Further, the second erroneous data can be restored based on the data in the target verification data block and the verification data to determine the expected data.

[0077] by Figure 3 For example, when the data in the first information data block is lost, c 1,1 ,c 1,2 ,c 1,3 Not available. At this point, you can determine c 1,3 is the data not carried in the second check data block, that is, the first error data, c 1,1 and c 1,2 It is the data that has been loaded into the second verification data block, that is, the second error data.

[0078] First, recover the first error data to determine c 1,3 The target data to be processed corresponds to the third column, so it can be based on {c 2,3 ,c 3,3 ,c 4,3 ,c 5,3 ,c 6,3 ,p 1,3}Using the properties of RS code, c 1,3 and p 2,3 ,p 3,3 To restore. That is, to get the same 1,3 The corresponding first expected data and the verification data p of the target data to be processed in the second verification data block 2,3 ,p 3,3 .

[0079] According to {c 2,1 ,p 2,3 + c 1,1 +c 2,1}, we can determine p 2,3 + c 1,1 , because when restoring the target data to be processed, p has been determined 2,3 The value of c can therefore be determined 1,1 The corresponding second expected data.

[0080] Similarly, we can determine 1,2 The corresponding second expected data is obtained, thereby determining the expected data of the error data block according to the first expected data and all the second expected data.

[0081] According to the above operation of recovering the error data block, it can be determined that in the process of recovering the error data block, {c 2,3 ,c 3,3 ,c 4,3 ,c 5,3 ,c 6,3 ,p 1,3， c 2,1 ,p 2,3 + c 1,1 +c 2,1， c 2,2 ,p 3,3 + c 1,2 +c 2,2}10 sub-data.

[0082] In the case of no piggybacking, to recover the above erroneous data block, for each data to be processed, n-3=9-3=6 sub-data are needed, that is, 3×6=18 sub-data are needed. Therefore, by piggybacking the information data to the verification data, the number of sub-data needed for data recovery can be effectively reduced.

[0083] Similarly, in Figure 3 Under the shown loading relationship, if any information data block has a transmission error, 10 sub-data are required to repair the erroneous data block. Compared with using 18 sub-data for recovery, the repair bandwidth is 10 / 18=55.6%, saving 44.4% of the repair bandwidth.

[0084] Through analysis and calculation, when the number of sub-data n in the data to be processed and the number of information data blocks k1 are determined, as the number of data to be processed in the data group to be processed increases, the number of information data that can be carried in the verification data will also increase. Compared with the ordinary forward data recovery method, the average repair bandwidth required to recover an erroneous data block is reduced more significantly. When the number of verification data blocks p is determined, as the number of sub-data in the data to be processed increases, the average repair bandwidth is reduced more significantly.

[0085] According to the embodiment of the present invention, since the information data block includes original data that is not carried in the second verification data block, when a transmission error occurs in the information data block, the original data that is not carried in the second verification data block can be restored first according to the properties of the verification code. The original data of other second verification data blocks that have been carried can be restored based on the carrying relationship, thereby ensuring the accuracy of data processing.

[0086] In the process of implementing the present invention, it is found that with a finite field For example, the data on this finite field can be regarded as a finite field The eight-element vector on is equivalent to a byte composed of eight bits of data. When a transmission error occurs, it may be that one or several of the eight-element vectors are wrong, not all eight data are wrong. When using the forward data recovery method for data recovery, it is necessary to use the byte-level data in other data blocks, that is, to use the byte-level data to recover the bit-level data, which is too costly.

[0087] According to the properties of trace mapping, let and yes exist For two groups of bases on , we can determine formula (2):

[0088] (2)

[0089] Then the above two groups of bases are dual bases, where yes arrive The trace mapping on , l represents the expansion number of the finite field. For any α∈ ,have Therefore, according to α can be uniquely determined. Preferably, l can be 8.

[0090] Through the above analysis, it is assumed that the data to be processed includes n sub-data, and the n sub-data are n-k1 original data bits to be transmitted and k1 cyclic redundancy check bits added according to the original data bits. Bytes in In case of transmission errors, just make sure Dual codeword , and make constitute exist A basis on , , , is the column multiplier of the dual code.

[0091] Since the trace function Linear property, for any , there is a relationship as shown in formula (3):

[0092] (3)

[0093] Because according to α can be uniquely determined, so according to You can determine , according to the set on the right side of formula (3) It can be seen that only by using You can restore Therefore, restore The required repair bandwidth is , the amount of information data that set t needs to provide is Through the above analysis, we can find the appropriate dual information data, that is, the set of check polynomials corresponding to the information data. , which can reduce the repair bandwidth.

[0094] The above check polynomial can be selected from l subspace polynomials of degree less than or equal to n-k1+1, and the subspace polynomial is based on A polynomial defined as zeros of a linear subspace W in .make express Kernel, Indicates L W Image, α represents the independent variable in the linear subspace W.

[0095] Assumptions , yes One of Victoria Linear subspace. Let ,remember ,but .So yes The linearized polynomial on is also a polynomial from to its own Linear mapping. And, yes The middle dimension is of Linear subspace.

[0096] For a given parameter RS ​​code, select one Victoria Linear subspace ,satisfy , where deg() represents the degree of the polynomial. exist A basis on , then for an invalid data , we have the following formula (4):

[0097] (4)

[0098] In this case, the length of the code to be restored is set to b, and we have Bit.

[0099] In the actual operation process of recovering lost data, the selection method of the check polynomial has nothing to do with the specific information of the code word bit, but only with the assignment point bit. Therefore, pre-calculation and table lookup can be used to further reduce communication delay.

[0100] According to an embodiment of the present invention, the data processing method also includes: determining an assignment point set based on the number of check data blocks, wherein the assignment point set includes multiple assignment points, and multiple check data blocks correspond to one assignment point respectively; determining multiple check polynomials for each assignment point, wherein the degree of the check polynomial is determined based on the number of check data blocks and the number of information data blocks; based on multiple check polynomials, respectively determining trace mappings for restoring erroneous data blocks that have transmission errors in the event that transmission errors occur in multiple information data blocks; and storing the trace mappings in a data recovery table.

[0101] In each system, the parameters of the RS code are usually determined, that is, the set of assignment points and the number of information data Therefore, a number less than or equal to The check polynomial ,in, .

[0102] Determine the data bits in multiple information data blocks separately In the event of a transmission error, the corresponding trace mapping is determined based on multiple check polynomials to obtain The above trace mapping is fixed, so multiple trace mappings determined for different data blocks and different data bits can be saved in the data recovery table, so that after a transmission error occurs, the trace mapping used for data restoration can be quickly determined from the data recovery table based on the data blocks and data bits with transmission errors.

[0103] According to an embodiment of the present invention, by determining the dual codeword corresponding to the data to be processed, determining the subspace polynomial corresponding thereto, and constructing a check polynomial based on the subspace polynomial, the corresponding trace mapping is restored for each data block in the data group to be processed, and the trace mapping is saved in a data recovery table, so that when data restoration is required, the trace mapping can be quickly determined and the corresponding operation can be performed to restore the data block, thereby improving the response speed of data processing and reducing the delay.

[0104] According to an embodiment of the present invention, based on the dependency relationship between multiple transmission nodes used to send multiple data blocks, a standard data block for data recovery of an erroneous data block is determined, including: determining a standard data block for data recovery of an erroneous data block from a pre-stored data recovery table, wherein the data recovery table includes multiple trace mappings, and the multiple trace mappings are determined based on a check polynomial, and the check polynomial is used to generate a dual codeword.

[0105] Based on the data blocks and data bits with transmission errors, the trace mapping used for data recovery of the data blocks and data bits can be queried from the data recovery table, and the standard data blocks to be used can be determined based on the trace mapping.

[0106] According to an embodiment of the present invention, based on the standard data block, the erroneous data block is restored to obtain expected data in the erroneous data block, including: when the number of erroneous data blocks is 1, based on the standard data block, based on multiple standard data blocks and trace mappings corresponding to the multiple standard data blocks, the erroneous data block is restored to obtain the expected data.

[0107] When the number of the error data block is 1, the expected data in the error data block can be obtained by substituting the data bits in the standard data block involved in the trace mapping into the trace mapping.

[0108] According to the embodiments of the present invention, the trace mapping is quickly determined by table lookup, and the erroneous data block is recovered according to the trace mapping and data without transmission errors, so that the data processing efficiency can be improved while ensuring the accuracy of data processing.

[0109] According to an embodiment of the present invention, the data processing method also includes: when the number of erroneous data blocks is greater than 1, for multiple erroneous data blocks, the erroneous data blocks are restored based on multiple standard data blocks and trace mappings corresponding to each of the multiple standard data blocks to obtain expected data with interference; data is exchanged between the multiple erroneous data blocks, and multiple expected data are determined based on the expected data with interference.

[0110] When the number of erroneous data blocks is greater than 1, since multiple erroneous data blocks may interfere with each other, for each erroneous data block, the data bits in the standard data block involved in the trace mapping are substituted into the trace mapping, and the obtained data is the expected data with interference.

[0111] According to an embodiment of the present invention, data is exchanged between multiple error data blocks, and multiple expected data are determined based on expected data with interference, including: repeating the following operations until the data in the multiple error data blocks are restored to expected data: determining a target error data block from the multiple error data blocks; transferring the expected data with interference of each of the other error data blocks except the target error data block among the multiple error data blocks to the target error data block; in the case of a restored error data block, transferring the expected data of the restored error data block to the target error data block; and based on the expected data of the restored error data block and the expected data with interference of each of the other error data blocks, restoring the expected data with interference of the target error data block to obtain the expected data, and determining the target error data block as the restored error data block.

[0112] Figure 4 A schematic diagram showing data exchange and interference elimination of multiple erroneous data blocks according to an embodiment of the present invention is shown.

[0113] like Figure 4 As shown, in the case where there are three error data blocks, a target error data block can be determined from multiple error data blocks, for example, the first error data block is determined as the target error data block. Since there is no restored error data block at this time, only the expected data with interference in each of the other error target data blocks except the target error data block is transferred to the first error data block, and the expected data with interference in the first error data block is restored based on the expected data with interference in the second error data block and the third error data block, so that the influence of the second error data block and the third error data block on the first error data block can be eliminated, and the expected data in the first error data block can be obtained, and the first error data block is determined as the restored error data block.

[0114] After completing the data recovery of the first error data block, a target error data block is determined from the second error data block and the third error data block, for example, the second error data block is determined as the target error data block. Since the first error data block is a restored error data block at this time, the expected data in the first error data block and the expected data with interference in the third error data block can be transferred to the second error data block, and the expected data with interference in the second error data block can be restored to obtain the expected data in the second error data block, and the second error data block can be determined as a restored error data block.

[0115] After completing the data recovery of the second error data block, since there is only the third error data block at this time, the third error data block is the target error data block, and there are no other error data blocks at this time, and there are only two restored error data blocks. Therefore, the expected data of the first error data block and the second error data block are transferred to the third error data block respectively, and the expected data with interference in the third error data block can be restored to obtain the expected data in the third error data block.

[0116] In another example, the restoration of multiple erroneous data bits is similar to the restoration process of multiple erroneous data blocks described above. Commonly used Cooperative repair of RS code, assuming that data bit 1, data bit 2 and data bit 3 are lost. ,Right now . Let the set of assignment points be ,Right now .

[0117] Using the domain promotion method, , where θ is The root, then constitute exist The basis on which yes exist There are roots on and .

[0118] Assumptions ,in yes Polynomials on For and ,make , and make the definition as shown in formula (5):

[0119] (5)

[0120] The following results can be obtained: for A group of base, for of base, for A group of base, for of base, for A group of base, for of base.

[0121] Therefore, cooperative repair can be achieved according to the domain promotion framework. The other 11 bits in the information data except the above three data bits are determined as the help bits for data recovery. Data bit 1, data bit 2 and data bit 3 take 4 bits from the help bits respectively, eliminate their respective partial interference items, and obtain their respective expected data with interference. Data bit 2 and data bit 3 take 4 bits from data bit 1 respectively, and then exchange 8 bits with each other, repair data bit 2 and data bit 3, and obtain the expected data of data bit 2 and data bit 3 respectively. Data bit 1 takes 4 bits from data bit 2 and data bit 3 respectively, repairs data bit 1, and obtains the expected data of data bit 1. Data bit 1, data bit 2 and data bit 3 can be repaired.

[0122] In the above process of repairing data bit 1, data bit 2 and data bit 3, the total repair bandwidth is 16 bits, and the average repair bandwidth of each data bit is 16 / 3 bits, which saves a lot of repair bandwidth compared with traditional repair methods.

[0123] In particular, if , then data bit 1, data bit 2 and data bit 3 can directly exchange 4 bits with each other during the data exchange phase to restore their respective expected data. At this time, the total repair bandwidth is 156 bits, and the average repair bandwidth per data bit is 52 bits, which saves 35% of the repair bandwidth compared with the ordinary repair method.

[0124] According to an embodiment of the present invention, when there are multiple erroneous data blocks or multiple erroneous data bits in an erroneous data block, multiple rounds of data exchange can be used to gradually eliminate the mutual influence between the multiple erroneous data blocks or multiple erroneous data bits to ensure the accuracy of data processing.

[0125] Since the data to be transmitted includes verification data for correctness verification and data recovery, this part of data does not belong to the data to be transmitted itself. The data sender can add this part of data to the data to be transmitted and encapsulate it before sending the data to obtain the data to be processed.

[0126] According to an embodiment of the present invention, the data to be processed is generated by a data sending end through the following operations: adding a first check code to the data to be transmitted to obtain intermediate data, wherein the intermediate data corresponds to multiple information data blocks; and based on the intermediate data and the second check code, obtaining the data to be processed, wherein the second check code corresponds to multiple check data blocks.

[0127] The first check code may be a check code for performing a cyclic redundancy check, and the data to be transmitted may be all the data that the data transmitter needs to send. Based on the data to be transmitted and the cyclic redundancy check principle, the first check code corresponding to the data to be transmitted is determined, and the first check code is added to the data to be transmitted, so that the intermediate data can be obtained.

[0128] According to an embodiment of the present invention, after the data receiving end determines the recovery result of the data group to be processed according to the data processing method, the intermediate data can be determined based on the reply result, and the correctness of the data to be transmitted can be further checked based on the first check code in the intermediate data.

[0129] The second check code may be a check code used for forward error correction. After the second check code is added to the intermediate data, the data to be processed may be obtained.

[0130] According to an embodiment of the present invention, by adding a first check code to the data to be transmitted, after the data receiving end receives the data to be processed and completes processing thereof, the accuracy of the data to be transmitted can be further detected according to the first check code, thereby further improving the accuracy and reliability of data processing.

[0131] According to an embodiment of the present invention, a first check code is added to the data to be transmitted to obtain intermediate data, including: constructing a primitive polynomial based on a finite field to which the data to be transmitted belongs, wherein the degree of the primitive polynomial is the same as the order of the finite field; constructing a first generating polynomial based on the roots of the primitive polynomial; determining a first check code based on the first generating polynomial; and adding the first check code to the data to be transmitted to obtain intermediate data.

[0132] Based on the order of the finite field to which the data to be transmitted belongs, the degree of the primitive polynomial is determined, and the roots of the primitive polynomial are determined. Since the roots of the primitive polynomial are the generators of all non-zero elements in the finite field, the roots of multiple generator polynomials can be determined based on the roots of the primitive polynomial and the degree of the primitive polynomial.

[0133] Based on the roots of the generating polynomial, a first generating polynomial can be constructed, and a check matrix corresponding to the first generating polynomial can be determined. Based on the check matrix, a first check code can be determined. Taking the cyclic redundancy check as an example, the first check code obtained is the cyclic redundancy check code, which can be added to the data to be transmitted to obtain the intermediate data.

[0134] Take a finite field For example, since the order of the finite field is 8, we can construct an 8th-order primitive polynomial. The root of the primitive polynomial is α, and we can determine And α is the generator of all non-zero elements in the finite field, that is They are different from each other, and the first check matrix can be determined as shown in formula (6):

[0135] (6)

[0136] According to an embodiment of the present invention, obtaining the data to be processed based on the intermediate data and the second check code includes: determining a second generating polynomial, wherein the degree of the second generating polynomial is determined based on the length of the second check code; determining an information polynomial, wherein the degree of the information polynomial is determined based on the length of the intermediate data; and determining the data to be processed based on the second generating polynomial and the second check code.

[0137] The data to be transmitted with a length of l is encoded by RS code to obtain the second generator matrix as shown in formula (7):

[0138] (7)

[0139] After obtaining the second generator matrix, the information vector can be determined according to the properties of the RS code. , and calculate C·G to get the RS code with a length of l, that is, the data to be processed.

[0140] In another example, a reversible linear transformation may be performed on the matrix G so that the first k1 columns of the matrix G become unit matrices, thereby obtaining a standard generator matrix.

[0141] According to an embodiment of the present invention, a first check code and a second check code corresponding to the data to be transmitted are generated based on the properties of the cyclic redundancy check and the forward data recovery, and the first check code and the second check code are added to the data to be transmitted to obtain the data to be processed. After receiving the data to be processed, the data receiving end can process the data to be processed according to the check data block composed of the second check code to recover the data block with transmission errors. The data to be transmitted is further checked for correctness according to the first check code to improve the transmission accuracy of the data to be transmitted.

[0142] Based on the above data processing method, the present invention also provides a data processing device. Figure 5 The device is described in detail.

[0143] Figure 5 A structural block diagram of a data processing device according to an embodiment of the present invention is shown.

[0144] like Figure 5 As shown, the data processing device 500 of this embodiment includes a data verification module 510 , a data determination module 520 , a data restoration module 530 and a result determination module 540 .

[0145] The data verification module 510 is used to perform correctness verification on the data group to be processed in response to receiving the data group to be processed sent by the data sending end, and obtain a verification result, wherein the data group to be processed includes multiple data to be processed with the same length, each data to be processed includes multiple sub-data, and multiple data with the same relative position in the multiple data to be processed constitute a data block, and the multiple data blocks include information data blocks and verification data blocks. In one embodiment, the data verification module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0146] The data determination module 520 is used to determine a standard data block for data recovery of the erroneous data block based on the dependency relationship between multiple transmission nodes used to send the multiple data blocks when the verification result indicates that there is an erroneous data block with a transmission error in the multiple information data blocks, and the erroneous data block includes the information data block. In one embodiment, the data determination module 520 can be used to perform the operation S210 described above, which will not be repeated here.

[0147] The data restoration module 530 is used to restore the error data block based on the standard data block to obtain the expected data in the error data block. In one embodiment, the data restoration module 530 can be used to perform the operation S210 described above, which will not be described in detail here.

[0148] The result determination module 540 is used to determine the recovery result of the to-be-processed data group based on the data blocks without transmission errors and the expected data in the error data blocks. In one embodiment, the result determination module 540 can be used to perform the operation S210 described above, which will not be described in detail here.

[0149] According to an embodiment of the present invention, the data determination module 520 includes a first data determination submodule. The first data determination submodule is used to determine k1-1 information data blocks and a first check data block other than the error data block as standard data blocks.

[0150] According to an embodiment of the present invention, the data restoration module 530 includes a second data determination submodule, a target data determination submodule, a first data restoration submodule, a third data determination submodule, a second data restoration submodule and a third data restoration submodule. The second data determination submodule is used to determine the first error data that is not carried to the second verification data block and the second error data that is carried to the second verification data block in the error data block. The target data determination submodule is used to determine the target data to be processed where the first error data is located. The first data restoration submodule is used to restore the first error data based on multiple standard data blocks to obtain the first expected data. The third data determination submodule is used to determine the target verification data block for carrying the second error data from the target data to be processed. The second data restoration submodule is used to determine the expected data based on the first expected data and the second expected data. The third data restoration submodule is used to determine the expected data based on the first expected data and the second expected data.

[0151] According to an embodiment of the present invention, the data processing device 500 further includes a data partitioning module and a data loading module. The data partitioning module is used to randomly partition the l original information data in the k1 information data blocks into data to be loaded and data not to be loaded, so that there is and only one data not to be loaded in the k1 information data blocks, and there are l-1 data not to be loaded in each of the multiple data to be processed. The data loading module is used to load the data to be loaded onto the original verification data of the data to be processed that is different from the data to be loaded.

[0152] According to an embodiment of the present invention, the data determination module 520 includes a fourth data determination submodule. The fourth data determination submodule is used to determine a standard data block for data recovery of an erroneous data block from a pre-stored data recovery table, wherein the data recovery table includes a plurality of trace mappings, the plurality of trace mappings are determined based on a check polynomial, and the check polynomial is used to generate a dual codeword.

[0153] According to an embodiment of the present invention, the data restoration module 530 includes a fourth data restoration submodule. The fourth data restoration submodule is used to restore the erroneous data block based on the standard data block, based on multiple standard data blocks and trace mappings respectively corresponding to the multiple standard data blocks to obtain expected data when the number of erroneous data blocks is 1.

[0154] According to an embodiment of the present invention, the data processing device 500 further includes a data pre-restoration module and a data exchange module. The data pre-restoration module is used to restore the error data blocks based on multiple standard data blocks and trace mappings corresponding to the multiple standard data blocks respectively, to obtain the expected data with interference, when the number of error data blocks is greater than 1. The data exchange module is used to exchange data between the multiple error data blocks, and determine the multiple expected data based on the expected data with interference.

[0155] According to an embodiment of the present invention, the data exchange module includes a first error data determination submodule, a second error data determination submodule, a first data transmission submodule, and a second data transmission submodule. The first error data determination submodule is used to determine a target error data block from a plurality of error data blocks. The second error data determination submodule is used to transmit the expected data with interference of each of the other error data blocks except the target error data block among the plurality of error data blocks to the target error data block. The first data transmission submodule is used to transmit the expected data of the restored error data block to the target error data block in the case where there is a restored error data block. The second data transmission submodule is used to restore the expected data with interference of the target error data block based on the expected data of the restored error data block and the expected data with interference of each of the other error data blocks, obtain the expected data, and determine the target error data block as the restored error data block.

[0156] According to an embodiment of the present invention, the data processing device 500 further includes a set determination module, a polynomial determination module, a mapping determination module and a mapping storage module. The set determination module is used to determine the assignment point set based on the number of check data blocks, wherein the assignment point set includes multiple assignment points, and multiple check data blocks correspond to one assignment point respectively. The polynomial determination module is used to determine multiple check polynomials for each assignment point, wherein the order of the check polynomial is determined based on the number of check data blocks and the number of information data blocks. The mapping determination module is used to determine, based on multiple check polynomials, respectively, trace mappings for restoring erroneous data blocks that have transmission errors in the case of transmission errors in multiple information data blocks. The mapping storage module is used to store the trace mappings in a data recovery table.

[0157] According to an embodiment of the present invention, the data processing device 500 further includes a first adding module and a second adding module. The first adding module is used to add a first check code to the data to be transmitted to obtain intermediate data, wherein the intermediate data corresponds to a plurality of information data blocks. The second adding module is used to obtain the data to be processed based on the intermediate data and the second check code, wherein the second check code corresponds to a plurality of check data blocks.

[0158] According to an embodiment of the present invention, the first adding module includes a first polynomial construction submodule, a second generating polynomial construction submodule, a check code determination submodule and a check code adding submodule. The first polynomial construction submodule is used to construct a primitive polynomial based on the finite field to which the data to be transmitted belongs, wherein the degree of the primitive polynomial is the same as the order of the finite field. The second generating polynomial construction submodule is used to construct a first generating polynomial based on the roots of the primitive polynomial. The check code determination submodule is used to determine the first check code according to the first generating polynomial. The check code adding submodule is used to add the first check code to the data to be transmitted to obtain intermediate data.

[0159] According to an embodiment of the present invention, the second adding module includes a third polynomial construction submodule, a fourth polynomial construction submodule and a second adding submodule. The third polynomial construction submodule is used to determine a second generating polynomial, wherein the degree of the second generating polynomial is determined based on the length of the second check code. The fourth polynomial construction submodule is used to determine an information polynomial, wherein the degree of the information polynomial is determined based on the length of the intermediate data. The second adding submodule is used to determine the data to be processed based on the second generating polynomial and the second check code.

[0160] Figure 6 A block diagram of an electronic device suitable for implementing a data processing method according to an embodiment of the present invention is shown.

[0161] like Figure 6 As shown, the electronic device 600 according to an embodiment of the present invention includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (R01) 602 or a program loaded from a storage part 608 to a random access memory (R01) 603. The processor 601 may include, for example, a general-purpose microprocessor (such as a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (for example, an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include an onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0162] In RAM 603, various programs and data required for the operation of electronic device 600 are stored. Processor 601, ROM 602 and RAM 603 are connected to each other via bus 604. Processor 601 performs various operations of the method flow according to the embodiment of the present invention by executing the program in ROM 602 and / or RAM 603. It should be noted that the program can also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 can also perform various operations of the method flow according to the embodiment of the present invention by executing the program stored in the one or more memories.

[0163] According to an embodiment of the present invention, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to the bus 604. The electronic device 600 may further include one or more of the following components connected to the input / output (I / O) interface 605: an input portion 606 including a keyboard, a mouse, etc.; an output portion 607 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage portion 608 including a hard disk, etc.; and a communication portion 609 including a network interface card such as a LAN card, a modem, etc. The communication portion 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 610 as needed, so that a computer program read therefrom is installed into the storage portion 608 as needed.

[0164] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiment; or may exist independently without being assembled into the device / apparatus / system. The above computer-readable storage medium carries one or more programs, and when the above one or more programs are executed, the method according to the embodiment of the present invention is implemented.

[0165] According to an embodiment of the present invention, the computer-readable storage medium may be a non-volatile computer-readable storage medium, for example, it may include but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present invention, the computer-readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, an apparatus or a device. For example, according to an embodiment of the present invention, the computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0166] The embodiment of the present invention also includes a computer program product, which includes a computer program, and the computer program contains program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data processing method provided by the embodiment of the present invention.

[0167] The computer program executes the above functions defined in the system / device of the embodiment of the present invention when it is executed by the processor 601. According to the embodiment of the present invention, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0168] In one embodiment, the computer program may rely on tangible storage media such as optical storage devices, magnetic storage devices, etc. In another embodiment, the computer program may also be transmitted and distributed in the form of signals on a network medium, and downloaded and installed through the communication part 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0169] In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, the above functions defined in the system of the embodiment of the present invention are performed. According to the embodiment of the present invention, the system, device, means, module, unit, etc. described above can be implemented by a computer program module.

[0170] According to an embodiment of the present invention, the program code for executing the computer program provided by the embodiment of the present invention can be written in any combination of one or more programming languages. Specifically, these computing programs can be implemented using high-level process and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, Java, C++, python, "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on the remote computing device, or entirely on the remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (e.g., using an Internet service provider to connect through the Internet).

[0171] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0172] It will be appreciated by those skilled in the art that the features described in the various embodiments of the present invention may be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, without departing from the spirit and teachings of the present invention, the features described in the various embodiments of the present invention may be combined and / or combined in various ways. All of these combinations and / or combinations fall within the scope of the present invention.

[0173] The embodiments of the present invention are described above. However, these embodiments are only for the purpose of illustration, and are not intended to limit the scope of the present invention. Although each embodiment is described above, it does not mean that the measures in each embodiment cannot be used in combination advantageously. Without departing from the scope of the present invention, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present invention.

Claims

1. A data processing method performed by a data receiving end, characterized in that: The method comprises: In response to receiving a data group to be processed sent by a data sending end, performing a correctness check on the data group to be processed to obtain a check result, wherein the data group to be processed includes a plurality of data to be processed with the same length, each of the data to be processed includes a plurality of sub-data, a plurality of sub-data with the same relative position in the plurality of data to be processed constitutes a data block, and the plurality of data blocks include an information data block and a check data block; In a case where the verification result indicates that there is an error data block with a transmission error among the multiple information data blocks, based on a dependency relationship between multiple transmission nodes used to send the multiple data blocks, determining a standard data block for performing data recovery on the error data block, wherein the standard data block includes other information data blocks except the error data block; Based on the standard data block, the error data block is restored to obtain expected data in the error data block; and A recovery result of the to-be-processed data group is determined based on the data blocks without transmission errors and the expected data in the erroneous data blocks.

2. The method according to claim 1, characterized in that: The dependency relationship includes a mounting relationship between the plurality of transmission nodes and / or a cooperation relationship between the plurality of transmission nodes. The said piggyback relationship indicates that at least one information data of the data to be processed is piggybacked onto the verification data of other data to be processed. The cooperative relationship means that when there are multiple erroneous data blocks, the data in the multiple erroneous data blocks are initially restored using other information data blocks, and the initially restored data in the multiple erroneous data blocks are exchanged to complete data recovery.

3. The method according to claim 1, characterized in that The multiple data blocks include information data blocks and check data blocks, wherein the number of the multiple data blocks is n, the multiple data blocks include k1 information data blocks and p check data blocks, k1+p=n, and k1, p and n are all integers greater than 1; The p check data blocks include 1 first check data block not carrying the information in the information data block and p-1 second check data blocks carrying the information in the information data block; The determining, based on the dependency relationship between the plurality of transmission nodes used to send the plurality of data blocks, a standard data block used to recover data from the erroneous data block comprises: The k1-1 information data blocks and the first check data block excluding the error data block are determined as the standard data blocks.

4. The method according to claim 3, characterized in that The plurality of information data blocks respectively include original information data not loaded into the second verification data block and original information data loaded into the second verification data block; The restoring the error data block based on the standard data block to obtain expected data in the error data block includes: Determine first error data in the error data block that is not carried to the second verification data block and second error data that is carried to the second verification data block; Determine the target data to be processed where the first erroneous data is located; Based on the plurality of standard data blocks, the first erroneous data is restored to obtain first expected data; Determine a target check data block for carrying the second error data from the target data to be processed; Based on the target verification data block and other original information data carried in the target verification data block, the second erroneous data is restored to obtain second expected data; and The expected data is determined based on the first expected data and the second expected data.

5. The method according to claim 4, characterized in that The number of the plurality of data to be processed is l, the k1 information data blocks each include l original information data, and the p verification data blocks each include l original verification data; The method further comprises: For l original information data in the k1 information data blocks, randomly divide the l original information data into data to be carried and data not to be carried, so that there is and only one data not to be carried in each of the k1 information data blocks, and there are l-1 data not to be carried in each of the multiple data to be processed; and For the plurality of data to be carried, the data to be carried is carried onto original verification data of data to be processed that is different from the data to be carried.

6. The method according to claim 1, characterized in that The determining, based on the dependency relationship between the plurality of transmission nodes used to send the plurality of data blocks, a standard data block used to recover data from the erroneous data block comprises: A standard data block for data recovery of the erroneous data block is determined from a pre-stored data recovery table, wherein the data recovery table includes a plurality of trace mappings, the plurality of trace mappings are determined based on a check polynomial, and the check polynomial is used to generate a dual codeword.

7. The method according to claim 6, characterized in that The restoring the error data block based on the standard data block to obtain expected data in the error data block includes: When the number of the error data blocks is 1, based on the plurality of standard data blocks and trace mappings respectively corresponding to the plurality of standard data blocks, the error data block is restored to obtain the expected data; When the number of the error data blocks is greater than 1, for the plurality of error data blocks, based on the plurality of standard data blocks and the trace mappings respectively corresponding to the plurality of standard data blocks, the error data blocks are restored to obtain the expected data with interference; Data is exchanged between the plurality of erroneous data blocks, and a plurality of expected data is determined based on the expected data with interference.

8. The method according to claim 7, characterized in that The exchanging data between the plurality of erroneous data blocks and determining the plurality of expected data based on the expected data with interference includes: Repeat the following steps until the data in the plurality of erroneous data blocks are restored to the expected data: determining a target erroneous data block from the plurality of erroneous data blocks; Transmitting the interfered expected data of each of the other error data blocks except the target error data block among the plurality of error data blocks to the target error data block; In case there is a restored error data block, transferring the expected data of the restored error data block to the target error data block; and Based on the expected data of the restored erroneous data block and the expected data with interference of each of the other erroneous data blocks, the expected data with interference of the target erroneous data block is restored to obtain the expected data, and the target erroneous data block is determined as the restored erroneous data block.

9. The method according to claim 6, characterized in that The method further comprises: Determine a value assignment point set based on the number of the verification data blocks, wherein the value assignment point set includes a plurality of value assignment points, and a plurality of the verification data blocks correspond to one of the value assignment points respectively; For each of the evaluation points, determining a plurality of check polynomials, wherein the degree of the check polynomials is determined based on the number of the check data blocks and the number of the information data blocks; Based on the multiple check polynomials, respectively determining a trace mapping for restoring an erroneous data block having a transmission error when a transmission error occurs in the multiple information data blocks; and The trace mapping is stored in the data recovery table.

10. The method according to any one of claims 1 to 9, characterized in that The data to be processed is generated by the data sending end through the following operations: Adding a first check code to the data to be transmitted to obtain intermediate data, wherein the intermediate data corresponds to the plurality of information data blocks; and The data to be processed is obtained based on the intermediate data and a second check code, wherein the second check code corresponds to a plurality of check data blocks.

11. The method according to claim 10, characterized in that The step of adding a first check code to the data to be transmitted to obtain intermediate data comprises: Based on the finite field to which the data to be transmitted belongs, construct a primitive polynomial, wherein the degree of the primitive polynomial is the same as the order of the finite field; Based on the roots of the primitive polynomial, construct a first generating polynomial; Determining the first check code according to the first generator polynomial; and The first check code is added to the data to be transmitted to obtain the intermediate data.

12. The method according to claim 11, characterized in that The obtaining the data to be processed based on the intermediate data and the second check code comprises: Determine a second generating polynomial, wherein the degree of the second generating polynomial is determined based on the length of the second check code; determining an information polynomial, wherein the degree of the information polynomial is determined based on the length of the intermediate data; and The data to be processed is determined based on the second generator polynomial and the second check code.

13. An electronic device, comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 12.

14. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images