Method and device for enhancing data reliability and storage medium

Abstract

The invention relates to a method and device for enhancing data reliability and a storage medium. The method comprises the following steps: grouping user data according to a set data length N to obtain a data segment D; in each data segment D, one piece of known verification data Pi is inserted every set g bits; storing the data segment D in combination with the verification data Pi; receiving read user data, acquiring read check data # imgabs0 # from the read user data according to the position of the check data, and calculating correction parameters of all check data positions of the user data according to the known check data Pi and the read check data # imgabs1 #; verifying whether the read user data is abnormal or not based on the correction parameters on all the verification data positions, and when the read user data is abnormal, estimating correction parameters of data bits of each data segment of the abnormally read user data according to the calculated correction parameters on the verification data positions; and correcting the received read user data according to the correction parameter of the data bit of the data segment.

Description

Technical Field

[0001] The present invention relates to the technical field of data reliability, and in particular to a method, device and storage medium for enhancing data reliability. Background Art

[0002] Insurance companies involve user data storage and usage scenarios during their operations. For example, insurance companies generate a large amount of important data during the business processing stage, including various policy information and customer information, such as name, identity information and policy amount. Front-end sales staff use the order recording APP to enter order information, including customer information, insurance type, premium, insured amount, etc., and the data is transmitted to the back-end server through the network. When the front-end needs to call customer data, it sends instructions to the back-end server, and the back-end server extracts the corresponding data from the device. Data during data storage may cause reliability defects due to various factors. The following are some common factors: Mechanical hard disks or solid-state drives may cause data loss due to wear, failure or accidental damage. Memory stick failure may cause data errors during the writing process. Unstable or interrupted power supply may cause data corruption. The file system may be damaged due to software errors, virus attacks or improper operations, resulting in data anomalies. Database software may cause data structure damage due to errors or defects. Vulnerabilities in the operating system may cause data storage problems. Network problems or hardware failures may cause data corruption during transmission. Software defects or hardware problems may cause data writing errors. When multiple users access and modify data at the same time, improper concurrency control may cause data inconsistency. Summary of the invention

[0003] In order to solve the above technical problem or at least partially solve the above technical problem, the present invention provides a method, device and storage medium for enhancing data reliability.

[0004] In a first aspect, the present invention provides a method for enhancing data reliability, comprising:

[0005] Before storing the user data in the device, the user data is grouped according to the set data length N to obtain data segments D; in each data segment D, a known check data P is inserted every set g bits. i , where g is an integer greater than or equal to 1 and less than or equal to N; combine the data segment D with the check data P i To store;

[0006] Receive read user data and obtain read verification data from the read user data according to the position of the verification data According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all check data positions of the user data; verify whether the read user data is abnormal based on the correction parameters at all check data positions. If abnormal, estimate the correction parameters of each data bit of each data segment of the abnormal read user data based on the calculated correction parameters at the check data positions; and correct the received read user data based on the correction parameters of the data bits of the data segment.

[0007] Furthermore, in each data segment D, a known check data P is inserted every set g bits. i When d1, d2, ...d g , d g+1 , d g+2 ,...d 2g , d 2g+1 , d 2g+2 ,...d 3g , ..., the data segment D, the check data P1 is set at the beginning of the data segment D, d g , d g+1 Set the verification data P2 between 2g , d 2g+1 Set the verification data P3...d kg , d kg+1 Set the verification data P3 between k+1 .

[0008] Furthermore, the known verification data P mentioned above i Verify data with readout Calculating the correction parameters at all check data positions of the user data includes: traversing each check data P i And the corresponding readout verification data For each pair of traversed check data P i And the corresponding readout verification data The correction parameters of the calibration data are calculated as:

[0009]

[0010] Furthermore, for the following: Read out user data and obtain the segment header value of the read out user data as the first read out verification data Collect subsequent readout verification data every set g bits

[0011] Furthermore, the read user data having a correction parameter at the verification data position not equal to 1 is regarded as abnormal read user data.

[0012] Furthermore, the correction parameter of each data bit of each data segment of the abnormally read user data is estimated based on the calculated correction parameter at the check data position, and the calculation formula is as follows:

[0013] in, D f d is the correction parameter for data bit d of data segment D, P f i The correction parameter for the low-order check data of data segment D, P f i+1 It is the correction parameter of the high-order check data of data segment D.

[0014] Furthermore, the correcting the received read user data according to the correction parameter of the data bit of the data segment includes:

[0015] Traverse all data segments of abnormally read user data; for each traversed data segment, calculate the data after each data bit is corrected according to the following formula:

[0016] D f d Correction parameters for abnormally reading data bit d of data segment D in user data, Read the data segment in the user data for abnormality The data bit d before correction, Read the data segment in the user data for abnormality The data bit d is the data after correction.

[0017] In a second aspect, the present invention provides a device for enhancing data reliability, comprising: at least one processing unit, the processing unit and a storage unit are interconnected via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, the method for enhancing data reliability is implemented.

[0018] In a third aspect, the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for enhancing data reliability is implemented.

[0019] The above technical solution provided by the embodiment of the present invention has the following advantages compared with the prior art:

[0020] Before storing the user data in the device, the application groups the user data according to the set data length N to obtain data segments D; in each data segment D, a known check data P is inserted every set g bits. i, where g is an integer greater than or equal to 1 and less than or equal to N; combine the data segment D with the check data P i Store; receive read user data, and obtain read verification data from the read user data according to the position of the verification data According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all check data positions of the user data; verify whether the read user data is abnormal based on the correction parameters at all check data positions. If abnormal, estimate the correction parameters of each data bit of each data segment of the abnormal read user data according to the correction parameters at the calculated check data positions; modify the received read user data according to the correction parameters of the data bits of the data segment. The present invention regularly adds check data bits before storing the original data. When reading data at the front end, the check data bits can be used to perform data verification on the read data. If each check data bit does not change, it can be judged that the data is not damaged and the original data is used; if the check data changes, it means that the data is damaged. The linear calculation method in the present invention is used to realize the recovery operation of the erroneous data, which enhances the reliability of the data to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0022] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0023] Figure 1 A flowchart of a method for enhancing data reliability provided by an embodiment of the present invention;

[0024] Figure 2 A schematic diagram of a data segment provided by an embodiment of the present invention;

[0025] Figure 3 A schematic diagram of a data segment for reading user data provided by an embodiment of the present invention;

[0026] Figure 4 A schematic diagram of a device for enhancing data reliability provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0028] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or device including the element.

[0029] Example 1

[0030] Based on the linear calculation method, a method for recovering data that has errors in storage is provided to make up for the shortcomings of the existing technology. Figure 1 As shown, the technology of the present invention realizes a method for enhancing data reliability. To achieve the above purpose, the specific technical solution adopted by the present invention is:

[0031] Before storing the user data in the device, the user data is grouped according to the set data length N to obtain data segments D, and the length of each obtained data segment is N.

[0032] In each data segment D, a known check data P is inserted every g bits. i , where g is an integer greater than or equal to 1 and less than or equal to N; that is, there is a check data for every g bits of data, which is used to calculate the correction parameter of the data. Combine the data segment D with the check data P i , where G is an integer greater than or equal to 1 and less than or equal to N. Figure 2 As shown, the data segment D of the example includes the following data: d1, d2, ...d g , d g+1 , d g+2 ,...d 2g , d 2g+1 , d 2g+2 ,...d 3g , ...set the check data P1 at the beginning of the data segment D, d g , d g+1 Set the verification data P2 between 2g , d2g+1 Set the verification data P3...d kg , d kg+1 Set the verification data P3 between k+1 .

[0033] If user data is damaged during transmission and storage, the check data in each group of data segments can be used to estimate the damage extent of the user data and then perform linear recovery.

[0034] In a specific implementation process, the read user data is received, and the read verification data is obtained from the read user data according to the position of the verification data. Figure 3 As shown, the user data is read as follows: Get the segment header value of the read user data as the first read verification data Collect subsequent readout verification data every set g bits

[0035] According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all the check data positions of the user data. Verify whether the read user data is abnormal based on the correction parameters at all the check data positions.

[0036] In case of an abnormality, the correction parameters of the data bits of each data segment are estimated according to the calculated correction parameters at the check data position; and the received read user data is corrected according to the correction parameters of the data bits of each data segment.

[0037] The calculation process of the correction parameters is as follows:

[0038] Traverse each verification data P i And the corresponding readout verification data For each pair of traversed check data P i And the corresponding readout verification data The correction parameters of the calibration data are calculated as:

[0039]

[0040] For the following: Read out user data and obtain the segment header value of the read out user data as the first read out verification data Collect subsequent readout verification data every set g bits

[0041] The read user data whose correction parameter at the check data position is not equal to 1 is regarded as abnormal read user data. In the case of abnormality, the correction parameter of the data bit of each data segment is estimated based on the calculated correction parameter at the check data position, and the calculation formula is as follows:

[0042] in, D f d is the correction parameter for data bit d of data segment D, P f i The correction parameter for the low-order check data of data segment D, P f i+1 is the correction parameter of the high-order check data of the data segment D. In the present invention, a linear estimation algorithm is used to calculate the correction parameter on each data bit of each data segment.

[0043] The step of correcting the received read user data according to the correction parameter of the data bit of the data segment comprises:

[0044] Traverse all data segments of abnormally read user data; for each traversed data segment, calculate the data after each data bit is corrected according to the following formula:

[0045] D f d Correction parameters for abnormally reading data bit d of data segment D in user data, Read the data segment in the user data for abnormality The data bit d before correction, Read the data segment in the user data for abnormality The data bit d is the data after correction.

[0046] The invention regularly adds check data bits before the original data is stored. When the front end reads the data, the check data bits can be used to check the read data. If each check data bit does not change, it can be determined that the data is not damaged and the original data is used; if the check data changes, it means that the data is damaged. The linear calculation method in the present invention is used to realize the recovery operation of the erroneous data, thereby enhancing the reliability of the data to a certain extent.

[0047] Example 2

[0048] See also Figure 4As shown, an embodiment of the present invention provides a device for enhancing data reliability, including: the processing unit and the storage unit are interconnected through a bus unit, and the storage unit can be used to store software programs, computer executable programs and modules, such as the software programs, computer executable programs and modules corresponding to a method for enhancing data reliability in an embodiment of the present invention. The processing unit implements the above-mentioned method for enhancing data reliability by running the software programs, computer executable programs and modules stored in the storage unit, including:

[0049] Before storing the user data in the device, the user data is grouped according to the set data length N to obtain data segments D; in each data segment D, a known check data P is inserted every set g bits. i , where g is an integer greater than or equal to 1 and less than or equal to N; combine the data segment D with the check data P i Store; receive read user data, and obtain read verification data from the read user data according to the position of the verification data According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all check data positions of the user data; verify whether the read user data is abnormal based on the correction parameters at all check data positions. If abnormal, estimate the correction parameters of each data bit of each data segment of the abnormal read user data based on the calculated correction parameters at the check data positions; and correct the received read user data based on the correction parameters of the data bits of the data segment.

[0050] Of course, the computer program stored in the storage unit of the device for enhancing data reliability provided by an embodiment of the present invention is not limited to the method operations described above, and can also execute related operations in the method for enhancing data reliability provided by any embodiment of the present invention.

[0051] Example 3

[0052] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed, the method for enhancing data reliability is implemented, including:

[0053] Before storing the user data in the device, the user data is grouped according to the set data length N to obtain data segments D; in each data segment D, a known check data P is inserted every set g bits. i , where g is an integer greater than or equal to 1 and less than or equal to N; combine the data segment D with the check data P iStore; receive read user data, and obtain read verification data from the read user data according to the position of the verification data According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all check data positions of the user data; verify whether the read user data is abnormal based on the correction parameters at all check data positions. If abnormal, estimate the correction parameters of each data bit of each data segment of the abnormal read user data based on the calculated correction parameters at the check data positions; and correct the received read user data based on the correction parameters of the data bits of the data segment.

[0054] A computer-readable storage medium provided by an embodiment of the present invention stores a computer program which is not limited to the method operations described above, but can also execute related operations in a method for enhancing data reliability provided by any embodiment of the present invention.

[0055] In the embodiments provided by the present invention, it should be understood that the disclosed structures and methods can be implemented in other ways. For example, the structural embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, structures or units, which can be electrical, mechanical or other forms.

[0056] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0057] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0058] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for enhancing data reliability, characterized in that: include: Before storing the user data in the device, the user data is grouped according to the set data length N to obtain data segments D; In each data segment D, a known check data P is inserted every g bits. i , where g is an integer greater than or equal to 1 and less than or equal to N; combine the data segment D with the check data P i To store; Receive read user data and obtain read verification data from the read user data according to the position of the verification data According to the known verification data P mentioned above i Verify data with readout Calculate the correction parameters at all check data positions of the user data; verify whether the read user data is abnormal based on the correction parameters at all check data positions. If abnormal, estimate the correction parameters of each data bit of each data segment of the abnormal read user data based on the calculated correction parameters at the check data positions; and correct the received read user data based on the correction parameters of the data bits of the data segment.

2. The method for enhancing data reliability according to claim 1, characterized in that: In each data segment D, a known check data P is inserted every g bits. i When d1, d2, ...d g , d g+1 , d g+2 ,...d 2g , d 2g+1 , d 2g+2 ,...d 3g , ..., the data segment D, the check data P1 is set at the beginning of the data segment D, d g , d g+1 Set the verification data P2 between 2g , d 2g+1 Set the verification data P3...d kg , d kg+1 Set the verification data P3 between k+1 .

3. The method for enhancing data reliability according to claim 1, characterized in that: According to the known verification data P mentioned above i Verify data with readout Calculating the correction parameters at all check data positions of the user data includes: traversing each check data P i And the corresponding readout verification data For each pair of traversed check data P i And the corresponding readout verification data The correction parameters of the calibration data are calculated as:

4. The method for enhancing data reliability according to claim 3, characterized in that: For the following: Read out user data and obtain the segment header value of the read out user data as the first read out verification data Collect subsequent readout verification data every set g bits 5. The method for enhancing data reliability according to claim 1, characterized in that: The read user data having a correction parameter not equal to 1 at the verification data position is regarded as abnormal read user data.

6. The method for enhancing data reliability according to claim 1, characterized in that: The correction parameter of each data bit of each data segment of the abnormally read user data is estimated based on the calculated correction parameter at the check data position. The calculation formula is as follows: in, D f d is the correction parameter for data bit d of data segment D, P f i The correction parameter for the low-order check data of data segment D, P f i+1 It is the correction parameter of the high-order check data of data segment D.

7. The method for enhancing data reliability according to claim 1, characterized in that: The step of correcting the received read user data according to the correction parameter of the data bit of the data segment comprises: Traverse all data segments of abnormally read user data; for each traversed data segment, calculate the data after each data bit is corrected according to the following formula: D f d Correction parameters for abnormally reading data bit d of data segment D in user data, Read the data segment in the user data for abnormality The data bit d before correction, Read the data segment in the user data for abnormality The data bit d is the data after correction.

8. A device for enhancing data reliability, characterized in that: include: At least one processing unit, the processing unit and the storage unit are interconnected via a bus unit, the storage unit stores a computer program, and when the computer program is executed by the processing unit, the method for enhancing data reliability as described in any one of claims 1-7 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for enhancing data reliability as described in any one of claims 1 to 7 is implemented.

Images