A high-speed data access method and device based on error correction

Abstract

The present invention discloses a high-speed data access method and device based on error correction, the method comprising: when writing data: grouping all original data according to a storage unit size, obtaining a first preset number of groups of original data as a segment, for a segment of original data, extracting one bit from each group to form a group of data to be encoded, generating a group of error correction encoded data based on the group of data to be encoded, until all bits of each group of original data are subjected to error correction encoding; writing a group of data to be encoded and a corresponding group of error correction encoded data bit by bit into a third number of logical units, writing one bit of data into each logical unit. The present invention realizes the correction of abnormal errors occurring in a single storage unit by using storage array encoding while ensuring data throughput, avoiding data errors caused by abnormalities in a single storage unit, and improving system reliability.

Description

Technical Field

[0001] The present invention relates to the technical field of data storage, and in particular to a high-speed data access method and device based on error correction. Background Art

[0002] With the development of solid-state storage technology, various high-speed solid-state flash memories are used in various instruments and equipment as data storage media. Since most solid-state storage devices are NAND FLASH, due to problems such as production processes, it is inevitable that some storage units cannot be used normally during production and use, that is, bad blocks appear. The bad blocks generated after solid-state storage leaves the factory can generally be removed by initialization scanning, storage space mapping, etc., but the bad blocks that appear during use need to be checked for errors in the memory through ECC checks, and the bad blocks that are checked are included in the storage space mapping management, and the storage space is remapped. This part of the work is generally managed by the file system. In some embedded high-speed application scenarios, due to the ultra-high data throughput (GByte / s), general file systems are no longer usable.

[0003] Existing data access is generally divided into two types: with file system and without file system:

[0004] 1. The method with a file system generally uses an embedded core processor with an operating system (Linux system) to manage peripheral solid-state storage devices. This method has the advantages of easy operation and high reliability, but due to limitations such as parallel processing capabilities, it cannot support usage scenarios that require ultra-high-speed read and write bandwidth.

[0005] 2. The method without file system can adopt the solution with FPGA as the core processing unit to directly drive the external storage device. This method has the advantages of high concurrent processing capability and ultra-high-speed read and write bandwidth, but the bad block problem of storage needs to be handled through relatively complex initialization, dynamic bad block checking, storage space mapping and other solutions, and the management complexity increases with the increase in the number of mounted storage units. Summary of the invention

[0006] In view of the problems existing in the above-mentioned prior art, the present invention provides a high-speed data access method and device based on error correction, which can ensure data reliability while ensuring that the data throughput meets the requirements of ultra-high-speed read and write bandwidth. The technical solution is as follows:

[0007] In a first aspect, a high-speed data access method based on error correction is provided, which is applied to read and write control of a flash solid-state storage device, comprising:

[0008] When writing data: all original data are grouped according to a storage unit size, a first preset number of groups of original data are obtained as a segment, for a segment of original data, one bit is extracted from each group to form a group of data to be encoded, and a group of error correction encoding data is generated based on the group of data to be encoded, until all bits of each group of original data are subjected to the error correction encoding process;

[0009] Writing a set of to-be-encoded data and a corresponding set of error correction coded data bit by bit into a third number of logical units (LUNs), with one bit of data being written into each logical unit;

[0010] When reading data, data is read from each logical unit and the original data is obtained after decoding and error correction.

[0011] In a possible implementation manner, after performing error detection and error correction on the read data, the method further includes:

[0012] For a storage unit with an erroneous data bit in the logic unit, an abnormal storage record is added;

[0013] Monitor whether the number of abnormal storage records in the same storage unit is greater than a preset threshold;

[0014] If so, generating an update request for a mapping table of physical addresses and logical addresses of the data storage space;

[0015] The mapping table is updated based on the update request in an idle state.

[0016] In a possible implementation, one logic unit (LUN) corresponds to one chip select enable signal input terminal to implement separate control of the read and write process of each logic unit.

[0017] In one possible implementation, when writing a set of to-be-encoded data and a corresponding set of error correction coded data bit by bit into a third number of logical units (LUNs) and reading data, the data reading and writing speed is improved by controlling the simultaneous activation of the third number of logical units (LUNs).

[0018] In a possible implementation, the error correction coding process uses a Hamming code coding algorithm.

[0019] In a possible implementation, the logic unit is located in a FLASH particle of a flash solid-state storage device, the flash solid-state storage device includes 8 FLASH particles, and one FLASH particle includes 4 logic units.

[0020] In a second aspect, a high-speed data access device based on error correction is provided, comprising:

[0021] The original data unit error correction coding unit is used to group all the original data according to a storage unit size, obtain a first preset number of groups of original data as a segment, extract one bit from each group of a segment of original data to form a group of data to be encoded, and generate a group of error correction coding data based on the group of data to be encoded, until all bits of each group of original data are subjected to the error correction coding process;

[0022] The original data and coded data writing unit writes a set of to-be-coded data and a corresponding set of error-correcting coded data into a third number of logical units (LUNs) bit by bit, with one bit of data being written into each logical unit;

[0023] The data reading unit is used to read data from each logical unit and obtain the original data through decoding and error correction.

[0024] In a possible implementation, the above-mentioned error correction-based high-speed data access device also includes: an abnormal storage unit management unit, which is used to add an abnormal storage record for a storage unit in which an erroneous data bit occurs in a logical unit; when the number of abnormal storage records for the same storage unit is greater than a preset threshold, an update request is generated for the mapping table of the physical address and the logical address of the data storage space, and the mapping table is updated based on the update request in an idle state.

[0025] In a third aspect, an application of the high-speed data access method in a high-speed camera based on FPGA is provided, wherein the FPGA realizes high-speed reading and writing of image data by controlling a solid-state storage array composed of a plurality of FLASH particles.

[0026] A high-speed data access method and device based on error correction of the present invention have the following beneficial effects: the present invention realizes the use of storage array coding to correct abnormal errors occurring in a single storage unit while ensuring data throughput, thereby avoiding data errors caused by abnormalities in a single storage unit and improving system reliability through the combination setting of multiple storage units, the setting of a method for selecting original data bits participating in an encoding process when error correction encoding is performed on the original data, and the setting of a method for writing the generated error correction code into the storage unit. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 is a flowchart of the error correction encoding and writing process of a segment of original data according to an embodiment of the present invention;

[0028] Figure 2 is a flow chart of reading data in an embodiment of the present invention;

[0029] Figure 3 It is a schematic diagram of a writing process for a set of data to be encoded in the present invention;

[0030] Figure 4It is a schematic diagram of the read and write control of the flash solid-state storage device by the FPGA in the high-speed camera of the present invention;

[0031] Figure 5 It is a structural block diagram of a high-speed data access device based on error correction in the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings. The described embodiments should not be regarded as limiting the present invention. All other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

[0033] The embodiment of the present invention discloses a high-speed data access method based on error correction, which is applied to the read and write control of the FPGA on the flash solid-state storage device in the FPGA-based high-speed camera. The FPGA realizes the reading and writing of high-speed image data by controlling the solid-state storage array composed of multiple FLASH particles. Specifically, the access process of high-speed image data includes: each high-speed image data is collected by a high-speed image acquisition sensor and transmitted to the FPGA, and the FPGA controls the solid-state storage array composed of 8 FLASH particles. The high-speed image sensor transmits the collected image data to the FPGA, and after receiving the original image data, the FPGA encodes the data. The encoded data is written into the solid-state storage array. After the data acquisition is completed, the image data is read out according to the data reading space given by the host computer, and the read image data is decoded and then the original image data is output to the host computer. Among them, the control method of the FPGA on the solid-state storage array composed of 8 FLASH particles,

[0034] When writing data:

[0035] Grouping all the original data according to a storage unit size, obtaining a first preset number of groups of original data as a segment;

[0036] The encoding and writing process for a piece of raw data includes:

[0037] (1) extracting one bit from each group to form a group of data to be encoded, and performing an error correction encoding process based on the group of data to be encoded to generate a group of error correction coded data;

[0038] (2) repeating step (1) until all bits of each set of original data have been subjected to the error correction coding process;

[0039] (3) writing a set of to-be-encoded data and a corresponding set of error correction coded data bit by bit into a third number of logical units (LUNs), with one bit of data being written into each logical unit, that is, enabling one storage unit in each logical unit to store one bit of data;

[0040] (4) Repeat step (3) until a segment of original data and the corresponding error correction coded data complete the writing process.

[0041] When reading data, data is read from each logical unit and the original data is obtained after decoding and error correction.

[0042] In the above writing process, one bit is extracted from each of the first preset number of groups of original data to form a group of data to be encoded, that is, the number of bits of a group of data to be encoded is the first preset number, and based on the first preset number of bits of data to be encoded and the preset error correction coding algorithm, the number of bits of a group of error correction coded data can be determined, and the number of bits of a group of error correction coded data is recorded as the second number;

[0043] In this embodiment, the first preset number is 11, and one storage unit of all logical units stores 8-bit data as an example. Then, the size of a group of original data is 8 bits, and the size of a segment of original data is 11*8 bits, that is, 11 bytes. The error correction coding and writing process for the 11 bytes is as follows:

[0044] Each group of original data is numbered as the first group, the second group, ... the eleventh group, and each bit of each group of data is numbered as the first bit, the second bit, ... the eighth bit;

[0045] Extract the first bit from each of the first to eleventh groups of original data to form the first group of data to be encoded, and take the Hamming code encoding algorithm as an example to encode the first group of data to be encoded to generate 5-bit error correction coded data;

[0046] Extracting the second bit from each of the first to eleventh groups of original data to form a second group of data to be encoded, and forming corresponding 5-bit error correction encoded data;

[0047] Similarly, extract the j-th group of data to be encoded (j is less than or equal to 8) from each of the 1st to 11th groups of original data to form corresponding 5-bit error correction encoded data;

[0048] Write the 11 bits of the first group of data to be encoded and the corresponding 5 bits of error correction encoded data, a total of 16 bits of data, into 16 logical units, and number each bit of storage space in the logical unit bit by bit, the first bit, the second bit, ... the eighth bit, and write the 16 bits of data into the first bit of each logical unit in turn;

[0049] The 11 bits of the second group of data to be encoded and the corresponding 5 bits of error correction encoded data are sequentially written into the second bit of each logical unit;

[0050] Similarly, the 11 bits of the j-th group of data to be encoded and the corresponding 5 bits of error correction encoded data are written into the j-th bit of each logical unit in sequence.

[0051] Correspondingly, the process of decoding and error correction when reading data can be:

[0052] The j-th bit data of the 1st to 16th logic units are read out one by one, and the read data are decoded and error corrected to obtain the original data.

[0053] In the embodiment of the present invention, the value of the second quantity is obtained by setting the value of the first preset quantity and selecting the error correction coding scheme, and the number of logical units required for the storage process of a segment of original data and the error correction coding data corresponding to the segment of original data, that is, the third quantity, is further obtained. For example, the above-mentioned access to a segment of original data requires 16 logical units. In this embodiment, in order to be able to flexibly adopt a variety of error correction coding algorithms to adapt to different third quantity requirements, the flash solid-state storage device is set to include 8 FLASH particles, and one FLASH particle includes 4 logical units.

[0054] The embodiment of the present invention sets a combination of multiple storage units, sets a method for selecting the original data bits involved in a coding process when the original data is subjected to error correction coding, and sets a method for writing the generated error correction code into the storage unit:

[0055] First, it realizes error correction processing on the read data to avoid data errors caused by damage to a single unit, and eliminates the impact of the failure of a certain storage unit on the entire system by encoding the original data.

[0056] On the second aspect, in this embodiment, the read and write process of each logic unit is individually controlled by corresponding one of the logic units (LUN) to one chip select enable signal input terminal. When a set of data to be encoded and a corresponding set of error correction coded data are written bit by bit into the third number of logic units (LUN) and when the data is read, the data read and write speed is increased by controlling the simultaneous activation of the third number of logic units (LUN), that is, the data processing bandwidth is increased by parallel control of the storage unit array. While ensuring high-speed access, the accuracy of data stored in the solid-state storage device is improved. When a small number of storage units fail due to life problems, the correctness of the read data can be guaranteed, thereby extending the service life of the storage system.

[0057] Thirdly, compared with the prior art design of improving data reliability by setting up a RAID redundant disk array, the implementation example of the present invention increases the physical storage units that can be operated simultaneously, refines the size of the unit storage unit, simplifies the control operation method, and improves the association between storage data blocks, thereby improving real-time processing capabilities.

[0058] On the basis of ensuring data throughput and data reliability as described above, after performing error detection and correction on the read data, this embodiment also includes management of normal and abnormal storage units in the entire flash solid-state storage device, specifically including:

[0059] For a storage unit with an erroneous data bit in the logic unit, an abnormal storage record is added;

[0060] Monitor whether the number of abnormal storage records in the same storage unit is greater than a preset threshold;

[0061] If so, generating an update request for a mapping table of physical addresses and logical addresses of the data storage space;

[0062] The mapping table is updated based on the update request in an idle state.

[0063] In order to facilitate data access on a normal storage unit according to the logical address and physical address of the mapping table when the flash storage device is enabled next time, in an embodiment of the present invention, the position information of the abnormal storage unit is statistically analyzed during the decoding process of each read data, based on multiple read and write operations on the flash solid-state storage device and multiple statistical results of the position information of the abnormal storage unit, when the number of abnormal storage records of the same storage unit is greater than a preset threshold, the storage unit at the position is determined to be an abnormal physical storage unit, and the mapping table of the physical address and the logical address is dynamically updated to remove the abnormal physical storage unit from the mapping table.

[0064] The embodiment of the present invention further provides a high-speed data access device based on error correction, comprising:

[0065] The original data unit error correction coding unit is used to group all the original data according to a storage unit size, obtain a first preset number of groups of original data as a segment, extract one bit from each group of a segment of original data to form a group of data to be encoded, and generate a group of error correction coding data based on the group of data to be encoded, until all bits of each group of original data are subjected to the error correction coding process;

[0066] The original data and coded data writing unit writes a set of to-be-coded data and a corresponding set of error-correcting coded data into a third number of logical units (LUNs) bit by bit, with one bit of data being written into each logical unit;

[0067] A data reading unit is used to read data from each logic unit and obtain the original data through decoding and error correction;

[0068] The abnormal storage unit management unit is used to add an abnormal storage record for a storage unit in which an erroneous data bit appears in a logical unit; when the number of abnormal storage records of the same storage unit is greater than a preset threshold, an update request is generated for the mapping table of the physical address and the logical address of the data storage space, and the mapping table is updated based on the update request in an idle state.

[0069] The specific definition of a high-speed data access device based on error correction in this embodiment can refer to the definition of a high-speed data access method based on error correction in the above text, which will not be repeated here. The above units can be implemented in software form, and the operations corresponding to the above units are executed based on FPGA calls.

[0070] The present invention is not limited to the above-mentioned specific implementation modes. Various changes made by ordinary technicians in this field based on the above-mentioned concepts without creative work are all within the protection scope of the present invention.

Claims

1. A high-speed data access method based on error correction, applied to read and write control of flash solid-state storage devices, characterized in that: include: When writing data: Grouping all the original data according to a storage unit size, obtaining a first preset number of groups of original data as a segment, extracting one bit from each group of a segment of original data to form a group of data to be encoded, and generating a group of error correction encoding data based on the group of data to be encoded, until all bits of each group of original data are subjected to the error correction encoding process; Based on the first preset number of bits of data to be encoded and the preset error correction coding algorithm, the number of bits of a set of error correction coded data obtained may be determined and recorded as a second number; Obtaining the number of logic units required for the storage process, i.e., a third number, based on the first preset number and the second number; Writing a set of to-be-encoded data and a corresponding set of error correction coded data into a third number of logic units bit by bit, with one bit of data being written into each logic unit; When reading data, data is read from each logical unit and the original data is obtained after decoding and error correction; Wherein, the logic unit is located in a FLASH particle of a flash solid-state storage device, the flash solid-state storage device includes 8 FLASH particles, and one FLASH particle includes 4 logic units; One of the logic units corresponds to one chip select enable signal input terminal to realize individual control of the reading and writing process of each logic unit; When writing a set of to-be-encoded data and a corresponding set of error correction coded data bit by bit into the third number of logic units and reading the data, the data reading and writing speed is improved by controlling the simultaneous activation of the third number of logic units.

2. The high-speed data access method based on error correction according to claim 1, characterized in that: include: After error detection and correction of the read data, it also includes: For a storage unit with an erroneous data bit in the logic unit, an abnormal storage record is added; Monitor whether the number of abnormal storage records in the same storage unit is greater than a preset threshold; If so, generating an update request for a mapping table of physical addresses and logical addresses of the data storage space; The mapping table is updated based on the update request in an idle state.

3. The high-speed data access method based on error correction according to claim 1, characterized in that: include: The error correction coding process adopts the Hamming code coding algorithm.

4. A high-speed data access device based on error correction, characterized in that: include: The original data unit error correction coding unit is used to group all the original data according to a storage unit size, obtain a first preset number of groups of original data as a segment, extract one bit from each group of a segment of original data to form a group of data to be encoded, and generate a group of error correction coding data based on the group of data to be encoded, until all bits of each group of original data are subjected to the error correction coding process; The original data and coded data writing unit writes a set of to-be-coded data and a corresponding set of error-correcting coded data into a third number of logic units bit by bit, with one bit of data being written into each logic unit; comprising: Based on the first preset number of bits of data to be encoded and the preset error correction coding algorithm, the number of bits of a set of error correction coded data obtained may be determined and recorded as a second number; Obtaining the number of logic units required for the storage process, i.e., a third number, based on the first preset number and the second number; When reading data, data is read from each logical unit and the original data is obtained after decoding and error correction; A data reading unit is used to read data from each logic unit and obtain the original data through decoding and error correction; Wherein, the logic unit is located in a FLASH particle of a flash solid-state storage device, the flash solid-state storage device includes 8 FLASH particles, and one FLASH particle includes 4 logic units; One of the logic units corresponds to one chip select enable signal input terminal to realize individual control of the reading and writing process of each logic unit; When writing a set of to-be-encoded data and a corresponding set of error correction coded data bit by bit into the third number of logic units and reading the data, the data reading and writing speed is improved by controlling the simultaneous activation of the third number of logic units.

5. The high-speed data access device based on error correction according to claim 4, characterized in that: Also includes: The abnormal storage unit management unit is used to add an abnormal storage record for the storage unit in which an error data bit occurs in the logic unit; When the number of abnormal storage records of the same storage unit is greater than a preset threshold, an update request is generated for a mapping table of physical addresses and logical addresses of the data storage space, and the mapping table is updated based on the update request in an idle state.

6. Application of the high-speed data access method according to any one of claims 1 to 3 in a high-speed camera based on FPGA, characterized in that: The FPGA realizes high-speed reading and writing of image data by controlling a solid-state storage array composed of a plurality of FLASH particles.

Images