Data storage method and data storage system based on CPU

Abstract

The invention discloses a data storage method and a data storage system based on a CPU (central processing unit), and relates to the technical field of storage control. The CPU is respectively connected with a cache module and an NAND flash memory; the method comprises the steps that the cache module continuously receives to-be-stored data; the CPU reads the data to be stored in the cache module and determines the type of the data to be stored; the CPU obtains the type and the residual capacity of the cache module, processes the to-be-stored data and writes the to-be-stored data into the NAND flash memory based on the type of the to-be-stored data, the type and the residual capacity of the cache module and a pre-constructed data processing model; wherein the type of the to-be-stored data comprises continuous data and random data; the types of the cache modules comprise a non-volatile cache module and a volatile cache module. According to the storage method and the system applying the storage method, the problems that the data writing speed is low, the writing amplification is large, and the resource utilization efficiency is low are solved.

Description

Technical Field

[0001] The present invention relates to the field of storage control technology, and in particular to a data storage method and a data storage system based on a CPU. Background Art

[0002] NAND flash memory is a non-volatile memory that can still save data after power failure. As solid-state drives gradually replace mechanical hard drives in the industrial application market, NAND flash memory is also widely used in various electronic products due to its excellent read and write performance, large storage capacity and low power consumption.

[0003] In the prior art, when using NAND flash memory for data storage, the data is usually written directly into the NAND flash memory without being processed. This method has the following defects: when processing random data, it is not sorted and is directly written into the NAND flash memory. When writing and garbage collection, it will cause a slower writing speed and a larger write amplification during garbage collection. In addition, some technologies set up a temporary storage area, but do not distinguish between continuous data and random data, resulting in low resource utilization efficiency. Summary of the invention

[0004] In order to solve one or more of the above technical problems, the present application provides a CPU-based data storage method and a data storage system.

[0005] In a first aspect, the present application provides a CPU-based data storage method, which adopts the following technical solution: The CPU is connected to the cache module and the NAND flash memory respectively; the method comprises: The cache module continuously receives data to be stored; The CPU reads the data to be stored in the cache module and determines the type of the data to be stored; The CPU reads the data to be stored in the cache module and determines the type of the data to be stored; The CPU obtains the type and remaining capacity of the cache module, and based on the type of the data to be stored, the type and remaining capacity of the cache module, and a pre-built data processing model, processes the data to be stored and writes it into the NAND flash memory; Among them, the types of data to be stored include continuous data and random data; the types of cache modules include non-volatile cache modules and volatile cache modules.

[0006] By adopting the above technical solution, the CPU executes the corresponding data processing process according to multiple factors such as the type of data to be stored, the type and remaining capacity of the cache module, and the operating status of its own system, performs targeted and refined processing on the data to be stored, and writes the processed data to be stored into the NAND flash memory, thereby improving the speed of writing data into the NAND flash memory, significantly reducing the write amplification factor of the NAND flash memory, improving resource utilization efficiency, improving the performance of the NAND flash memory, and extending the service life of the NAND flash memory.

[0007] In a specific possible implementation scheme, the data processing model includes a first data processing sub-model, a second data processing sub-model, a third data processing sub-model and a fourth data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored based on the fourth data processing sub-model and then writes it into the NAND flash memory.

[0008] In a specific implementation scheme, the CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model, which specifically includes: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, and then the CPU moves the data to be stored in the cache module to the NAND flash memory; If not, the CPU directly writes the data to be stored in the cache module into the NAND flash memory.

[0009] By adopting the above technical solution, when the type of the cache module is a non-volatile cache module, since the non-volatile cache module has the characteristic of long-term data storage, therefore, for continuous data, when the system is idle, the continuous data is directly moved and written into the NAND flash memory as much as possible, which can reduce the occupancy of the cache module, reduce the occupancy of the CPU system, avoid the problem of a large flash memory write amplification factor due to a busy CPU system, avoid system congestion, and ensure the reliability of data storage.

[0010] In a specific implementation scheme, the CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model, which specifically includes: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, then the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory; If not, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory.

[0011] By adopting the above technical solution, when the type of cache module is a non-volatile cache module and the data to be stored is random data, it is first determined whether the remaining capacity of the cache module is sufficient. If not, the CPU quickly performs data integration and movement operations on the data to be stored; if sufficient, the CPU controls the cache module to continue to temporarily store the received data to be stored, and at the same time, the CPU performs data integration and movement operations according to its own system operating status when the system is not busy.

[0012] In a specific implementation scheme, the CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model, which specifically includes: The CPU directly moves the data to be stored in the cache module to the NAND flash memory.

[0013] By adopting the above technical solution, when the type of cache module is a volatile cache module, the data stored in the volatile cache module will be cleared when the system is in an unexpected situation such as power failure or restart. Therefore, in order to improve the security of data transmission and storage, if the data to be stored is continuous data, the continuous data will be directly moved and written into the NAND flash memory.

[0014] In a specific possible implementation scheme, the CPU processes the data to be stored and writes it into the NAND flash memory based on the fourth data processing sub-model, specifically including: The CPU determines, based on the remaining capacity information of the cache module, whether the remaining capacity of the cache module is lower than a first preset capacity threshold; If yes, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory; If not, the CPU controls the cache module to temporarily store the received data to be stored, and determines whether the system is busy based on the operating status of the system itself; If it is idle, the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory; If it is busy, the CPU will continue to monitor the operating status of its own system until the system is idle, then the CPU will integrate the data to be stored in the cache module and move the integrated data to be stored to the NAND flash memory.

[0015] By adopting the above technical solution, if the data to be stored is random data, when the remaining capacity of the volatile cache module is relatively small, the CPU quickly integrates the random data into a large file and then moves it to the NAND flash memory, so as to avoid the problem of data loss caused by the inability to continue to store data when the remaining capacity of the cache module is insufficient. When the remaining capacity of the volatile cache module is relatively large, it means that the volatile cache module can continue to store data at this time, and the CPU will perform data integration and movement when its own system is idle.

[0016] In a specific implementation scheme, the data processing model further includes a power-off protection processing sub-model; the CPU is used to detect the power-off risk of the cache module based on the power-off protection processing sub-model and execute corresponding protection measures, specifically including: If the type of the cache module is a volatile cache module, the CPU acquires the power supply voltage of the cache module in real time, and determines whether the cache module has a power failure risk based on the fluctuation of the power supply voltage; If so, the CPU moves the data to be stored in the cache module to the NAND flash memory.

[0017] By adopting the above technical solution and taking into account the waveform of the power supply voltage of the volatile cache module, when the power supply voltage is unstable, the volatile cache module may be at risk of power failure. In order to avoid data loss, the CPU quickly moves the data to be stored in the cache module to the NAND flash memory for storage, making the data transmission process safer and more reliable.

[0018] In a specific possible implementation scheme, when the type of the cache module is a non-volatile cache module, the cache module is an area divided inside the NAND flash memory or a device external to the NAND flash memory; When the type of the cache module is a volatile cache module, the cache module is a device external to the NAND flash memory.

[0019] By adopting the above technical solution, when the cache module is a non-volatile cache type, the technicians in this field can divide a part of the area inside the NAND flash memory as the cache module, or choose to use an external NAND flash memory device as the cache module; when the cache module is a volatile cache type, the technicians in this field can choose an external NAND flash memory device as the cache module.

[0020] In a specific implementation scheme, the process of the CPU integrating the data to be stored in the cache module specifically includes: The CPU divides each of the data to be stored into corresponding data sets based on the data size of the data to be stored; The CPU sorts the data to be stored in each of the data sets based on the sorting rules corresponding to each of the data sets to generate a corresponding data sequence; The CPU matches and combines the data to be stored in each of the data sequences to form integrated data to be stored.

[0021] In a second aspect, the present application provides a data storage system, which adopts the following technical solution: the system includes a CPU, a cache module and a NAND flash memory, and the data storage system applies the CPU-based data storage method described in the above-mentioned first aspect or any possible implementation scheme of the first aspect.

[0022] In summary, the technical solution of this application includes at least the following beneficial technical effects: 1. The CPU executes the corresponding data processing process according to the type of data to be stored, the type and remaining capacity of the cache module, and the operating status of its own system, performs targeted and refined processing on the data to be stored, and writes the processed data to be stored into the NAND flash memory, thereby increasing the speed of writing data into the NAND flash memory, significantly reducing the write amplification factor of the NAND flash memory, improving resource utilization efficiency, improving the performance of the NAND flash memory, and extending the service life of the NAND flash memory; 2. When the cache module is a non-volatile cache module, since the non-volatile cache module has the characteristic of long-term data storage, for continuous data, when the system is idle, the continuous data is directly moved and written into the NAND flash memory as much as possible, which can reduce the occupancy of the cache module and the CPU system, avoid the problem of a large NAND flash memory write amplification factor caused by a busy CPU system, avoid system congestion, and ensure the reliability of data storage; for random data, when the remaining capacity of the cache module is insufficient, the CPU quickly performs data integration and movement operations on the data to be stored. If it is sufficient, the CPU controls the cache module to continue to temporarily store the received data to be stored, and at the same time, the CPU performs data integration and movement operations when the system is not busy; 3. When the cache module is a volatile cache module, the data stored in the volatile cache module will be cleared when the system is powered off or restarted, etc., in order to improve the security of data transmission and storage, for continuous data, the continuous data is directly moved and written into the NAND flash memory; for random data, when the remaining capacity of the cache module is relatively small, the CPU quickly performs integration and movement operations to avoid the problem of data loss caused by the inability to continue to store data when the remaining capacity of the cache module is insufficient. When the remaining capacity of the cache module is relatively large, it means that the cache module can continue to store data at this time, then the CPU will perform data integration and movement operations when its own system is idle. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] Figure 1 This is a schematic diagram of the connection relationship of the CPU in the embodiment of the present application; Figure 2 This is the main flow chart of the data storage method in the embodiment of the present application; Figure 3 is a flowchart of processing data to be stored when the cache module is a non-volatile cache module in an embodiment of the present application; Figure 4 is a flowchart of processing data to be stored when the cache module is a volatile cache module in an embodiment of the present application; Figure 5 It is a schematic diagram of the data integration process in the embodiment of the present application. DETAILED DESCRIPTION

[0024] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0025] The present application embodiment provides a data storage method based on CPU, referring to Figure 1 , the CPU is connected to the cache module and the NAND flash memory respectively; specifically, the cache module is a high-speed cache module.

[0026] Reference Figure 2 , the data storage method comprises: The cache module continuously receives data to be stored; The CPU reads the data to be stored in the cache module and determines the type of the data to be stored; The CPU obtains the type and remaining capacity of the cache module, and based on the type of the data to be stored, the type and remaining capacity of the cache module, and a pre-built data processing model, writes the data to be stored into the NAND flash memory after processing; Among them, the types of data to be stored include continuous data and random data; the types of cache modules include non-volatile cache modules and volatile cache modules.

[0027] In a possible implementation, the CPU may include two data input interfaces, one data input interface is connected to the cache module, and the other data input interface is used to directly receive the data to be stored. That is to say, the CPU can not only determine the type of data to be stored by reading the data to be stored in the cache module, but also determine the type of data to be stored by directly receiving and reading the data to be stored.

[0028] In a possible implementation, the data processing model includes a first data processing sub-model, a second data processing sub-model, a third data processing sub-model and a fourth data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored based on the fourth data processing sub-model and then writes it into the NAND flash memory.

[0029] In one possible implementation, refer to Figure 3 , the CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model, specifically including: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, and then the CPU moves the data to be stored in the cache module to the NAND flash memory; If not, the CPU directly writes the data to be stored in the cache module into the NAND flash memory.

[0030] Through the above process, when the type of cache module is a non-volatile cache module, since the non-volatile cache module has the characteristic of long-term data storage, therefore, for continuous data, when the system is idle, the continuous data is directly moved and written into the NAND flash memory as much as possible, which can reduce the occupancy of the cache module and the CPU system, avoid the problem of a large NAND flash memory write amplification factor due to a busy CPU system, avoid system congestion, and ensure the reliability of data storage.

[0031] In one possible implementation, refer to Figure 3 , the CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model, specifically including: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, then the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory; If not, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory.

[0032] In the above process, when the type of cache module is a non-volatile cache module and the data to be stored is random data, it is first determined whether the remaining capacity of the cache module is sufficient. If not, the CPU quickly performs data integration and movement operations on the data to be stored; if sufficient, the CPU controls the cache module to continue to temporarily store the received data to be stored. At the same time, the CPU performs data integration and movement operations according to its own system operating status when the system is not busy.

[0033] Since each random data is a relatively small file, the CPU integrates the data to be stored in the cache module into a large file and then moves it to the NAND flash memory. This integration and movement process can not only improve the writing speed of random data, but also can increase the speed by more than 10 times compared to the method of directly writing random data into the NAND flash memory, which significantly improves the writing speed of random data, and significantly reduces the write amplification factor and extends the service life of the NAND flash memory.

[0034] In one possible implementation, refer to Figure 4 , the CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model, specifically including: The CPU directly moves the data to be stored in the cache module to the NAND flash memory.

[0035] When the cache module type is a volatile cache module, the data stored in the volatile cache module will be cleared when the system is powered off or restarted, etc., in order to improve the security of data transmission and storage, if the data to be stored is continuous data, the continuous data will be directly moved and written into the NAND flash memory.

[0036] In one possible implementation, refer to Figure 4 , the CPU processes the data to be stored and writes it into the NAND flash memory based on the fourth data processing sub-model, specifically including: The CPU determines, based on the remaining capacity information of the cache module, whether the remaining capacity of the cache module is lower than a first preset capacity threshold; If yes, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory; If not, the CPU controls the cache module to temporarily store the received data to be stored, and determines whether the system is busy based on the operating status of the system itself; If it is idle, the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory; If it is busy, the CPU will continue to monitor the operating status of its own system until the system is idle, then the CPU will integrate the data to be stored in the cache module and move the integrated data to be stored to the NAND flash memory.

[0037] In the above process, if the data to be stored is random data, when the remaining capacity of the cache module is relatively small, the CPU will quickly integrate the random data into a large file and then move it to the NAND flash memory to avoid the problem of data loss caused by the inability to continue to store data when the remaining capacity of the cache module is insufficient. When the remaining capacity of the cache module is relatively large, it means that the cache module can continue to store data at this time, and the CPU will perform data integration and movement when its own system is idle.

[0038] In a possible implementation, the determination of the first preset capacity threshold can be adjusted accordingly by those skilled in the art according to the performance and / or actual usage of the cache module; illustratively, for a cache module with lower performance, the first preset capacity threshold can be set to 20% of the total capacity of the cache module, that is, when the current remaining capacity of the cache module is lower than 20% of the total capacity, it can be considered that the remaining capacity of the cache module is already at a low level and cannot continue to receive data to be stored; for a volatile cache module with higher performance, the first preset capacity threshold can be set to 10% of the total capacity of the cache module, that is, when the current remaining capacity of the cache module is lower than 10% of the total capacity, it can be considered that the remaining capacity of the cache module is already at a low level and cannot continue to receive data to be stored.

[0039] In a possible implementation, the data processing model further includes a power-off protection processing sub-model; the CPU is used to detect the power-off risk of the cache module based on the power-off protection processing sub-model, and execute corresponding protection measures, specifically including: If the type of the cache module is a volatile cache module, the CPU acquires the power supply voltage of the cache module in real time, and determines whether the cache module has a power failure risk based on the fluctuation of the power supply voltage; If so, the CPU moves the data to be stored in the cache module to the NAND flash memory.

[0040] The above process takes into account the waveform of the power supply voltage of the volatile cache module. When the power supply voltage is unstable, the volatile cache module may be at risk of power failure. In order to avoid data loss, the CPU quickly moves the data to be stored in the cache module to the NAND flash memory for storage, making the data transmission process safer and more reliable.

[0041] In a possible implementation manner, the process of the CPU integrating the data to be stored in the cache module specifically includes the following steps S1-S3: S1, the CPU, based on the data size D of the data to be stored size , each of the data to be stored is divided into corresponding data sets.

[0042] Specifically, in this step, the data to be stored can be divided based on a pre-set data set and the corresponding data ranges for each data set. For example, the pre-set data sets include a first data set, a second data set, and a third data set. The first data set, the second data set, and the third data set respectively correspond to different data ranges, and the data ranges increase in sequence. For example, the data range corresponding to the first data set is (0, A], the data range corresponding to the second data set is (A, B], and the data range corresponding to the third data set is (B, C); and A < B ≤ S < C, where S is the page size of the NAND flash memory; C is the file size threshold, which can be considered as the value for dividing continuous data and random data, and A and B are two set boundary values.

[0043] S2. The CPU sorts the data to be stored in each data set based on the sorting rules corresponding to each data set, generating corresponding data sequences.

[0044] In this step, the sorting rules corresponding to each data set are different. Continuing with the above example, for the first data set, its sorting rule is: according to the data size D size of each data to be stored in the first data set, sort each data to be stored, generating a first data sequence; preferably, the sorting can be performed in ascending order.

[0045] For the second data set, its sorting rule is: First, based on the data size D size of each data to be stored in the second data set, the page size S of the NAND flash memory, and the multiple calculation equation, calculate the multiple n corresponding to each data to be stored; where the multiple calculation equation is: (n - 1)*S < D size ≤ n*S, n ≥ 1, and n is an integer. Then, based on the page size S of the NAND flash memory, the data size D size of each data to be stored, and the corresponding multiple n, calculate the data difference △D corresponding to each data to be stored; where △D = n*S - D size . Finally, based on the data difference △D corresponding to each data to be stored, sort each data to be stored, generating a second data sequence; preferably, based on the data difference △D corresponding to each data to be stored, sort the corresponding data to be stored in ascending order.

[0046] For the third data set, its sorting rule is the same as that of the second data set, which is also to first calculate the multiple n, then calculate the data difference △D, and finally sort the data to be stored according to the data difference △D, which will not be elaborated here.

[0047] S3, matching and combining the data to be stored in each of the data sequences to form integrated data to be stored.

[0048] In this step, the data to be stored in each of the data sequences are matched and combined, including steps S31-S32: S31, marking the data to be stored in the second data sequence and the third data sequence as data to be matched; S32, for each data to be stored in the first data sequence, searching for matching items from the data to be matched in accordance with the data matching rules, and integrating the data to be stored and the corresponding matching items into one data to form integrated data to be stored.

[0049] Specifically, the data matching rule includes: for the data to be stored in the first data sequence, searching for matching items from each of the data to be matched in turn, if the data size of the data to be stored in the first data sequence is greater than the sum of the data sizes of the data to be matched D size-sum , satisfying: T<D size-sum ≤n*S; then the data to be matched is determined to be a matching item of the data to be stored in the first data sequence. Wherein, T is a preset ratio threshold value, which is related to S, for example, T=90%S; n is the multiple n corresponding to the data to be matched calculated above. It should be noted that when searching for matching items from each of the data to be matched in sequence in this step, it can be searched in sequence from the second data sequence first. If not found, it can be searched in sequence from the third data sequence.

[0050] Through the above process, the functions of integrating small data into big data and integrating random addresses into continuous addresses are realized, thereby ensuring the improvement of random data writing and reading performance.

[0051] It can be understood by those skilled in the art that for the data to be stored that cannot be matched, that is, the data to be stored for which no matching items are found in the first data sequence, and the data in the second data sequence and the third data sequence that are not used as matching items, they are marked as data to be integrated again, and after waiting for the next batch of data to be stored to be divided, they are merged into the corresponding data set, and the above integration process is performed again. The next batch of data to be stored still uses the same integration method. That is, for the data to be stored for which no matching items are found in the first data sequence, they are merged into the first data set of the next batch; for the data in the second data sequence that are not used as matching items, they are merged into the second data set of the next batch; for the data in the third data sequence that are not used as matching items, they are merged into the third data set of the next batch, which will not be repeated here. With the continuous receipt of data, the fusion of new data and unmatched data is also continuously realized.

[0052] Combine the following Figure 5 Explanation of the above integration process: The data range of the first data set is preset to (0, 200 Byte], the data range of the second data set is preset to (200 Byte, 600 Byte], and the data range of the third data set is preset to (600 Byte, 1200 Byte); and the page size S of the NAND flash memory is 512 Byte.

[0053] First, based on the data size D of the data to be stored size , and divide each data to be stored into a corresponding data set respectively, and the data to be stored in the first data set includes P1, P2, P3, and P4, and the data size of P1 is 20Byte, the data size of P2 is 45Byte, the data size of P3 is 50Byte, and the data size of P4 is 30Byte; the data to be stored in the second data set includes Q1, Q2, Q3, and Q4, and the data size of Q1 is 300Byte, the data size of Q2 is 400Byte, the data size of Q3 is 350Byte, and the data size of Q4 is 550Byte; the data to be stored in the third data set includes R1, R2, R3, and R4, and the data size of R1 is 700Byte, the data size of R2 is 650Byte, the data size of R3 is 1100Byte, and the data size of R4 is 710Byte.

[0054] If the data are arranged in ascending order, after the data to be stored in the first data set are sorted, the first data sequence {P1, P4, P2, P3} can be obtained; after the data to be stored in the second data set are sorted, the second data sequence {Q2, Q3, Q1, Q4} can be obtained; after the data to be stored in the third data set are sorted, the third data sequence {R4, R1, R2, R3} can be obtained.

[0055] After obtaining the three data sequences, firstly for P1, matching items are searched in turn from the second data sequence and the third data sequence, so as to form integrated data to be stored; the matching process is not described in detail here.

[0056] In a possible implementation, after the data to be stored and the corresponding matching items are integrated into one data in step S32 to form the integrated data to be stored, the data to be stored and the corresponding matching items can be marked with a partition identifier, and a mapping relationship can be established based on the integrated data to be stored; so that when reading subsequently, the integrated data to be stored can be first read out based on the mapping table, and then the corresponding data can be extracted based on the partition identifier. Figure 5As shown, if Q2 is a match for P1, an isolate identifier is used between Q2 and P1.

[0057] In a possible implementation, in order to avoid the risk of power outages, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory every first preset period T1, and may also establish a mapping table to mark the integrated data to be stored; further, the CPU may also release the integrated data to be stored from its own cache every second preset period T2; further, the CPU may also move the unmatched data to be stored to the NAND flash memory every third preset period T3, and temporarily store it in its own cache, waiting for a new matching opportunity for the next batch of data.

[0058] The ideal situation achieved by applying this integration method is that the data size of the data to be stored in the first data sequence is less than 0.5S, and the data size of the data to be matched in the second data sequence is between 0.5S and S. In this way, the two data can be combined into data of size S after integration and then moved.

[0059] In a possible implementation manner, before the cache module continues to receive the data to be stored, the method further includes: The CPU determines a file size threshold C based on a first data processing capacity of the CPU; the file size threshold C is used to distinguish between continuous data and random data; the first data processing capacity represents the capacity allocated by the CPU for processing the data to be stored.

[0060] Specifically, if the file size of the data to be stored exceeds the file size threshold C, the CPU determines that the data to be stored is continuous data; if the file size of the data to be stored does not exceed the file size threshold C, the CPU determines that the data to be stored is random data.

[0061] Therefore, after the staff allocates the first data processing capacity in the CPU according to actual needs, the CPU automatically determines the file size threshold C to facilitate subsequent determination of the type of data to be stored.

[0062] Therefore, the data storage method of the present application executes the corresponding data processing process through the CPU according to multiple factors such as the type of data to be stored, the type and remaining capacity of the cache module, and the operating status of the system itself, so as to perform targeted and refined processing on the data to be stored, and write the processed data to be stored into the NAND flash memory, thereby improving the speed of writing data into the NAND flash memory, significantly reducing the write amplification factor of the NAND flash memory, improving resource utilization efficiency, improving the performance of the NAND flash memory, and extending the service life of the NAND flash memory.

[0063] In a possible implementation manner, when the type of the cache module is a non-volatile cache module, the cache module is an area divided inside the NAND flash memory or is a device external to the NAND flash memory; When the type of the cache module is a volatile cache module, the cache module is a device external to the NAND flash memory.

[0064] The above-mentioned NAND flash memory external device is other devices connected to the NAND flash memory. Therefore, when the cache module is a non-volatile cache type, those skilled in the art can divide a part of the area inside the NAND flash memory as the cache module, or choose to use a NAND flash memory external device as the cache module; when the cache module is a volatile cache type, those skilled in the art can choose a NAND flash memory external device as the cache module.

[0065] An embodiment of the present application provides a data storage system, the system including a CPU, a cache module and a NAND flash memory, and the data storage system applies the above-mentioned CPU-based data storage method.

[0066] The above are all preferred embodiments of the present application, and the protection scope of the present application is not limited thereto. Therefore, any equivalent changes made according to the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A CPU-based data storage method, characterized in that: The CPU is connected to the cache module and the NAND flash memory respectively; the data storage method comprises: The cache module continuously receives data to be stored; The CPU reads the data to be stored in the cache module and determines the type of the data to be stored; The CPU obtains the type and remaining capacity of the cache module, and based on the type of the data to be stored, the type and remaining capacity of the cache module, and a pre-built data processing model, writes the data to be stored into the NAND flash memory after processing; Among them, the types of data to be stored include continuous data and random data; the types of cache modules include non-volatile cache modules and volatile cache modules.

2. The CPU-based data storage method according to claim 1, characterized in that: The data processing model includes a first data processing sub-model, a second data processing sub-model, a third data processing sub-model and a fourth data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model; When the type of the cache module is a non-volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is continuous data, the CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model; When the type of the cache module is a volatile cache module and the type of the data to be stored is random data, the CPU processes the data to be stored based on the fourth data processing sub-model and then writes it into the NAND flash memory.

3. The CPU-based data storage method according to claim 2, characterized in that: The CPU processes the data to be stored and writes it into the NAND flash memory based on the first data processing sub-model, specifically including: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, and the CPU moves the data to be stored in the cache module to the NAND flash memory; If not, the CPU directly writes the data to be stored in the cache module into the NAND flash memory.

4. The CPU-based data storage method according to claim 2, characterized in that: The CPU processes the data to be stored and writes it into the NAND flash memory based on the second data processing sub-model, specifically including: The CPU determines whether the system is busy based on the operating status of its own system; If yes, the CPU controls the cache module to temporarily store the received data to be stored, and continuously monitors the operating status of its own system until the system is idle, at which time the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory; If not, the CPU integrates the data to be stored in the cache module, and moves the integrated data to be stored to the NAND flash memory.

5. The CPU-based data storage method according to claim 2, characterized in that: The CPU processes the data to be stored and writes it into the NAND flash memory based on the third data processing sub-model, specifically including: The CPU directly moves the data to be stored in the cache module to the NAND flash memory.

6. The CPU-based data storage method according to claim 2, characterized in that: The CPU processes the data to be stored and writes it into the NAND flash memory based on the fourth data processing sub-model, specifically including: The CPU determines, based on the remaining capacity information of the cache module, whether the remaining capacity of the cache module is lower than a first preset capacity threshold; If yes, the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory; if no, the CPU controls the cache module to temporarily store the received data to be stored and determines whether the system is busy based on its own system operation status; If it is idle, the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory; if it is busy, the CPU continuously monitors the operating status of its own system until the system is idle, and the CPU integrates the data to be stored in the cache module and moves the integrated data to be stored to the NAND flash memory.

7. The CPU-based data storage method according to claim 2, characterized in that: The data processing model also includes a power-off protection processing sub-model; The CPU is used to detect the power-off risk of the cache module based on the power-off protection processing sub-model and execute corresponding protection measures, specifically including: If the type of the cache module is a volatile cache module, the CPU acquires the power supply voltage of the cache module in real time, and determines whether the cache module has a power failure risk based on the fluctuation of the power supply voltage; If so, the CPU moves the data to be stored in the cache module to the NAND flash memory.

8. The CPU-based data storage method according to claim 4, characterized in that: The process of the CPU integrating the data to be stored in the cache module specifically includes: The CPU divides each of the data to be stored into corresponding data sets based on the data size of the data to be stored; The CPU sorts the data to be stored in each of the data sets based on the sorting rules corresponding to each of the data sets to generate a corresponding data sequence; The CPU matches and combines the data to be stored in each of the data sequences to form integrated data to be stored.

9. The CPU-based data storage method according to claim 1, characterized in that: When the type of the cache module is a non-volatile cache module, the cache module is an area divided inside the NAND flash memory or a device external to the NAND flash memory; When the type of the cache module is a volatile cache module, the cache module is a device external to the NAND flash memory.

10. A data storage system, characterized in that: The data storage system comprises a CPU, a cache module and a NAND flash memory, and the data storage system applies the CPU-based data storage method described in any one of claims 1 to 9.

Images