Nonvolatile data storage method and system

Abstract

The invention belongs to the technical field of data storage, and particularly relates to a storage method and system of nonvolatile data, which realize high efficiency and safety of dynamic data storage by distributing at least two Flash pages for data storage, writing a basic information block into the head of each Flash page and associating a key value information block for each data object. According to the method, the length of the data entry can be dynamically adjusted, the problem that data may be lost when power failure occurs in the read-write process is solved through the strategy of writing first and erasing second, and meanwhile the purpose of prolonging the service life of Flash is achieved through dynamic management of the data entry and reasonable scheduling between pages.

Description

Technical Field

[0001] The present invention belongs to the technical field of data storage, and in particular relates to a non-volatile data storage method and system. Background Art

[0002] In embedded microcontroller (MCU) applications, NORFlash is widely used as a storage medium for program code and static data because of its ability to support random access and fast startup. However, when it comes to the storage of dynamic data, the limitations of NORFlash become particularly prominent. Existing technical solutions usually adopt two ways to deal with this challenge: Each piece of data occupies a separate page: Although this method simplifies the data management and erasure process, it leads to a huge waste of storage space because in most cases a single piece of data is much smaller than the Flash page size.

[0003] Pre-allocate fixed space to store multiple data: Although this method improves space utilization, when modifying or deleting data, other data needs to be moved to a new page, which increases the number of erase times and accelerates the loss of Flash. At the same time, there is a risk of data loss due to unexpected power outages during data migration.

[0004] The main problem of the above-mentioned prior arts is that they fail to effectively balance the storage efficiency and the life of the Flash. In particular, when processing dynamic data, how to reduce the number of erase times of the Flash page becomes an urgent problem to be solved. Summary of the invention

[0005] The purpose of the present invention is to provide a non-volatile data storage method and system, which can not only dynamically adjust the length of data entries, but also solve the problem of possible data loss when power is off during the reading and writing process through the strategy of writing first and then erasing.

[0006] To achieve the above-mentioned purpose, the present invention provides a storage method for non-volatile data on one hand, comprising: allocating at least two Flash pages for data storage; writing a basic information block at the head of each Flash page, the basic information block comprising a start mark, a page size, a delete mark, a current page swap count value, a reserved word and an end mark; each data object is associated with a key value information block, the key value information block comprising a start mark, a key value unique number, a length of an object data block, an address value for storing the object data block, a CRC16 check value of the object data block, a CRC16 check value of the key value information block, a delete mark and an end mark; the data blocks of the data object are stored starting from the end of the Flash page, and the key value information blocks are written sequentially from the starting position of the page.

[0007] Preferably, it also includes: when writing new data, determining the starting address of the new data according to the most recent data block address and the new data length; updating the relevant information in the key value information block, including the length, address value and CRC16 check value of the object data block.

[0008] Preferably, the method further includes: if there is an old version of the data, setting a deletion mark in the key value information block of the old version of the data to mark that the data has been deleted.

[0009] Preferably, the method further includes: when reading data of a specific key value, traversing all key value information blocks from the beginning, finding the last matching key value information block, parsing the length and address of the data therefrom, and reading the corresponding data object.

[0010] Preferably, the method further includes: when the current Flash page is full, all new key value information and data objects are moved to another free page, and then the old page is erased.

[0011] Preferably, the method further includes: adopting a write-first-erase-later strategy to ensure that data is not lost when a power outage occurs during the data migration process.

[0012] Another aspect of the present invention provides a non-volatile data storage system, comprising: At least two Flash pages for storing data objects and their corresponding key-value information blocks; The control module is used for performing storage of non-volatile data.

[0013] Preferably, the control module is also used to initialize the Flash page, set the basic information block and the key value information block, and manage the reading and writing operations of the data.

[0014] Preferably, the control module is further used to traverse the basic information blocks in all Flash page headers when the system is powered on to determine the available storage resources.

[0015] Preferably, the control module is further used to search for valid key-value information blocks and reallocate them to new page addresses during page data arrangement, while ignoring key-value information blocks marked for deletion, so as to achieve wear leveling.

[0016] Technical effects and advantages of the present invention: Compared with the prior art, the present invention provides a non-volatile data storage method and system, which has the following advantages: The present invention achieves high efficiency and security of dynamic data storage by allocating at least two Flash pages for data storage, writing a basic information block in the header of each Flash page, and associating a key value information block with each data object. The method can not only dynamically adjust the length of data entries, but also solve the problem of possible data loss when power is off during the reading and writing process through the strategy of writing first and then erasing, and at the same time, through the dynamic management of data entries and the reasonable scheduling between pages, the purpose of extending the service life of Flash is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 A flowchart of a method for storing non-volatile data of the present invention; Figure 2 The figure is a working flow chart of the non-volatile data storage system of the present invention. DETAILED DESCRIPTION

[0018] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. The specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0019] The present invention provides a non-volatile data storage method, which achieves high efficiency and security of dynamic data storage by allocating at least two Flash pages for data storage, writing a basic information block in the header of each Flash page, and associating a key value information block with each data object. The method can not only dynamically adjust the length of data entries, but also solve the problem of data loss when power is off during the reading and writing process through the strategy of writing first and then erasing. At the same time, through the dynamic management of data entries and the reasonable scheduling between pages, the purpose of extending the service life of Flash is achieved. The details are as follows: like Figure 1 As shown, a non-volatile data storage method in this embodiment includes the following steps: Allocate at least two Flash pages for data storage. By allocating multiple Flash pages, redundant data storage and wear leveling can be achieved to avoid wear caused by excessive use of a single page.

[0020] A basic information block is written at the head of each Flash page, and the basic information block includes a start flag, a page size, a delete flag, a current page swap count value, a reserved word, and an end flag; this information helps the system to quickly identify the page status and perform effective data management and page scheduling.

[0021] Each data object is associated with a key information block, which includes a start mark, a key unique number, the length of the object data block, the address value of the object data block, the CRC16 check value of the object data block, the CRC16 check value of the key information block, a deletion mark and an end mark; this design enables each data object to be tracked and managed independently, facilitating data retrieval and updating.

[0022] The data blocks of the data objects are stored from the end of the Flash page, and the key value information blocks are written sequentially from the beginning of the page. This layout reduces the impact of data writing on the stored data, and also facilitates data search and deletion operations.

[0023] In another embodiment, the method further includes: when writing new data, determining the starting address of the new data according to the most recent data block address and the new data length; and updating the relevant information in the key value information block, including the length, address value and CRC16 check value of the object data block. In this way, it is possible to ensure that the new data is correctly written to the unused space to avoid overwriting the existing data, and at the same time update the relevant information of the key value information block to maintain the consistency of the data.

[0024] In another embodiment, the method further includes: if there is an old version of data, setting a deletion mark in the key value information block of the old version of data to mark the data as deleted. This mechanism allows the system to skip the data marked as deleted in subsequent operations, thereby reducing unnecessary read operations and improving efficiency.

[0025] In another embodiment, when reading data of a specific key value, all key value information blocks are traversed from the beginning to find the last matching key value information block, the length and address of the data are parsed therefrom, and the corresponding data object is read. By traversing the key value information blocks from the beginning, the latest data version can be accurately located to ensure that the latest and valid data is read.

[0026] In another embodiment, when the current Flash page is full, all new key value information and data objects are moved to another free page, and then the old page is erased. In this way, the page cleaning work can be completed without affecting the existing data, avoiding the wear caused by frequent erasing.

[0027] In another embodiment, the method further includes: adopting a write-first-then-erase strategy to ensure that data will not be lost when a power outage occurs during the data migration process. During the data migration process, data is first written to a new page and then the old page is erased. Even if a power outage occurs during the writing process, data will not be lost, thereby enhancing data security.

[0028] In summary, the life of Flash is significantly extended by wear leveling and reducing unnecessary erase operations. The write-before-erase strategy ensures data integrity under any circumstances and avoids data loss caused by power failure. Reasonable data layout and management mechanism make data read and write operations more efficient and reduce invalid read and write operations. The design of key-value information blocks enables data objects to be tracked independently, facilitates data update and deletion, and improves the flexibility and maintainability of the system.

[0029] In this embodiment, a non-volatile data storage system is also provided, including: At least two Flash pages for storing data objects and their corresponding key-value information blocks; The control module is used to execute the above-mentioned non-volatile data storage method.

[0030] In another embodiment, the control module is further used to initialize the Flash page, set the basic information block and the key value information block, and manage the reading and writing operations of the data.

[0031] In another embodiment, the control module is further configured to traverse the basic information blocks in all Flash page headers when the system is powered on to determine available storage resources.

[0032] In another embodiment, the control module is further configured to search for valid key value information blocks and reallocate them to new page addresses during page data arrangement, while ignoring key value information blocks marked for deletion, so as to achieve wear leveling.

[0033] like Figure 2 As shown, the storage system works in the following way: The system must allocate more than 2 Flash pages to store data. Data is stored in Flash pages. When a page is full, the next free page is written, and 1 free page is reserved as a swap page when the data is full. In order to facilitate the search of data objects each time, a key value is assigned to each object. When initializing Flash, first erase the Flash page, and then write a basic information block at the head of the page.

[0034] The basic information block fields include: 1. Start flag, 2. Page size, 3. Delete flag, 4. Current page swap count value, 5. Reserved word, 6. End flag. Each time the system is powered on, it first traverses the basic information blocks of all Flash page headers.

[0035] The key value information block consists of the following fields: 1. Start mark, 2. Key value unique number, 3. Object data block length, 4. Address value storing object data block, 5. CRC16 check value of object data block, 7. CRC16 check value of key value information block, 8. Delete mark, 9. End mark. The key value information block is written sequentially from the beginning of the page (following the position after the basic information block). Each key value information block corresponds to an object data block. Object data blocks are stored starting from the end of the page.

[0036] When the system needs to write a new object data, the last object data block address is read from the previous key value information block as the end address, and the length of the current object data block is added to get the start address of the current object data block. Then the length information, address information, CRC16 checksum information, key value unique number, etc. of the object data are written to the new key value information block position. If there is an old version of the object data, the "delete flag" in the key value block corresponding to the old version of the object data is set to indicate that the old data has been deleted. (Using the characteristic that Flash can only change from 1 to 0 when writing).

[0037] Each time you need to read object data (referring to the unique number of key values), you will traverse all key value information blocks from the beginning to find the last key value corresponding to the "unique number of key values". Then, you will parse the length and address position of the object data from the key value information block, and then read the object data from the object data address.

[0038] When the Flash page is full, all new key values ​​and object data are moved to another Flash page first, and then the old Flash is erased. The write-first-erase mechanism ensures that no data is lost during the data page migration process.

[0039] This storage method has been verified on the Silicon Labs MCU platform. Because this method is optimized for the storage characteristics of NOR Flash (direct read and write operations can be performed by address, data bits can only change from 1 to 0, and erased by page), it theoretically supports all NOR Flash.

[0040] The present invention uses key values ​​to bind object data and introduces multiple Flash pages for data exchange, which can achieve wear balance of Flash pages and solve the problem of data loss due to power failure. Because NOR Flash needs to be erased by page, and the data bit can only be changed from 1 to 0 when writing. If the power is accidentally cut off during the page data exchange process, the data of the page will be lost because the data in the memory has not yet completed writing to the Flash.

[0041] This storage method uses a write-first-erase-later strategy to solve the problem of data loss caused by unexpected power outages during data operations. Because the number of page erases and writes of NOR Flash is effective, it is generally less than 100,000 times. In order to avoid long-term erase and write operations on fixed pages, the dirty data marking method is used. If a piece of object data needs to be changed or deleted, the page erase operation is not performed first. Instead, the "deletion mark" in the key value information block of the object data is set first, and then a new key value information block is written after the blank space of the last key value information block, and the new object data is written in the data area.

[0042] When a page is full, write the new key data block and object data block to another free page. When only the last free page is left, perform page data sorting. The process of page data sorting is to scan all non-empty pages, find valid key data blocks, and when the key data block is valid, reallocate the key data block to the object data block to the new page address. Then write the key data and object data to the new page address. For the key information block with the "deletion mark" set, ignore it directly. When all the valid data of the old page has been moved to the new page, erase the old page. Then repeat the process for the next page. Through this method, all Flash pages will be used evenly.

[0043] In addition, the present invention also provides a terminal device. A non-volatile data storage method involved in this embodiment is mainly applied to the terminal device, which can be a PC, a portable computer, a mobile terminal, or other device with display and processing functions.

[0044] Specifically, the terminal device may include a processor (such as a CPU), a communication bus, a user interface, a network interface, and a memory. The communication bus is used to realize the connection and communication between these components; the user interface may include a display screen (Display), an input unit such as a keyboard (Keyboard); the network interface may optionally include a standard wired interface, a wireless interface (such as a WI-FI interface); the memory may be a high-speed RAM memory, or a stable memory (non-volatile memory), such as a disk memory, and the memory may optionally be a storage device independent of the aforementioned processor.

[0045] Among them, the memory stores a readable storage medium, and the readable storage medium stores a non-volatile data storage program. The processor can call the non-volatile data storage program stored in the memory and execute a non-volatile data storage method provided by an embodiment of the present invention.

[0046] It is understood that a readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer-readable storage media can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples of computer-readable storage media (a non-exhaustive list) include: portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), static random access memories (SRAM), portable compact disk read-only memories (CD-ROM), digital versatile disks (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or raised structures in grooves on which instructions are stored, and any suitable combination of the above. The computer-readable storage medium used here is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0047] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0048] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0049] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for storing non-volatile data, characterized in that: include: Allocate at least two Flash pages for data storage; Write a basic information block at the head of each Flash page, wherein the basic information block includes a start flag, a page size, a delete flag, a current page swap count value, a reserved word, and an end flag; Each data object is associated with a key information block, wherein the key information block includes a start mark, a key unique number, the length of the object data block, an address value storing the object data block, a CRC16 check value of the object data block, a CRC16 check value of the key information block, a delete mark, and an end mark; The data blocks of the data object are stored starting from the end of the Flash page, and the key value information blocks are written sequentially from the beginning of the page.

2. A method for storing non-volatile data according to claim 1, characterized in that: Also includes: When writing new data, the starting address of the new data is determined based on the most recent data block address and the length of the new data; Update the relevant information in the key value information block, including the length, address value and CRC16 check value of the object data block.

3. A method for storing non-volatile data according to claim 1, characterized in that: Also includes: If an old version of the data exists, a deletion mark in the key value information block of the old version of the data is set to mark that the data has been deleted.

4. The method for storing non-volatile data according to claim 1, characterized in that: Also includes: When reading data of a specific key value, traverse all key value information blocks from the beginning, find the last matching key value information block, parse the length and address of the data from it, and read the corresponding data object.

5. The method for storing non-volatile data according to claim 1, characterized in that: Also includes: When the current Flash page is full, all new key value information and data objects are moved to another free page, and then the old page is erased.

6. The method for storing non-volatile data according to claim 1, characterized in that: Also includes: The write-before-erase strategy is adopted to ensure that data will not be lost in the event of a power outage during data migration.

7. A non-volatile data storage system, characterized in that: include: At least two Flash pages for storing data objects and their corresponding key-value information blocks; The control module is used for performing storage of non-volatile data.

8. A non-volatile data storage system according to claim 7, characterized in that: The control module is also used to initialize the Flash page, set the basic information block and the key value information block, and manage the reading and writing operations of the data.

9. The non-volatile data storage system according to claim 7, characterized in that: The control module is also used to traverse the basic information blocks in all Flash page headers when the system is powered on to determine the available storage resources.

10. The non-volatile data storage system according to claim 7, characterized in that: The control module is also used to find valid key value information blocks and reallocate them to new page addresses during page data arrangement, while ignoring key value information blocks marked for deletion, so as to achieve wear leveling.

Images