Packet data persistence and updating method based on nonvolatile memory

Abstract

The invention discloses a grouped data persistence and updating method based on a nonvolatile memory, which is suitable for a storage management scene in which local data needs to be frequently updated, and aims at optimizing the data updating efficiency and prolonging the service life of the memory. The invention provides a grouped data persistence and updating method based on additional writing, aiming at solving the problem that the service life of a traditional nonvolatile memory is shortened due to block erasure when the traditional nonvolatile memory is used for processing frequent updating of data. According to the method, only the changed data in the written grouped data is added, so that the whole storage block is prevented from being repeatedly erased and written, the abrasion of the nonvolatile memory is effectively reduced, and the storage efficiency is improved. Meanwhile, according to the method, a power failure protection function is realized, a wear leveling function is realized through a delay memory recovery strategy, and the service life of the memory is further prolonged.

Description

Technical Field

[0001] The present invention belongs to the technical field of information storage, and in particular relates to a method for persisting and updating packet data based on a non-volatile memory. Background Art

[0002] In existing data storage methods based on non-volatile memory, when a certain data needs to be updated, if the memory supports byte-by-byte erasing (such as EEPROM), the original data location is modified directly; if the memory does not support byte-level erasing (such as FLASH), the entire page of content containing the new data is written to a new storage location. In scenarios where data is frequently updated, this method will increase the number of times the memory is erased and written, affecting the service life of the memory, and the write amplification effect will also increase the time it takes to save data. Summary of the invention

[0003] The main purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and provide a method for persisting and updating packet data based on non-volatile memory. When a certain data item in the packet data needs to be updated, the method appends the updated item of the data item to the end of the packet data in the form of a data record, rather than overwriting the data at the original position; when reading data, the records are read forward one by one from the end of the packet data, and the latest data record is returned. When storing packet data, the method can save data at the granularity of independent data records. In this way, the number of times non-volatile memories such as FLASH are erased can be effectively reduced, thereby extending the life of the memory, and significantly saving time when saving updated data.

[0004] In order to achieve the above object, the present invention adopts the following technical solutions: A method for persisting and updating packet data based on a non-volatile memory, wherein the packet data is encapsulated into a data record in the form of a key-value pair, wherein the data record is a basic unit for persisting the packet data in the non-volatile memory and is updated or read separately; the non-volatile memory is divided into a super block and a data block, wherein the super block includes at least two continuous storage blocks; the data block includes a metadata block, M partition data blocks and M partition additional blocks, which are used to store M groups of packet data, wherein M is the number of partitions, and M≥1; the super block, the metadata block and the partition additional block respectively store different management data, and the management data are all stored in the form of management records; each storage block in the super block is a super block, which is used to store the management record of the storage block address of the metadata block; the metadata The block is used to store management records of metadata, and the metadata includes a partition metadata table and a bitmap; the partition metadata table includes partition metadata of all partitions, and the partition metadata includes feature information, a starting address, an allocated space length, and an additional block address of the partition; the bitmap includes the status of all storage blocks in the data block area, which is used to implement the wear leveling function of the memory; the status of the storage blocks in the data block area includes free, in use, to be recycled, and damaged; every 2 bits in the bitmap correspond to the status of a storage block, and the storage block is the minimum erase granularity unit of the non-volatile memory; the partition additional block stores the management record of the written space length of the partition data block, which is used to locate the end of the written data of the partition data block when writing or reading the data record; the packet data persistence and update method includes the following steps: A partition data block and a corresponding partition additional block store a group of grouped data, each partition data block includes an integer number of storage blocks, each data record occupies an integer number of continuous programming blocks, and the programming block is the minimum programming granularity unit of the non-volatile memory; when updating the data record of the grouped data, an additional writing method is adopted at the end of the written space, including writing the key and value of the data record; if the grouped data is saved for the first time, step S201 is executed, otherwise the grouped data is updated and step S202 is executed; S201, the process of saving the group data for the first time is: S2011, allocating an empty item from the partition metadata table, the empty item being the partition metadata whose partition characteristic information is 0, and updating the characteristic information of the empty item to the characteristic information of the grouped data; S2012, applying for allocation of new partition data blocks; S2013, writing all the data in the grouped data into the data area of ​​the partitioned data block one by one in the form of data records; S2014, allocating a free storage block in the data block area as a partition additional block for the grouped data, and updating the written space length record of the corresponding partition data block; S2015, updating the allocated partition metadata, and updating the metadata record in the metadata block, and the saving operation ends; S202: If it is not the first time to save the group data, the process of updating the group data is as follows: S2021, locating the next programming block at the end of the valid data in the data area of ​​the partition data block according to the length of the written space of the corresponding partition additional block; S2022, check whether the data of the next programming block at the end of the valid data is the erased value of the non-volatile memory and calculate whether the remaining space in the data area of ​​the partition data block is sufficient to append and write new data records. If the data of the next programming block at the end of the valid data is the erased value of the non-volatile memory and the remaining space is sufficient, write new data records one by one from the next programming block at the end of the valid data, and update the written space length record in the corresponding partition additional block, and the update operation ends; otherwise, apply to allocate a new partition data block and write all the latest data records of the current group data in the data area; the update operation ends; When reading data records of grouped data, search for data records from the end of the written data forward, first determine whether the key of the data record matches, and if it matches, return the value corresponding to the key, otherwise continue searching forward; since each updated data record is appended to the end of the metadata, the first data record matched from the back to the front is the latest data record corresponding to the key.

[0005] Furthermore, each storage block in the super block area is a super block, which are respectively the first super block, the second super block, ..., the Nth super block according to the memory address from low to high; In the process of storage update, in order to ensure the consistency of data, the version number is stored in each super block, and the maximum version number is set as the valid super block to ensure the orderly update of data. v The programming blocks store the version number, and the remaining programming blocks store management records, where N v is the number of programming blocks occupied by the version number. The management record in the super block is a metadata block address record, which stores the storage block address of the metadata block; The super block with the largest version number in the super block area is a valid super block, and the rest are invalid super blocks; Super blocks are used cyclically. When a valid super block is full, the next adjacent super block is taken as the new valid super block. The version number of the new valid super block is the version number of the old valid super block plus 1. If the current super block is the Nth super block, the next adjacent super block is the 1st super block. The cyclic use of multiple super blocks can prevent super blocks from being over-erased and realize the wear leveling function of the super block area. Further, the management data in the super block, metadata block and partition additional block are stored in the form of management records, the management record includes two status fields and management data, each status field occupies one programming block, and each management record occupies an integer number of consecutive programming blocks; When updating the management record, an append write method is adopted. During the writing process, the first status field is first written as the pending write state, and then the management data part of the record is written. After the management data is written, the second status field is written as the completed state. When updating the management record, the second status field of each management record is read from the front to the back. When the second status field of the first management record is not a management record in the completed state, the first status field of the management record is read to check whether the first status field is the erased value of the non-volatile memory and calculate whether the remaining space of the storage block is sufficient to append the new management record. If so, the new management record is appended and written from the address of the first status field. Otherwise, Then, if the new management record is a metadata block address record in a super block, a new valid super block is selected, the new valid super block is erased, the new management record is written into the new valid super block, and the version number of the new valid super block is updated; if the new management record is a management record in a metadata block, an application is made in the data block area to allocate a new storage block and write the new management record, and the metadata block address record in the valid super block is updated; if the new management record is a management record in a partition additional block, an application is made in the data block area to allocate a new storage block and write the new management record, and the metadata management record in the metadata block is updated; through the two status fields of the management record, the atomic operation of updating the management record can be realized, and the power-off protection function can be realized; When reading the latest management record, the second status field of each management record is read from the front to the back. When the second status field of the last management record is a management record in the completed state, the management data of the record is returned.

[0006] Furthermore, when the packet data persistence and update method is first executed, an initialization operation needs to be performed to ensure that after the non-volatile memory is powered on, a valid block can be selected according to the version number of the super block, so as to prepare for subsequent data read and write operations. The initialization process is as follows: S101, the non-volatile memory is powered on, and the version numbers of the first super block and the second super block are read; S102, if the version numbers of the first and second super blocks are both non-volatile memory erase values, indicating that the super block and the metadata block have not been initialized, the first super block is selected as the valid super block, a storage block in the data block area is allocated as the metadata block, the partition metadata table and the bitmap are initialized, the partition feature information, the starting address, the length of the allocated space, and the partition additional block address of all partition metadata in the partition metadata table are set to 0, the bitmap status corresponding to the metadata block in the data block area is set to be in use, the bitmap status of other storage blocks is set to be idle, the initialized metadata management record is written into the metadata block, and then the metadata block address record is written into the valid super block, and the version number of the valid super block is set to 1; otherwise, S103 is executed; S103, if only the version number of the first super block is the erased value of the non-volatile memory, the effective super block is the Nth super block; otherwise, execute S104; S104, read the version number of the first super block, and then read the version numbers of the second, third, ..., Nth super blocks one by one. If the version number of a super block read is not equal to the version number of the previous super block plus one, the valid super block is the previous super block; if the version numbers of the second super block to the Nth super block are all equal to the version number of the previous super block plus one, the valid super block is the Nth super block; when the valid super block is confirmed, it means that a valid metadata block already exists in the data block area.

[0007] Furthermore, the partition data block is used to store the corresponding grouped data. The storage blocks in the partition data block are linked in a linked list form to effectively avoid the storage fragmentation problem. Each storage block in the partition data block consists of a forward link address, a backward link address and a data area, wherein the forward link address and the backward link address occupy N of the storage block respectively. a Programming blocks, where N a It is the number of programming blocks occupied by the link address, the forward link address stores the address of the previous storage block in the partition data block, the backward link address stores the address of the next storage block in the partition data block, the forward link address and the backward link address constitute a bidirectional linked list of multiple storage blocks, the forward link address of the first storage block in the partition data block and the backward link address of the last storage block are the erase values ​​of the non-volatile memory; the data area is used to store data records of the packet data.

[0008] Furthermore, when reading data records of grouped data, the written space length of the corresponding partition additional block is used to locate the end of the valid data in the data area of ​​the partition data block, and the key of each data record in the data area is read forward from the end address of the valid data. When the first matching key is read, the value corresponding to the key is returned. This method utilizes the existing valid data length and storage order, reduces the scanning of invalid data, avoids the inefficient operation of comparing each item from the beginning, and significantly optimizes the query performance, especially when the amount of data is large.

[0009] Furthermore, in order to realize the power-off protection function of the non-volatile memory, when the packet data needs to be updated, the partition data block is updated first, and then the partition additional block, metadata block, and super block are updated in sequence; when reading the data record of the packet data, the super block is read first, and then the metadata block, partition additional block, and partition data block are read in sequence; When a power outage occurs during the packet data update process, since the update of the partition additional block, metadata block or super block is not completed, the non-volatile memory will start from the super block and read and restore the valid data of the most recently valid metadata block, partition additional block, and partition data block in sequence when it is powered on next time; in this way, the unfinished operations before the power outage can be canceled and the valid data before the power outage can be restored.

[0010] Furthermore, in order to extend the service life of the non-volatile memory, the group data persistence and update method uses a delayed memory recovery strategy to implement the wear leveling function of the non-volatile memory, so that the number of erasures and writes of each storage block in the data area is more balanced; the process of allocating storage blocks in the data block area to metadata blocks, partition data blocks or partition additional blocks is: S301, search the bitmap to see if there are enough free storage blocks; if there are enough free storage blocks, execute step S302. Otherwise, execute step S305; S302, erasing a required number of idle storage blocks; S303, for the storage block of the partitioned data block, update the forward link address and the backward link address of the storage block to form a bidirectional linked list of multiple storage blocks; if the write-after-read check fails when updating the link address, update the corresponding storage block status to damaged, and re-execute step S301; S304, updating the bitmap, changing the status of the corresponding newly allocated storage block from free to in use, and changing the status of the old storage block from in use to pending recycling, and the allocation process ends; S305, uniformly change the states of all storage blocks to be recycled to the idle state, but do not perform a uniform erasing operation; search from the bitmap whether there are enough idle storage blocks; if there are enough idle storage blocks, execute S302; otherwise, prompt insufficient memory and end the allocation process.

[0011] Furthermore, when writing a record to a storage block in the data block area, if the read-after-write verification fails, the management record in the metadata block is updated, and the status of the corresponding storage block in the bitmap is marked as damaged, so as to avoid reallocation of the damaged storage block during subsequent use, and apply for a new storage block in the data block area to complete the write operation.

[0012] Compared with the prior art, the present invention has the following advantages and beneficial effects: (1) The present invention proposes a method for persisting and updating packet data based on a non-volatile memory. By appending the updated data to the end of the original data instead of overwriting the original data, the number of memory erase operations is effectively reduced, thereby extending the service life of the memory.

[0013] (2) The grouped data persistence and update method based on non-volatile memory proposed in the present invention only needs to append the updated data during the update, which reduces the storage time for each data update and improves the efficiency of data storage operations. It is particularly suitable for scenarios where small amounts of data need to be updated frequently.

[0014] (3) The packet data persistence and update method based on non-volatile memory proposed in the present invention supports power-off protection by checking the status field, and supports wear leveling by delaying memory recycling strategy, thereby extending the service life of the memory. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0016] Figure 1 It is a structural diagram of an object storage area of ​​a packet data persistence and update method based on a non-volatile memory disclosed in the present invention; Figure 2 It is a super block structure diagram of the packet data persistence and update method based on non-volatile memory disclosed in the present invention; Figure 3 It is a partition data block structure diagram of the grouped data persistence and update method based on non-volatile memory disclosed in the present invention; Figure 4 It is a metadata block management record structure diagram of the packet data persistence and update method based on non-volatile memory disclosed in the present invention; Figure 5 is a flowchart of saving object attribute records in Example 1 of the present invention; Figure 6 This is a comparison chart of the number of memory blocks required for writing complete object data and additionally writing updated attribute data when object data is updated in Example 1 of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0018] Example 1 This embodiment discloses an object persistence and update method of an object-oriented programming language based on a non-volatile memory. For an object-oriented programming language, the data storage function provided by it can divide all the data to be stored into multiple groups of grouped data in units of objects, and encapsulate each attribute field of the object into a data record; in the non-volatile memory, the data record is persistently stored in the form of a key-value pair, and is appended, updated or read separately; in this embodiment, the on-chip Flash of STM32F103ZET6 is used as an example of a non-volatile memory, but the present invention does not limit the non-volatile memory; the minimum erase granularity unit of the on-chip Flash main memory of STM32F103ZET6 is 2K bytes, the minimum programming granularity unit is 2 bytes, and 64K bytes are allocated as the object storage area, the storage block is 2K bytes, and the programming block is 2 bytes; the object persistence and update method includes: The object storage area is divided into a super block area and a data block area; The super block area includes 2 consecutive storage blocks; the data block area includes a metadata block, M partition data blocks and M partition additional blocks, the partition data block is used as an object data block, and the partition additional block is used as an object additional block; an object data block and a corresponding object additional block are used to store data of an object; in this embodiment, M is 6, supporting the storage of up to 6 object data; The management data in the superblock, metadata block and object additional block are all stored in the form of management records, which include 2 status fields and management data. Each status field occupies one programming block. The records in the superblock and object additional block occupy 4 consecutive programming blocks, and the records in the metadata block occupy 48 consecutive programming blocks. Each storage block in the super block area is a super block, which is marked as the first super block and the second super block from low to high addresses; each super block starts with N v The programming blocks store the version number of the block, and the remaining locations store metadata block address records; in this embodiment, N v Take 2, and N takes 2; the super block with the largest version number in the super block area is the valid super block, and the rest are invalid super blocks; super blocks are used cyclically, and when a valid super block is full, another super block is taken as the new valid super block, and the version number of the new valid super block is the version number of the old valid super block plus 1.

[0019] The metadata block includes a partition metadata table and a bitmap. The partition metadata table is used as an object metadata table. The object metadata table includes object metadata of all objects. The object metadata includes object identifier, object data start address, object allocation space length, and object additional block address. The bitmap includes the status of all storage blocks in the data block area. The status of storage blocks in the data block area includes four types: free, in use, to be recycled, and damaged. The object data block includes at least one storage block. Each storage block in the object data block consists of a forward link address, a backward link address and a data area. In this embodiment, the number of programming blocks occupied by the link address is N. a Take 1, the forward link address and the backward link address are located in the first programming block and the second programming block of the storage block respectively, the forward link address stores the address of the previous storage block in the object data block, and the backward link address stores the address of the next storage block in the object data block, forming a bidirectional linked list of multiple storage blocks; the forward link address of the first storage block in the object data block and the backward link address of the last storage block are the Flash erase value 0xFFFF; the data area is used to store object data records, and each attribute of the object corresponds to a data record; The object additional block is used to store the written space length record of the data area in the object data block where the object is located; Initialization steps: In this embodiment, when the Flash is powered on and initialized, in order to improve the program execution efficiency, the object metadata table and the bitmap are read into the RAM cache, and then written back to the Flash after the reading or saving operation is completed; the process is as follows: T21, initialize the valid super block; T211, read the version numbers of the first super block and the second super block in the super block area; T212. If the version numbers of the first and second super blocks are both the Flash erase value 0xFFFFFFFF, the first super block is selected as the valid super block, and the metadata block needs to be initialized, and step T22 is executed; otherwise, step T213 is executed; T213. If only the version number of the first super block is the Flash erase value 0xFFFFFFFF, the valid super block is the second super block, and step T23 is executed; otherwise, step T214 is executed; T214, read the version number of the first super block and the version number of the second super block. If the version number of the second super block is not equal to the version number of the previous super block plus one, the valid super block is the first super block, and step T23 is executed; if the second super block is equal to the version number of the previous super block plus one, the valid super block is the second super block, and step T23 is executed; T22, initialize metadata block; T221, allocate a storage block in the data block area as a metadata block, initialize the object metadata table and bitmap in the RAM cache, set the object identifier, object data start address, object allocation space length, and object additional block address to 0, set the bitmap status corresponding to the metadata block in the data block area to in use, and set the bitmap status of other storage blocks to free; T222, write the object metadata table and bitmap in the RAM cache into the metadata block in the form of metadata management records, then write the address record of the metadata block into the first super block, and write the version number of the first super block to 1, and the initialization operation ends; T23, according to the metadata block address record in the valid super block, locate the storage block where the metadata block is located, read the object metadata table and bitmap of the metadata block into the RAM cache, and the initialization operation ends; The process of saving object data is as follows Figure 5 When saving object properties, if it is the first time to save the object, execute step T31; otherwise, if the object properties need to be updated, execute step T32; T31. The steps for saving object data for the first time are: T311. Allocate an empty item from the object metadata table. The empty item is the object metadata whose object ID is 0. Update the object ID of the empty item to the ID of the object. T312, apply for allocation of new object data block, allocate 3 free storage blocks, write the forward link addresses of the allocated storage blocks except the first storage block as the address of the previous storage block, write the backward link addresses of the allocated storage blocks except the last storage block as the address of the next storage block, and mark all the corresponding storage blocks of the bitmap in the RAM cache as being in use; T313. Serialize all attributes in the object, using the attribute name as the key and the attribute value as the value, and fill the serialized data of each attribute to an integer multiple of the programming block length. In this embodiment, the data record is filled to 16 bytes, and the data records are written one by one into the data area of ​​the object data block; T314, apply for allocation of a new object additional block, allocate 1 free storage block, and update the written space length record of the corresponding object data block; T315. Update the allocated object metadata, and update the metadata record in the RAM cache. Append the latest metadata record in the RAM cache to the metadata block. The first save operation of the object ends. T32. If it is not the first time to save the object data, append the updated object attributes to the end of the valid data of the original object. The steps are as follows: T321. Serialize the updated object attributes, using the attribute name as the key and the attribute value as the value, and fill the serialized data of each attribute to an integer multiple of the programming block length. In this embodiment, the data record is filled to 16 bytes; T322, locate the next programming block after the valid data in the data area of ​​the object data block through the written space length of the corresponding object additional block, check whether the data of the next programming block after the valid data is the Flash erase value 0xFFFF, and calculate whether the remaining space in the data area of ​​the object data block is sufficient to append new data records. If the data of the next programming block after the valid data is 0xFFFF and the remaining space is sufficient, write the updated data records one by one from the next programming block after the valid data, and update the written space length record in the corresponding object additional block, and the update operation ends; otherwise, save all current attributes of the object according to steps T312 to T314, update the object metadata, append the latest metadata in the RAM cache to the metadata block, and the save operation ends; The above-mentioned method of appending object update data effectively reduces the number of memory block applications compared to writing the entire object data, such as Figure 6 ; When reading object attributes, search and read from the back to the front in the data area of ​​the object data block. The specific steps are: T41. Locate the corresponding object additional block in the RAM cache by using the object metadata, locate the end of the valid data in the data area of ​​the object data block by using the written space length of the corresponding object additional block, and read the data record by shifting forward 16 bytes from the end address of the valid data in the data area; T42, read the attribute key, determine whether it matches the key of the attribute to be read, if it matches, read the attribute value and return it; if it does not match, read the previous data record by 16 bytes forward, and repeat T42; The update steps of management records are as follows: Read the second status field of each management record from front to back. When the first management record whose second status field is not in the completed state is read, read the first status field of the management record, check whether the first status field is the Flash erase value 0xFFFF and calculate whether the remaining space of the storage block is sufficient to append the new management record. If so, append the new management record from the address of the first status field; otherwise, if the new management record is a metadata block address record in the super block, select another super block as the new valid super block, erase the new valid super block, and write the new management record in the 3rd programming block of the new valid super block. The super block management record is updated by writing the value of the current super block version number plus 1 into the 1st and 2nd programming blocks of the new valid super block, and the super block management record update operation is completed; if the new management record is a management record in the metadata block, an application is made in the data block area to allocate a new storage block and write the new management record, and the metadata block address record in the valid super block is updated, and the metadata block management record update operation is completed; if the new management record is a management record in the object additional block, an application is made in the data block area to allocate a new object additional block, and the new management record is written in the new storage block, and the metadata management record in the metadata block is updated, and the object additional block management record update operation is completed.

[0020] The steps for reading management records are as follows: The second status field of each management record is read from the front to the back. When the last management record whose second status field is in the completed state is read, the management data of the record is returned.

[0021] When the object attribute data needs to be updated, the object data block is updated first, and then the object additional block, metadata block, and super block are updated in sequence; when reading the data record of the object attribute data, the super block is read first, and then the metadata block, object additional block, and object data block are read in sequence; when a power outage occurs during the object attribute data update process, since the update of the object additional block, metadata block, or super block is not completed, the non-volatile memory starts from the super block when it is powered on next time, and reads and restores the valid data of the most recently valid metadata block, object additional block, and object data block in sequence.

[0022] The steps of allocating a storage block in the data block area to a metadata block, an object data block or an object additional block are as follows: T81, search the bitmap to see if there are enough free storage blocks; if there are enough free storage blocks, execute T82; otherwise, execute T85; T82, erasing a required number of storage blocks in idle state; T83, for the storage blocks of the object data block, write the forward link addresses of the remaining storage blocks except the first storage block in the allocated storage blocks as the address of the previous storage block, and write the backward link addresses of the remaining storage blocks except the last storage block in the allocated storage blocks as the address of the next storage block, so as to form a bidirectional linked list of multiple storage blocks; if the read-after-write check fails when updating the link address, update the corresponding storage block status to damaged, and re-execute T81; T84, update the bitmap, change the status of the corresponding newly allocated storage block from free to in use, and change the status of the old storage block from in use to pending recycling, and the allocation process ends; T85. Change the states of all storage blocks to be recycled to free states, but do not perform a unified erase operation; search the bitmap to see if there are enough free storage blocks; if there are enough free storage blocks, execute T82; otherwise, prompt that there is insufficient memory and the allocation process ends.

[0023] The memory delayed recycling strategy used in the above step T84 changes the status of the old storage block from being in use to being recycled instead of being erased and reused immediately, thereby achieving a better wear leveling effect; Table 1 shows the storage block erasure count data obtained by updating the object attribute data 200 times using the delayed recycling and random window strategies for the storage blocks in this embodiment, wherein the random window strategy is the wear leveling strategy used by the embedded file system Littlfs.

[0024] Table 1. Data table of storage block erase times using two wear leveling strategies: delayed recycling and sliding window

[0025] When writing a record to a storage block in the data block area, if the read-after-write check fails, the management record in the metadata block is updated, the status of the corresponding storage block in the bitmap is marked as damaged, and a new storage block is requested in the data block area to complete the write operation.

[0026] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for persisting and updating packet data based on non-volatile memory, characterized in that: The packet data is encapsulated into a data record in the form of a key-value pair, wherein the data record is a basic unit of persistent storage of the packet data in a non-volatile memory, and is updated or read separately; the non-volatile memory is divided into a super block and a data block, wherein the super block includes at least two consecutive storage blocks; the data block includes a metadata block, M partition data blocks and M partition additional blocks, which are used to store M groups of packet data, wherein M is the number of partitions, and M≥1; the super block, the metadata block and the partition additional block respectively store different management data, and the management data are all stored in the form of management records; each storage block in the super block is a super block, which is used to store the management record of the storage block address of the metadata block; the metadata block is used to store the management record of the metadata The metadata includes a partition metadata table and a bitmap; the partition metadata table includes partition metadata of all partitions, and the partition metadata includes feature information, a starting address, an allocated space length, and an additional block address of the partition; the bitmap records the status of all storage blocks in the data block area, and is used to implement the wear leveling function of the memory; the status of the storage blocks in the data block area includes free, in use, to be recycled, and damaged; every 2 bits in the bitmap correspond to the status of a storage block, and the storage block is the minimum erase granularity unit of the non-volatile memory; the partition additional block stores the management record of the written space length of the partition data block, and is used to locate the end of the written data of the partition data block when writing or reading the data record; the packet data persistence and update method includes the following steps: A partition data block and a corresponding partition additional block store a group of grouped data, each partition data block includes an integer number of storage blocks, each data record occupies an integer number of continuous programming blocks, and the programming block is the minimum programming granularity unit of the non-volatile memory; When updating the data record of the grouped data, an additional writing method is adopted at the end of the written space, including writing the key and value of the data record; if the grouped data is saved for the first time, step S201 is executed, otherwise the grouped data is updated and step S202 is executed; S201, the process of saving the group data for the first time is: S2011, allocating an empty item from the partition metadata table, the empty item being the partition metadata whose partition characteristic information is 0, and updating the characteristic information of the empty item to the characteristic information of the grouped data; S2012, applying for allocation of new partition data blocks; S2013, writing all the data in the grouped data into the data area of ​​the partitioned data block one by one in the form of data records; S2014, allocating a free storage block in the data block area as a partition additional block for the grouped data, and updating the written space length record of the corresponding partition data block; S2015, updating the allocated partition metadata, and updating the metadata record in the metadata block, and the saving operation ends; S202: If it is not the first time to save the group data, the process of updating the group data is as follows: S2021, locating the next programming block at the end of the valid data in the data area of ​​the partition data block according to the length of the written space of the corresponding partition additional block; S2022, check whether the data of the next programming block at the end of the valid data is the erased value of the non-volatile memory and calculate whether the remaining space in the data area of ​​the partition data block is sufficient to append and write new data records. If the data of the next programming block at the end of the valid data is the erased value of the non-volatile memory and the remaining space is sufficient, write new data records one by one from the next programming block at the end of the valid data, and update the written space length record in the corresponding partition additional block, and the update operation ends; otherwise, apply to allocate a new partition data block and write all the latest data records of the current group data in the data area; the update operation ends; When reading data records of grouped data, locate the end of valid data in the data area of ​​the partition data block through the length of written space of the corresponding partition additional block, and search for data records from the end of the valid data forward. First, determine whether the key of the data record matches. If it matches, return the value corresponding to the key, otherwise continue to search forward; the first data record matched from the back to the front is the latest data record corresponding to the key.

2. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: Each storage block in the super block area is a super block, and the super blocks are respectively the first super block, the second super block, ..., the Nth super block from low to high according to the memory address; The first to the Nth block in the super block v The programming blocks store the version number, and the remaining programming blocks store management records, where N v is the number of programming blocks occupied by the version number. The management record in the super block is a metadata block address record, which stores the storage block address of the metadata block; The super block with the largest version number in the super block area is a valid super block, and the rest are invalid super blocks; Super blocks are used cyclically. When a valid super block is full, the next adjacent super block is taken as the new valid super block. The version number of the new valid super block is the version number of the old valid super block plus 1. If the current super block is the Nth super block, the next adjacent super block is the 1st super block.

3. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: The management data in the super block, metadata block and partition additional block are stored in the form of management records, which include two status fields and management data, each status field occupies one programming block, and each management record occupies an integer number of consecutive programming blocks; When updating the management record, an append write method is adopted. During the writing process, the first status field is first written as a pending write state, and then the management data part of the record is written. After the management data is written, the second status field is written as a completed state; when updating the management record, the second status field of each management record is read from front to back. When a management record whose second status field is not in the completed state is read in the first management record, the first status field of the management record is read, and it is checked whether the first status field is an erase value of the non-volatile memory and whether the remaining space of the storage block is sufficient to append the new management record. If so, the new management record is appended from the address of the first status field. Otherwise, if the new management record is a metadata block address record in a super block, a new valid super block is selected, the new valid super block is erased, the new management record is written into the new valid super block, and the version number of the new valid super block is updated; if the new management record is a management record in a metadata block, an application is made in the data block area to allocate a new storage block to write the new management record, and the metadata block address record in the valid super block is updated; If the new management record is a management record in the partition additional block, then apply for allocation of a new storage block in the data block area to write the new management record, and update the metadata management record in the metadata block; When reading the latest management record, the second status field of each management record is read from the front to the back. When the second status field of the last management record is a management record in the completed state, the management data of the record is returned.

4. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: When the group data persistence and update method is executed for the first time, an initialization operation needs to be performed, and the process is as follows: S101, the non-volatile memory is powered on, and the version numbers of the first super block and the second super block are read; S102, if the version numbers of the first and second super blocks are both non-volatile memory erase values, then select the first super block as the valid super block, allocate a storage block in the data block area as the metadata block, initialize the partition metadata table and bitmap, set the partition feature information, start address, allocation space length, and partition additional block address of all partition metadata in the partition metadata table to 0, set the bitmap status corresponding to the metadata block in the data block area to be in use, set the bitmap status of other storage blocks to be idle, write the initialized metadata management record into the metadata block, then write the metadata block address record into the valid super block, and set the version number of the valid super block to 1; Otherwise, execute S103; S103, if only the version number of the first super block is the erased value of the non-volatile memory, the effective super block is the Nth super block; otherwise, execute S104; S104, read the version number of the first super block, and then read the version numbers of the second, third, ..., Nth super blocks one by one. If the version number of a super block read is not equal to the version number of the previous super block plus one, the effective super block is the previous super block; If the version numbers of the 2nd to Nth super blocks are all equal to the version number of the previous super block plus one, then the valid super block is the Nth super block; when the valid super block is confirmed, it means that there is a valid metadata block in the data block area.

5. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: The partition data block is used to store the corresponding grouped data. Each storage block in the partition data block consists of a forward link address, a backward link address and a data area, wherein the forward link address and the backward link address occupy N of the storage block respectively. a Programming blocks, where N a It is the number of programming blocks occupied by the link address, the forward link address stores the address of the previous storage block in the partition data block, the backward link address stores the address of the next storage block in the partition data block, the forward link address and the backward link address constitute a bidirectional linked list of multiple storage blocks, the forward link address of the first storage block in the partition data block and the backward link address of the last storage block are the erase values ​​of the non-volatile memory; the data area is used to store data records of the packet data.

6. The method for persisting and updating packet data based on non-volatile memory according to any one of claims 1 to 5, characterized in that: When the group data needs to be updated, the partition data block is updated first, and then the partition additional block, metadata block, and super block are updated in sequence; when reading the data record of the group data, the super block is read first, and then the metadata block, partition additional block, and partition data block are read in sequence; When a power outage occurs during the packet data update process, since the update of the partition additional block, metadata block or super block is not completed, the non-volatile memory will read and restore the valid data of the most recent valid metadata block, partition additional block, and partition data block in sequence, starting from the super block, when it is powered on next time.

7. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: In the group data persistence and update method, the process of allocating storage blocks in the data block area to metadata blocks, partition data blocks or partition additional blocks is as follows: S301, searching from the bitmap whether there are enough free storage blocks; if there are enough free storage blocks, executing step S302, otherwise executing step S305; S302, erasing a required number of idle storage blocks; S303, for the storage block of the partitioned data block, update the forward link address and the backward link address of the storage block to form a bidirectional linked list of multiple storage blocks; if the write-after-read check fails when updating the link address, update the corresponding storage block status to damaged, and re-execute step S301; S304, updating the bitmap, changing the status of the corresponding newly allocated storage block from free to in use, and changing the status of the old storage block from in use to pending recycling, and the allocation process ends; S305, uniformly change the states of all storage blocks to be recycled to the idle state, but do not perform a uniform erasing operation; search from the bitmap whether there are enough idle storage blocks; if there are enough idle storage blocks, execute S302; otherwise, prompt insufficient memory and end the allocation process.

8. The method for persisting and updating packet data based on non-volatile memory according to claim 1, characterized in that: When writing a record to a storage block in the data block area, if the read-after-write check fails, the management record in the metadata block is updated, the status of the corresponding storage block in the bitmap is marked as damaged, and a new storage block is requested in the data block area to complete the write operation.

Images