Metadata control method, apparatus and electronic device

Abstract

This application provides a metadata control method, apparatus, and electronic device. The method includes: responding to a metadata change request, writing specified metadata into a target metadata page in a target cache area, and setting a target data bit in a dirty page bitmap corresponding to the target metadata page; and performing a metadata synchronization operation when data synchronization conditions are met. The metadata synchronization operation includes: writing metadata from at least one metadata page corresponding to at least one set data bit in the dirty page bitmap into a first storage area, and resetting at least one data bit. The data synchronization conditions include: the data transmission channel is in an idle state.

Description

Technical Field

[0001] This application relates to the field of storage systems, and more specifically, to a metadata control method, apparatus, and electronic device. Background Technology

[0002] In storage systems, metadata needs to be written for data storage and control. However, real-time metadata synchronization to maintain data consistency can easily interfere with other processes in the system, affecting the stability of business operations. Furthermore, metadata writing can easily cause write amplification, which can reduce the lifespan of storage media, slow down system response speed, and reduce system stability.

[0003] This shows that the metadata control methods in related technologies suffer from poor system stability. Summary of the Invention

[0004] This application provides a metadata control method, apparatus, and electronic device to at least solve the technical problem of poor system stability in metadata control methods in related technologies.

[0005] According to one aspect of the embodiments of this application, a metadata control method is provided, comprising: responding to a metadata change request, writing specified metadata into a target metadata page in a target cache area, and setting a target data bit in a dirty page bitmap corresponding to the target metadata page; and performing a metadata synchronization operation when a data synchronization condition is met, wherein the metadata synchronization operation comprises: writing metadata from at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into a first storage area, and resetting the at least one data bit, wherein the data synchronization condition comprises: the data transmission channel being in an idle state.

[0006] According to another aspect of the embodiments of this application, a metadata control device is also provided, comprising: a first execution unit, configured to, in response to a metadata change request, write specified metadata into a target metadata page in a target cache area, and set a target data bit in a dirty page bitmap corresponding to the target metadata page; and a second execution unit, configured to, when data synchronization conditions are met, perform a metadata synchronization operation, wherein the metadata synchronization operation comprises: writing metadata from at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into a first storage area, and resetting the at least one data bit, wherein the data synchronization conditions include: the data transmission channel being in an idle state.

[0007] According to another aspect of the embodiments of this application, a computer-readable storage medium is also provided, wherein a computer program is stored therein, wherein the computer program is configured to perform the steps in any of the above method embodiments when executed by a processor.

[0008] According to another aspect of the embodiments of this application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, causing the computer device to perform the steps in any of the method embodiments described above.

[0009] According to another aspect of the embodiments of this application, an electronic device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor is configured to perform the steps of any of the above method embodiments through the computer program.

[0010] This application addresses the issue of poor system stability in metadata control methods by responding to metadata change requests, writing specified metadata to a target metadata page in a target cache area, and setting the target data bit in the dirty page bitmap corresponding to the target metadata page. Under certain data synchronization conditions, a metadata synchronization operation is performed, comprising: writing metadata from at least one metadata page corresponding to at least one set data bit in the dirty page bitmap to a first storage area, and resetting at least one data bit. The data synchronization conditions include: the data transmission channel being idle. By enabling on-demand, batch writing, reducing the writing frequency, and avoiding interference with other system services, this application solves the technical problem of poor system stability in related metadata control methods, thereby improving system stability. Attached Figure Description

[0011] Figure 1 This is a schematic diagram illustrating an application scenario of a metadata control method according to an embodiment of this application;

[0012] Figure 2 This is a flowchart illustrating an optional metadata control method according to an embodiment of this application;

[0013] Figure 3 This is a flowchart illustrating another optional metadata control method according to an embodiment of this application;

[0014] Figure 4 This is a flowchart illustrating another optional metadata control method according to an embodiment of this application;

[0015] Figure 5This is a flowchart illustrating another optional metadata control method according to an embodiment of this application;

[0016] Figure 6 This is a structural block diagram of an optional metadata control device according to an embodiment of this application;

[0017] Figure 7 This is a computer system architecture block diagram of an optional electronic device according to an embodiment of this application. Detailed Implementation

[0018] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0019] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0020] According to one aspect of the embodiments of this application, a metadata control method is provided. Optionally, in this embodiment, the above-described metadata control method may be applied to, but is not limited to, [examples of other methods]. Figure 1 The hardware environment shown includes terminal device 102 and server 104. Server 104 can be connected to terminal device 102 via a network and can be used to provide services (e.g., application services, etc.) to terminal device 102 or clients installed on terminal device 102. A database can be set up on server 104 or independently of server 104 to provide data storage services for server 104.

[0021] The aforementioned network may include, but is not limited to, at least one of the following: wired network and wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: wide area network (WAN), metropolitan area network (MAN), and local area network (LAN). The aforementioned wireless network may include, but is not limited to, at least one of the following: Wireless Fidelity (WIFI) and Bluetooth. Terminal device 102 may be, but is not limited to, a personal computer (PC), mobile phone, tablet computer, etc. Server 104 may be, but is not limited to, a cloud server, server cluster, or other server types.

[0022] The metadata control method of this application embodiment can be executed by server 104, terminal device 102, or jointly by server 104 and terminal device 102. Alternatively, the metadata control method of this application embodiment can be executed by a client installed on the terminal device 102.

[0023] Taking the metadata control method in this embodiment executed by server 104 as an example, Figure 2 This is a flowchart illustrating an optional metadata control method according to an embodiment of this application, such as... Figure 2 As shown, the process of this method may include the following steps:

[0024] In step S202, in response to the metadata change request, the specified metadata is written to the target metadata page in the target cache area, and the target data bit in the dirty page bitmap is set to match the target metadata page.

[0025] Step S204: If the data synchronization conditions are met, perform a metadata synchronization operation. The metadata synchronization operation includes: writing the metadata of at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into the first storage area, and resetting at least one data bit. The data synchronization conditions include: the data transmission channel is in an idle state.

[0026] The metadata control method in this embodiment can be applied to the field of storage systems, specifically to scenarios involving the storage and synchronization of metadata.

[0027] In storage systems, metadata serves as crucial information controlling the mapping between logical block addresses and physical block addresses, block status, write / erase counts, bad block management, and file system structure. Its integrity directly determines the availability and data reliability of the storage system. To ensure that metadata is not lost in abnormal scenarios such as power outages, it needs to be written to non-volatile storage media. Typically, when metadata changes, the updated content is immediately written to the non-volatile storage media via the storage controller.

[0028] However, the physical characteristics of non-volatile storage media in storage systems typically require write operations to be performed on a page-by-page basis, and an erase operation must be performed on the target block before writing. Because erase operations are characterized by high latency, non-interruption, and exclusive resource usage, each metadata update triggers a complete garbage collection process: reading valid data, merging new data, erasing the original block, and writing the new block. This process not only consumes data transmission channels, cache bandwidth, and processing cycles, but may also compete with read / write requests for business data in the system for system bus resources, causing I / O path congestion, which in turn leads to fluctuations in business execution latency, decreased throughput, and compromises the stability of the system's real-time response.

[0029] Furthermore, metadata is typically updated in a fine-grained, high-frequency manner, such as changes to individual records in a mapping table, setting bad block markers, or increasing the number of erase / write cycles. Because metadata updates require triggering the movement and rewriting of the entire block, the actual amount of data written to the non-volatile storage medium may exceed the amount of data requested by the host, resulting in write amplification. This accelerates the consumption of erase / write cycles on the non-volatile storage medium and shortens its lifespan.

[0030] Frequent garbage collection and metadata write operations can also increase system load, consume limited system resources, reduce cache hit efficiency, and increase firmware scheduling overhead, leading to longer system response cycles, accumulated task scheduling delays, and ultimately system performance degradation and instability. When multiple concurrent business requests and metadata synchronization operations are triggered simultaneously, the system may enter a resource contention phase, affecting the reliability and availability of the equipment.

[0031] This shows that the metadata control methods in related technologies suffer from poor system stability.

[0032] To at least address some of the technical issues, in this embodiment, in response to a metadata change request, specified metadata is written to a target metadata page in the target cache area, and the target data bit in the dirty page bitmap corresponding to the target metadata page is set. When data synchronization conditions are met, a metadata synchronization operation is performed. This metadata synchronization operation includes: writing metadata from at least one metadata page corresponding to at least one set data bit in the dirty page bitmap to a first storage area, and resetting at least one data bit. The data synchronization conditions include: the data transmission channel being in an idle state. Because on-demand, batch writing is implemented, the writing frequency is reduced, and other system services are not interfered with. Therefore, the technical problem of poor system stability in metadata control methods in related technologies can be solved, achieving the effect of improving system stability.

[0033] Optionally, in response to a metadata change request, the specified metadata to be updated can be written to a target metadata page in a target cache area. This target cache area can be composed of high-speed volatile storage media, such as random access memory (RAM), specifically dynamic random access memory (DRAM) or static random access memory (SRAM). This embodiment does not limit the specific type of RAM used. Volatile storage media have low access latency, supporting low-latency, high-throughput write operations.

[0034] Optionally, after writing the metadata to the target metadata page, the dirty page bitmap can be updated synchronously, setting the bit fields corresponding to the target metadata page. Here, the dirty page bitmap can be used for metadata consistency control; it can be an array of bits, with each bit mapping to a physical or logical index of a metadata page in the target cache area, used to identify whether the page has been modified since the last synchronization.

[0035] Optionally, the set operation can be an atomic write, which is completed by the firmware control logic in the memory controller. It does not depend on the external storage medium, does not trigger any non-volatile write operations, and does not occupy storage bus resources.

[0036] Through the above process, the metadata change and persistence process can be decoupled, the logical modification of metadata is limited to the cache domain, and its state is tracked through bitmap, avoiding immediate writing to the underlying storage medium.

[0037] Optionally, when the data synchronization conditions are met, a metadata synchronization operation can be performed, wherein the metadata synchronization operation includes: writing the metadata in at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into the first storage area, and resetting at least one data bit.

[0038] Optionally, data synchronization conditions may include: the data transmission channel being in an idle state. Specifically, the data transmission channel being in an idle state can mean that the system is not executing other services, or that no service I / O requests have been received within a consecutive preset time window, and the programming queue is empty, there are no pending transactions, and the flash array has no ongoing read, write, or erase operations.

[0039] Optionally, the system can periodically check whether the data synchronization conditions are met. For example, it can check whether the system is in an idle state every 5 seconds. When the system is detected to be in an idle state and is still within the continuous idle time window that allows the synchronization operation to be performed, the metadata synchronization operation is performed.

[0040] Optionally, if the system detects any business I / O request or bus activity within a preset time window, it may not trigger the metadata synchronization process, retain the dirty page bitmap state, and wait for the next cycle detection.

[0041] Optionally, the metadata synchronization operation may include reading the dirty page bitmap, traversing all its data bits, and identifying all set bit fields. Each set data bit corresponds to a modified metadata page in the target cache area. This metadata page can be a physically contiguous storage unit in dynamic random access memory (DRAM) or static random access memory (SRAM), and its content can be a metadata structure that has not been persisted since the most recent change.

[0042] Optionally, the metadata in at least one of the set metadata pages can be written to the first storage area, which can be done in batches.

[0043] Optionally, the first storage area can be a storage area on a non-volatile storage medium, located within a fixed address space reserved by the system, and physically isolated from the user data area. Write operations can be performed on a page-by-page basis, with the content of each metadata page being written to the target page all at once.

[0044] For example, the first storage area may be a storage area on a NAND Flash storage medium, which has high storage density and supports large-capacity data block read and write.

[0045] Optionally, after all target metadata pages have been written, a bitmap reset operation can be performed to clear the data bits in the dirty page bitmap corresponding to the synchronized metadata pages bit by bit.

[0046] Through the embodiments of this application, in response to a metadata change request, specified metadata is written to a target metadata page in a target cache area, and the target data bit in the dirty page bitmap corresponding to the target metadata page is set. When the data synchronization condition is met, a metadata synchronization operation is performed, wherein the metadata synchronization operation includes: writing the metadata in at least one metadata page corresponding to at least one set data bit in the dirty page bitmap to a first storage area, and resetting at least one data bit. The data synchronization condition includes: the data transmission channel is in an idle state. This can solve the technical problem of poor system stability in the metadata control method in related technologies, and achieve the effect of improving system stability.

[0047] In one exemplary embodiment, after writing the specified metadata to the target metadata page in the target cache region, the above method further includes:

[0048] The control increments the dirty page counter value by 1;

[0049] The data synchronization conditions also include:

[0050] The dirty page counter reaches a preset counting threshold; and / or,

[0051] The time difference between the current time and the time of the last metadata synchronization operation reaches a preset time difference threshold; and / or,

[0052] A specified event signal is received, wherein the specified event signal is used to indicate at least one of the following: bad block marking completed, firmware upgrade, hardware error interruption.

[0053] Optionally, after writing the specified metadata to the target metadata page in the target cache region, the dirty page counter can be incremented by 1. Here, the dirty page counter can be a counting structure in an internal register or cache, used to count the total number of metadata pages marked as dirty in the target cache region in real time.

[0054] Optionally, after setting the data bits in the dirty page bitmap, the count value of the dirty page counter can be reduced accordingly.

[0055] In this embodiment, the triggering conditions (i.e., data synchronization conditions) for metadata synchronization operations may further include: the dirty page counter reaches a preset counting threshold; and / or, the time difference between the current time and the time of the last metadata synchronization operation reaches a preset time difference threshold; and / or, a specified event signal is received.

[0056] Optionally, the preset counting threshold can be a value pre-configured by the system based on the target cache area capacity, metadata page size, and write performance parameters of the first storage area. When the dirty page counter reaches or exceeds this threshold, it can be determined that the amount of metadata to be synchronized has accumulated to a critical level that affects recovery efficiency, triggering the synchronization process to avoid the loss of a large amount of metadata due to power failure, thereby ensuring data integrity.

[0057] Optionally, the preset time difference threshold can be a fixed time interval, for example, 5 seconds. It can be based on the system's real-time clock module, so that metadata can be persisted according to a defined time period, preventing metadata status from being stuck in the volatile cache for a long time due to long-term lack of synchronization.

[0058] Optionally, metadata synchronization can be performed upon receiving a specified event signal. This specified event signal can be generated by the system firmware or hardware exception handling module and can be used to indicate high-priority system state changes, including but not limited to: bad block marking completion, firmware upgrade operation initiation, and hardware error interruption triggering. When the system detects any of the above events, it can interrupt the regular synchronization scheduling process and initiate metadata synchronization. Here, bad block marking completion means that the physical state of the storage medium has undergone an irreversible change and must be immediately fixed to a non-volatile area to prevent the mapping table from becoming invalid after a reboot; firmware upgrades involve changes to the metadata structure version, and the old version of metadata must be ensured to be completely and persistently persisted before the upgrade; hardware error interruption (such as power fluctuations or controller malfunctions) indicates that the system may be about to shut down abnormally, requiring metadata synchronization to be completed first to reduce data consistency risks.

[0059] Optionally, the specified event signal can also be used to indicate other events, such as system shutdown, but this embodiment does not limit this.

[0060] Optionally, the first storage area may also include historical checkpoints to record updates to the first storage area, which may also be recorded in the header information of the first storage area.

[0061] Optionally, the metadata synchronization operation may be triggered only when multiple conditions in the data synchronization conditions are met. For example, the metadata synchronization operation may be triggered only when the dirty page counter reaches a preset counting threshold and the data transmission channel is idle. Alternatively, the metadata synchronization operation may be triggered only when the time difference between the current time and the time of the last metadata synchronization operation reaches a preset time difference threshold and the data transmission channel is idle.

[0062] For example, such as Figure 3 As shown, the system initially operates normally. In response to business modifications to RAM metadata, it can perform trigger condition checks. Trigger conditions can include reaching the water level threshold, the timeout of the last synchronization execution, and receiving a specified event signal. If any trigger condition is met, the NAND bus status can be checked. If it is busy, it will wait briefly or delay before performing trigger condition checks again. If it is idle, it will perform synchronization, write the data of dirty pages in RAM to NAND, update the corresponding NAND checkpoint or header information, clear the RAM dirty page mark, and reset the timer.

[0063] This embodiment demonstrates how multi-condition triggering of metadata synchronization operations can ensure the timeliness of metadata persistence while avoiding frequent synchronizations caused by accidental triggering of a single condition, thereby improving system performance and data reliability.

[0064] In one exemplary embodiment, the above method further includes:

[0065] In response to receiving a specified instruction, lock the metadata in the target cache area, where the specified instruction is one of the following: a shutdown instruction or a restart instruction;

[0066] Perform an erase operation on the second storage area and write the metadata in the target cache area into the second storage area. The second storage area and the first storage area are different storage areas.

[0067] Optionally, in response to receiving a specified instruction, a metadata locking operation can be performed to lock the metadata in the target cache area. The specified instruction can be a shutdown instruction or a reboot instruction, which can be generated by the system power management module or the firmware bootloader. After receiving the specified instruction, all new metadata change requests can be paused, write operations to the target cache area can be stopped, and an atomic barrier instruction can be used to ensure that all pending metadata write operations have been committed to the cache.

[0068] Optionally, access permissions for all metadata pages in the target cache area can be locked to stop read and write access, keeping the metadata state static during subsequent operations and preventing inconsistencies in synchronized data due to concurrent modifications.

[0069] Optionally, after the target cache region is successfully locked, an atomic write operation can be performed to write the metadata in the target cache region to the second storage region. Here, an atomic write operation means breaking down the operation into indivisible smallest atomic units, ensuring that the operation can only succeed completely or fail completely, without intermediate states, in order to avoid partial execution exceptions under concurrency.

[0070] Optionally, the second storage area can also be a storage area of ​​non-volatile storage media. The second storage area and the first storage area can be different storage areas. It can serve as a mirror layer for metadata and differ from the first storage area in terms of functional positioning and access characteristics. For example, the second storage area can be a NOR Flash storage medium.

[0071] Optionally, before writing the metadata in the target cache area to the second storage area, an erase operation can be performed on the second storage area to ensure that the area to be written in the second storage area is in its initial state.

[0072] Optionally, after writing the metadata, a data integrity check can be performed on the second storage area to ensure that the written content is consistent with the metadata at the time of locking. If the check fails, an error log can be recorded, the current data in the original second storage area can be retained without performing a rollback or retry, or a data rollback can be performed; this embodiment does not limit this.

[0073] This embodiment enables the recovery of metadata in power outage scenarios by persisting it completely and consistently in the second storage area before powering off or restarting, thereby improving system stability and reliability.

[0074] In one exemplary embodiment, the second storage area includes a first partition and a second partition;

[0075] Perform an erase operation on the second storage area and write the metadata in the target cache area to the second storage area, including:

[0076] Based on the first sequence number of the first partition and the first sequence number of the second partition, the target partition is determined from the first partition and the second partition. The first sequence number is used to identify the update order of different partitions in the second storage area. The target partition is the partition with the smaller first sequence number between the first partition and the second partition.

[0077] Perform an erase operation on the target partition and write the metadata in the target cache area to the target partition.

[0078] Optionally, the second storage area may include the first partition and the second partition. When synchronizing metadata to the second storage area, the serial numbers of the two partitions can be read during writing, and the partition with the smaller serial number can be selected for erasure and overwriting.

[0079] Optionally, each partition may contain a sequence number field with a fixed offset, namely the first sequence number. The first sequence number may be an unsigned integer value or a value of other types. This embodiment does not limit this.

[0080] Optionally, the first sequence number can be used to identify the update order of the last successful write of metadata to the partition, and its value monotonically increases with each write operation.

[0081] For example, the value of the first sequence number can also decrease monotonically with each write operation. In this case, when writing metadata to the second storage area, the metadata can be written to the partition with the larger first sequence number.

[0082] Optionally, when performing a metadata write operation, the serial number fields of the first and second partitions can be read to obtain their current first serial numbers. After reading, a serial number comparison operation can be performed to determine the numerical relationship between the first serial number of the first partition and the first serial number of the second partition. The target partition can be the partition with the smaller first serial number; if the two partition serial numbers are equal, for example, if neither has an initial serial number, then either one can be chosen as the target partition.

[0083] Optionally, after selecting the target partition, an erase operation can be performed on the target partition to write the metadata in the target cache area to the target partition. This can be done by using a direct memory access channel to completely write all the metadata in the locked target cache area to the target partition, and the write operation is performed according to a predefined format of the metadata structure.

[0084] This embodiment achieves dynamic selection of the write target by comparing the column number values, so that each update only applies to the older version of the partition, while the newer version of the partition retains valid metadata. This is beneficial for recovering data from the newer version of the partition in case of write errors, and ensures the atomicity, consistency and durability of metadata when the system is shut down or restarted, thereby improving the stability and reliability of the system.

[0085] In one exemplary embodiment, after writing the metadata in the target cache region to the target partition, the above method further includes:

[0086] Read the metadata from the target partition and verify the metadata in the target partition;

[0087] If the metadata verification in the target partition passes, update the first serial number of the target partition, wherein the updated first serial number of the target partition is greater than the first serial number of the other partition in the first and second partitions, excluding the target partition.

[0088] Optionally, after reading is complete, the data read from the target partition can be verified according to a predefined metadata verification algorithm. This can be integrity verification, security verification, or other verification content. Verification algorithms include, but are not limited to, Cyclic Redundancy Check (CRC) and Error Correcting Code (ECC), as well as hash-based checksums, etc. This embodiment does not limit these.

[0089] Here, CRC is an algorithm that calculates a fixed-length check code based on the data content. It is mainly used to detect errors that may occur after data transmission or storage. ECC, on the other hand, adds redundant check information to the original data to detect and correct errors that occur during data transmission or storage.

[0090] Optionally, the scope of the verification data may include, but is not limited to, fields such as metadata body, structure header information, block status, mapping table, and erase / write count. The verification calculation result can be compared with the checksum field generated synchronously during writing and stored at the end of the metadata. If the calculation result is completely consistent with the stored checksum, the verification is deemed to have passed; if there is any bit difference or the number of ECC correction failures exceeds the threshold, the verification is deemed to have failed.

[0091] Optionally, if the metadata verification in the target partition passes, a serial number update operation can be performed.

[0092] For example, the current serial numbers of the first and second partitions can be read to determine the serial number of the other partition besides the target partition, which is then denoted as the reference serial number. The updated target partition serial number is set to the reference serial number plus one to ensure that the target partition serial number is greater than the serial number of the other partition, thus establishing a version order relationship.

[0093] Optionally, after the serial number is updated, the new value can be written to the serial number field in the header of the target partition, and hardware ECC verification can be performed to verify the correctness of the write.

[0094] Optionally, if the serial number writing fails, it can be left unretrieved and the metadata content can be left unchanged. The target partition metadata can be kept in an intermediate state where it has been verified but the version has not been updated, in order to maintain the fault tolerance of the recovery mechanism. Here, the system can still determine the validity of the metadata based on the checksum when it starts up again. The inconsistent serial number will not cause the recovery to fail, but will only affect the accuracy of the version determination.

[0095] Optionally, the power-down permission signal can be sent and the power-down can be initiated only after the data has been written to the target partition and verified. For example, ... Figure 4 As shown, upon receiving a shutdown or restart signal, new I / O requests can be suspended and the RAM metadata cache locked. The sequence numbers (Seq) of areas A and B in the NOR space are read and compared. If the sequence number of area A is smaller, area A is selected as the target partition; if the sequence number of area B is smaller, area B is selected as the target partition. The target NOR space is then erased and a full write is performed, writing the metadata from RAM to the target NOR space. After writing, a new checksum can be calculated and written, followed by a write integrity check. If the check passes, the system atomically increments the sequence number field of the target partition to max(first sequence number of area A, second sequence number of area B) + 1, completing the version update. If the check fails, the system does not modify the sequence number, retaining the original content of the target partition, ensuring that the system can still recover valid metadata from another partition.

[0096] By updating the serial number after the metadata content has been verified to be complete and valid, this embodiment ensures that there is a copy of the metadata in the second storage area that can be identified as the latest valid copy, thereby achieving highly reliable and unambiguous recovery of metadata when the system restarts.

[0097] In one exemplary embodiment, the above method further includes:

[0098] During the initialization phase, the metadata in the first partition and the metadata in the second partition are read and verified. The first partition and the second partition are different partitions in the second storage area, and the second storage area and the first storage area are different storage areas.

[0099] If one of the metadata in the first partition and the metadata in the second partition passes the verification, the verified metadata is read and written to the target cache area.

[0100] If the metadata in the first partition and the metadata in the second partition both pass the verification, read the metadata in the partition with the larger first sequence number in the first partition and the second partition, and write the read metadata into the target cache area. The first sequence number is used to identify the update order of different partitions in the second storage area.

[0101] If the metadata in the first partition and the metadata in the second partition both fail to pass verification, the metadata backup units in the first storage area are traversed, the metadata in the metadata backup unit with the highest second sequence number is read, and the read metadata is written to the target cache area. The second sequence number is used to identify the update order of different metadata backup units in the first storage area.

[0102] In this embodiment, metadata recovery can be performed during the system initialization phase, which can be a hierarchical cascaded recovery process.

[0103] Optionally, metadata can be read and verified on the first and second partitions in the second storage area first. Similar to the previous embodiments, the first and second partitions can be two physically independent, address-isolated partitions in the second storage area, used to store the shutdown final state metadata image.

[0104] Optionally, the metadata structures of the headers of the first and second partitions can be read separately using direct memory mapping, including the mapping table, block status, erase / write count, checksum, and sequence number fields. The read process can enable hardware ECC verification to ensure that data is correctly transferred from non-volatile media to system memory, and all read operations are completed internally by the controller, avoiding latency caused by the host bus.

[0105] Optionally, after reading is complete, integrity checks can be performed on the metadata in the first and second partitions respectively. The check algorithm uses a predefined checksum mechanism, and the check scope may include, but is not limited to, the metadata body, management structure header, block mapping information, and erase / write statistics fields, but does not include the sequence number field itself. The check results can be recorded in the system status register.

[0106] Optionally, if only one of the first and second partitions passes metadata verification, that partition can be used as a valid metadata source. The verified metadata content in that partition can be directly read and written to the target cache area, and the target cache area will then enter an operational state, and the system initialization process will continue.

[0107] Optionally, if the metadata of both the first and second partitions passes verification, the serial number fields of the first and second partitions can be read. These serial number fields identify the update order of the last successful write to each partition, and their values ​​monotonically increase with each shutdown write. By comparing the serial number values ​​of the two partitions, the partition with the larger serial number can be selected as the source of the latest version's metadata. Its metadata can then be fully read and written to the target cache area, ensuring that when both partitions are valid, the system prioritizes loading the latest state, avoiding overwriting of older versions due to abnormal shutdowns.

[0108] Optionally, if the metadata verification of both the first and second partitions fails, a secondary recovery process can be initiated, in which data can be recovered from the first storage area.

[0109] Optionally, each metadata backup unit may include a second sequence number to identify the update sequence of the backup unit, and its value may also be monotonically increasing.

[0110] Optionally, the first storage area may also include historical checkpoints, which can read the location of the most recent checkpoint record and read the metadata in the corresponding metadata backup unit.

[0111] Optionally, the metadata backup units in the first storage area can be traversed, the metadata in the metadata backup unit with the highest second sequence number can be read, and the read metadata can be written to the target cache area. Alternatively, the header information of each metadata backup unit can be read, the data in it can be verified, all valid backup units that pass the verification can be filtered out, and then the unit with the highest second sequence number in the valid backup units can be selected as the most recent valid backup unit, its metadata content can be read, and after verification, it can be written to the target cache area to complete the second-level recovery.

[0112] This embodiment, through multiple recovery mechanisms, ensures the determinism and consistency of metadata loading, thereby improving the stability and reliability of the system.

[0113] In one exemplary embodiment, the above method further includes:

[0114] Verify the metadata backup unit in the first storage area;

[0115] If all metadata backup units in the first storage area fail verification, the metadata structure is reconstructed based on the block information in the physical storage block, and the reconstructed metadata is loaded into the target cache area.

[0116] Optionally, the metadata backup units in the first storage area can also be verified. The verification process is similar to that in the previous embodiment. It can be that the metadata backup units in the first storage area are traversed sequentially and verified. The unit with the highest second sequence number among the valid metadata backup units is selected for reading. Alternatively, the metadata backup unit with the highest second sequence number can be selected first for verification. If the verification passes, the metadata is read. If the verification fails, the metadata unit with the highest second sequence number among the remaining metadata backup units is selected for verification. The process is repeated in this manner.

[0117] Optionally, if all metadata backup units in the first storage area fail verification, the physical blocks can be traversed for reconstruction.

[0118] Optionally, the complete refactoring process can be executed autonomously by the system controller, without relying on external metadata or logs.

[0119] Optionally, the main data area and OOB (Out-of-Band) area of ​​each physical storage block can be read block by block in physical block address order. For each physical block, the raw metadata stored in its OOB area can be parsed, including the Logical Block Addressing (LBA) mapping identifier, write status flag, bad block flag, and write timestamp. If the LBA identifier in the OOB area is valid and the block is not marked as a bad block, the LBA information of the physical block can be parsed into the corresponding logical block index. The logical block number can be the result of LBA divided by 64 (with 64 sectors as a logical block unit), and the address of the physical block is recorded in the corresponding entry in the mapping table accordingly. If there is no valid LBA information in the OOB or the LBA identifier is abnormal, the block is determined to be an unallocated free block and the processing is skipped.

[0120] Optionally, during the mapping relationship construction process, LBA conflicts can be detected: if multiple physical blocks are mapped to the same logical block, the system will use the physical block with the newest last written timestamp as the reference and overwrite the original mapping entry; if the timestamp information is missing, the block with the lower physical address can be used as the valid mapping to ensure determinism.

[0121] Optionally, the bad block flag bit of each physical block can be read. If a block is marked as invalid, its physical address is recorded in the bad block list of the metadata structure, and its status is set to bad.

[0122] Optionally, after completing the traversal and parsing of physical blocks, a complete metadata structure can be constructed, which may include, but is not limited to, a logical block mapping table, a bad block bitmap, a global erase / write count, and a free block list. This structure can be derived from physical layer information without relying on any original backup data.

[0123] Optionally, consistency checks can be performed on the newly constructed metadata, including but not limited to verifying the integrity of the mapping table, the continuity of logical blocks, and the consistency of bad block markings. If the checks pass, the reconstructed metadata structure is written to the target cache area, completing the three-level recovery.

[0124] For example, such as Figure 5 As shown, upon power-on reset, the storage controller can be initialized, the header information of areas A and B in the NOR data can be read and the NOR data validity can be verified. If the data in at least one partition is valid, the serial numbers of the valid areas are compared, and the NOR area with the largest serial number is selected to load the metadata into RAM. If all verifications fail, the NAND metadata area is downgraded to scan, the most recent valid checkpoint is found, and the corresponding dataset is verified. If valid, the hardest NAND checkpoint data is loaded into RAM. If verification fails, the metadata is reconstructed through a full scan to recover the data, and the system is ready.

[0125] Optionally, after loading the metadata into RAM, the data area logs can be replayed for fine-tuning. This can involve reading and parsing the log records stored in the data area, and modifying the loaded metadata structure based on the change operations recorded in the logs.

[0126] This embodiment demonstrates how reconstructing metadata when backup metadata has failed can improve system reliability and stability.

[0127] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

[0128] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods according to the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / random access memory (RAM), magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0129] According to another aspect of the embodiments of this application, a metadata control device is also provided, which can be used to implement the metadata control method provided in the above embodiments, and will not be repeated hereafter. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0130] Figure 6 This is a structural block diagram of an optional metadata control device according to an embodiment of this application, such as... Figure 6 As shown, the metadata control device includes:

[0131] The first execution unit 602 is configured to respond to a metadata change request by writing the specified metadata into the target metadata page in the target cache area and setting the target data bit in the dirty page bitmap that corresponds to the target metadata page;

[0132] The second execution unit 604 is used to perform a metadata synchronization operation when the data synchronization conditions are met. The metadata synchronization operation includes writing the metadata in at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into the first storage area and resetting at least one data bit. The data synchronization conditions include that the data transmission channel is in an idle state.

[0133] It should be noted that the first execution unit 602 in this embodiment can be used to execute the above step S202, and the second execution unit 604 in this embodiment can be used to execute the above step S204.

[0134] The embodiments provided in this application, in response to a metadata change request, write specified metadata into a target metadata page in a target cache area and set the target data bit in the dirty page bitmap corresponding to the target metadata page; when the data synchronization conditions are met, perform a metadata synchronization operation, wherein the metadata synchronization operation includes: writing metadata from at least one metadata page corresponding to at least one set data bit in the dirty page bitmap into a first storage area and resetting at least one data bit. The data synchronization conditions include: the data transmission channel is in an idle state. This can solve the technical problem of poor system stability in metadata control methods in related technologies and achieve the effect of improving system stability.

[0135] In one exemplary embodiment, the above-described apparatus further includes:

[0136] The control unit is used to control the incrementing of the dirty page counter by 1 after the specified metadata is written to the target metadata page in the target cache area;

[0137] The data synchronization conditions also include:

[0138] The dirty page counter reaches a preset counting threshold; and / or,

[0139] The time difference between the current time and the time of the last metadata synchronization operation reaches a preset time difference threshold; and / or,

[0140] A specified event signal is received, wherein the specified event signal is used to indicate at least one of the following: bad block marking completed, firmware upgrade, hardware error interruption.

[0141] In one exemplary embodiment, the above-described apparatus further includes:

[0142] A locking unit is used to lock metadata in a target cache area in response to receiving a specified instruction, wherein the specified instruction is one of the following: a shutdown instruction or a restart instruction;

[0143] The third execution unit is used to perform an erase operation on the second storage area and write the metadata in the target cache area into the second storage area, wherein the second storage area and the first storage area are different storage areas.

[0144] In one exemplary embodiment, the second storage area includes a first partition and a second partition;

[0145] The third execution unit includes:

[0146] The determination module is used to determine the target partition from the first partition and the second partition based on the first serial number of the first partition and the first serial number of the second partition, wherein the first serial number is used to identify the update order of different partitions in the second storage area, and the target partition is the partition with the smaller first serial number between the first partition and the second partition;

[0147] The execution module is used to perform an erase operation on the target partition and write the metadata in the target cache area to the target partition.

[0148] In one exemplary embodiment, the above-described apparatus further includes:

[0149] The fourth execution unit is used to read the metadata in the target partition and verify the metadata in the target partition after writing the metadata in the target cache area to the target partition.

[0150] An update unit is used to update the first serial number of the target partition if the metadata verification in the target partition passes, wherein the updated first serial number of the target partition is greater than the first serial number of another partition in the first and second partitions other than the target partition.

[0151] In one exemplary embodiment, the above-described apparatus further includes:

[0152] The fifth execution unit is used to read the metadata in the first partition and the metadata in the second partition during the initialization phase, and to verify the metadata in the first partition and the metadata in the second partition. The first partition and the second partition are different partitions in the second storage area, and the second storage area and the first storage area are different storage areas.

[0153] The write unit is used to read the verified metadata and write it to the target cache area if one of the metadata in the first partition and the metadata in the second partition passes the verification.

[0154] The sixth execution unit is used to read the metadata of the partition with the larger first sequence number in the first partition and the second partition, and write the read metadata into the target cache area, provided that the metadata in the first partition and the metadata in the second partition have both been verified. The first sequence number is used to identify the update order of different partitions in the second storage area.

[0155] The seventh execution unit is used to traverse the metadata backup units in the first storage area, read the metadata in the metadata backup unit with the highest second sequence number, and write the read metadata into the target cache area if the metadata in the first partition and the metadata in the second partition both fail to pass the verification. The second sequence number is used to identify the update order of different metadata backup units in the first storage area.

[0156] In one exemplary embodiment, the above-described apparatus further includes:

[0157] The verification unit is used to verify the metadata backup units in the first storage area;

[0158] The eighth execution unit is used to reconstruct the metadata structure based on the block information in the physical storage block and load the reconstructed metadata into the target cache area if all the metadata backup units in the first storage area fail to pass the verification.

[0159] It should be noted that the above modules can be implemented by software or hardware. For the latter, they can be implemented in the following ways, but are not limited to: all the above modules are located in the same processor; or, the above modules are located in different processors in any combination.

[0160] According to another aspect of the embodiments of this application, a computer-readable storage medium is provided, the computer-readable storage medium including a stored program, wherein the program executes the steps in any of the above method embodiments when it is run.

[0161] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, ROMs, RAMs, portable hard drives, magnetic disks, or optical disks.

[0162] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor is configured to perform the steps of any of the method embodiments described above via the computer program. In an exemplary embodiment, the electronic device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0163] Specific examples in this embodiment can be found in the examples described in the above embodiments and exemplary implementations, and will not be repeated here.

[0164] According to another aspect of the embodiments of this application, a computer program product is also provided, comprising a computer program / instructions containing program code for performing the methods shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit 701, it performs various functions provided in the embodiments of this application. The sequence numbers of the embodiments of this application above are merely descriptive and do not represent the superiority or inferiority of the embodiments.

[0165] Figure 7 A schematic block diagram of a computer system architecture for implementing embodiments of the present application is shown. Figure 7 As shown, the computer system 700 includes a Central Processing Unit (CPU) 701, which performs various appropriate actions and processes based on programs stored in ROM 702 or loaded into RAM 703 from storage section 708. Random access memory 703 also stores various programs and data required for system operation. The CPU 701, ROM 702, and RAM 703 are interconnected via bus 704. Input / output (I / O) interface 705 is also connected to bus 704.

[0166] The following components are connected to the I / O interface 705: an input section 706 including a keyboard, mouse, etc.; an output section 707 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 708 including a hard disk, etc.; and a communication section 709 including a network interface card, such as a local area network card or modem, etc. The communication section 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the input / output interface 705 as needed. A removable medium 711, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 710 as needed so that computer programs read from it can be installed into the storage section 708 as needed.

[0167] Specifically, according to embodiments of this application, the processes described in the various method flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 709, and / or installed from removable medium 711. When the computer program is executed by central processing unit 701, it performs various functions defined in the system of this application.

[0168] It should be noted that, Figure 7 The computer system 700 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0169] Obviously, those skilled in the art should understand that the modules or steps of this application described above can be implemented using general-purpose computing devices. They can be centralized on a single computing device or distributed across a network of multiple computing devices. They can be implemented using computer-executable program code, and thus can be stored in a storage device for execution by a computing device. In some cases, the steps shown or described can be performed in a different order than those described herein, or they can be fabricated as separate integrated circuit modules, or multiple modules or steps can be fabricated as a single integrated circuit module. Thus, this application is not limited to any particular combination of hardware and software.

[0170] The above are merely preferred embodiments of this application and are not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the principles of this application should be included within the protection scope of this application.

Claims

1. A metadata control method, characterized in that, include: In response to a metadata change request, the specified metadata is written to the target metadata page in the target cache area, and the target data bit in the dirty page bitmap corresponding to the target metadata page is set. When the data synchronization conditions are met, a metadata synchronization operation is performed, wherein the metadata synchronization operation includes: writing the metadata of at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into a first storage area, and resetting the at least one data bit, wherein the data synchronization conditions include: the data transmission channel is in an idle state.

2. The method according to claim 1, characterized in that, After writing the specified metadata to the target metadata page in the target cache region, the method further includes: The control increments the dirty page counter value by 1; The data synchronization conditions also include: The dirty page counter reaches a preset counting threshold; and / or, The time difference between the current time and the time of the last execution of the metadata synchronization operation reaches a preset time difference threshold; and / or, A specified event signal is received, wherein the specified event signal is used to indicate at least one of the following: bad block marking completed, firmware upgrade, hardware error interruption.

3. The method according to claim 1, characterized in that, The method further includes: In response to receiving a specified instruction, the metadata in the target cache area is locked, wherein the specified instruction is one of the following: a shutdown instruction or a restart instruction; An erase operation is performed on the second storage area, and the metadata in the target cache area is written to the second storage area, wherein the second storage area and the first storage area are different storage areas.

4. The method according to claim 3, characterized in that, The second storage area includes a first partition and a second partition; The step of performing an erase operation on the second storage area and writing the metadata in the target cache area into the second storage area includes: Based on the first serial number of the first partition and the first serial number of the second partition, a target partition is determined from the first partition and the second partition, wherein the first serial number is used to identify the update order of different partitions of the second storage area, and the target partition is the partition with the smaller first serial number among the first partition and the second partition; Perform an erase operation on the target partition and write the metadata in the target cache area to the target partition.

5. The method according to claim 4, characterized in that, After writing the metadata in the target cache region to the target partition, the method further includes: Read the metadata in the target partition and verify the metadata in the target partition; If the metadata verification in the target partition passes, the first serial number of the target partition is updated, wherein the updated first serial number of the target partition is greater than the first serial number of the first partition and another partition in the second partition other than the target partition.

6. The method according to claim 1, characterized in that, The method further includes: During the initialization phase, metadata from the first partition and metadata from the second partition are read, and the metadata from the first partition and metadata from the second partition are verified. The first partition and the second partition are different partitions in the second storage area, and the second storage area and the first storage area are different storage areas. If one of the metadata in the first partition and the metadata in the second partition passes the verification, the verified metadata is read and written to the target cache area; If the metadata in the first partition and the metadata in the second partition both pass the verification, read the metadata in the partition with the larger first sequence number in the first partition and the second partition, and write the read metadata into the target cache area. The first sequence number is used to identify the update order of different partitions in the second storage area. If the metadata in the first partition and the metadata in the second partition both fail the verification, the metadata backup units in the first storage area are traversed, the metadata in the metadata backup unit with the highest second sequence number is read, and the read metadata is written to the target cache area. The second sequence number is used to identify the update order of different metadata backup units in the first storage area.

7. The method according to claim 6, characterized in that, The method further includes: Verify the metadata backup unit in the first storage area; If all metadata backup units in the first storage area fail verification, the metadata structure is reconstructed based on the block information in the physical storage block, and the reconstructed metadata is loaded into the target cache area.

8. A metadata control device, characterized in that, include: The first execution unit is configured to, in response to a metadata change request, write the specified metadata into the target metadata page in the target cache area, and set the target data bit in the dirty page bitmap corresponding to the target metadata page; The second execution unit is used to perform a metadata synchronization operation when the data synchronization conditions are met. The metadata synchronization operation includes: writing the metadata of at least one metadata page corresponding to at least one data bit set in the dirty page bitmap into a first storage area, and resetting the at least one data bit. The data synchronization conditions include: the data transmission channel is in an idle state.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the steps of the method according to any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 7.

Images