A digital-twin-based full-flash metadata management method

Abstract

The application discloses a kind of based on digital twinning full flash metadata management method, it is related to data storage and metadata management technical field, including the following steps: collection metadata operation and generate metadata operation sequence;Dynamic metadata association graph and double-state metadata digital twinning are constructed;Metadata addressing path and metadata key path closure are generated by improved Todyformer model;According to closure topography, generate candidate metadata management action set and causal action mask;Shadow state transfer and mask tree search are executed by improved EfficientZero V2 model, determine target metadata management trajectory;According to target trajectory, update metadata, write in submission anchor point and switch effective mapping root node;According to submission result, calibrate digital twinning, and restore the metadata association relationship that is not closed when fault.The application improves the consistency and management efficiency of metadata addressing, submission and fault recovery.

Description

Technical Field

[0001] This invention relates to the field of data storage and metadata management technology, and in particular to an all-flash metadata management method based on digital twins. Background Technology

[0002] With the increasing demand for low-latency and high-concurrency storage from cloud computing, databases, and artificial intelligence businesses, metadata such as address mapping pages, version records, and transaction logs in all-flash systems continues to increase. Metadata addressing, updating, and fault recovery are gradually becoming key aspects affecting system performance and consistency.

[0003] Existing technologies have proposed solutions for caching indexes, persistence, and log recovery. For example, CN118245387A uses directories, segments, and buckets to construct a hash index and reduces the access latency of all-flash arrays through locking, persistence of valid bits, and version verification; CN115576489A uses hash linked lists, data buffers, and metadata databases to solidify NVMe all-flash data; CN102012849A recovers the flash database through log areas, checkpoints, and boot scans.

[0004] However, continuous metadata operations in actual business often involve addressing dependencies, version dependencies, and transaction commit dependencies simultaneously. Existing solutions mostly revolve around single index access, cache read / write, or log scans, making it difficult to accurately define the scope of metadata that the current transaction must process. When actions such as cache residency, mapping merging, transaction splitting, and version release coexist, there is a lack of mechanisms to constrain the order of actions as metadata state changes, which can easily lead to invalid actions, persistence order conflicts, or atomic transaction corruption. After a failure, it is also difficult to complete the unclosed addressing, version, and transaction commit relationships based on the necessary metadata scope of the current transaction, resulting in a large recovery scan scope, coarse rollback granularity, and unstable recovery time.

[0005] Therefore, how to provide a digital twin-based all-flash metadata management method is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an all-flash metadata management method based on digital twins. This invention fully utilizes digital twins, graph neural networks, and reinforcement learning techniques to construct a dynamic metadata association graph and a bi-state metadata digital twin, generate metadata critical path closures, and plan the target metadata management trajectory, thereby achieving orderly metadata submission and fault recovery. It has the advantages of high addressing efficiency, strong transaction consistency, high submission reliability, and fast fault recovery speed.

[0007] A method for managing all-flash metadata based on digital twins according to an embodiment of the present invention includes the following steps:

[0008] Step 1: Collect metadata operations in the all-flash system and generate a metadata operation sequence according to the operation timing.

[0009] Step 2: Construct a dynamic metadata association graph based on addressing dependencies, version dependencies, and transaction commit dependencies, and establish a bi-state metadata digital twin containing actual state and shadow state;

[0010] Step 3: Input the metadata operation sequence and dynamic metadata association graph into the improved Todyformer model, generate metadata addressing paths through dependency gating, local message passing and cross-segment temporal attention, expand dependencies according to path confidence, and generate metadata critical path closures with transaction commit barriers and reachability of effective mapping root nodes as closure conditions.

[0011] Step 4: Generate a set of candidate metadata management actions and a causal action mask based on the node topology and operation prerequisite relationships of the metadata critical path closure;

[0012] Step 5: Input the metadata critical path closure, candidate metadata management action set, and causal action mask into the improved EfficientZero V2 model. Generate action sequence branches through closure state encoding, shadow state transition, and mask tree search. Prune action sequence branches that violate persistence order and atomic transaction integrity to determine the target metadata management trajectory.

[0013] Step 6: Update the metadata according to the topological order of the target metadata management trajectory and the metadata critical path closure, write the commit anchor, switch the valid mapping root node after persistence confirmation, and generate the metadata commit result;

[0014] Step 7: Calibrate the bi-state metadata digital twin based on the metadata submission results. In case of failure, restore the unclosed metadata associations within the critical path closure of the metadata based on the submission anchor point.

[0015] Optionally, step one specifically includes:

[0016] Obtain metadata operations from the mapping management interface, version management interface, and transaction management interface, and convert the metadata operations into metadata events that include operation identifier, transaction identifier, metadata object identifier, operation type, operation timestamp, operation sequence number within the transaction, and preceding metadata object identifier;

[0017] The operation timestamps of metadata events are unified using a hybrid logical clock, duplicate metadata events are merged using hash digests, and missing preceding metadata object identifiers are supplemented based on transaction logs.

[0018] The metadata events are sorted according to the hybrid logical clock, the in-transaction operation sequence number, and the metadata operation dependency order to generate a metadata operation sequence.

[0019] Optionally, step two specifically includes:

[0020] Metadata nodes are established using address mapping pages, version records, transaction log records, transaction commit records, and mapping root records. Addressing dependency edges, version dependency edges, and transaction commit dependency edges are established based on the lookup relationship, version inheritance relationship, and persistence order relationship between metadata nodes, respectively, to generate a dynamic metadata association graph.

[0021] Incrementally update metadata nodes and dependency edges according to the metadata operation sequence, and synchronize metadata nodes and dependency edges that have been persisted and can be reached from the current valid mapping root node to the actual state.

[0022] The copy-on-write method is used to establish shadow states based on the actual state and to establish mutually isolated shadow state branches. The dynamic metadata association graph, the actual state, the shadow state, and the node mapping relationship between the actual state and the shadow state are combined to form a bi-state metadata digital twin.

[0023] Optionally, step three specifically includes:

[0024] An improved Todyformer model is constructed, comprising a metadata manipulation embedding layer, a dependency type gating layer, a relationship graph attention layer, a cross-segment temporal attention layer, a metadata addressing path decoding layer, and a closure generation layer;

[0025] The metadata operation embedding layer encodes the metadata operation sequence and the actual state to generate metadata operation representations and metadata node representations;

[0026] The dependency type gating layer generates dependency gating values ​​based on metadata operation representations, metadata node representations, and dependency types. The relational graph attention layer performs message aggregation on addressing dependencies, version dependencies, and transaction commit dependencies based on the dependency gating values ​​to generate local dependency representations.

[0027] The cross-segment temporal attention layer performs temporal association on different metadata operation segments and merges local dependency representations to generate metadata association representations.

[0028] The metadata addressing path decoding layer uses a beam search to expand step by step from the metadata node associated with the current metadata operation to generate a set of candidate metadata addressing paths. The path confidence is generated based on the node selection probability and dependency gating value in each candidate metadata addressing path, and the metadata addressing path is determined based on the path confidence.

[0029] The closure generation layer generates metadata critical path closures based on metadata addressing paths, dynamic metadata association graphs, and currently valid mapping root nodes.

[0030] Optionally, the closure generation layer specifically includes:

[0031] Add the metadata nodes in the metadata addressing path to the closure node set, and expand the adjacent valid dependency edges according to the path confidence.

[0032] When extending to the transaction commit node of the current transaction, a transaction commit barrier is established, the necessary preceding metadata nodes that are missing before the transaction commit barrier are added to the closure node set, and the dependency edges from the transaction commit node of the current transaction to other transaction metadata nodes are blocked.

[0033] Search for reachable nodes along the addressing dependency edges from the current valid mapping root node. For closure nodes that cannot be reached from the current valid mapping root node, supplement intermediate metadata nodes along the reverse addressing dependency until the current valid mapping root node can reach all necessary metadata nodes.

[0034] Mark the metadata addressing path node, necessary preceding metadata node, and supplementary intermediate metadata node as necessary metadata nodes;

[0035] Detect dependency cycles and version conflicts in the closure node set. When conflicting version nodes belong to different transactions, retain the version branch with the same identifier as the current transaction. When conflicting version nodes belong to the same transaction, retain the version branch with the later version number. When the version numbers are the same, retain the version branch with the higher path confidence and mark the conflicting dependency edge corresponding to the excluded version branch.

[0036] Perform topological sorting on the set of closure nodes. When all necessary metadata nodes are located before the transaction commit barrier of the current transaction, the current valid mapping root node can reach all necessary metadata nodes, and the topological sorting is successful, generate a metadata critical path closure that includes necessary metadata node markers, transaction commit barriers, conflict dependency edge markers, and the topological order of closure nodes.

[0037] Optionally, step four specifically includes:

[0038] The path contribution is generated based on the occurrence ratio and path position of each metadata node in the candidate metadata addressing path set, and cache residency action and cache removal action are generated based on the path contribution.

[0039] Based on the transaction ownership, logical address continuity, and version relationship of the closure node, map merging action, transaction splitting action, and version chain compression action are generated. Based on the necessary metadata node tags, commit preparation action, effective mapping root node switching action, and old version release action are generated in sequence to form a candidate metadata management action set.

[0040] For each candidate metadata management action, establish a set of preceding nodes, a set of preceding actions, a read set, and a write set. Generate a causal action mask based on the preceding node status, the preceding action status, the version validity status, and the read / write conflict status. Only expose candidate metadata management actions whose dependency conditions have been met, and recalculate the causal action mask after the shadow status changes.

[0041] Optionally, step five specifically includes:

[0042] An improved EfficientZero V2 model is constructed, comprising a closure state encoding layer, a candidate action encoding layer, a shadow state transition layer, a management cost prediction layer, a policy value output layer, a mask tree search layer, and a metadata consistency constraint layer.

[0043] The closure state encoding layer generates a closure state vector based on the critical path closure of the metadata, and the candidate action encoding layer generates a candidate action vector based on the candidate metadata management action set.

[0044] The shadow state transition layer predicts the next closure state based on the closure state vector and candidate action vector, and updates the metadata node, dependency edge, necessary metadata node persistence state and mapping root node state in the corresponding shadow state branch.

[0045] The management cost prediction layer generates metadata addressing cost, persistence cost, transaction wait cost, and fault recovery cost based on the shadow state branch, and normalizes and weights each cost to generate the management cost.

[0046] The strategy value output layer generates action prior values ​​and shadow state values. The mask tree search layer infers candidate metadata management action sequences based on causal action masks. The metadata consistency constraint layer prunes action sequence branches that violate persistence order or atomic transaction integrity. The target metadata management trajectory is determined based on the management cost and shadow state value of the remaining action sequence branches.

[0047] Optionally, the mask tree search layer uses the current shadow state as the root node of the search tree, only considers the candidate metadata management actions that expose the causal action mask as extensible actions, and uses the PUCT selection function to determine the current extensible action;

[0048] After executing the current extended action, generate the next shadow state, and redetermine the previous node state, previous action state, version validity state, and read / write conflict state based on the next shadow state, and update the causal action mask;

[0049] The corresponding action sequence branch is pruned when the metadata consistency constraint layer is committed before the successor metadata node completes persistence, the effective mapping root node is switched before all necessary metadata nodes complete persistence, or the old version metadata node is released before the new version metadata node can be reached by the new mapping root node.

[0050] If the set of nodes to be committed corresponding to the commit preparation action does not cover all necessary metadata nodes, or if the transaction split action cuts off the transaction commit dependency edge that needs to maintain atomic commit, prune the corresponding action sequence branch;

[0051] The action sequence that performs persistent order verification and atomic transaction integrity verification is sorted according to management cost and shadow state value to determine the target metadata management trajectory.

[0052] Optionally, step six specifically includes:

[0053] Candidate metadata management actions are executed according to the target metadata management trajectory and the topological order of closure nodes. A new version of the metadata page is generated using the copy-on-write method, and the persistent state of the necessary metadata nodes is updated.

[0054] When performing the commit preparation action, a metadata critical path closure summary is generated based on the closure node identifier, version information and dependency type. The metadata critical path closure summary, the original valid mapping root node address, the new mapping root node address, the set of persisted metadata nodes and the commit sequence number are combined to form the commit anchor.

[0055] Persist the new version metadata page and commit anchor. When it is confirmed that the new version metadata page and commit anchor have been persisted, the set of persisted metadata nodes covers all necessary metadata nodes, and the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the commit anchor, execute the effective mapping root node switching action.

[0056] After the effective mapping root node switch is completed, the old version release action is executed. Based on the target metadata management trajectory execution status, effective mapping root node status, commit anchor status, and the final status of each metadata node in the metadata critical path closure, the metadata commit result is generated.

[0057] Optionally, step seven specifically includes:

[0058] Compare the metadata submission result with the shadow state prediction result corresponding to the target metadata management trajectory, update the actual state according to the node mapping relationship between the actual state and the shadow state, and form an online calibration sample from the state deviation.

[0059] The path prediction results of the improved Todyformer model are calibrated based on online calibration samples, and the model parameters of the improved EfficientZero V2 model are updated.

[0060] When a failure occurs, the metadata critical path closure is reconstructed based on the valid mapped root node record and the commit anchor with the largest commit sequence number that has passed the integrity check, and the unattached metadata nodes, unclosed version metadata nodes and unclosed commit transactions are identified.

[0061] When all necessary metadata nodes are persisted and the new mapping root node can reach all necessary metadata nodes, the switch to the effective mapping root node is completed. If there are necessary metadata nodes that have not been persisted, the original effective mapping root node remains valid, and new version metadata nodes that are not referenced by the effective mapping root node or the effective version are reclaimed.

[0062] Complete the addressing dependency edges based on the unattached metadata nodes, complete the version dependency edges based on the unclosed version metadata nodes, complete the transaction commit dependency edges based on the unclosed committed transactions, and determine the metadata nodes to continue committing or rollback.

[0063] Metadata recovery results are generated when the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the commit anchor, the current valid mapping root node can reach all necessary metadata nodes, and there are no unclosed transaction commit dependency edges. The dynamic metadata association graph and bi-state metadata digital twin are updated based on the metadata recovery results.

[0064] The beneficial effects of this invention are:

[0065] This invention constructs a bi-state metadata digital twin containing actual and shadow states, unifying addressing dependencies, version dependencies, and transaction commit dependencies onto a dynamic metadata association graph. It then utilizes an improved Todyformer model to generate metadata addressing paths, and combines transaction commit barriers, effective mapping of root node reachability, version conflict handling, and topology sorting to form a metadata critical path closure. This technology addresses the problems of existing solutions that struggle to accurately define the necessary metadata scope for the current transaction, easily overlooking preceding nodes or introducing irrelevant transaction nodes. It transforms metadata management from full scanning or partial empirical judgment to closure-based processing oriented towards transaction boundaries and mapping reachability, reducing the involvement of irrelevant metadata in addressing, persistence, and recovery, and improving the accuracy of metadata processing scope and the integrity of transaction commits.

[0066] This invention further generates a set of candidate metadata management actions and a causal action mask based on the metadata critical path closure. It then uses an improved EfficientZero V2 model to deduce action sequences in shadow states, dynamically restricting actions that do not meet preconditions and pruning branches that violate persistence order and atomic transaction integrity. Simultaneously, it employs commit anchors to bind closure digests, necessary metadata nodes, and the state of the mapped root node. This linkage mechanism solves the problems of sequential conflicts, version misreleases, and excessively large fault recovery ranges when multiple types of metadata actions are performed concurrently. It ensures that the generated target metadata management trajectory has direct execution conditions and, after a fault occurs, completes addressing dependencies, version dependencies, and transaction commit dependencies respectively. This provides a reliable technical foundation for reducing metadata write overhead, narrowing the recovery range, and improving the continuous operation capability of all-flash systems. Attached Figure Description

[0067] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0068] Figure 1 This is a flowchart of an all-flash metadata management method based on digital twins proposed in this invention;

[0069] Figure 2 This is a framework diagram of the improved Todyformer model in the all-flash metadata management method based on digital twin proposed in this invention;

[0070] Figure 3 This is a framework diagram of the improved EfficientZero V2 model in the all-flash metadata management method based on digital twins proposed in this invention. Detailed Implementation

[0071] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0072] refer to Figures 1-3 A method for managing metadata based on digital twins using all-flash memory includes the following steps:

[0073] Step 1: Collect metadata operations in the all-flash system and generate a metadata operation sequence according to the operation timing.

[0074] Step 2: Construct a dynamic metadata association graph based on addressing dependencies, version dependencies, and transaction commit dependencies, and establish a bi-state metadata digital twin containing actual state and shadow state;

[0075] Step 3: Input the metadata operation sequence and dynamic metadata association graph into the improved Todyformer model, generate metadata addressing paths through dependency gating, local message passing and cross-segment temporal attention, expand dependencies according to path confidence, and generate metadata critical path closures with transaction commit barriers and reachability of effective mapping root nodes as closure conditions.

[0076] Step 4: Generate a set of candidate metadata management actions and a causal action mask based on the node topology and operation prerequisite relationships of the metadata critical path closure;

[0077] Step 5: Input the metadata critical path closure, candidate metadata management action set, and causal action mask into the improved EfficientZero V2 model. Generate action sequence branches through closure state encoding, shadow state transition, and mask tree search. Prune action sequence branches that violate persistence order and atomic transaction integrity to determine the target metadata management trajectory.

[0078] Step 6: Update the metadata according to the topological order of the target metadata management trajectory and the metadata critical path closure, write the commit anchor, switch the valid mapping root node after persistence confirmation, and generate the metadata commit result;

[0079] Step 7: Calibrate the bi-state metadata digital twin based on the metadata submission results. In case of failure, restore the unclosed metadata associations within the critical path closure of the metadata based on the submission anchor point.

[0080] In this embodiment, step one specifically includes:

[0081] Obtain metadata operations from the mapping management interface, version management interface, and transaction management interface, and convert the metadata operations into metadata events. The metadata events include operation identifier, transaction identifier, metadata object identifier, operation type, operation timestamp, operation sequence number within the transaction, and previous metadata object identifier.

[0082] A hybrid logical clock is used to unify the operation timestamps generated by different storage controllers. The hybrid logical clock includes physical time and logical count. When the physical time of the current metadata event is later than the physical time of the previous metadata event, the logical count is restored to the initial value. When the physical times are the same, the logical count is incremented.

[0083] The SHA-256 hash function is used to perform digest processing on the operation identifier, transaction identifier, metadata object identifier and operation type, and duplicate metadata events with the same digest result are merged.

[0084] When a metadata event lacks a preceding metadata object identifier, the transaction identifier is used to determine the most recent metadata operation involving the same metadata object before the current metadata operation from the transaction log, and the corresponding metadata object identifier is written into the preceding metadata object identifier. When the transaction ownership cannot be determined, an independent transaction identifier is configured.

[0085] Metadata events are stably sorted according to the hybrid logical clock, intra-transaction operation sequence number, and metadata operation dependency order to generate a metadata operation sequence that retains the operation timing, transaction ownership, and preceding dependencies.

[0086] In this embodiment, step two specifically includes:

[0087] Metadata nodes are established using address mapping pages, version records, transaction log records, transaction commit records, and mapping root records. When a metadata node records the lookup entry or storage location of another metadata node, an addressing dependency edge is established from the metadata node that records the lookup entry to the metadata node being looked up. When a metadata node is a predecessor or parent version of another metadata node, a version dependency edge is established from the predecessor version node to the successor version node. When a metadata node needs to be persisted before another metadata node, a transaction commit dependency edge is established from the metadata node that persists first to the metadata node that persists later.

[0088] Configure transaction identifiers, dependency numbers, and valid states for addressing dependency edges, version dependency edges, and transaction commit dependency edges. Use an adjacency table to store the incoming and outgoing edges of metadata nodes, and build a dependency edge index based on the metadata object identifier, transaction identifier, and dependency type to generate a dynamic metadata association graph.

[0089] Incremental updates are performed on the dynamic metadata association graph according to the metadata operation sequence. The metadata creation operation establishes metadata nodes, the metadata modification operation adds or updates dependent edges, the metadata invalidation operation marks the corresponding metadata node as pending recycling, and the transaction commit operation marks the corresponding transaction commit dependent edge as closed.

[0090] The metadata nodes that have been persisted and can be reached from the current valid mapping root node along the addressing dependency edge, along with their corresponding valid dependency edges, will be synchronized to the actual state. The actual state records the version state, cache state, persistence state, transaction state, and valid dependency relationships.

[0091] The write-on-write method is used to establish shadow states based on the actual state. When the shadow state is initially established, the metadata node index and dependency edge index associated with the current metadata operation are copied. When the shadow state of the metadata node changes for the first time, the corresponding complete state is copied. And the shadow state branches that are isolated from each other are established for different shadow state transition processes.

[0092] The dynamic metadata association graph, actual state, shadow state, and node mapping relationship between actual state and shadow state are combined to form a bi-state metadata digital twin.

[0093] In this embodiment, step three specifically includes:

[0094] An improved Todyformer model is constructed, which includes a metadata operation embedding layer, a dependency type gating layer, a relationship graph attention layer, a cross-segment temporal attention layer, a metadata addressing path decoding layer, and a closure generation layer.

[0095] The metadata operation sequence is divided into metadata operation segments according to the transaction commit operation. When the metadata operations contained in a single transaction exceed the single input capacity of the model, a sliding slicing method with overlapping areas is used to divide the metadata operation segments so that continuous metadata operations that cross the segment boundary are simultaneously retained in adjacent metadata operation segments.

[0096] The metadata operation embedding layer performs embedding encoding on the operation type, transaction identifier, metadata object type, and operation order, and then performs linear projection, GELU activation, and LayerNorm processing in sequence to generate a metadata operation representation; it also performs embedding encoding and linear projection on the metadata node type, version status, cache status, persistence status, and transaction status in the actual state to generate a metadata node representation.

[0097] The dependency type gating layer concatenates the source metadata node representation, the target metadata node representation, the dependency type representation, and the current metadata operation representation, and generates a dependency gating value through linear transformation and Sigmoid function processing. When the dependency edge is in an invalid state or the transaction to which the dependency edge belongs is outside the processing range of the current transaction, the corresponding dependency gating value is set to zero.

[0098] The relational graph attention layer performs scaling dot product attention for addressing dependencies, version dependencies, and transaction commit dependencies respectively. It maps the target metadata node representation to a query vector, maps the adjacent source metadata node representations to key vectors and value vectors, and adds the dependency gating value as an attention bias to the scaling dot product result. After Softmax normalization and weighted aggregation, it generates aggregated messages for each dependency type. Then, it generates local dependency representations through linear projection, residual connection, and LayerNorm processing.

[0099] The cross-segment temporal attention layer uses Time2Vec to encode the mixed logical clock difference between metadata operation segments, performs temporal attention calculation on the local dependency representation of the current metadata operation segment and the local dependency representation of the historical metadata operation segment to generate a cross-segment dependency representation, and uses Sigmoid gating to fuse the local dependency representation and the cross-segment dependency representation to generate a metadata association representation;

[0100] The metadata addressing path decoding layer takes the metadata node associated with the current metadata operation as the starting node, and uses beam search to expand the candidate successor metadata nodes step by step. It jointly processes the metadata association representation, node selection probability and dependency gating value of the candidate successor metadata nodes to generate a set of candidate metadata addressing paths.

[0101] For each candidate metadata addressing path, the path length is normalized by taking the logarithm of the selection probability of each node and the dependency gate value, and then the path confidence is generated by using an exponential function. The candidate metadata addressing path with the highest path confidence that reaches the transaction commit node, the current valid mapping root node, or has no expandable successor metadata node is determined as the metadata addressing path.

[0102] Specifically, the initial path confidence threshold is set to 0.65. When the recall rate of necessary metadata nodes in historical metadata operations is less than 99%, the path confidence threshold is reduced by 0.02. When the recall rate of necessary metadata nodes reaches 99% and the proportion of redundant closure nodes exceeds 20%, the path confidence threshold is increased by 0.02. The path confidence threshold is limited to between 0.50 and 0.80.

[0103] The closure generation layer adds metadata nodes in the metadata addressing path to the closure node set, and uses a maximum priority queue to arrange the effective dependency edges adjacent to the closure node set in descending order of path confidence. Effective dependency edges whose path confidence reaches the current path confidence threshold and their corresponding successor metadata nodes are added to the closure node set.

[0104] When the dependency extends to the transaction commit node of the current transaction, a transaction commit barrier is established, the necessary preceding metadata nodes that were missing before the transaction commit barrier are added to the closure node set, and the dependency edges from the transaction commit node of the current transaction to other transaction metadata nodes are blocked.

[0105] Perform a breadth-first search along the addressing dependency edges from the current valid mapping root node to generate the root node reachable set; if the closure node does not belong to the root node reachable set, search for the intermediate metadata node connecting the current valid mapping root node along the reverse addressing dependency from the closure node, and add the intermediate metadata node to the closure node set until the current valid mapping root node can reach all necessary metadata nodes.

[0106] Mark the metadata addressing path nodes, the necessary preceding metadata nodes added before the transaction commit barrier, and the intermediate metadata nodes added to satisfy the reachability conditions of the valid mapping root node as necessary metadata nodes.

[0107] The Tarjan strongly connected component algorithm is used to detect dependency cycles and version conflicts in the closure node set. When conflicting version nodes belong to different transactions, the version branch with the same identifier as the current transaction is retained. When conflicting version nodes belong to the same transaction, the version branch with the later version number is retained. When the version numbers are the same, the version branch with the higher path confidence is retained. The dependency edges corresponding to the excluded version branches are marked as conflicting dependency edges.

[0108] The Kahn topological sorting algorithm is used to sort the set of closure nodes. Metadata nodes with an in-degree of zero are added to the topological queue. After each metadata node is taken out, the in-degree of the corresponding successor metadata node is reduced until the closure nodes are sorted.

[0109] When all necessary metadata nodes are located before the transaction commit barrier of the current transaction, the current valid mapping root node can reach all necessary metadata nodes, and the Kahn topology sorting is successful, a metadata critical path closure is generated. The metadata critical path closure includes a set of closure nodes, a set of closure dependency edges, necessary metadata node markers, a transaction commit barrier, the current valid mapping root node, conflict dependency edge markers, and the topology order of closure nodes.

[0110] Training samples are formed by extracting the actual access node sequence and actual transaction commit relationship from historical metadata operation records. The selection results of candidate successor metadata nodes are trained using cross-entropy loss, the dependency edge path attribution results are trained using binary cross-entropy loss, and the necessary metadata nodes are constrained to enter the metadata critical path closure using closure omission loss. The transaction commit barrier, effective mapping root node reachability judgment, version conflict handling, and dependency cycle handling are kept as non-training rules.

[0111] In this embodiment, step four specifically includes:

[0112] Based on the candidate metadata addressing path set and metadata critical path closure, generate cache residency action, cache remove action, mapping merge action, transaction split action, version chain compression action, commit preparation action, effective mapping root node switching action, and old version release action;

[0113] The path occurrence ratio is generated by dividing the number of times the metadata node appears in the candidate metadata addressing path set by the number of candidate metadata addressing paths. The path occurrence ratio is then combined with the reverse depth weight of the metadata node in the candidate metadata addressing path to generate the path contribution.

[0114] Based on the path contribution from high to low, the metadata nodes whose cumulative path contribution reaches 80% of the cumulative path contribution of all closure nodes are identified as key resident nodes. A cache residency action is generated for key resident nodes that do not reside in the metadata cache, and a cache removal action is generated for cache nodes that have not entered the metadata key path closure and have not been accessed within the preset historical period.

[0115] Perform continuous interval scanning on mapping metadata nodes with the same transaction identifier, consecutive logical addresses, and the same version sequence number to generate mapping merging actions. Mapping metadata nodes associated with conflicting dependency edges do not participate in mapping merging.

[0116] When the metadata critical path closure exceeds the capacity of a single transaction, the transaction splitting position is enumerated according to the topological order of the closure nodes. The transaction splitting position that cuts the transaction commit dependency edge that needs to maintain atomic commit is excluded, and the transaction splitting position with the fewest number of dependency edges is selected to generate the transaction splitting action.

[0117] When a version dependency chain contains three or more consecutive version nodes, the middle version node is identified as the version node to be compressed. When the version node to be compressed does not have a valid reference outside the closure, is not associated with conflicting dependency edges, and the necessary mapping relationship can be transferred to the adjacent valid version node, a version chain compression action is generated.

[0118] Generate a commit preparation action based on the necessary metadata node markers, generate a valid mapping root node switching action based on the completion status of the commit preparation action, generate an old version release action based on the reachability status of the new version metadata node relative to the new mapping root node, and combine the candidate metadata management actions into a candidate metadata management action set.

[0119] For each candidate metadata management action, establish a set of preceding nodes, a set of preceding actions, a read set, and a write set. Generate a causal action mask based on the status of preceding nodes, the status of preceding actions, the version validity status, and the read / write conflict status. When all preceding nodes meet the required status, all preceding actions are completed, the metadata version in the read set is valid, and there is no write conflict in the write set, the corresponding candidate metadata management action is opened. When any condition is not met, the corresponding candidate metadata management action is closed.

[0120] Specifically, the commit preparation action is only enabled when all necessary metadata nodes enter the persistent state, the effective mapping root node switching action is only enabled when all necessary metadata nodes have been persisted and the commit anchor has been written, and the old version release action is only enabled when the new version metadata node can be reached by the new mapping root node and the old version metadata node has no valid reference.

[0121] After the shadow state changes, the causal action mask is regenerated based on the updated preceding node state, preceding action state, version validity state, and read / write conflict state, forming a candidate metadata management action set and a causal action mask.

[0122] In this embodiment, step five specifically includes:

[0123] An improved EfficientZero V2 model is constructed, which includes a closure state encoding layer, a candidate action encoding layer, a shadow state transition layer, a management cost prediction layer, a policy value output layer, a mask tree search layer, and a metadata consistency constraint layer.

[0124] The closure state encoding layer performs relational graph attention encoding on the critical path closure of metadata. It uses the current valid mapping root node representation as the query vector and the closure node representation as the key vector and value vector to perform Softmax attention pooling. It also performs joint encoding on the necessary metadata node markings, transaction commit barriers and closure node topology order to generate the closure state vector.

[0125] The candidate action encoding layer encodes the action type, operation object, preceding node, preceding action, read set, and write set of the candidate metadata management actions to generate candidate action vectors.

[0126] The shadow state transition layer uses an action-conditional residual multilayer perceptron to concatenate the closure state vector and the candidate action vector, and then generates the state change through linear transformation, LayerNorm and GELU activation processing. The state change is then added to the closure state vector by residual to generate the predicted next closure state.

[0127] When performing candidate metadata management actions, the metadata nodes, dependency edges, necessary metadata node persistence states, and mapping root node states are updated in the corresponding shadow state branches according to the deterministic processing rules of the candidate metadata management actions. The updated shadow states are then input into the closure state encoding layer to generate the verification closure state. The cosine consistency loss constraint is used to predict the next closure state to be consistent with the verification closure state.

[0128] The management cost prediction layer generates metadata addressing cost, persistence cost, transaction wait cost, and fault recovery cost based on the shadow state branches. The metadata addressing cost is generated based on the number of addressing layers from the current valid mapping root node to the metadata node to be accessed. The persistence cost is generated based on the number of newly added metadata pages and commit records. The transaction wait cost is generated based on the number of unsatisfied transaction commit dependencies. The fault recovery cost is generated based on the number of unclosed metadata nodes and dependent edges. The various costs are normalized and weighted to generate the management cost.

[0129] The strategy value output layer generates the action prior value and shadow state value of candidate metadata management actions;

[0130] The mask tree search layer uses PUCT tree search, with the current shadow state as the root node of the search tree. It only considers candidate metadata management actions with open causal action masks as scalable actions. It calculates the PUCT selection result based on the average value of the action, the prior value of the action, the number of visits to the parent node, and the number of visits to the child node, and selects the candidate metadata management action with the largest PUCT selection result to generate the next shadow state.

[0131] After generating the next shadow state, the causal action mask is regenerated based on the previous node state, previous action state, version validity state, and read / write conflict state in the next shadow state.

[0132] The metadata consistency constraint layer performs persistence order verification and atomic transaction integrity verification based on the topological order of closure nodes, necessary metadata node tags, and transaction commit barriers.

[0133] When a successor metadata node is committed before the preceding metadata node completes persistence, when an effective mapping root node is switched before all necessary metadata nodes complete persistence, or when an old version metadata node is released before the new version metadata node can be reached by the new mapping root node, the corresponding action sequence branch is pruned.

[0134] If the set of nodes to be committed corresponding to the commit preparation action does not cover all necessary metadata nodes, or if the transaction split action cuts off the transaction commit dependency edge that needs to maintain atomic commit, prune the corresponding action sequence branch;

[0135] The action sequence branches that pass the persistence order verification and atomic transaction integrity verification are evaluated. The normalized shadow state value is subtracted from the normalized management cost to generate a trajectory evaluation value. The action sequence branch with the highest trajectory evaluation value is determined as the target metadata management trajectory.

[0136] Training samples are formed by extracting the pre-execution closure state, actual execution action, post-execution closure state, and actual management cost from historical metadata management records. The management cost prediction layer and shadow state value are trained using Huber loss, the action prior value is trained using cross-entropy loss, and the shadow state transition layer is trained using cosine consistency loss. The causal action mask generation rule, persistence order verification rule, and atomic transaction integrity verification rule are kept as non-training constraints.

[0137] In this embodiment, step six specifically includes:

[0138] Execute candidate metadata management actions according to the action execution order in the target metadata management trajectory and the topological order of closure nodes in the metadata critical path closure;

[0139] When performing a cache residency action, the corresponding metadata node is loaded into the metadata cache area and the cache state in the actual state is updated; when performing a cache remove action, the metadata node with updated content is written to the flash storage device and the corresponding cache space is released; when performing a mapping merge action, multiple mapping updates are written to the same new version metadata page in logical address order; when performing a transaction split action, sub-transactions are established and committed sequentially according to the transaction split position; when performing a version chain compression action, the necessary mapping relationship is transferred to the adjacent valid version node, and the version node to be compressed is set to the state to be reclaimed after the version dependency relationship is persisted.

[0140] The new version metadata page is generated using the copy-on-write method. The current valid version metadata page is not overwritten. Each write operation records the storage address, transaction identifier, version number, and CRC64 check value. The persistence status of the necessary metadata nodes is updated according to the write completion status.

[0141] When performing the commit preparation action, the closure node identifiers, version information and dependency types are arranged according to the topological order of the closure nodes. The SHA-256 hash function is used to generate the metadata critical path closure digest. The metadata critical path closure digest, the original valid mapping root node address, the new mapping root node address, the set of persistent metadata nodes and the commit sequence number are combined to form the commit anchor.

[0142] The commit anchor is appended to the commit log, and the NVMe Flush command is used to perform a persistent refresh of the new version metadata page and the commit anchor. The persistent confirmation of the new version metadata page and the persistent confirmation of the commit anchor are returned by the flash storage device.

[0143] When performing a valid mapping root node switch action, the valid mapping root node record is stored in the primary root node slot and the backup root node slot. The new mapping root node address, commit sequence number and verification information are written to the non-current valid root node slot. After the non-current valid root node slot is persisted, the valid root node slot identifier is updated.

[0144] Specifically, the effective mapping root node switching action is performed only when the new version metadata page and the commit anchor have been persisted, the set of persisted metadata nodes covers all necessary metadata nodes, and the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the commit anchor. If any condition is not met, the original effective mapping root node remains valid.

[0145] When performing the old version release action, the old version metadata nodes that are no longer referenced by the new mapping root node or other valid versions will be set to the pending recycling state, and the commit anchor will be updated to the committed state.

[0146] The metadata submission result is generated based on the execution status of the target metadata management trajectory, the status of the effective mapping root node, the status of the submission anchor point, and the final status of each metadata node within the metadata critical path closure.

[0147] In this embodiment, step seven specifically includes:

[0148] Compare the metadata submission results with the shadow state prediction results corresponding to the target metadata management trajectory, and compare the metadata node states, dependency edge states, persistent states and mapping root node states item by item according to the node mapping relationship between the actual state and the shadow state.

[0149] When the actual execution result is consistent with the shadow state prediction result, the state changes in the corresponding shadow state branch are merged into the actual state; when the actual execution result is inconsistent with the shadow state prediction result, the actual state is updated according to the metadata submission result, and the inconsistent metadata nodes, dependency edges and candidate metadata management actions are combined into an online calibration sample.

[0150] The path prediction results of the improved Todyformer model are calibrated using a temperature scaling method based on the predicted probability of candidate successor metadata nodes in the online calibration samples and the actual access results. The path confidence threshold is adjusted based on the recall rate of necessary metadata nodes and the proportion of closure redundant nodes.

[0151] The actual closure state and actual management cost in the online calibration samples are written into the priority experience replay buffer. The sample priority is set according to the prediction error, and the shadow state transition layer, management cost prediction layer and policy value output layer of the improved EfficientZero V2 model are updated according to the incremental update cycle.

[0152] When a storage controller restarts, metadata management service fails, or transaction commit is interrupted, CRC64 integrity checks are performed on the valid mapped root node records in the primary root node slot and the standby root node slot, and the valid mapped root node record with the largest commit sequence number is selected from the valid mapped root node records that pass the integrity check.

[0153] Starting from the end of the commit log, examine the commit anchors backwards and select the commit anchor with the largest commit sequence number that passes the integrity check as the most recent complete commit anchor. Then, reconstruct the metadata critical path closure based on the metadata critical path closure summary, the original valid mapping root node address, the new mapping root node address, and the set of persisted metadata nodes in the most recent complete commit anchor.

[0154] Metadata nodes that have been persisted but cannot be reached from the current valid mapping root node are identified as unattached metadata nodes. Metadata nodes that have generated subsequent versions but whose corresponding version dependency edges have not been persisted are identified as unclosed version metadata nodes. Transactions that have not been persisted by all necessary metadata nodes but have been generated by transaction commit nodes are identified as unclosed commit transactions.

[0155] When all necessary metadata nodes have been persisted and the new mapping root node can reach all necessary metadata nodes, the effective mapping root node switch is completed; if there are necessary metadata nodes that have not been persisted, the original effective mapping root node remains valid, and the new version metadata nodes that are not referenced by the original effective mapping root node or other effective versions are added to the queue to be reclaimed.

[0156] Complete the addressing dependency edges from the current valid mapping root node to the corresponding metadata node based on the unattached metadata nodes; complete the version dependency edges from the previous version node to the subsequent version node based on the unclosed version metadata nodes; complete the transaction commit dependency edges from the previous metadata node to the subsequent metadata node based on the unclosed committed transactions.

[0157] If all necessary metadata nodes corresponding to an unclosed transaction have been persisted and the commit anchor is complete, continue to execute the transaction commit. If necessary metadata nodes have not been persisted or the commit anchor is incomplete, roll back the uncompleted transaction and set the metadata nodes not referenced by the original valid mapping root node or valid version to a pending recycling state.

[0158] Recalculate the metadata critical path closure summary. If the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the most recent complete commit anchor, the current valid mapping root node can reach all necessary metadata nodes, and there are no unclosed transaction commit dependency edges, generate the metadata recovery result.

[0159] Update the actual state of the dynamic metadata association graph and the bi-state metadata digital twin based on the metadata recovery results, and delete the shadow state branches that were not completed before the failure occurred.

[0160] Example 1: To verify the feasibility of this invention in practice, it was applied to a dual-controller all-flash database storage system. The system is configured with eight enterprise-grade NVMe solid-state drives, a metadata page size of 16KB, a metadata cache capacity of 32GB, and approximately two million metadata nodes pre-established. The workload includes random address mapping updates, continuous version overwrite, and multi-object transaction commits. A single transaction contains 32 to 96 metadata operations, and the number of concurrent transactions is 128.

[0161] During system operation, metadata operations are collected from the mapping management interface, version management interface, and transaction management interface, sorted using a hybrid logical clock, and a metadata operation sequence is generated. Metadata nodes are established based on address mapping pages, version records, transaction log records, transaction commit records, and mapping root records. A dynamic metadata association graph is generated according to addressing dependencies, version dependencies, and transaction commit dependencies, and a shadow state for copy-on-write is established based on the actual state.

[0162] The improved Todyformer model divides the metadata operation sequence into metadata operation fragments, each containing a maximum of 256 metadata operations. Adjacent fragments retain 32 overlapping operations, and the initial path confidence threshold is set to 0.65. After generating a set of candidate metadata addressing paths, the model restricts cross-transaction dependency expansion through transaction commit barriers and supplements necessary metadata nodes based on the reachability of the effective mapping root node, generating a metadata critical path closure. The processing capacity of a single metadata critical path closure is set to 512 metadata nodes, and the cache access history period is set to the most recent 5000 metadata operations.

[0163] Based on the candidate metadata addressing path set and metadata critical path closure, generate cache residency actions, cache remove actions, mapping merge actions, transaction split actions, version chain compression actions, commit preparation actions, valid mapping root node switching actions, and old version release actions, and generate causal action masks based on the preceding node status, preceding action status, version validity status, and read / write conflict status.

[0164] The improved EfficientZero V2 model deduces candidate metadata management actions in the shadow state branches, with the weights for metadata addressing cost, persistence cost, transaction wait cost, and fault recovery cost set to 0.25, 0.30, 0.20, and 0.25, respectively. After each shadow state transition, a causal action mask is regenerated, and action sequence branches that violate persistence order or atomic transaction integrity are pruned to determine the target metadata management trajectory.

[0165] Generate a new version metadata page and commit anchor point according to the target metadata management trajectory and closure node topology order. After the new version metadata page and commit anchor point are persisted, the persisted metadata node set covers all necessary metadata nodes, and the metadata critical path closure digest is verified to be consistent, switch the valid mapping root node.

[0166] The test was executed continuously for ten rounds, each processing 100,000 metadata transactions, and randomly injected with controller restarts, transaction commit interruptions, and tail corruption of the commit log, resulting in 500 fault recovery attempts. Comparison Method A uses full metadata dependency scanning, fixed action rules, and full log replay recovery; Comparison Method B uses the unmodified Todyformer model to predict metadata addressing paths, without setting transaction commit barriers, effective root node reachability completion, or metadata critical path closures; Comparison Method C uses the unmodified EfficientZero V2 model to generate management actions, without setting causal action masks, persistence order verification, or atomic transaction integrity pruning.

[0167] Table 1. Comprehensive Test Results of Different Metadata Management Methods

[0168]

[0169] Comparative method A does not perform model derivation and has the lowest action planning latency, but it requires scanning a large range of metadata relationships. Compared with comparative method A, the P99 metadata addressing latency of this invention is reduced by 51.5%, the number of pages written for single-transaction persistence is reduced by 55.7%, the number of fault recovery scan nodes is reduced by 94.8%, and the fault recovery success rate is improved by 4.1 percentage points, indicating that the metadata critical path closure can significantly narrow the scope of metadata addressing, persistence, and fault recovery.

[0170] While method B offers faster path generation, it lacks transaction commit barriers and effective mapping root node reachability completion. Compared to method B, this invention reduces the number of persistent write pages by 41.3%, the number of fault recovery scan nodes by 88.5%, and the fault recovery success rate by 2.8 percentage points, demonstrating that the closure generation mechanism can avoid omitting necessary preceding metadata nodes and introducing irrelevant transaction nodes.

[0171] Comparative method C can proactively reduce the number of persistent write pages, but it lacks causal action masking and metadata consistency pruning, leading to an increase in invalid action sequences. Compared to comparative method C, the action planning latency of this invention is reduced by 61.1%, P99 metadata addressing latency is reduced by 25.6%, and the number of fault recovery scan nodes is reduced by 81.9%, indicating that causal action masking can significantly compress the tree search space and ensure that the target metadata management trajectory satisfies the persistence order and atomic transaction integrity.

[0172] This embodiment demonstrates that the present invention limits the necessary processing scope by defining the critical path closure of metadata, reduces invalid action sequences by using causal action masks and metadata consistency constraints, and enables fault recovery to complete addressing dependency edges, version dependency edges, and transaction commit dependency edges by associating closure digests, necessary metadata nodes, and valid mapping root node states through commit anchors. This results in significant comprehensive technical effects in terms of metadata addressing efficiency, persistent write volume, and fault recovery reliability.

[0173] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for managing metadata based on digital twins using all-flash memory, characterized in that: Includes the following steps: Step 1: Collect metadata operations in the all-flash system and generate a metadata operation sequence according to the operation timing. Step 2: Construct a dynamic metadata association graph based on addressing dependencies, version dependencies, and transaction commit dependencies, and establish a bi-state metadata digital twin containing actual state and shadow state; Step 3: Input the metadata operation sequence and dynamic metadata association graph into the improved Todyformer model, generate metadata addressing paths through dependency gating, local message passing and cross-segment temporal attention, expand dependencies according to path confidence, and generate metadata critical path closures with transaction commit barriers and reachability of effective mapping root nodes as closure conditions. Step 4: Generate a set of candidate metadata management actions and a causal action mask based on the node topology and operation prerequisite relationships of the metadata critical path closure; Step 5: Input the metadata critical path closure, candidate metadata management action set, and causal action mask into the improved EfficientZero V2 model. Generate action sequence branches through closure state encoding, shadow state transition, and mask tree search. Prune action sequence branches that violate persistence order and atomic transaction integrity to determine the target metadata management trajectory. Step 6: Update the metadata according to the topological order of the target metadata management trajectory and the metadata critical path closure, write the commit anchor, switch the valid mapping root node after persistence confirmation, and generate the metadata commit result; Step 7: Calibrate the bi-state metadata digital twin based on the metadata submission results. In case of failure, restore the unclosed metadata associations within the critical path closure of the metadata based on the submission anchor point.

2. The all-flash memory metadata management method based on digital twins as described in claim 1, characterized in that, Step one specifically includes: Obtain metadata operations from the mapping management interface, version management interface, and transaction management interface, and convert the metadata operations into metadata events that include operation identifier, transaction identifier, metadata object identifier, operation type, operation timestamp, operation sequence number within the transaction, and preceding metadata object identifier; The operation timestamps of metadata events are unified using a hybrid logical clock, duplicate metadata events are merged using hash digests, and missing preceding metadata object identifiers are supplemented based on transaction logs. The metadata events are sorted according to the hybrid logical clock, the in-transaction operation sequence number, and the metadata operation dependency order to generate a metadata operation sequence.

3. The all-flash memory metadata management method based on digital twins as described in claim 1, characterized in that, Step two specifically includes: Metadata nodes are established using address mapping pages, version records, transaction log records, transaction commit records, and mapping root records. Addressing dependency edges, version dependency edges, and transaction commit dependency edges are established based on the lookup relationship, version inheritance relationship, and persistence order relationship between metadata nodes, respectively, to generate a dynamic metadata association graph. Incrementally update metadata nodes and dependency edges according to the metadata operation sequence, and synchronize metadata nodes and dependency edges that have been persisted and can be reached from the current valid mapping root node to the actual state. The copy-on-write method is used to establish shadow states based on the actual state and to establish mutually isolated shadow state branches. The dynamic metadata association graph, the actual state, the shadow state, and the node mapping relationship between the actual state and the shadow state are combined to form a bi-state metadata digital twin.

4. The all-flash memory metadata management method based on digital twin as described in claim 1, characterized in that, Step three specifically includes: An improved Todyformer model is constructed, comprising a metadata manipulation embedding layer, a dependency type gating layer, a relationship graph attention layer, a cross-segment temporal attention layer, a metadata addressing path decoding layer, and a closure generation layer; The metadata operation embedding layer encodes the metadata operation sequence and the actual state to generate metadata operation representations and metadata node representations; The dependency type gating layer generates dependency gating values ​​based on metadata operation representations, metadata node representations, and dependency types. The relational graph attention layer performs message aggregation on addressing dependencies, version dependencies, and transaction commit dependencies based on the dependency gating values ​​to generate local dependency representations. The cross-segment temporal attention layer performs temporal association on different metadata operation segments and merges local dependency representations to generate metadata association representations. The metadata addressing path decoding layer uses a beam search to expand step by step from the metadata node associated with the current metadata operation to generate a set of candidate metadata addressing paths. The path confidence is generated based on the node selection probability and dependency gating value in each candidate metadata addressing path, and the metadata addressing path is determined based on the path confidence. The closure generation layer generates metadata critical path closures based on metadata addressing paths, dynamic metadata association graphs, and currently valid mapping root nodes.

5. A method for managing all-flash metadata based on digital twins according to claim 4, characterized in that, The closure generation layer specifically includes: Add the metadata nodes in the metadata addressing path to the closure node set, and expand the adjacent valid dependency edges according to the path confidence. When extending to the transaction commit node of the current transaction, a transaction commit barrier is established, the necessary preceding metadata nodes that are missing before the transaction commit barrier are added to the closure node set, and the dependency edges from the transaction commit node of the current transaction to other transaction metadata nodes are blocked. Search for reachable nodes along the addressing dependency edges from the current valid mapping root node. For closure nodes that cannot be reached from the current valid mapping root node, supplement intermediate metadata nodes along the reverse addressing dependency until the current valid mapping root node can reach all necessary metadata nodes. Mark the metadata addressing path node, necessary preceding metadata node, and supplementary intermediate metadata node as necessary metadata nodes; Detect dependency cycles and version conflicts in the closure node set. When conflicting version nodes belong to different transactions, retain the version branch with the same identifier as the current transaction. When conflicting version nodes belong to the same transaction, retain the version branch with the later version number. When the version numbers are the same, retain the version branch with the higher path confidence and mark the conflicting dependency edge corresponding to the excluded version branch. Perform topological sorting on the set of closure nodes. When all necessary metadata nodes are located before the transaction commit barrier of the current transaction, the current valid mapping root node can reach all necessary metadata nodes, and the topological sorting is successful, generate a metadata critical path closure that includes necessary metadata node markers, transaction commit barriers, conflict dependency edge markers, and the topological order of closure nodes.

6. The all-flash memory metadata management method based on digital twin as described in claim 1, characterized in that, Step four specifically includes: The path contribution is generated based on the occurrence ratio and path position of each metadata node in the candidate metadata addressing path set, and cache residency action and cache removal action are generated based on the path contribution. Based on the transaction ownership, logical address continuity, and version relationship of the closure node, map merging action, transaction splitting action, and version chain compression action are generated. Based on the necessary metadata node tags, commit preparation action, effective mapping root node switching action, and old version release action are generated in sequence to form a candidate metadata management action set. For each candidate metadata management action, establish a set of preceding nodes, a set of preceding actions, a read set, and a write set. Generate a causal action mask based on the preceding node status, the preceding action status, the version validity status, and the read / write conflict status. Only expose candidate metadata management actions whose dependency conditions have been met, and recalculate the causal action mask after the shadow status changes.

7. The all-flash metadata management method based on digital twins according to claim 1, characterized in that, Step five specifically includes: An improved EfficientZero V2 model is constructed, comprising a closure state encoding layer, a candidate action encoding layer, a shadow state transition layer, a management cost prediction layer, a policy value output layer, a mask tree search layer, and a metadata consistency constraint layer. The closure state encoding layer generates a closure state vector based on the critical path closure of the metadata, and the candidate action encoding layer generates a candidate action vector based on the candidate metadata management action set. The shadow state transition layer predicts the next closure state based on the closure state vector and candidate action vector, and updates the metadata node, dependency edge, necessary metadata node persistence state and mapping root node state in the corresponding shadow state branch. The management cost prediction layer generates metadata addressing cost, persistence cost, transaction wait cost, and fault recovery cost based on the shadow state branch, and normalizes and weights each cost to generate the management cost. The strategy value output layer generates action prior values ​​and shadow state values. The mask tree search layer infers candidate metadata management action sequences based on causal action masks. The metadata consistency constraint layer prunes action sequence branches that violate persistence order or atomic transaction integrity. The target metadata management trajectory is determined based on the management cost and shadow state value of the remaining action sequence branches.

8. A method for managing all-flash metadata based on digital twins according to claim 7, characterized in that, The mask tree search layer uses the current shadow state as the root node of the search tree, and only considers the candidate metadata management actions with open causal action masks as extensible actions, and uses the PUCT selection function to determine the current extensible action; After executing the current extended action, generate the next shadow state, and redetermine the previous node state, previous action state, version validity state, and read / write conflict state based on the next shadow state, and update the causal action mask; The corresponding action sequence branch is pruned when the metadata consistency constraint layer is committed before the successor metadata node completes persistence, the effective mapping root node is switched before all necessary metadata nodes complete persistence, or the old version metadata node is released before the new version metadata node can be reached by the new mapping root node. If the set of nodes to be committed corresponding to the commit preparation action does not cover all necessary metadata nodes, or if the transaction split action cuts off the transaction commit dependency edge that needs to maintain atomic commit, prune the corresponding action sequence branch; The action sequence that performs persistent order verification and atomic transaction integrity verification is sorted according to management cost and shadow state value to determine the target metadata management trajectory.

9. A method for managing all-flash metadata based on digital twins according to claim 1, characterized in that, Step six specifically includes: Candidate metadata management actions are executed according to the target metadata management trajectory and the topological order of closure nodes. A new version of the metadata page is generated using the copy-on-write method, and the persistent state of the necessary metadata nodes is updated. When performing the commit preparation action, a metadata critical path closure summary is generated based on the closure node identifier, version information and dependency type. The metadata critical path closure summary, the original valid mapping root node address, the new mapping root node address, the set of persisted metadata nodes and the commit sequence number are combined to form the commit anchor. Persist the new version metadata page and commit anchor. When it is confirmed that the new version metadata page and commit anchor have been persisted, the set of persisted metadata nodes covers all necessary metadata nodes, and the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the commit anchor, execute the effective mapping root node switching action. After the effective mapping root node switch is completed, the old version release action is executed. Based on the target metadata management trajectory execution status, effective mapping root node status, commit anchor status, and the final status of each metadata node in the metadata critical path closure, the metadata commit result is generated.

10. A method for managing all-flash metadata based on digital twins according to claim 1, characterized in that, Step seven specifically includes: Compare the metadata submission result with the shadow state prediction result corresponding to the target metadata management trajectory, update the actual state according to the node mapping relationship between the actual state and the shadow state, and form an online calibration sample from the state deviation. The path prediction results of the improved Todyformer model are calibrated based on online calibration samples, and the model parameters of the improved EfficientZero V2 model are updated. When a failure occurs, the metadata critical path closure is reconstructed based on the valid mapped root node record and the commit anchor with the largest commit sequence number that has passed the integrity check, and the unattached metadata nodes, unclosed version metadata nodes and unclosed commit transactions are identified. When all necessary metadata nodes are persisted and the new mapping root node can reach all necessary metadata nodes, the switch to the effective mapping root node is completed. If there are necessary metadata nodes that have not been persisted, the original effective mapping root node remains valid, and new version metadata nodes that are not referenced by the effective mapping root node or the effective version are reclaimed. Complete the addressing dependency edges based on the unattached metadata nodes, complete the version dependency edges based on the unclosed version metadata nodes, complete the transaction commit dependency edges based on the unclosed committed transactions, and determine the metadata nodes to continue committing or rollback. Metadata recovery results are generated when the recalculated metadata critical path closure summary is consistent with the metadata critical path closure summary in the commit anchor, the current valid mapping root node can reach all necessary metadata nodes, and there are no unclosed transaction commit dependency edges. The dynamic metadata association graph and bi-state metadata digital twin are updated based on the metadata recovery results.

Images