Disk flushing method, apparatus, equipment, media and products based on two-dimensional transaction linked lists

By constructing a two-dimensional transaction linked list and performing disk flushing according to the file and transaction operation sequence, the performance and consistency issues of cross-node access in the cluster file system are resolved, improving concurrency performance and maintaining data consistency.

CN123115067APending Publication Date: 2026-11-17CETC JINCANG (BEIJING) TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202611390757.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-11-17

AI Technical Summary

Technical Problem

Existing cluster file systems use a full disk flushing mode in cross-node access scenarios, which leads to inefficient file conflict handling, long disk flushing blocking time, and insufficient correlation processing, making it difficult to achieve a balance between performance and consistency.

Method used

A disk flushing method based on a two-dimensional transaction linked list is adopted. A two-dimensional transaction linked list containing file linked lists and transaction linked lists is constructed. By locating the two-dimensional transaction linked list of the file to be flushed, the associated metadata is identified and the file is flushed to disk in sequence according to the transaction operation time, thereby reducing the disk flushing range and lock blocking.

Benefits of technology

It reduces disk I/O load, decreases the scope of log lock blocking, improves the concurrent processing performance of the cluster file system, and maintains data and metadata consistency.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN123115067A_ABST
    Figure CN123115067A_ABST
Patent Text Reader

Abstract

This application provides a disk flushing method, apparatus, device, medium, and product based on a two-dimensional transaction linked list, relating to the fields of clustered file systems and distributed storage technology. Addressing the problems of increased disk I / O load, expanded lock wait range, and insufficient consistency between file data and associated metadata such as parent directories and resource groups caused by overall disk flushing in concurrent scenarios in existing clustered file systems, a disk flushing mechanism that organizes transactions by file is proposed. This method establishes a two-dimensional transaction linked list composed of file and transaction sequence dimensions. When a disk flushing requirement arises, it quickly locates the transaction set corresponding to the target file, identifies associated metadata based on the operation objects involved in the transaction, and then performs targeted persistence on the target file data and its associated metadata according to the transaction sequence. This scheme can reduce the disk flushing scope, reduce irrelevant transaction blocking, alleviate I / O amplification, and improve the concurrent processing performance and data consistency of the clustered file system.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the fields of clustered file systems and distributed storage technology, and in particular to a disk flushing method, apparatus, device, medium and product based on a two-dimensional transaction linked list. Background Technology

[0002] In a typical cluster environment, multiple compute nodes collaborate to complete data read and write tasks by sharing storage resources. In such scenarios, file system performance and reliability are critical indicators, especially under high concurrency access and frequent cross-node operations, where rapid persistence of file metadata and ensuring data consistency become core challenges. Current cluster file systems typically employ a multi-log model, where each node maintains an independent log file to avoid performance overhead from global locks. Within a single node, transaction metadata is first written to a cache, and log locks control the serialization of disk flushing operations, thereby ensuring transaction consistency within a single node. When a disk flush condition is triggered, the system performs a full disk flush, forcibly writing all log caches and data caches of that node to disk at once.

[0003] In existing technologies, when different nodes access the same file, a full disk flush is required, leading to unnecessary input / output overhead and lock wait time. Furthermore, in Order mode, all log write requests are blocked during disk flushing, resulting in decreased concurrency performance. When file modification is required, related metadata such as parent directories and resource groups involved in the file modification are not synchronously flushed to disk, potentially causing data inconsistency.

[0004] Therefore, the existing cluster file system's log flushing mechanism adopts a full flush mode, which leads to problems such as file conflicts causing inefficient processing, long flushing blocking time, and insufficient correlation processing in cross-node access scenarios, making it difficult to achieve a balance between performance and consistency of the cluster file system. Summary of the Invention

[0005] This application provides a disk flushing method, apparatus, device, medium, and product based on a two-dimensional transaction linked list, which solves the technical problem that the existing cluster file system's log flushing mechanism adopts a full flushing mode, resulting in file conflicts in cross-node access scenarios, leading to inefficient processing, long flushing blocking time, and insufficient correlation processing, making it difficult to achieve a balance between performance and consistency in the cluster file system.

[0006] Firstly, this application provides a disk flushing method based on a two-dimensional transaction linked list, including:

[0007] Construct a two-dimensional transaction linked list, which contains a file linked list and a transaction linked list. Each transaction linked list is sorted according to the operation time of the transaction.

[0008] In response to the disk flushing requirement, locate the two-dimensional transaction linked list corresponding to the file to be flushed;

[0009] Based on the operation objects of the transactions in the two-dimensional transaction chain list, identify the associated metadata corresponding to the file to be flushed to disk.

[0010] Based on the operation sequence of transactions in the two-dimensional transaction chain, the data of the file to be flushed to disk and the associated metadata of the file to be flushed to disk are sequentially written to disk.

[0011] In one possible implementation, a two-dimensional transaction linked list is constructed, including:

[0012] In response to the first operation on a file, the creation of nodes in the file list is triggered, and each node in the file list corresponds to a file.

[0013] Add the transactions related to the current file to the transaction list according to the operation sequence of the transactions related to the current file.

[0014] In one possible embodiment, the associated metadata includes the parent directory and resource group associated with the transaction. Based on the operation objects of the transaction in the two-dimensional transaction chain, the associated metadata corresponding to the file to be flushed to disk is identified, including:

[0015] Determine whether the current transaction's operation involves a parent directory or resource group;

[0016] When the current transaction operates on a parent directory or resource group, the metadata of the parent directory or resource group is identified as associated metadata.

[0017] In one possible embodiment, the method further includes:

[0018] By using localized locking, requests for transactions unrelated to the file to be flushed to disk are not locked, allowing these unrelated requests to continue requesting log space.

[0019] In one possible implementation, before constructing the two-dimensional transaction linked list, the following steps are also included:

[0020] Based on historical access logs, analyze the access patterns of historical files to predict the potential conflict probability of the current file.

[0021] Assign a priority tag to the current file based on the probability of potential conflicts.

[0022] In one possible embodiment, according to the operation sequence of transactions in the two-dimensional transaction chain, the data of the file to be flushed to disk and the associated metadata corresponding to the file to be flushed to disk are sequentially written to disk, including:

[0023] Based on the priority marker, the two-dimensional transaction list corresponding to the first priority file is written to disk, and the two-dimensional transaction list corresponding to the second priority file is written to disk in batches, wherein the first priority is higher than the second priority.

[0024] Secondly, this application provides a disk flushing device based on a two-dimensional transaction linked list, comprising:

[0025] The two-dimensional transaction linked list construction module is used to construct a two-dimensional transaction linked list. The two-dimensional transaction linked list contains a file linked list and a transaction linked list. Each transaction linked list is sorted according to the operation time of the transaction.

[0026] The two-dimensional transaction list positioning module is used to locate the two-dimensional transaction list corresponding to the file to be flushed to disk in response to the disk flushing requirement.

[0027] The associated metadata identification module is used to identify the associated metadata corresponding to the file to be flushed to disk based on the operation object of the transaction in the two-dimensional transaction chain list.

[0028] The data write-to-disk module is used to write the data of the file to be written to disk and the associated metadata of the file to be written to disk in sequence according to the operation sequence of the transactions in the two-dimensional transaction chain.

[0029] Thirdly, this application provides an electronic device, including: a processor and a memory communicatively connected to the processor;

[0030] The memory stores the instructions that the computer executes;

[0031] The processor executes computer-executable instructions stored in memory to implement any of the methods of the first aspect.

[0032] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method of any one of the first aspects.

[0033] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method of any one of the first aspects.

[0034] The disk flushing method, apparatus, device, medium, and product based on a two-dimensional transaction linked list provided in this application construct a two-dimensional transaction linked list containing a file linked list and a transaction linked list. When a disk flushing requirement arrives, the method locates the two-dimensional transaction linked list corresponding to the file to be flushed, identifies the associated metadata of the file to be flushed based on the operation object of the transaction, and sequentially writes the data and associated metadata of the file to be flushed to disk according to the operation sequence of the transaction. This can ensure the consistency and timing of data and related metadata flushing, while converging the flushing scope to the target file and its associated content, thereby reducing disk I / O load, reducing the log lock blocking range, and improving the concurrent processing performance of the cluster file system. Attached Figure Description

[0035] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0036] Figure 1 This is a schematic diagram illustrating the relationship between multiple log structures and concurrency.

[0037] Figure 2 A flowchart illustrating a disk flushing method based on a two-dimensional transaction linked list, provided for an embodiment of this application;

[0038] Figure 3 This is a schematic diagram of the structure of an existing transaction linked list;

[0039] Figure 4 A schematic diagram illustrating a novel data structure for transactions provided in an embodiment of this application;

[0040] Figure 5 A schematic diagram illustrating the data structure of each file node in a file linked list, provided in an embodiment of this application;

[0041] Figure 6a This is a diagram illustrating how requests are blocked when the log is flushed.

[0042] Figure 6b A schematic diagram illustrating the blocking of requests during log flushing under a new transactional data structure provided in this application embodiment;

[0043] Figure 7 A schematic diagram of a disk flushing device based on a two-dimensional transaction linked list provided in an embodiment of this application;

[0044] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application.

[0045] The accompanying drawings have illustrated specific embodiments of this application, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concept of this application to those skilled in the art through reference to specific embodiments. Detailed Implementation

[0046] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in any of the foregoing embodiments.

[0047] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of the relevant data all comply with the relevant laws, regulations, and standards of the relevant countries and regions, have taken necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation access points for users to choose to authorize or refuse.

[0048] Furthermore, the technical solution involved in this application, which involves big data analysis of user information (including but not limited to personal biometrics, identity data, consumption data, asset data, electronic terminal operation data, etc.) and the use of artificial intelligence technology for automated decision-making, and makes decisions that have a significant impact on personal rights based on the results of automated decision-making, provides users with corresponding operation entry points for users to choose to agree to or reject the results of automated decision-making; if the user chooses to reject, the process will proceed to the expert decision-making process.

[0049] It should be noted that the disk flushing method, apparatus, device, medium and product based on two-dimensional transaction linked list provided in this application can be used in the fields of cluster file system and distributed storage technology, and can also be used in any field other than cluster file system and distributed storage technology. The application fields of the disk flushing method, apparatus, device, medium and product based on two-dimensional transaction linked list in this application are not limited.

[0050] Clustered file systems involve distributed storage and transaction consistency processing technologies, and are commonly used in distributed databases, cloud storage platforms, and large-scale data processing systems. Multiple nodes perform file read / write operations, metadata updates, and log persistence on shared storage. The system typically includes node-local logs, caches, and concurrent access control mechanisms for shared files.

[0051] Most existing cluster file systems employ a process of writing independent node logs and caches, combined with disk flushing. After a node receives a file modification transaction, it first writes the relevant data and metadata to its local cache and records the transaction content in its log. When nodes access the same file, a synchronization operation is triggered, or the log space reaches a certain limit, disk flushing is then performed to write the cached content to disk.

[0052] In concurrent scenarios, this type of solution typically employs a full disk flushing approach. This means that if a file triggers a cross-node access conflict, the relevant nodes often need to flush all their logs and caches to disk simultaneously, rather than only processing the modifications to the conflicting file. For this data-first-then-metadata-flushing model, if there are sequential large I / O writes, the duration of a single flush will increase significantly.

[0053] Figure 1This is a diagram illustrating the relationship between multiple log structures and concurrency, such as... Figure 1 As shown, the relationship between multi-log structure and concurrency in the existing file cluster system is as follows: 1. Multiple processes, or a single process with multiple threads, generate multiple concurrent accesses to the file system. The file system processes each request with a separate transaction. 2. Metadata and data involved in each transaction can be written to the cache first. 3. There are several ways to trigger disk flushing: the first is when multiple nodes access the same file, requiring the node currently writing to flush to disk; the second is periodic execution, where disk flushing must be performed when the specified log execution cycle is reached; the third is capacity-based, where disk flushing is forced when the log usage exceeds a certain percentage and the free log space is insufficient. 4. During disk flushing, in order mode, data must be written to the data area first. After the data is successfully flushed to disk, the metadata is written to the log. 5. Different nodes write their own log files, and the data is uniformly flushed to the data area.

[0054] The resulting problems are that disk I / O load increases significantly with each disk flush, the log lock wait range expands, and transactions unrelated to the currently conflicting file may be blocked. If the associated metadata such as parent directories and resource groups involved in file modification are not synchronously identified and processed, it can also lead to inconsistencies in directory entries, permission chains, or resource configurations, affecting the concurrent performance and data consistency of the cluster file system.

[0055] To address the aforementioned technical challenges, a file-based transaction organization approach is adopted. This maps dispersed transaction relationships to file and transaction lists, making it easier to directly locate the transaction set corresponding to the target file when a disk flushing requirement arises. By combining transaction operation object identification with associated metadata, the scope to be flushed can be narrowed down to the target file and its associated content, and the flushing can be completed sequentially according to the transaction timeline.

[0056] In a cluster architecture with shared storage, this processing path makes the disk flushing operation revolve around the file to be flushed, reducing the impact on other transactions within the node, narrowing the scope of lock blocking, alleviating the I / O amplification problem caused by the overall disk flush, and maintaining a consistent update relationship between file data and associated metadata.

[0057] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0058] Figure 2 A flowchart illustrating a disk flushing method based on a two-dimensional transaction linked list provided in this application is shown below. Figure 2 As shown, the method includes:

[0059] S201: Construct a two-dimensional transaction linked list, which contains a file linked list and a transaction linked list. Each transaction linked list is sorted according to the operation sequence of the transaction.

[0060] In this application, a two-dimensional transaction linked list serves as a data structure for organizing transactions at the file granularity. It comprises a file linked list and transaction linked lists attached to each file linked list node. The file linked list records files that have undergone operations and serves as an index structure at the file level. The transaction linked list records transactions related to the corresponding file and maintains the chronological order of these transactions. Specifically, the execution entity can be the transaction management module on any storage node in the cluster file system. This transaction management module is connected to the local log module, cache management module, and shared storage access module. Upon receiving a file modification transaction, the node first parses the transaction description information. This description information includes at least the transaction identifier, target file identifier, operation type, operation object, transaction generation timestamp or monotonically increasing sequence number, data modification location, metadata modification items, and the transaction's record location in the local log. Subsequently, using the target file identifier as the search key, the corresponding file linked list node is found within the file linked list, and the current transaction is associated with the corresponding transaction linked list for that file. To ensure that the subsequent disk flushing order is consistent with the transaction execution order, each transaction node in the transaction list stores an operation timing field. This operation timing field can be expressed using an incrementing transaction sequence number within the node, a logical clock value, or a log append offset, as long as it can characterize the order in which related transactions occur for the same file. In one possible embodiment, file list nodes and transaction nodes can also store relevant information to support list organization, transaction association, and disk flushing. The file list can be organized using a doubly linked structure to support node insertion, removal, and traversal; the transaction list can be organized using a unidirectional or doubly linked structure with append-to-tail, where nodes are appended to the tail in the order they are received. If the same transaction modifies multiple files simultaneously, the transaction is split into multiple file sub-transactions, each attached to the corresponding file's transaction list, and associated with a common transaction identifier, so that each file list node only retains the transaction view directly related to that file. Based on the above analysis, it can be seen that by reorganizing the transaction relationships scattered in node logs and caches into a two-dimensional transaction linked list consisting of file linked lists and transaction linked lists, the organizational relationship between files and transactions can be established. Subsequently, transactions related to the same file can be continuously gathered into the same transaction linked list, thus providing a structural foundation for locating the range to be flushed to disk by file, identifying associated metadata, and performing sequential flushing according to time sequence.

[0061] S202: In response to the disk flushing request, locate the two-dimensional transaction list corresponding to the file to be flushed.

[0062] In this application, when a disk flushing request arrives, not all transactions within a node are processed uniformly. Instead, the associated two-dimensional transaction list is located around the target file corresponding to the triggering event. A disk flushing request can be triggered by concurrent conflict detection results from cross-node access to the same file, by application-layer synchronization operations, by local log space reaching a threshold, cache page reclamation requests, lease reclamation events, or shared storage consistency verification events. Upon receiving a disk flushing request, the execution entity first extracts the identifier of the file to be flushed. The file identifier can be a pathname, file handle, inode (index node) number, or an internal object number obtained after path resolution. If the disk flushing request is reported by the cross-node conflict detection module, this module simultaneously reports the conflicting file identifier, conflicting node identifier, and conflict time point. If the disk flushing request is triggered by a Sync (synchronize) call, the system call processing module reports the target file descriptor and parses it to obtain the file identifier. If the disk flushing request is triggered by log space conditions, the log management module outputs the set of files that need to be prioritized for log release, and then locates each file in the set one by one. During the location process, the transaction management module performs a search in the file linked list based on the file identifier to obtain the corresponding file linked list node, and further determines the transaction linked list associated with that file linked list node and the range of transactions not yet flushed to disk. To shorten the search path, an index structure can be used to improve location efficiency. If there are concurrent writes or repeated scheduling, a concurrency control mechanism can be used to keep the current flushing processing range stable. If the target file linked list node is not found in the file linked list, it is determined that the current file has no transactions to be flushed to disk, and the location result is returned as empty. The file linked list here represents the index structure that records all files that have been operated on, the transaction linked list here represents the transaction sequence corresponding to the target file, and the two-dimensional transaction linked list here represents the two-dimensional mapping relationship from file to transaction formed by the aforementioned two types of linked lists. Based on the above analysis, it can be seen that after the flushing requirement occurs, directly locating the file linked list node and transaction linked list corresponding to the target file can converge the flushing processing range to the set of transactions associated with the target file, without extending to the cache and log records of other unrelated files within the node, so that subsequent locking, sorting, and flushing actions revolve around the target file.

[0063] S203: Identify the associated metadata of the file to be flushed to disk based on the operation object of the transaction in the two-dimensional transaction chain list.

[0064] In this application, after obtaining the transaction list corresponding to the file to be flushed, the associated metadata that needs to be processed together with the file data is further identified based on the operation objects recorded in each transaction node of the transaction list. Here, the operation object represents the scope of system objects actually affected by the transaction, which, in addition to the file data page itself, can also include various metadata objects related to file modification. The execution entity traverses the transaction list from head to tail, reading the operation type and operation object set for each transaction node. Based on different operation types and operation objects, the metadata objects related to the current transaction are determined and included in the associated metadata scope to be flushed. To ensure that the identification results can directly participate in the flushing process, the system establishes an associated metadata set during traversal. The set elements include at least the metadata object identifier, object type, the time sequence position of the last modification by the current transaction list, the page reference in the cache, and the record position in the log. For cases where the same metadata object is repeatedly modified by multiple transactions, multiple set elements are not created repeatedly; instead, the last modification time and dirty page reference of the corresponding element of the metadata object are updated, so that only the final content to be persisted is written during subsequent flushing, maintaining the time sequence constraint. If a transaction involves multiple operation objects, these objects are parsed and merged into the associated metadata set. For example, in a specific implementation, the corresponding directory-type metadata, space allocation-type metadata, access control-type metadata, or other metadata objects related to file modification can be identified based on the actual impact of the transaction. The transaction list acts as a transaction sequence carrier in this step, while the operation object field serves as the basis for identifying associated metadata. File list nodes can simultaneously maintain a cache index of metadata objects related to the current file to accelerate object reference retrieval. Based on the above analysis, identifying metadata objects directly related to file modification item by item according to the operation objects in the transaction node ensures that the scope to be flushed to disk covers not only the file data itself but also the metadata objects actually modified in the current transaction sequence. This provides clear boundaries for subsequent persistence processing and maintains a correspondence with the transaction content.

[0065] S204: Based on the operation sequence of transactions in the two-dimensional transaction chain, flush the data of the file to be flushed to disk and the associated metadata of the file to be flushed to disk in sequence.

[0066] In this application, after locating the target file and identifying the associated metadata, the disk flushing is performed sequentially according to the operation order in the transaction chain. Specifically, the transaction management module first generates the current flushing sequence based on the tail boundary of the transaction chain recorded when the flushing begins. Each item in the sequence corresponds to a transaction node and carries references to the data page and metadata object associated with that transaction. Subsequently, a file-level flushing lock is added to the file chain node to be flushed, and an object-level short-term persistent lock is added to the identified associated metadata object. The scope of the lock only covers the current file to be flushed and its associated metadata set, and does not extend to other file chain nodes. During the flushing process, starting from the incomplete position at the head of the transaction chain, transaction nodes are retrieved one by one according to the operation order. For each transaction node, the data page corresponding to the transaction is first flushed from the node cache to the target data block in the shared storage. Then, the associated metadata page changed by the transaction is written to the metadata area in the shared storage. Finally, the persistence completion flag or commit bit in the local log is updated. If a transaction node contains only metadata updates and no data page modifications, the corresponding metadata page is written directly. If a transaction node contains only data page modifications and does not cause new metadata changes, only the corresponding data page is refreshed, and the transaction's timing position is maintained as complete. The sequential flushing here does not involve writing all dirty pages in a single batch within a node; instead, it strictly follows the operation order of the target file's transaction chain, ensuring that the data of transaction i and the associated metadata caused by transaction i are processed before proceeding to transaction i+1. To reduce overwrite issues caused by timing overlap, the system verifies whether the current cached version of the object corresponds to the last modified version of the current transaction node before writing each object page. If the object is modified by a new transaction not included in the current tail boundary during the current flushing process, only the stable version up to the current boundary is written, and the updated version is retained in the cache and transaction chain for the next flush. After a transaction node is flushed to disk, the most recently flushed time, the count of unflushed transactions, and the corresponding log reclamation position in the file linked list node are updated. When multiple consecutive transactions at the head of the transaction linked list have completed flushing, these transaction nodes are removed from the transaction linked list. If there are no unflushed transactions under that file linked list node, the file linked list node can be marked as free and reclaimed later. The two-dimensional transaction linked list provides sequential traversal and range limitation capabilities in this step. The operation time sequence field of the transaction linked list directly determines the order in which data pages and metadata pages are written, while the file linked list node limits the flushing to a single target file. Based on the above analysis, writing the target file's data and the identified associated metadata sequentially to shared storage according to the transaction operation time sequence ensures that the flushing order is consistent with the actual execution order of the transactions and restricts the flushing action to the target file and its associated objects. This allows for file-by-file convergence persistence processing in a shared storage cluster scenario, reducing the intervention of transactions unrelated to currently conflicting files.

[0067] In one implementation scenario, a scheme based on a two-dimensional linked list transaction structure is proposed to reduce the granularity of log flushing to disk, thereby improving file system performance while ensuring data consistency. This scheme abandons the requirement that all data in the log must be written to disk when a log flush is triggered. Instead, it flushes only the file to be flushed, along with all related metadata such as parent directories and resource groups, all at once. This significantly reduces the granularity of flushing and improves response performance.

[0068] Specifically, the establishment of a two-dimensional transaction structure for metadata. Figure 3 Here is a schematic diagram of the structure of an existing transaction linked list, such as... Figure 3 As shown, the original transaction structure is a linked list structure. New transactions are continuously appended to the end of the transaction. When flushing to disk, all transactions are flushed to disk from the beginning according to the time sequence.

[0069] Figure 4 This is a schematic diagram of a new data structure for transactions provided in an embodiment of this application, such as... Figure 4 As shown, the new transaction structure is a two-dimensional structure. All the files being operated on are recorded through a linked list. Each file has its own independent linked list structure that records all transactions related to that file. The transactions related to the file are arranged in chronological order.

[0070] Figure 5 This application provides a schematic diagram illustrating the data structure of each file node in a file linked list, as shown in the embodiments of this application. Figure 5 As shown, the new transaction data structure has become multiple transaction lists based on files: the file list is used to record all the files that have been operated on. The condition for adding a file to the list is that the file is operated on for the first time and that the operation on that file is not associated with any other file transactions; the file list is added in the order of the first operation on the file, and the transaction for that file is used as the head of the transaction list. Subsequent transactions related to that file will be appended to the transaction list for that file.

[0071] When a subsequent transaction is added, it is first determined whether the file in the transaction exists in the file list. If it does not exist, it is determined whether there is a relationship between its parent directory, resource group and the existing file list. If there is, it is added to the transaction list of that file. If not, a file node is added to the file list, and a transaction list is created for that file.

[0072] When there is a mandatory disk flushing requirement, if the file exists in the file list, the corresponding transaction list is found based on the file list. During disk flushing, all transactions related to that file are flushed in sequence. Data flushing occurs before metadata flushing. Based on the metadata of the file in the file list, the file and associated files are found and their data is flushed first, followed by the metadata. This reduces the granularity of disk flushing and ensures the integrity of transactions.

[0073] Figure 6a This is a diagram illustrating how requests are blocked during log flushing, as shown below. Figure 6a As shown, in the scenario of requesting logs, the existing scenario is that if the logs are flushed to disk, all write requests are blocked. Figure 6b This application provides a schematic diagram illustrating the blocking of requests during log flushing under a novel transactional data structure, as shown in the embodiments of this application. Figure 6b As shown, if the log request is unrelated to the file being flushed to disk, the request can be responded to, improving concurrent response speed. For scenarios where inter-node operations on the same file require lock acquisition, resulting in a full log flush, the log is no longer fully flushed; instead, the linked list of file-related transactions is flushed to disk on a file-by-file basis. Similarly, for single-node requests to perform a file sync operation, a full log flush is no longer performed; only file-related transactions are flushed.

[0074] In another implementation scenario, a disk flushing method based on a two-dimensional transaction linked list is provided, applied to cross-node file access and transaction processing in a distributed database cluster. Multiple database nodes share storage resources and frequently access the same files across nodes, including inventory table files and log files. In an online business system instance, both node A and node B access the shared inventory file F. Node A is responsible for handling inventory deduction operations for user orders, while node B is responsible for handling inventory replenishment operations. The metadata of file F includes file attributes such as permissions, the directory structure of its parent directory D, and the storage configuration of its resource group G. Before constructing the two-dimensional transaction linked list, the system first analyzes the access patterns of historical files based on historical access logs, extracting the access frequency, concurrent access relationships, transaction crossover situations, and time distribution characteristics of different files, predicting the potential conflict probability of the current file, and assigning priority tags to the current file based on the potential conflict probability. In conjunction with this online business system, the system uses an access frequency evaluation module to detect whether the number of cross-node access conflicts for file F exceeds a threshold. When the number of cross-node access conflicts for file F exceeds the threshold, the transaction list F_trans is marked as high priority so that when node A performs disk flushing, it will prioritize processing F_trans rather than the transaction list of low-frequency file accesses.

[0075] In one possible implementation, the system splits the two-dimensional transaction linked list into a file linked list and a transaction linked list attached to the file linked list nodes. For the first operation on file F, namely the inventory deduction transaction T1, the creation of a file linked list node is triggered, creating a file linked list node F_node, and establishing a transaction linked list F_trans under F_node. After a user places an order, node A modifies the inventory value of file F from 100 to 80. This inventory deduction transaction T1 is appended to F_trans according to the operation sequence, ensuring that multiple transactions in the transaction linked list are maintained in a chronological order. Besides file F, the operation of T1 also involves updating the directory entries of the parent directory D, including the size and modification time of file F, and adjusting the storage configuration of resource group G. Therefore, after reading the operation objects of T1, the system identifies the metadata of the parent directory D and the resource group G, associates the metadata of the parent directory D and the resource group G with the transaction linked list of file F, and includes it in the subsequent content to be flushed to disk. In this way, the transaction list F_trans of file F only contains transactions directly related to F, while also including the metadata updates of parent directory D and resource group G in the scope of associated processing.

[0076] When a new transaction related to the current file is received, the system adds the transaction to the transaction list corresponding to the current file according to the operation sequence of the transaction. In this online business system, the new transaction T2 is an inventory replenishment operation for node B. Node B needs to query the current inventory value of file F to trigger the replenishment process. The system first checks whether the file involved in T2 already exists in the file list. If file F already exists in the file list, it further determines whether the parent directory D and resource group G are associated with the metadata in the existing file list. If the association is established, T2 is merged into the transaction list F_trans corresponding to F_node. If the association is not established, a new file list node is created. When node B accesses file F, if the modification of node A has not yet been flushed to disk, node B will read the old value 100, thus causing a business logic error. At this time, the system detects a cross-node access conflict, specifically, when node B reads file F, it finds that its metadata has not been written to disk, thus triggering node A's flush operation.

[0077] Upon receiving a disk flush request, the system first locates the two-dimensional transaction list corresponding to the file F to be flushed. During the flush control period, localized locking is used to constrain only the transaction access paths related to the file F. When node A executes the flush operation, it only locks transaction requests involving file F, including node B's inventory replenishment transaction T2, while not locking unrelated transaction requests, allowing unrelated transaction requests to continue requesting log space. In the same scenario, other nodes, such as node C, can continue to execute transactions on unrelated files. For example, when node C modifies the permissions of file G, its transaction list G_trans is not related to file F, so it can continue to request log space without waiting for node A's flush to complete. After locating the transaction list corresponding to file F, the system reads the operation objects of each transaction in F_trans and determines whether the operation object of the current transaction involves a parent directory or resource group. When the operation object of the current transaction involves the parent directory D, the metadata of the parent directory D is identified as associated metadata. When the operation object of the current transaction involves the resource group G, the metadata of the resource group G is identified as associated metadata. If the same transaction involves both the parent directory D and the resource group G, the metadata corresponding to both is identified together and included in the content to be flushed to disk.

[0078] After identifying the associated metadata, the system performs disk write control based on the operation sequence in the transaction chain. When file F is assigned the first priority based on the analysis results of historical access logs and the detection results of the access frequency evaluation module, the two-dimensional transaction chain F_trans corresponding to file F is directly retrieved, and the data is written to disk sequentially according to the transaction order in F_trans. In this example, when a cross-node access conflict triggers the disk flushing condition of node A, the system only performs a disk flushing operation on file F and its associated parent directory D and resource group G, rather than a full disk flush. Following the temporal order of the transaction chain F_trans, the data of file F is first written to disk or to the log, where the written data of file F is the inventory value 80. Then, the directory entries updated for the parent directory D and the configuration adjustments for resource group G corresponding to each transaction are sequentially written to disk or to the log. The directory entries updated by node A include the size and modification time of file F, and the configuration adjustments for resource group G correspond to the storage configuration of resource group G. During the disk write process, the data of file F, the metadata of parent directory D, and the metadata of resource group G are always committed according to the temporal relationship of the corresponding transactions. After the write is completed, the lock is released.

[0079] For files assigned to the second priority, the system organizes the two-dimensional transaction lists corresponding to multiple second-priority files and performs batch disk write-to-disk operations. Following the operation sequence within each transaction list, the system writes the data and identified associated metadata of each file to disk sequentially. Corresponding to file F being marked as high priority and flushed to disk first, the transaction lists of low-frequency access files are included in the second-priority file set for batch disk write-to-disk processing. In this distributed database cluster, inventory table files, log files, and other shared files can all have their potential conflict probabilities predicted based on access frequency, concurrent access relationships, transaction overlap, and time distribution characteristics obtained from historical access log analysis. Priority tags are then assigned between the first and second priorities accordingly. When node A performs disk flush-to-disk operations, it prioritizes processing the high-priority F_trans transactions, rather than the transaction lists of low-frequency access files, while ensuring that node C's log space allocation for the transaction corresponding to the permission modification of file G is not affected by the disk flush-to-disk control of file F.

[0080] This embodiment provides a disk flushing method based on a two-dimensional transaction linked list, comprising: constructing a two-dimensional transaction linked list, which includes a file linked list and a transaction linked list, with each transaction linked list ordered according to the operation sequence of the transactions; locating the two-dimensional transaction linked list corresponding to the file to be flushed in response to the disk flushing requirement; identifying the associated metadata corresponding to the file to be flushed based on the operation objects of the transactions in the two-dimensional transaction linked list; and flushing the data of the file to be flushed and the associated metadata corresponding to the file to be flushed sequentially according to the operation sequence of the transactions in the two-dimensional transaction linked list. In this application, by recording the operated files with a file linked list, aggregating related transactions of the same file with a transaction linked list and maintaining the operation sequence, and combining the direct location of the target file's two-dimensional transaction linked list and the identification of associated metadata when disk flushing is triggered, the disk flushing process is limited to the target file's transaction set, and the persistent writing of data pages and metadata pages is advanced along the transaction occurrence sequence. Therefore, transactions within a node that are unrelated to the current file to be flushed do not need to participate in the current disk flushing process, and the file data and associated metadata are persisted according to the same transaction sequence, which is suitable for concurrent access control, log reclamation, and consistency maintenance scenarios in shared storage cluster file systems.

[0081] Based on the aforementioned embodiments, a two-dimensional transaction linked list is further constructed, including: in response to the first operation on a file, triggering the creation of nodes in the file linked list, with each node in the file linked list corresponding to each file; and adding transactions related to the current file to the transaction linked list according to the operation sequence corresponding to the transactions related to the current file.

[0082] In practical implementation, upon receiving the first operation for a file, the system first checks if a corresponding node exists in the current two-dimensional transaction list based on the file identifier. If no corresponding node is found, a new file list node is created and attached to the file list, while the transaction list head or tail pointer is initialized for that node. If the file list node already exists, it is reused to avoid duplicate file-level entry points. Transactions related to the current file are written to the transaction list associated with that node based on the transaction creation time, commit time, or reception order. Transaction records can include transaction number, operation type, target object, timestamp, and associated metadata identifiers, thus forming a stable temporal sequence of transactions for the same file.

[0083] In this structure, the creation of file list nodes is triggered by the first access to a file, ensuring that each operated file has an independent transaction categorization entry point. After the transaction list is appended with records according to the operation sequence of related transactions for the current file, the transaction set under that file node can be directly read when locating a specific file later, and the corresponding sequential transaction sequence for that file can be obtained. Thus, the two-dimensional transaction list simultaneously achieves file-level location and transaction-level sorting, providing a foundation for subsequent convergence of the disk flushing range by file.

[0084] After adopting the above construction method, a clear mapping relationship is formed between file-level nodes and transaction-level records. Related transactions of the same file can be centrally merged and maintain temporal consistency, facilitating precise management by target file in a shared storage cluster environment. This reduces the traversal and interference of irrelevant transactions, ensuring that file-related data and its associated metadata are organized consistently, thereby supporting subsequent disk flushing processing in transaction order.

[0085] Based on the aforementioned embodiments, the associated metadata further includes the parent directory and resource group associated with the transaction. According to the operation object of the transaction in the two-dimensional transaction chain list, the associated metadata corresponding to the file to be flushed to disk is identified, including: determining whether the operation object of the current transaction involves the parent directory or resource group; if the operation object of the current transaction involves the parent directory or resource group, determining that the metadata of the parent directory or resource group is identified as associated metadata.

[0086] In the two-dimensional transaction linked list, the file linked list is used to locate the target file, and the transaction linked list is used to record transactions related to that file in chronological order. After determining the file to be flushed to disk, the system reads the current transaction in the corresponding transaction linked list and parses the operation object of the current transaction. If the operation object is directory creation, directory renaming, directory mounting change, or directory attribute modification, a mapping relationship is established between the operation object and the parent directory metadata; if the operation object is resource quota adjustment, resource group binding change, or resource group attribute update, a mapping relationship is established between the operation object and the resource group metadata.

[0087] When the parsing results indicate that the current transaction's operation object involves a parent directory or resource group, the system registers the metadata corresponding to that parent directory or resource group in the associated metadata set and uses it as the synchronous write object for the file to be flushed to disk. This process can be completed by the transaction parsing module, which extracts the object type, object identifier, and associated path information from the transaction record and queries the directory metadata table or resource group metadata table based on the object identifier to obtain the metadata entry corresponding to the current transaction. In practical applications, the transaction parsing module can also use a hash index to locate the metadata entry; this application does not limit this approach.

[0088] After generating the associated metadata set, the system, according to the temporal relationship in the transaction chain, includes the data of the file to be flushed to disk along with the identified parent directory or resource group metadata into the flush queue, and completes the writing within the same persistence cycle. Thus, the file content maintains consistency with its upper-level directory information and resource configuration status, and the flushing scope only extends to the associated metadata actually involved in the current transaction.

[0089] With this implementation method, the system can directly identify the metadata of the parent directory or resource group based on the transaction operation object, reduce the occurrence of irrelevant metadata entering the disk flushing scope, and enable file data and associated metadata to be flushed to disk synchronously, thereby maintaining the consistency between directory hierarchy, resource configuration and file status.

[0090] In one possible implementation, the method further includes: using localized locking to prevent locking of unrelated transaction requests that are not related to the file to be flushed to disk, allowing unrelated transaction requests to continue to request log space.

[0091] In its implementation, after locating the file to be flushed, the system identifies transaction requests related to that file using a two-dimensional transaction linked list and locks the log write entry points corresponding to these requests, while keeping the request channels open for transaction requests for other files. The control unit can determine whether a transaction is irrelevant based on the file identifier, transaction identifier, and linked list node relationships. If deemed irrelevant, the system directly allows the log request, enabling the file to continue acquiring writable log positions and completing subsequent writes before the flushing process is finished. This control process can be executed by the lock management module on the node side. The lock management module can maintain a file-level lock table or a transaction association table to ensure that the lock scope only covers the transaction set corresponding to the file to be flushed.

[0092] Through the aforementioned localized lock control, the flushing action only constrains transaction requests related to the file to be flushed. Unrelated transaction requests can still request log space in parallel and enter the normal write path, thus enabling the system to maintain the continuous commit capability of other transactions during the flushing of the target file. Since the lock contention scope converges to the local transaction set, other transactions within the node are no longer blocked as a whole, thereby maintaining the continuity of log request and transaction processing, and cooperating with the targeted flushing supported by the two-dimensional transaction linked list.

[0093] With this approach, the locking scope during disk flushing is limited to transactions related to the file to be flushed. Unrelated transactions can continue to request log space, the node's concurrent write capability is maintained, and the impact of the overall disk flushing on other transactions is compressed to a local scope. At the same time, it helps to maintain the consistency between log requests and transaction timing.

[0094] Based on the aforementioned embodiments, before constructing the two-dimensional transaction linked list, the method further includes: analyzing the access patterns of historical files based on historical access logs to predict the potential conflict probability of the current file; and assigning a priority tag to the current file based on the potential conflict probability.

[0095] In practice, the system first extracts access features corresponding to the current file from historical access logs. These features include access time intervals, number of consecutive writes, number of concurrent cross-node accesses, read-write alternation frequency, and modification records associated with the parent directory or resource group. Subsequently, the system inputs the extracted access features into a preset conflict assessment model. This model can employ a rule-based model, a threshold mapping model, or a probability prediction model trained on historical samples to output the potential conflict probability of the current file. If the current file is frequently accessed alternately by multiple nodes within a preset observation window, triggers shared metadata updates immediately after a write, or enters the disk flushing queue multiple times, the corresponding potential conflict probability is determined to be high.

[0096] After obtaining the potential conflict probability, the system assigns priority tags to the current file according to the probability range. These priority tags include at least three categories: high, medium, and low, or numerical tags are used to record their relative processing order. The higher the potential conflict probability of the current file, the higher its priority tag appears, thus allowing the transaction node corresponding to that file to be assigned a higher scheduling weight when constructing the subsequent two-dimensional transaction linked list. If the system uses a specific model of probability prediction module, such as an integrated analysis module with historical feature normalization and threshold mapping units, this module can reduce the adaptation cost between different log formats. In practical applications, other models of this component can also be selected, and this application does not limit this choice.

[0097] This implementation predicts the probability of conflict for the current file using historical access logs and converts the probability into priority markers. This allows the subsequent organization of the two-dimensional transaction list and disk flushing scheduling to be directly sorted based on file conflict tendencies. Consequently, files with higher conflict probabilities are prioritized in transaction organization and persistence, leading to their associated transaction sets entering the processing sequence earlier.

[0098] By adopting this approach, the system can complete the conflict probability layering before constructing the two-dimensional transaction linked list, ensuring that the file-level transaction organization is consistent with the subsequent disk flushing order, reducing the waiting time of highly conflicting files during the disk flushing stage, and making the scheduling order of the transaction set more in line with historical access patterns.

[0099] Based on the aforementioned embodiments, further, according to the operation sequence of transactions in the two-dimensional transaction chain, the data of the file to be flushed to disk and the associated metadata corresponding to the file to be flushed to disk are sequentially written to disk, including: according to the priority marker, writing the two-dimensional transaction chain corresponding to the first priority file to disk, and writing the two-dimensional transaction chain corresponding to the second priority file to disk in batches, wherein the first priority is higher than the second priority.

[0100] This processing method uses priority tags to group and schedule two-dimensional transaction linked lists. High-priority files are persisted with their data and associated metadata first, while low-priority files are flushed to disk in batches. This limits the flushing scope to the target file set and ensures consistency between transaction timing and metadata update relationships. This allows for more centralized write scheduling on shared storage, reduces synchronization waits for irrelevant files, and lowers the disk load caused by repeated flushes triggered by low-priority transactions.

[0101] Figure 7 A schematic diagram of a disk flushing device based on a two-dimensional transaction linked list provided in this application embodiment is shown below. Figure 7 As shown, the disk flushing device 70 based on a two-dimensional transaction linked list provided in this embodiment includes:

[0102] The two-dimensional transaction linked list construction module 701 is used to construct a two-dimensional transaction linked list. The two-dimensional transaction linked list contains a file linked list and a transaction linked list. Each transaction linked list is sorted according to the operation time of the transaction.

[0103] The two-dimensional transaction list positioning module 702 is used to locate the two-dimensional transaction list corresponding to the file to be flushed in response to the disk flushing requirement.

[0104] The associated metadata identification module 703 is used to identify the associated metadata corresponding to the file to be flushed based on the operation object of the transaction in the two-dimensional transaction chain list.

[0105] The data write-to-disk module 704 is used to write the data of the file to be written to disk and the associated metadata of the file to be written to disk in sequence according to the operation sequence of the transactions in the two-dimensional transaction chain list.

[0106] In one possible implementation, the two-dimensional transaction linked list construction module 701 is specifically used to: trigger the creation of nodes in the file linked list in response to the first operation on the file, with each node in the file linked list corresponding to each file; and add the transactions related to the current file to the transaction linked list according to the operation sequence of the transactions related to the current file.

[0107] In one possible implementation, the associated metadata includes the parent directory and resource group associated with the transaction. The associated metadata identification module 703 is specifically used to: determine whether the operation object of the current transaction involves the parent directory or resource group; and if the operation object of the current transaction involves the parent directory or resource group, determine that the metadata of the parent directory or resource group is identified as associated metadata.

[0108] In one possible implementation, the disk flushing device based on a two-dimensional transaction linked list is also specifically used to: control the disk flushing with localized locks, not lock unrelated transaction requests that are not related to the file to be flushed, and allow unrelated transaction requests to continue to request log space.

[0109] In one possible implementation, the disk flushing device based on a two-dimensional transaction linked list is also specifically used to: analyze the access patterns of historical files based on historical access logs, predict the potential conflict probability of the current file, and assign a priority tag to the current file based on the potential conflict probability.

[0110] In one possible implementation, the data persistence module 704 is specifically used to: persist the two-dimensional transaction list corresponding to the first priority file to disk according to the priority marker, and persist the two-dimensional transaction list corresponding to the second priority file to disk in batches, wherein the first priority is higher than the second priority.

[0111] The disk flushing device based on a two-dimensional transaction linked list provided in this embodiment can execute the method provided in the above method embodiment. Its implementation principle and technical effect are similar, and will not be described in detail here.

[0112] Figure 8 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 8 As shown, the electronic device 80 may include a memory 801 and a processor 802. Optionally, the electronic device may also include a transceiver 803, wherein the memory 801 and the processor 802 communicate with each other; for example, the memory 801, the processor 802 and the transceiver 803 may communicate via a communication bus 804, the memory 801 is used to store a computer program, and the processor 802 executes the computer program to implement the method of the above embodiments.

[0113] Optionally, the aforementioned processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps in the method embodiments disclosed in this application can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0114] This application also provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the methods in any of the above method embodiments.

[0115] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the methods in any of the above method embodiments.

[0116] All or part of the steps in the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a readable memory. When the program is executed, it performs the steps of the above method embodiments; and the aforementioned memory (storage medium) includes: read-only memory (ROM), RAM, flash memory, hard disk, solid-state drive, magnetic tape, floppy disk, optical disk, and any combination thereof.

[0117] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processing unit of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0118] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0119] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0120] Obviously, those skilled in the art can make various modifications and variations to the embodiments of this application without departing from the spirit and scope of this application. Therefore, if these modifications and variations to the embodiments of this application fall within the scope of this application and its equivalents, this application also intends to include these modifications and variations.

[0121] In this application, the term "comprising" and its variations can refer to non-limiting inclusion; the term "or" and its variations can refer to "and / or". The terms "first", "second", etc., in this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0122] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to this application.

[0123] It should be further noted that although the steps in the flowchart are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowchart may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0124] It should be understood that the above-described device embodiments are merely illustrative, and the device of this application can also be implemented in other ways. For example, the division of units / modules in the above embodiments is only a logical functional division, and there may be other division methods in actual implementation. For example, multiple units, modules, or components may be combined, or integrated into another system, or some features may be ignored or not executed.

[0125] Furthermore, unless otherwise specified, the functional units / modules in the various embodiments of this application can be integrated into one unit / module, or each unit / module can exist physically separately, or two or more units / modules can be integrated together. The integrated units / modules described above can be implemented in hardware or as software program modules.

[0126] When integrated units / modules are implemented in hardware, the hardware can be digital circuits, analog circuits, etc. The physical implementation of the hardware structure includes, but is not limited to, transistors, memristors, etc. Unless otherwise specified, the processor can be any suitable hardware processor, such as a CPU, GPU, FPGA, DSP, and ASIC, etc. Unless otherwise specified, the storage unit can be any suitable magnetic or magneto-optical storage medium, such as Resistive Random Access Memory (RRAM), Dynamic Random Access Memory (DRAM), Static Random Access Memory (SRAM), Enhanced Dynamic Random Access Memory (EDRAM), High-Bandwidth Memory (HBM), Hybrid Memory Cube (HMC), etc.

[0127] If the integrated unit / module is implemented as a software program module and sold or used as an independent product, it can be stored in a computer-readable storage device (CMD). Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a memory and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this application. The aforementioned memory includes various media capable of storing program code, such as a USB flash drive, read-only memory (ROM), random access memory (RAM), portable hard drive, magnetic disk, or optical disk.

[0128] In the above embodiments, the descriptions of each embodiment have their own emphasis. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments. The technical features of the above embodiments can be combined arbitrarily. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as the combination of these technical features does not contradict each other, it should be considered within the scope of this specification.

[0129] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0130] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope.

Claims

1. A disk flushing method based on a two-dimensional transaction linked list, characterized in that, The method includes: Construct a two-dimensional transaction linked list, which includes a file linked list and a transaction linked list, and each of the transaction linked lists is sorted according to the operation time sequence of the transaction; In response to the disk flushing requirement, locate the two-dimensional transaction linked list corresponding to the file to be flushed; Based on the operation object of the transaction in the two-dimensional transaction chain list, identify the associated metadata corresponding to the file to be flushed to disk. According to the operation sequence of the transactions in the two-dimensional transaction chain, the data of the file to be flushed to disk and the associated metadata of the file to be flushed to disk are sequentially written to disk.

2. The method according to claim 1, characterized in that, The construction of the two-dimensional transaction linked list includes: In response to the first operation on a file, the creation of nodes in the file linked list is triggered, and each node in the file linked list corresponds to each file. According to the operation sequence of the transactions related to the current file, add the transactions related to the current file to the transaction list.

3. The method according to claim 1, characterized in that, The associated metadata includes the parent directory and resource group associated with the transaction. Identifying the associated metadata corresponding to the file to be flushed to disk based on the operation object of the transaction in the two-dimensional transaction chain list includes: Determine whether the current transaction's operation involves the parent directory or resource group; If the current transaction involves the parent directory or resource group, the metadata of the parent directory or resource group is identified as the associated metadata.

4. The method according to claim 1, characterized in that, The method further includes: By using localized locking, requests for transactions unrelated to the file to be flushed to disk are not locked, allowing such unrelated requests to continue requesting log space.

5. The method according to claim 1, characterized in that, Before constructing the two-dimensional transaction linked list, the following steps are also included: Based on historical access logs, analyze the access patterns of historical files to predict the potential conflict probability of the current file. Assign a priority tag to the current file based on the potential conflict probability.

6. The method according to claim 5, characterized in that, The step of sequentially writing the data of the file to be flushed to disk and the associated metadata corresponding to the file to be flushed to disk according to the operation sequence of the transactions in the two-dimensional transaction chain list includes: Based on the priority marker, the two-dimensional transaction list corresponding to the first priority file is written to disk, and the two-dimensional transaction list corresponding to the second priority file is written to disk in batches, wherein the first priority is higher than the second priority.

7. A disk flushing device based on a two-dimensional transaction linked list, characterized in that, The device includes: A two-dimensional transaction linked list construction module is used to construct a two-dimensional transaction linked list, which includes a file linked list and a transaction linked list, and each of the transaction linked lists is sorted according to the operation time sequence of the transaction. The two-dimensional transaction list positioning module is used to locate the two-dimensional transaction list corresponding to the file to be flushed to disk in response to the disk flushing requirement. The associated metadata identification module is used to identify the associated metadata corresponding to the file to be flushed based on the operation object of the transaction in the two-dimensional transaction chain list. The data write-to-disk module is used to write the data of the file to be written to disk and the associated metadata of the file to be written to disk in sequence according to the operation sequence of the transactions in the two-dimensional transaction chain.

8. An electronic device, characterized in that, include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, are used to implement the method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method of any one of claims 1 to 6.