Buffer management method, device, equipment and medium based on object storage

Abstract

The embodiment of the present invention provides a buffer management method, device, equipment and medium based on object storage, which relates to the technical field of database system. The method includes dividing the metadata of the database into root metadata and secondary metadata according to the hierarchical structure; for each data modification operation in the buffer, generating a log corresponding to each data modification operation; mapping the buffer to a virtual directory of the object storage, storing the log under the virtual directory according to the second version number, and storing the data modification record of the buffer in the HashMap of the shared memory; when updating the metadata, maintaining the reference relationship between the root metadata and the secondary metadata; when reading the buffer data, reading the valid log versions in sequence, playing back step by step according to the version number, generating data corresponding to the latest version number and returning it to the user. The application improves the consistency and storage efficiency of data, and realizes the efficient operation of the database engine on the object storage.

Description

Technical Field

[0001] The present invention relates to the technical field of database systems, and in particular to a buffer management method, device, equipment and medium based on object storage. Background Art

[0002] With the rapid development of cloud computing and distributed storage, object storage has been widely used in large-scale data management and cloud-native database engines due to its scalability, high availability, and low cost. However, the design concept of object storage is more inclined to sequential access to large files, while database systems usually need to efficiently support random access, partial modification, and low-latency reading and writing, which makes object storage and database buffer management have significant technical contradictions.

[0003] Traditional database systems rely on file systems or block storage to implement buffer management, allowing random read and write of pages. However, when database storage is migrated to object storage, random access and partial write operations face the following technical challenges:

[0004] No support for random writes: Object storage generally does not support modifying the contents of an object. Any update requires rewriting the entire object.

[0005] High latency: The access latency of object storage is higher than that of the local file system, which is not friendly to small files and frequent reading and writing;

[0006] Metadata storage limitations: To overcome the limitations of object storage, metadata is usually stored on a separate server. Although this solution partially solves the random read and write problem, it limits the size of metadata and makes it difficult to support more complex database storage engine designs.

[0007] Existing database systems mainly use the following solutions to deal with these problems:

[0008] Independent metadata server: Existing cloud-native database systems (such as Amazon Aurora) store metadata on dedicated servers. Although fast access is possible, there are certain limitations on storage scale and scalability;

[0009] Log-based storage model: All operations are recorded by writing transaction logs, and the logs are replayed to form the latest state during recovery. This method is suitable for scenarios with high transaction consistency requirements, but frequent replay operations bring significant performance overhead;

[0010] Full version generation: Generate full snapshots periodically to reduce playback costs. This method requires more storage space, and the snapshot generation operation itself may affect system performance.

[0011] These existing solutions solve the challenges of random access and partial writes to a certain extent, but there is still room for optimization in terms of storage efficiency, performance consistency, and scalability. Summary of the invention

[0012] In view of this, an embodiment of the present invention provides a buffer management method based on object storage to solve the technical problem in the prior art that when database storage is migrated to object storage, storage efficiency, performance consistency and scalability are still low. The method includes:

[0013] Dividing the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer;

[0014] For each data modification operation of the buffer, generating a log corresponding to each data modification operation;

[0015] Mapping the buffer to a virtual directory of the object storage, storing the log in the virtual directory according to the second version number, and storing the data modification record of the buffer in a HashMap of the shared memory;

[0016] When the metadata is updated, the reference relationship between the root metadata and the secondary metadata is maintained;

[0017] When reading data from the buffer, valid log versions are read in sequence, and the data corresponding to the latest second version number is gradually played back to generate data corresponding to the latest second version number and returned to the user.

[0018] The embodiment of the present invention also provides a buffer management device based on object storage to solve the technical problem in the prior art that when database storage is migrated to object storage, storage efficiency, performance consistency and scalability are still low. The device includes:

[0019] A partitioning module, used for partitioning the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, wherein the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer;

[0020] A log generation module, used for generating a log corresponding to each data modification operation for each data modification operation in the buffer;

[0021] A buffer mapping module, used for mapping the buffer to a virtual directory of the object storage, storing the log in the virtual directory according to the second version number, and storing the data modification record of the buffer in the HashMap of the shared memory;

[0022] A metadata updating module, used for maintaining the reference relationship between the root metadata and the secondary metadata when updating the metadata;

[0023] The reading module is used to read the valid log versions in sequence when reading the data in the buffer, replay them step by step according to the second version number, generate data corresponding to the latest second version number and return them to the user.

[0024] An embodiment of the present invention also provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements any of the above-mentioned object storage-based buffer management methods when executing the computer program, so as to solve the technical problem in the prior art that when database storage is migrated to object storage, storage efficiency, performance consistency and scalability are still low.

[0025] An embodiment of the present invention also provides a computer-readable storage medium, which stores a computer program for executing any of the above-mentioned object storage-based buffer management methods to solve the technical problem in the prior art that when database storage is migrated to object storage, storage efficiency, performance consistency and scalability are still low.

[0026] Compared with the prior art, the beneficial effects achieved by at least one of the above technical solutions adopted in the embodiments of this specification include at least: dividing the metadata of the database into root metadata and secondary metadata according to the hierarchical structure, the root metadata records the first version number and the pointer to the secondary metadata, and the secondary metadata is managed through the buffer; for each data modification operation of the buffer, a log corresponding to each data modification operation is generated; the buffer is mapped to a virtual directory of the object storage, the log is stored in the virtual directory according to the second version number, and the data modification record of the buffer is stored in the HashMap of the shared memory; when the metadata is updated, the reference relationship between the root metadata and the secondary metadata is maintained; when the data in the buffer is read, the valid log version is read in sequence, and the second version number is gradually played back, and the data corresponding to the latest second version number is generated and returned to the user. The present application manages the metadata hierarchically, supports multiple repairs and reference relationship updates, ensures that the reference relationship between the root metadata and the secondary metadata remains consistent, thereby ensuring the consistency of the overall data; by setting the buffer mapping and logging mechanism, the efficient logging and persistence of the buffer are realized, the frequency and cost of object storage access are reduced, and the storage efficiency is improved. Therefore, the data buffer management method of the present application realizes the efficient operation of the database engine on object storage based on multiple mechanisms such as metadata hierarchical management and log-style recording. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0028] Figure 1 is a flow chart of a buffer management method based on object storage provided by an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of a hierarchical structure of metadata provided by an embodiment of the present invention;

[0030] Figure 3 This is a schematic diagram of a log format provided by an embodiment of the present invention;

[0031] Figure 4 is a structural block diagram of a computer device provided by an embodiment of the present invention;

[0032] Figure 5 It is a structural block diagram of a buffer management device based on object storage provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0034] The following describes the implementation methods of the present application through specific examples, and those skilled in the art can easily understand other advantages and effects of the present application from the contents disclosed in this specification. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. The present application can also be implemented or applied through other different specific implementation methods, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present application. It should be noted that, in the absence of conflict, the following embodiments and the features in the embodiments can be combined with each other. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without making creative work belong to the scope of protection of the present application.

[0035] In an embodiment of the present invention, a buffer management method based on object storage is provided, such as Figure 1 As shown, the method includes:

[0036] Step S101, dividing the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer;

[0037] Step S102: for each data modification operation in the buffer, generate a log corresponding to each data modification operation;

[0038] Step S103, mapping the buffer to a virtual directory of the object storage, storing the log in the virtual directory according to the second version number, and storing the data modification record of the buffer in the HashMap of the shared memory;

[0039] Step S104: when the metadata is updated, the reference relationship between the root metadata and the secondary metadata is maintained;

[0040] Step S105: when reading the data in the buffer, read the valid log versions in sequence, replay them step by step according to the second version number, generate data corresponding to the latest second version number and return them to the user.

[0041] In this embodiment, by hierarchical management of metadata, multiple repairs and reference relationship updates are supported, ensuring that the reference relationship between the root metadata and the secondary metadata remains consistent, thereby ensuring the consistency of the overall data; by setting the buffer mapping and logging mechanism, efficient logging and persistence of the buffer are achieved, the frequency and cost of object storage access are reduced, and storage efficiency is improved. Therefore, the data buffer management method of the present application realizes the efficient operation of the database engine on object storage based on multiple mechanisms such as metadata hierarchical management and log-style logging.

[0042] In specific implementation, the metadata hierarchy refers to Figure 2 As shown in the figure, the metadata of the database is organized in a hierarchical structure. Each table corresponds to a root metadata (RootMeta), which records the type and first version number of its sub-files. For metadata that is repaired multiple times, such as secondary metadata, buffer management is used to support high-frequency modification. For one-time write data, such as indexes or initial files, they are directly written to the object storage.

[0043] Specifically, a root metadata page includes a version number and a pointer to the secondary metadata (SubmetaX). For example, the Version under the RootMeta page in the figure records the version of the current metadata, metapnt1, metapnt2,..., metapntN, which are pointers to different secondary metadata pages. The Version under the SubmetaX page records the version of the current sub-metadata, datapnt1, datapnt2,..., datapntM, which are pointers to the actual data file (Datafile). Datafile is the actual data file stored in the database. By organizing metadata in a hierarchical structure, data in the database can be efficiently managed and accessed; each metadata page (including root metadata and sub-metadata) has a version number, which helps to track and manage different versions of metadata; by using pointers (metapnt and datapnt), specific sub-metadata pages or data files can be quickly located, thereby improving the efficiency of data access. This metadata hierarchical management mechanism not only improves the flexibility and scalability of data management, but also ensures the consistency and integrity of data.

[0044] In one embodiment, maintaining the reference relationship between the root metadata and the secondary metadata includes:

[0045] When the secondary metadata is modified, it triggers an update of the first version number of the root metadata.

[0046] In this embodiment, during the metadata update process, the reference relationship is maintained. For example, the modification of the secondary metadata will trigger the update of the root page version number, thereby ensuring overall consistency.

[0047] In specific implementation, maintaining reference relationships means that when a metadata item is updated, other related metadata items are also properly adjusted to reflect the latest status. This includes but is not limited to: updating version numbers and updating pointers. Updating pointers means that if the location of secondary metadata changes, the pointer in the root metadata needs to be updated to the new location. Ensure that all operations that rely on these metadata can obtain the latest and most accurate information.

[0048] In one embodiment, storing the log in the virtual directory according to the second version number includes:

[0049] When the preset conditions are met, all logs are written at once to the virtual directory corresponding to the object storage according to the second version number according to the records in the HashMap.

[0050] In specific implementation, the preset condition can be that when a transaction is committed or the buffer space exceeds the threshold, the log is written to the object storage at one time, thereby ensuring the consistency and persistence of the data. The log is stored in the virtual directory by version number, indicating that each modification has its unique identifier, forming a traceable history record, so that the historical changes of the data can be tracked.

[0051] The log format can be from the existing format of the database or a custom format. For example, Figure 3 A log format shown in FIG. 1 can use Version: N to represent a log entry with version number N, and Item1, Item2,..., Itemj to represent that the log entry with version number N contains multiple items from Item1 to Itemj; and Version: M to represent a log entry with version number M, and Item1, Item2,..., Itemi to represent that the log entry with version number M contains multiple items from Item1 to Itemi. Checkpoints are used to record a specific time point of the system status.

[0052] In specific implementation, the modification records of the buffer are stored in a HashMap in shared memory to support concurrent access. That is, the modification records of the data in the buffer (i.e. which data items have been modified and their new values) will be temporarily stored in a HashMap located in shared memory. HashMap is a data structure based on hash table implementation, which provides efficient insertion, deletion and search operations. The role of HashMap in this embodiment is to provide a fast access and update mechanism to support concurrent operations and ensure that all ongoing modifications can be tracked.

[0053] In one embodiment, for the log playback process, in order to reduce the overhead of frequent playback, when the log volume is large, a full version will be generated and stored in the object storage.

[0054] In one embodiment, the method further comprises:

[0055] When data is flushed, the first version number of the root metadata is updated;

[0056] If the first version number of the read data is smaller than the current first version number, the read data is discarded.

[0057] In specific implementation, data flushing refers to the process of writing data in memory (including data in the buffer) to persistent storage (object storage). When data is flushed, the version number of RootMeta is updated to ensure data consistency. If the version number of the read data is smaller than the version number of RootMeta, the data is inconsistent and needs to be discarded. The version number of RootMeta is used as the basis for consistency judgment, which simplifies complex lock management operations.

[0058] In one embodiment, the method further comprises:

[0059] Set a transaction number for each data record;

[0060] When reading data, the data records are filtered based on the transaction number according to the content of the current transaction, and the relevant data records visible to the current transaction are returned.

[0061] In practice, all metadata modifications are visible immediately. When performing a read operation, the system checks the transaction number of each data record and decides which records should be visible to the current transaction based on the context of the current transaction (for example, the time point when the transaction started). This filtering mechanism ensures that even if there are multiple concurrent transactions in progress, each transaction can only see the data state that matches its time point, thereby achieving correct isolation and consistency.

[0062] In one embodiment, in the case of sharing data across multiple clusters, how to ensure the consistency and synchronization of data, an update strategy is provided, which specifically includes the following steps:

[0063] When the write cluster modifies the data, the write cluster sends an update notification to the read cluster, and the read cluster abandons the data currently in the buffer and reloads the latest data from the write cluster or the object storage.

[0064] In this embodiment, the active notification strategy can immediately make the read cluster aware of data changes and respond in a timely manner. However, each write operation needs to send a notification, which may increase network communication overhead.

[0065] Furthermore, in order to ensure data consistency and synchronization when data is shared across multiple clusters, another update strategy is provided, which specifically includes the following steps:

[0066] When the transaction starts to execute, the read cluster directly reads the first version number of the root metadata from the object storage, and compares the read first version number with the first version number saved in the buffer;

[0067] If the read first version number is newer than the first version number stored in the buffer, the read cluster synchronizes the secondary metadata.

[0068] In this embodiment, a cross-cluster update strategy for version comparison is provided, which reduces unnecessary network communication because version comparison only checks the version number once at the beginning of a transaction. Through the active notification strategy and the cross-cluster update strategy for version comparison, the system can flexibly adjust its behavior in different application scenarios to achieve optimal performance and reliability and realize data synchronization in a distributed environment.

[0069] The object storage-based buffer management method of this application is designed to efficiently support random access, partial writing, and ensure data consistency and visibility. By combining innovative technologies such as metadata hierarchical management, log-style recording, and cross-cluster update strategies, the efficient operation of the database engine on object storage is achieved. The purpose is to: provide a flexible and efficient metadata management mechanism that supports multiple repairs and reference relationship updates; achieve efficient logging and persistence of the buffer, reduce the frequency and cost of object storage access; ensure data consistency and immediate visibility, and provide reliable access guarantees in concurrent transaction environments; support distributed deployment and high availability through cross-cluster update strategies.

[0070] Therefore, compared with the prior art, the object storage-based buffer management method of the present application has the following characteristics:

[0071] 1. Efficiency: Reduce frequent access to object storage and reduce latency through log caching and playback mechanisms;

[0072] 2. Scalability: Supports hierarchical metadata management and large-scale distributed storage scenarios, breaking through the limitations of independent metadata servers;

[0073] 3. Consistency: Based on version number control and disk flushing mechanism, data consistency is ensured and implementation logic is simplified;

[0074] 4. Flexibility: Supports multiple log formats and customizable strategies to meet the needs of different database engines;

[0075] 5. Cross-cluster sharing: Data synchronization in a distributed environment is achieved through two strategies: active notification and version comparison.

[0076] In this embodiment, a computer device is provided, such as Figure 4 As shown, it includes a memory 401, a processor 402, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, any of the above-mentioned object storage-based buffer management methods is implemented.

[0077] Specifically, the computer device may be a computer terminal, a server or a similar computing device.

[0078] In this embodiment, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program for executing any of the above-mentioned object storage-based buffer management methods.

[0079] Specifically, computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0080] Based on the same inventive concept, an object-based storage buffer management device is also provided in an embodiment of the present invention, as described in the following embodiment. Since the principle of solving the problem by the object-based storage buffer management device is similar to that of the object-based storage buffer management method, the implementation of the object-based storage buffer management device can refer to the implementation of the object-based storage buffer management method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements the predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0081] Figure 5 is a structural block diagram of a buffer management device based on object storage according to an embodiment of the present invention, such as Figure 5 As shown, it includes: a division module 501, a log generation module 502, a buffer mapping module 503, a metadata update module 504 and a reading module 505. The structure is described below.

[0082] A division module 501 is used to divide the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, wherein the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer;

[0083] A log generation module 502, configured to generate a log corresponding to each data modification operation for each data modification operation in the buffer;

[0084] A buffer mapping module 503 is used to map the buffer to a virtual directory of the object storage, store the log in the virtual directory according to the second version number, and store the data modification record of the buffer in the HashMap of the shared memory;

[0085] The metadata updating module 504 is used to maintain the reference relationship between the root metadata and the secondary metadata when the metadata is updated;

[0086] The reading module 505 is used to read the valid log versions in sequence when reading the data in the buffer, replay them step by step according to the second version number, generate data corresponding to the latest second version number and return them to the user.

[0087] In one embodiment, the buffer mapping module 503 is further used to:

[0088] When the preset conditions are met, all logs are written at once to the virtual directory corresponding to the object storage according to the second version number according to the records in the HashMap.

[0089] In one embodiment, the metadata updating module 504 is further configured to:

[0090] When the secondary metadata is modified, it triggers an update of the first version number of the root metadata.

[0091] In one embodiment, the apparatus further comprises:

[0092] An update module, used for updating the first version number of the root metadata when performing data flushing;

[0093] The discarding module is used to discard the read data if the first version number of the read data is smaller than the current first version number.

[0094] In one embodiment, the apparatus further comprises:

[0095] The transaction number setting module is used to set the transaction number for each data record;

[0096] The filtering module is used to filter the data records based on the transaction number according to the content of the current transaction when reading data, and return the relevant data records visible to the current transaction.

[0097] In one embodiment, the apparatus further comprises:

[0098] The active notification module is used for when the write cluster modifies the data, the write cluster sends an update notification to the read cluster, and the read cluster abandons the data in the current buffer and reloads the latest data from the write cluster or the object storage.

[0099] In one embodiment, the apparatus further comprises:

[0100] A version comparison module, configured to, when a transaction starts to be executed, cause the read cluster to directly read the first version number of the root metadata from the object storage, and compare the read first version number with the first version number stored in the buffer;

[0101] A synchronization module is used to synchronize the secondary metadata with the read cluster if the read first version number is newer than the first version number stored in the buffer. The embodiment of the present invention achieves the following technical effects: the metadata of the database is divided into root metadata and secondary metadata according to the hierarchical structure, the root metadata records the first version number and the pointer to the secondary metadata, and the secondary metadata is managed through the buffer; for each data modification operation in the buffer, a log corresponding to each data modification operation is generated; the buffer is mapped to a virtual directory of the object storage, the log is stored in the virtual directory according to the second version number, and the data modification record of the buffer is stored in the HashMap of the shared memory; when the metadata is updated, the reference relationship between the root metadata and the secondary metadata is maintained; when the data in the buffer is read, the valid log versions are read in sequence, and the data corresponding to the latest second version number is gradually played back according to the second version number, and the data corresponding to the latest second version number is generated and returned to the user. This application manages metadata in a hierarchical manner, supports multiple repairs and reference relationship updates, ensures that the reference relationship between the root metadata and the secondary metadata remains consistent, and thus ensures the consistency of the overall data; by setting up a buffer mapping and logging mechanism, efficient logging and persistence of the buffer are achieved, the frequency and cost of object storage access are reduced, and storage efficiency is improved. Therefore, the data buffer management method of this application realizes the efficient operation of the database engine on object storage based on multiple mechanisms such as metadata hierarchical management and log-style logging.

[0102] Obviously, those skilled in the art should understand that the modules or steps of the above-mentioned embodiments of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order from that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. In this way, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0103] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A buffer management method based on object storage, characterized in that: include: Dividing the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer; For each data modification operation of the buffer, generating a log corresponding to each data modification operation; Mapping the buffer to a virtual directory of the object storage, storing the log in the virtual directory according to the second version number, and storing the data modification record of the buffer in a HashMap of the shared memory; When the metadata of the database is updated, the reference relationship between the root metadata and the secondary metadata is maintained; The maintaining of the reference relationship between the root metadata and the secondary metadata includes: triggering an update of the first version number of the root metadata when the secondary metadata is modified; When reading data from the buffer, valid log versions are read in sequence, and the data corresponding to the latest second version number is gradually played back to generate data corresponding to the latest second version number and returned to the user.

2. The object storage-based buffer management method according to claim 1, characterized in that: The step of storing the log in the virtual directory according to the second version number includes: When the preset conditions are met, all logs are written at once to the virtual directory corresponding to the object storage according to the second version number according to the records in the HashMap.

3. The object storage-based buffer management method according to claim 1, characterized in that: The method further comprises: When data is flushed, the first version number of the root metadata is updated; If the first version number of the read data is smaller than the current first version number, the read data is discarded.

4. The object storage-based buffer management method according to claim 1, characterized in that: The method further comprises: Set a transaction number for each data record; When reading data, the data records are filtered based on the transaction number according to the content of the current transaction, and the relevant data records visible to the current transaction are returned.

5. The object storage-based buffer management method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the write cluster modifies the data, the write cluster sends an update notification to the read cluster, and the read cluster abandons the data currently in the buffer and reloads the latest data from the write cluster or the object storage.

6. The object storage-based buffer management method according to any one of claims 1 to 4, characterized in that: The method further comprises: When the transaction starts to execute, the read cluster directly reads the first version number of the root metadata from the object storage, and compares the read first version number with the first version number saved in the buffer; If the read first version number is newer than the first version number stored in the buffer, the read cluster synchronizes the secondary metadata.

7. A buffer management device based on object storage, characterized in that: include: A partitioning module, used for partitioning the metadata of the database into root metadata and secondary metadata according to a hierarchical structure, wherein the root metadata records a first version number and a pointer to the secondary metadata, and the secondary metadata is managed through a buffer; A log generation module, used for generating a log corresponding to each data modification operation for each data modification operation in the buffer; A buffer mapping module, used for mapping the buffer to a virtual directory of the object storage, storing the log in the virtual directory according to the second version number, and storing the data modification record of the buffer in the HashMap of the shared memory; A metadata updating module, used for maintaining the reference relationship between the root metadata and the secondary metadata when updating the metadata of the database; The maintaining of the reference relationship between the root metadata and the secondary metadata includes: triggering an update of the first version number of the root metadata when the secondary metadata is modified; The reading module is used to read the valid log versions in sequence when reading the data in the buffer, replay them step by step according to the second version number, generate data corresponding to the latest second version number and return them to the user.

8. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the object storage-based buffer management method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program for executing the object storage-based buffer management method according to any one of claims 1 to 6.

Images