Instant snapshot creation and reading methods, apparatuses, and devices

Abstract

The application discloses a kind of transient snapshot creation and reading method, device and equipment, it is related to data processing technical field, comprising: when receiving snapshot creation request, current snapshot is created based on snapshot creation request, and current snapshot is inserted into snapshot table;When receiving historical snapshot reading request, corresponding historical snapshot timestamp is obtained from snapshot table based on historical snapshot reading request;Inquiry is carried out based on historical snapshot timestamp, and target checkpoint list is obtained;Based on target checkpoint list, load the data object existing in historical snapshot timestamp to construct target snapshot partition.Because the problem that long creating time caused by full data replication in traditional snapshot technology and low reading performance caused by full historical data scanning is solved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method, apparatus and device for creating and reading instantaneous snapshots. Background Technology

[0002] As enterprise data volumes continue to grow and mixed workload application scenarios expand, database systems need to be able to capture data status at the millisecond level and read historical data at high performance without significantly impacting the performance of online business. This is necessary to support key functions such as data version management, real-time analysis, and rapid recovery, and to meet the needs for high-frequency, fine-grained data snapshot creation and querying.

[0003] Existing database snapshot technologies mainly employ full replication or incremental snapshot methods. Creating a snapshot requires large-scale data scanning and replication, resulting in a snapshot creation process that can take anywhere from several minutes to several hours, severely impacting business continuity. At the same time, reading historical data requires full scanning and state reconstruction, which is inefficient and cannot meet real-time query requirements.

[0004] Therefore, existing technologies suffer from problems such as excessively long snapshot creation time and low snapshot reading performance, failing to provide efficient and real-time access to historical data states while ensuring business continuity.

[0005] The above content is only used to help understand the technical solution of this application and does not represent an admission that the above content is prior art. Summary of the Invention

[0006] The main objective of this application is to provide a method, apparatus, and device for creating and reading instantaneous snapshots, aiming to solve the technical problem of how to improve the efficiency of snapshot creation and reading.

[0007] To achieve the above objectives, this application proposes a method for creating and reading instantaneous snapshots, the method comprising:

[0008] Upon receiving a snapshot creation request, a current snapshot is created based on the snapshot creation request, and the current snapshot is inserted into the snapshot table;

[0009] Upon receiving a historical snapshot read request, the corresponding historical snapshot timestamp is obtained from the snapshot table based on the historical snapshot read request;

[0010] A list of target checkpoints is obtained by querying based on the historical snapshot timestamps.

[0011] Based on the target checkpoint list, data objects existing at the historical snapshot timestamps are loaded to construct the target snapshot partition.

[0012] In one embodiment, upon receiving a snapshot creation request, creating a current snapshot based on the snapshot creation request and inserting the current snapshot into the snapshot table includes:

[0013] Upon receiving a snapshot creation request, a snapshot identifier and a snapshot timestamp are generated;

[0014] The snapshot level and snapshot name are determined based on the snapshot creation request, wherein the snapshot level includes cluster level, account level, database level and table level;

[0015] Extract the corresponding object identification information based on the snapshot level;

[0016] The current snapshot is determined based on the snapshot identifier, the snapshot timestamp, the snapshot name, the snapshot level, and the object identifier information;

[0017] Insert the current snapshot into the snapshot table.

[0018] In one embodiment, upon receiving a historical snapshot read request, retrieving the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot read request includes:

[0019] Upon receiving a historical snapshot read request, determine the historical snapshot name or query timestamp based on the historical snapshot read request;

[0020] Search for historical snapshots in the snapshot table based on the historical snapshot name or the query timestamp;

[0021] Extract the historical snapshot timestamp from the historical snapshot.

[0022] In one embodiment, the step of querying based on the historical snapshot timestamp to obtain the target checkpoint list includes:

[0023] The historical snapshot timestamp is sent to the transaction node, and the checkpoint list returned by the transaction node based on the historical snapshot timestamp is obtained, wherein the checkpoint list includes multiple checkpoints;

[0024] If the end time of the last checkpoint in the checkpoint list is greater than or equal to the historical snapshot timestamp, the checkpoint list is used as the target checkpoint list.

[0025] In one embodiment, the step of loading data objects containing the historical snapshot timestamps based on the target checkpoint list to construct a target snapshot partition includes:

[0026] Obtain the target checkpoints contained in each of the target checkpoint lists, wherein each target checkpoint includes multiple data object information, and the data object information includes data object ID, creation timestamp, and deletion timestamp;

[0027] When the creation timestamp of the data object information is less than or equal to the historical snapshot timestamp and the deletion timestamp of the data object information is greater than the historical snapshot timestamp, the data object corresponding to the data object ID of the data object information is obtained;

[0028] Construct the target snapshot partition based on the data object.

[0029] In one embodiment, before obtaining the target checkpoints contained in each of the target checkpoint lists, the method further includes:

[0030] Use the data object ID corresponding to the target checkpoint list and the historical snapshot timestamp as query conditions;

[0031] Based on the query conditions, perform a query in the snapshot usage cache;

[0032] When the corresponding snapshot partition is found in the snapshot usage cache, the snapshot partition is used as the target snapshot partition;

[0033] If no corresponding snapshot partition is found in the snapshot usage cache, the step of obtaining the target checkpoints contained in each target checkpoint list is performed.

[0034] In one embodiment, after constructing the target snapshot partition based on the data object, the method further includes:

[0035] Based on the snapshot table, multiple active snapshot timestamps are obtained;

[0036] Based on the target checkpoint, information on multiple data objects is obtained;

[0037] The creation timestamp and deletion timestamp of each data object are compared with the timestamp of each active snapshot to obtain the comparison results;

[0038] Based on the comparison results, determine the objects that can be deleted;

[0039] Add the removable object to the tombstone record.

[0040] In one embodiment, after loading data objects containing the historical snapshot timestamps based on the target checkpoint list to construct the target snapshot partition, the method further includes:

[0041] Get the total number of data objects, the data size of each data object, and the percentage of rows deleted from each data object;

[0042] Based on the data size and the proportion of deleted rows, a vacuum score is obtained;

[0043] When the vacuum score is greater than a preset score threshold, the data object is added to the queue to be merged.

[0044] The data storage status is determined to meet the merge triggering conditions based on the total number of data objects, the data size of the data objects, and the vacuum degree score.

[0045] When the data storage status meets the merge triggering conditions, data merging is performed based on the queue to be merged.

[0046] In addition, to achieve the above objectives, this application also proposes a snapshot creation and reading device, which includes: a creation module, used to create a current snapshot based on a snapshot creation request when a snapshot creation request is received, and to insert the current snapshot into a snapshot table;

[0047] The reading module is used to obtain the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot reading request when a historical snapshot reading request is received.

[0048] The query module is used to perform queries based on the historical snapshot timestamps to obtain a list of target checkpoints;

[0049] The loading module is used to load data objects existing at the historical snapshot timestamps based on the target checkpoint list to construct the target snapshot partition.

[0050] In addition, to achieve the above objectives, this application also proposes a device for creating and reading instantaneous snapshots, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the instantaneous snapshot creation and reading method described above.

[0051] In addition, to achieve the above objectives, this application also proposes a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the instantaneous snapshot creation and reading method described above.

[0052] In addition, to achieve the above objectives, this application also provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the instantaneous snapshot creation and reading method described above.

[0053] Upon receiving a snapshot creation request, this application creates a current snapshot based on the request and inserts it into a snapshot table. Upon receiving a historical snapshot read request, it retrieves the corresponding historical snapshot timestamp from the snapshot table based on the request. It then queries the historical snapshot timestamp to obtain a target checkpoint list. Based on the target checkpoint list, it loads data objects existing at the historical snapshot timestamps to construct a target snapshot partition. This solves the problems of long creation times due to full data replication and low read performance due to full historical data scanning in traditional snapshot technologies. Compared with existing technologies, this application achieves millisecond-level snapshot creation by inserting metadata records only into the snapshot table, and reconstructs historical states by querying checkpoint entries that cover snapshot timestamps and loading only relevant data objects. This results in an order-of-magnitude improvement in snapshot creation time from minutes to milliseconds, and snapshot read performance approaching normal read performance levels. Attached Figure Description

[0054] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0055] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0056] Figure 1 This is a flowchart illustrating an embodiment of the method for creating and reading instantaneous snapshots in this application.

[0057] Figure 2 This is a flowchart illustrating Embodiment 2 of the method for creating and reading instantaneous snapshots in this application.

[0058] Figure 3 This is a schematic diagram of the module structure of the instantaneous snapshot creation and reading device according to an embodiment of this application;

[0059] Figure 4 This is a schematic diagram of the device structure of the hardware operating environment involved in the instantaneous snapshot creation and reading method in the embodiments of this application.

[0060] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0061] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0062] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0063] The main solution of this application embodiment is as follows: upon receiving a snapshot creation request, a current snapshot is created based on the snapshot creation request, and the current snapshot is inserted into the snapshot table; upon receiving a historical snapshot read request, the corresponding historical snapshot timestamp is obtained from the snapshot table based on the historical snapshot read request; a target checkpoint list is obtained by querying based on the historical snapshot timestamp; and based on the target checkpoint list, data objects existing at the historical snapshot timestamps are loaded to construct the target snapshot partition. In this embodiment, for ease of description, the instantaneous snapshot creation and reading system is used as the execution subject for the following description.

[0064] Because existing database snapshot technologies mainly use full replication or incremental snapshots, large-scale data scanning and replication are required when creating a snapshot, resulting in a snapshot creation process that can take several minutes to several hours, seriously affecting business continuity. At the same time, reading historical data requires full scanning and state reconstruction, which is inefficient and cannot meet the requirements of real-time queries.

[0065] This application provides a solution that addresses the problems of long creation times due to full data replication and low read performance due to full historical data scanning in traditional snapshot technologies. Compared with existing technologies, this application achieves millisecond-level snapshot creation by inserting metadata records only into the snapshot table, and reconstructs the historical state by querying checkpoint entries that cover the snapshot timestamp and loading only the relevant data objects. This results in an order-of-magnitude improvement in snapshot creation time from minutes to milliseconds, and snapshot read performance approaching the level of normal read performance.

[0066] It should be noted that the executing entity in this embodiment can be a computing service device with data processing, network communication, and program execution functions, such as a tablet computer, personal computer, or mobile phone, or an electronic device capable of performing the above functions, such as a snapshot creation and retrieval system. The following description uses a snapshot creation and retrieval system as an example to illustrate this embodiment and the subsequent embodiments.

[0067] Based on this, embodiments of this application provide a method for creating and reading instantaneous snapshots, referring to... Figure 1 , Figure 1 This is a flowchart illustrating the first embodiment of the instantaneous snapshot creation and reading method of this application.

[0068] In this embodiment, the instantaneous snapshot creation and reading method includes steps S10~S40:

[0069] Step S10: Upon receiving a snapshot creation request, create the current snapshot based on the snapshot creation request and insert the current snapshot into the snapshot table;

[0070] It should be noted that a snapshot creation request is an operation instruction initiated by a user or the system to create a historical state record at a specified data level; the current snapshot refers to the record generated corresponding to the request, which contains snapshot metadata information; and the snapshot table is a system table in the database used to centrally store all snapshot metadata information.

[0071] Understandably, since traditional snapshot creation requires full data copying, resulting in long creation time and impacting business continuity, step S10, which only inserts metadata records into the snapshot table to complete snapshot creation, avoids data scanning and copying operations, thereby improving snapshot creation speed and achieving millisecond-level response.

[0072] In one feasible implementation, step S10 may include: upon receiving a snapshot creation request, generating a snapshot identifier and a snapshot timestamp; determining a snapshot level and a snapshot name based on the snapshot creation request, wherein the snapshot level includes cluster level, account level, database level, and table level; extracting corresponding object identification information based on the snapshot level; determining the current snapshot based on the snapshot identifier, the snapshot timestamp, the snapshot name, the snapshot level, and the object identification information; and inserting the current snapshot into the snapshot table.

[0073] It should be noted that the snapshot identifier is a globally unique number used to uniquely identify the snapshot; the snapshot timestamp is the physical timestamp at the moment the snapshot was created, used to mark the time point of historical state; the snapshot level refers to the level of data scope covered by the snapshot, where: cluster level means the snapshot covers all data in the entire database cluster, suitable for full cluster backup and recovery scenarios, supporting critical business scenarios such as disaster recovery; account level means the snapshot covers all data under a specific account, suitable for account-level backup and recovery in multi-tenant environments, supporting tenant data migration and isolation scenarios; database level means the snapshot covers all data within a specific database, suitable for database-level backup and recovery, supporting database migration and version management scenarios; table level means the snapshot covers only a single data table, suitable for table-level backup and recovery, particularly suitable for multi-version data management operations, supporting fine-grained data version control; the snapshot name is a user-defined, readable identifier used to identify the snapshot; object identification information refers to the unique identifier of the data object corresponding to the snapshot level, such as table identifier, database identifier, account identifier, or cluster identifier.

[0074] Specifically, the system queries the system directory to obtain the corresponding object identifier (such as table ID) based on the snapshot level (such as "table level") and object name (such as "db1.orders") specified in the snapshot creation request. Then, it combines the generated snapshot identifier with the timestamp to assemble a complete snapshot metadata record, which is the current snapshot.

[0075] For example, the system is designed with a unified snapshot table as a centralized storage for snapshot metadata. The snapshot table adopts a relational storage model and contains the following key fields:

[0076] snapshot_id: A unique identifier for the snapshot (UUID type, globally unique);

[0077] sname: Snapshot name (user-readable identifier, primary key);

[0078] ts: Snapshot timestamp (physical timestamp, nanosecond precision);

[0079] level: Snapshot level (enumeration types: cluster level, account level, database level, and table level);

[0080] account_name: Account name (for account-level, database-level, and table-level snapshots);

[0081] database_name: Database name (for database-level and table-level snapshots);

[0082] table_name: The table name (for table-level snapshots);

[0083] obj_id: Object ID (associated account / database / table ID).

[0084] The entire snapshot creation process involves only one database write operation, taking milliseconds, and includes the following steps:

[0085] Execute the snapshot creation command: `CREATE SNAPSHOT sp1 FOR TABLE orders`;

[0086] Generate a unique snapshot ID (UUID): `550e8400-e29b-41d4-a716-446655440000`;

[0087] Get the physical timestamp of the current transaction as the snapshot timestamp: `1234567890123456789`

[0088] The system inserts a metadata record into the snapshot table: `(sp1, 550e8400-e29b-41d4-a716-446655440000, 1234567890123456789, 'table', 'orders', 1001)`.

[0089] In this embodiment, a snapshot is created by inserting only metadata without copying the actual data, which solves the performance bottleneck and business interruption problem caused by full data copying in the traditional snapshot creation process.

[0090] The above are merely feasible implementations of step S10 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S10.

[0091] Step S20: Upon receiving a historical snapshot read request, obtain the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot read request;

[0092] It should be noted that a historical snapshot read request is an operation command initiated by the user or the system to read the data status at a certain historical point in time; the historical snapshot timestamp refers to the physical timestamp corresponding to the request, representing the specific moment of the historical state.

[0093] For example, in a scenario where a user accidentally deletes important data and needs to quickly restore it to the state it was in one hour ago, the system needs to verify whether a snapshot from one hour ago exists and whether the user has the permission to restore it. For instance, it could retrieve the timestamp of the snapshot `sp_hour_ago`.

[0094] Execute the following query based on the snapshot name sp_hour_ago: `SELECT * FROM orders {SNAPSHOT = 'sp_hour_ago'} WHERE amount>1000`;

[0095] The system queries the snapshot table and finds the snapshot record: `(sp_hour_ago, T_snap =1234567890000000000, 'table', orders)`;

[0096] Get the snapshot timestamp: T_snap = 1234567890000000000.

[0097] Understandably, traditional historical data reading requires scanning the entire historical record and reconstructing the state, resulting in low query performance and long response time. Therefore, step S20 is performed to quickly obtain accurate timestamp information from a unified snapshot table, which avoids complex queries and associations on scattered metadata, thereby improving the efficiency of locating historical snapshots and providing an accurate time reference for subsequent state reconstruction.

[0098] In one feasible implementation, step S20 may include: upon receiving a historical snapshot read request, determining a historical snapshot name or query timestamp based on the historical snapshot read request; searching for a historical snapshot in the snapshot table based on the historical snapshot name or the query timestamp; and extracting the historical snapshot timestamp from the historical snapshot.

[0099] It should be noted that the historical snapshot name is a string name specified by the user when creating the snapshot to identify the snapshot; the query timestamp is a physical timestamp directly specified by the user to represent the historical moment that the user wants to query; and the historical snapshot refers to a complete metadata record stored in the snapshot table that corresponds to the historical snapshot name or the query timestamp.

[0100] Specifically, the system parses the historical snapshot read request and extracts the snapshot name or timestamp parameter from the request. If the parameter is a snapshot name, the system uses the name as the query condition to perform an exact match in the snapshot table and obtain the corresponding snapshot record. If the parameter is a timestamp, the system uses the timestamp as the query condition to find the record whose snapshot timestamp is not greater than the value and is the closest to it as the historical snapshot. Finally, the system extracts the historical snapshot timestamp from the timestamp field of the record.

[0101] In this embodiment, a centralized snapshot table supports efficient snapshot queries based on name or timestamp, solving the problems of complex queries and slow location caused by the scattered storage of snapshot metadata in traditional methods.

[0102] The above are merely feasible implementations of step S20 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S20.

[0103] Step S30: Query based on the historical snapshot timestamp to obtain a list of target checkpoints;

[0104] It should be noted that the target checkpoint list is an ordered set containing one or more checkpoints, where the time range of each checkpoint covers the historical snapshot timestamp, providing complete data object information for reconstructing the data state at that point in time.

[0105] It is understandable that, since the data state corresponding to the historical snapshot time point may be scattered across multiple checkpoints in different time periods, directly reading a single checkpoint may lead to data omissions or inconsistencies. Therefore, step S30 is performed to obtain all checkpoints covering the timestamp, which can avoid the problem of incomplete data during the state reconstruction process, thereby improving the accuracy and consistency of historical data queries.

[0106] In one feasible implementation, step S30 may include: sending the historical snapshot timestamp to the transaction node and obtaining a checkpoint list returned by the transaction node based on the historical snapshot timestamp, wherein the checkpoint list includes multiple checkpoints; when the end time of the last checkpoint in the checkpoint list is greater than or equal to the historical snapshot timestamp, the checkpoint list is used as the target checkpoint list.

[0107] It should be noted that a transaction node is a distributed database component responsible for managing the creation of transactions and checkpoints; a checkpoint is a persistent record of the state of all data objects in the database within a certain period of time. Each checkpoint includes a start time and an end time, identifying its valid time range; the end time of the last checkpoint in the checkpoint list refers to the upper limit of the time range recorded by the last checkpoint after sorting the list by time.

[0108] Specifically, the system sends the historical snapshot timestamp as a query condition to the transaction node. The transaction node searches its maintained checkpoint metadata for all checkpoints with a start time less than or equal to the timestamp and an end time greater than the timestamp, sorts them by start time, and returns them as a checkpoint list. After receiving the list, the system verifies whether the end time of the last checkpoint is not less than the historical snapshot timestamp to ensure that all data changes before that time point have been persisted. If the condition is met, the list is determined as the target checkpoint list.

[0109] For example, the system sends a snapshot read request to the transaction node to query the checkpoint entries that can serve the timestamp T_snap;

[0110] The transaction node returns a list of checkpoints: `[ Ckp1 (T_start = 1000, T_end = 2000), Ckp2(T_start = 2000 , T_end = 3000 ) , ... , Ckp_n ( T_start = T_snap - 1000 ,T_end=T_snap+1000)]`; The system verifies the checkpoints: the end time T_end of the last checkpoint Ckp_n is >= T_snap, and the verification passes.

[0111] In this embodiment, the complete checkpoint sequence covering the snapshot timestamp is obtained through transaction node coordination, which solves the problem of incomplete data source for historical state reconstruction caused by segmented storage of checkpoints, and ensures data consistency and reconstructability of snapshot reads.

[0112] The above are merely feasible implementations of step S30 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S30.

[0113] Step S40: Based on the target checkpoint list, load the data object that exists at the historical snapshot timestamp to construct the target snapshot partition.

[0114] It should be noted that a data object is a basic data unit stored in a database, containing its actual data content, metadata, and timestamp information; the target snapshot partition is a logical storage structure reconstructed based on historical snapshot timestamps, reflecting the data consistency status at that point in time.

[0115] Specifically, the system iterates through each checkpoint in the target checkpoint list, reading the metadata of the data objects it contains one by one. For each data object, its creation and deletion timestamps are compared with the historical snapshot timestamps. Only when the creation timestamp is not greater than the historical snapshot timestamp and the deletion timestamp is greater than the historical snapshot timestamp (or the deletion timestamp is empty) is the actual data content of the data object loaded from persistent storage. Then, all data objects that meet the conditions are organized according to the original table structure to form a complete partition view corresponding to the historical time point, i.e., the target snapshot partition. The constructed target snapshot partition is registered in the snapshot usage cache, and the combination of the data table identifier and the historical snapshot timestamp is used as the cache key for subsequent identical requests to be quickly reused, avoiding repeated construction.

[0116] When performing data recovery, it is necessary to ensure that the entire recovery process is executed within a single transaction to guarantee atomicity. It is also necessary to ensure that the corresponding recovery operations are performed according to the snapshot level: at the cluster level, all accounts, databases, and tables are recovered; at the account level, all databases and tables under that account are recovered; at the database level, all tables under that database are recovered; and at the table level, a single table is recovered.

[0117] Restore data based on the snapshot sp_hour_ago using the CLONE syntax: `CREATE TABLE orders_restored CLONE orders {SNAPSHOT = 'sp_hour_ago'}`.

[0118] The CLONE operation internally uses a snapshot read mechanism:

[0119] ① Determine the snapshot timestamp: Query by snapshot name or use the timestamp directly;

[0120] ② Based on the snapshot timestamp, use the snapshot read process to read the data of the source table orders at the snapshot time point;

[0121] ③ Write the read data into the target table orders_restored.

[0122] Understandably, since the traditional method requires a full scan and loading of all historical data when reading historical snapshots, resulting in high memory usage and poor read performance, step S40 is performed to incrementally reconstruct the state by loading only the data objects that exist at the historical snapshot timestamp. This avoids full data loading and unnecessary object processing, thereby improving the memory efficiency and response speed of snapshot reads.

[0123] In one feasible implementation, step S40 may include: obtaining the total number of data objects, the data size of the data objects, and the percentage of deleted rows for the data objects; obtaining a vacuum score based on the data size and the percentage of deleted rows; adding the data objects to a queue to be merged when the vacuum score is greater than a preset score threshold; determining whether the data storage status meets the merge triggering conditions based on the total number of data objects, the data size of the data objects, and the vacuum score; and merging data based on the queue to be merged when the data storage status meets the merge triggering conditions.

[0124] It should be noted that: the total number of data objects refers to the total number of data objects currently existing in the system; the data size of a data object refers to the amount of storage space it occupies; the deletion row ratio of a data object refers to the proportion of data rows marked for deletion in that object out of the total number of rows; the vacuum score is a quantitative indicator used to evaluate the storage efficiency of data objects, calculated based on the data size and the deletion row ratio; the preset scoring threshold is the scoring threshold value used to determine whether an object needs to be merged; the queue to be merged is a list storing data objects to be merged; the data storage status is a description of the current degree of storage space fragmentation and the urgency of merging; and the merge triggering conditions refer to the set of rules that must be met to trigger the background merge operation, which typically includes the number of objects, the total size, and the scoring threshold.

[0125] Specifically, the system calculates a vacuum score for each data object, which combines the object size with the proportion of deleted rows to reflect the degree of waste in its storage space. Objects with scores exceeding a preset threshold are added to the queue to be merged. At the same time, the system monitors the total number of data objects, the total data size, and the overall score distribution. When these indicators reach the preset threshold, the system determines that the data storage status meets the merge trigger conditions. At this time, the system selects multiple objects from the queue to be merged, sorts and merges them, generates a new merged object and updates the relevant metadata, and finally cleans up invalid data in the original objects.

[0126] For example, to further reduce storage costs, the system asynchronously merges small objects into larger objects in the background, reducing storage fragmentation and improving storage utilization. The system calculates a vacuum score for each data object to assess whether the object needs to be merged; the calculation formula is as follows:

[0127] VacuumScore = (Number of rows deleted / Total number of rows) * Object size * Weighting factor;

[0128] When the vacuum score of a data object exceeds a preset score threshold, the object is marked as an object to be merged and added to the merge queue.

[0129] For small objects in the queue to be merged, such as objects whose data size is less than the threshold (e.g., 10MB), merging is triggered when the number of small objects exceeds the tolerance threshold or the total size exceeds the merging threshold; for large objects in the queue to be merged, such as objects whose data size is greater than the threshold (e.g., 10MB), large objects in the queue to be merged are merged first.

[0130] When comparing data differences between two snapshots or tables during the merge execution phase, the identifiers or time points of the two target snapshots or tables are obtained. Based on the snapshot identifiers or time points, the data table states at the corresponding times are obtained through the snapshot read process. The states of the two data tables are compared to identify the types of data change operations (insert, update, and delete) that occurred during the process. A difference description file is generated based on the data change operations, which records detailed information about the data change operations. Based on the difference description file, the data change operations recorded therein are applied to the target data table. This process includes: parsing the difference description file to extract the data change operation sequence; verifying whether the target data table has a base snapshot corresponding to the difference description file; and sequentially executing the data change operation sequence on the target data table to ensure that the data state after application is consistent with expectations.

[0131] In this embodiment, by introducing an intelligent merging mechanism based on vacuum degree scoring after the snapshot partition is built, the problems of storage fragmentation and low space utilization caused by long-term data addition, deletion and modification are solved, and continuous optimization of storage space and cost reduction are achieved.

[0132] The above are merely feasible implementations of step S40 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S40.

[0133] This embodiment provides a method for creating and reading instantaneous snapshots. Upon receiving a snapshot creation request, a current snapshot is created based on the request and inserted into a snapshot table. Upon receiving a historical snapshot read request, the corresponding historical snapshot timestamp is retrieved from the snapshot table based on the request. A target checkpoint list is obtained by querying the historical snapshot timestamps. Based on the target checkpoint list, data objects existing at the historical snapshot timestamps are loaded to construct a target snapshot partition. This solves the problems of long creation times due to full data copying and low read performance due to full historical data scanning in traditional snapshot technologies. Compared with existing technologies, this application achieves millisecond-level snapshot creation by inserting metadata records only into the snapshot table, and reconstructs historical states by querying checkpoint entries that cover snapshot timestamps and loading only relevant data objects. This achieves an order-of-magnitude improvement in snapshot creation time from minutes to milliseconds, and snapshot read performance approaching normal read performance levels.

[0134] Based on the first embodiment of this application, in the second embodiment of this application, the content that is the same as or similar to that in the first embodiment described above can be referred to the above description, and will not be repeated hereafter. Based on this, please refer to... Figure 2 The instantaneous snapshot creation and reading method further includes steps S41 to S43 in step S40:

[0135] Step S41: Obtain the target checkpoints contained in each of the target checkpoint lists, wherein the target checkpoints include multiple data object information, and the data object information includes data object ID, creation timestamp and deletion timestamp;

[0136] It should be noted that a target checkpoint is a checkpoint extracted from the target checkpoint list. Each checkpoint contains a set of metadata for all data objects within a certain time period. Data object information is the description information of each data object recorded in the checkpoint. Among them, the data object ID is a number used to uniquely identify the data object, the creation timestamp is the physical timestamp when the data object was created, the deletion timestamp is the physical timestamp when the data object was marked for deletion, and if it is empty, it means that the object still exists.

[0137] Specifically, a new snapshot partition state object needs to be created and initialized to an empty state so that checkpoints can be added to the snapshot partition state object later; checkpoint entries are streamed to reconstruct historical states, starting from checkpoint Ckp1, and data object information is loaded.

[0138] It is understandable that since data objects may be modified or deleted multiple times at different points in time, and their state changes over time, step S41, by extracting object metadata containing timestamp information from the checkpoint, can avoid the inability to determine the existence state of the object at a specific point in time when reconstructing the historical state, thus providing a basis for subsequent accurate filtering based on timestamps.

[0139] In one feasible implementation, before step S41, the following steps may be included: using the data object ID corresponding to the target checkpoint list and the historical snapshot timestamp as query conditions; performing a query in the snapshot usage cache based on the query conditions; when a corresponding snapshot partition is found in the snapshot usage cache, using the snapshot partition as the target snapshot partition; when no corresponding snapshot partition is found in the snapshot usage cache, performing the step of obtaining the target checkpoints contained in each of the target checkpoint lists.

[0140] It should be noted that the query conditions are retrieval key values ​​composed of data object identifiers (such as table ID, database ID, etc.) and historical snapshot timestamps; snapshot cache is a cache that uses a least recently used eviction policy to store built snapshot partitions to accelerate repeated access.

[0141] Specifically, the system generates a cache key based on the data object identifier (e.g., database ID, table ID) and historical snapshot timestamps involved in the request, and searches for a matching snapshot partition in the snapshot usage cache. If a match is found, the partition is directly returned as the target snapshot partition, skipping the subsequent loading and rebuilding steps. If no match is found, step S41 is executed to obtain checkpoints and their contained data object information one by one from the target checkpoint list.

[0142] In this embodiment, by introducing a snapshot cache to avoid repeatedly building snapshot partitions at the same point in time, the performance overhead caused by repeatedly performing state reconstruction when frequently accessing the same historical snapshot is solved, which significantly improves the response speed of snapshot reads and the overall efficiency of the system.

[0143] The above are merely feasible implementations of step S41 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S41.

[0144] Step S42: When the creation timestamp of the data object information is less than or equal to the historical snapshot timestamp and the deletion timestamp of the data object information is greater than the historical snapshot timestamp, obtain the data object corresponding to the data object ID of the data object information;

[0145] It should be noted that the data object corresponding to the data object ID refers to the actual data content located and read from persistent storage based on the data object ID, including all its data rows and metadata.

[0146] It is understandable that since the data state at the historical snapshot time point consists only of the data objects that exist at that time point, step S42, which uses timestamp conditions to accurately filter out the data objects that actually exist at the historical snapshot time point, can avoid loading deleted or uncreated object data, thereby reducing unnecessary IO reads and memory usage and improving the efficiency of state reconstruction.

[0147] Step S43: Construct a target snapshot partition based on the data object.

[0148] Specifically, the system reorganizes the acquired data objects that exist at the historical snapshot timestamp according to the logical structure of the original data table to form a complete and consistent snapshot partition view corresponding to the historical time point. The target snapshot partition contains all the filtered data objects and their metadata, and is arranged internally according to the primary key or storage order of the original table, supporting direct query operations. After construction, the partition is assigned an identifier associated with the historical snapshot timestamp and can be used for subsequent data reading or cache storage.

[0149] Understandably, since the target snapshot partition needs to fully reflect the consistent view of data at the historical snapshot time point, step S43 is performed. By reconstructing the filtered data objects into logical partitions according to the original organization method, the complexity and consistency problems caused by directly operating the original storage structure can be avoided, thereby providing a user interface and query performance consistent with normal reads.

[0150] In one feasible implementation, step S43 may include: obtaining multiple active snapshot timestamps based on the snapshot table; obtaining multiple data object information based on the target checkpoint; comparing the creation timestamp and deletion timestamp of each data object information with each active snapshot timestamp to obtain a comparison result; determining deletable objects based on the comparison result; and adding the deletable objects to the tombstone record.

[0151] It should be noted that the active snapshot timestamp refers to the set of timestamps corresponding to all snapshots that have not yet been deleted; the comparison result is a conclusion to determine whether the object is referenced by any snapshot by comparing the timestamp range of the data object with the active snapshot timestamp; a deletable object refers to a data object that is not referenced by any active snapshot; a tombstone record is a metadata entry used to mark deletable objects and record their deletion timestamps for subsequent garbage collection and cleanup.

[0152] Specifically, the system queries the snapshot table for all snapshot records that have not been deleted, extracts their timestamps to form an active snapshot timestamp list; then it iterates through the data object information in the target checkpoint, comparing the creation timestamp and deletion timestamp of each object with the active snapshot timestamp list one by one. If the object's timestamp range intersects with any active snapshot timestamp, it is determined to be referenced and should be retained; otherwise, it is determined to be a deletable object; finally, the primary key and deletion timestamp of the deletable object are written into the tombstone record for subsequent processing in the garbage collection process.

[0153] For example, for each data object, a hierarchical query and binary search algorithm is used to collect applicable snapshots to obtain active table-level snapshots. The corresponding list of active snapshot timestamps is [1200, 1500]. The creation timestamp of data object obj1 is 1000 and the deletion timestamp is 1300. Its time range [1000, 1300) intersects with both active snapshot timestamps 1200 and 1500, so obj1 is referenced and needs to be retained. The creation timestamp of data object obj2 is 1400 and the deletion timestamp is 1600. Its time range [1400, 1600) only intersects with 1500, so obj2 is also referenced. The creation timestamp of data object obj3 is 1700 and the deletion timestamp is 1800. Its time range does not intersect with any active snapshot timestamp, so it is determined to be a deletable object and added to the tombstone record.

[0154] In this embodiment, by combining the active snapshot timestamp and the data object timestamp range for accurate lifecycle determination, the problem that traditional garbage collection mechanisms cannot identify snapshot dependencies and may accidentally delete referenced data is solved, ensuring the long-term validity of snapshot data and the effective reclamation of system storage space.

[0155] The above are merely feasible implementations of step S43 provided in this embodiment. This embodiment does not specifically limit the specific implementation of step S43.

[0156] This embodiment provides a method for creating and reading instantaneous snapshots. It obtains target checkpoints from a list of target checkpoints, where each target checkpoint includes multiple data object information, including a data object ID, a creation timestamp, and a deletion timestamp. When the creation timestamp of a data object is less than or equal to the historical snapshot timestamp and the deletion timestamp is greater than the historical snapshot timestamp, it obtains the data object corresponding to the data object ID. A target snapshot partition is then constructed based on the data object. This method solves the problems of traditional snapshot reads requiring full loading of historical data and incurring high memory and I / O overhead. Compared with existing technologies, this application achieves near-normal read performance and efficient memory usage through a timestamp-based incremental data loading and cache reuse mechanism.

[0157] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the instantaneous snapshot creation and reading method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0158] This application also provides a device for creating and reading instantaneous snapshots; please refer to [reference needed]. Figure 3 The instantaneous snapshot creation and reading device includes:

[0159] The creation module 10 is used to create a current snapshot based on the snapshot creation request when a snapshot creation request is received, and to insert the current snapshot into the snapshot table;

[0160] The reading module 20 is used to obtain the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot reading request when a historical snapshot reading request is received;

[0161] The query module 30 is used to perform a query based on the historical snapshot timestamp to obtain a list of target checkpoints;

[0162] Loading module 40 is used to load data objects existing in the historical snapshot timestamps based on the target checkpoint list to construct target snapshot partitions.

[0163] The instantaneous snapshot creation and reading apparatus provided in this application, employing the instantaneous snapshot creation and reading method in the above embodiments, can solve the technical problem of how to improve the efficiency of snapshot creation and reading. Compared with the prior art, the beneficial effects of the instantaneous snapshot creation and reading apparatus provided in this application are the same as those of the instantaneous snapshot creation and reading method provided in the above embodiments, and other technical features in the instantaneous snapshot creation and reading apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0164] The creation module 10 is further configured to, upon receiving a snapshot creation request, generate a snapshot identifier and a snapshot timestamp; determine the snapshot level and snapshot name according to the snapshot creation request, wherein the snapshot level includes cluster level, account level, database level, and table level; extract the corresponding object identifier information based on the snapshot level; determine the current snapshot according to the snapshot identifier, the snapshot timestamp, the snapshot name, the snapshot level, and the object identifier information; and insert the current snapshot into the snapshot table.

[0165] The reading module 20 is further configured to, upon receiving a historical snapshot reading request, determine the historical snapshot name or query timestamp based on the historical snapshot reading request; search for a historical snapshot in the snapshot table based on the historical snapshot name or the query timestamp; and extract the historical snapshot timestamp from the historical snapshot.

[0166] The query module 30 is further configured to send the historical snapshot timestamp to the transaction node and obtain the checkpoint list returned by the transaction node based on the historical snapshot timestamp, wherein the checkpoint list includes multiple checkpoints; when the end time of the last checkpoint in the checkpoint list is greater than or equal to the historical snapshot timestamp, the checkpoint list is used as the target checkpoint list.

[0167] The loading module 40 is further configured to obtain target checkpoints contained in each of the target checkpoint lists, wherein the target checkpoints include multiple data object information, and the data object information includes a data object ID, a creation timestamp, and a deletion timestamp; when the creation timestamp of the data object information is less than or equal to the historical snapshot timestamp and the deletion timestamp of the data object information is greater than the historical snapshot timestamp, the data object corresponding to the data object ID of the data object information is obtained; and a target snapshot partition is constructed based on the data object.

[0168] The loading module 40 is further configured to use the data object ID corresponding to the target checkpoint list and the historical snapshot timestamp as query conditions; perform a query in the snapshot usage cache based on the query conditions; when a corresponding snapshot partition is found in the snapshot usage cache, use the snapshot partition as the target snapshot partition; when no corresponding snapshot partition is found in the snapshot usage cache, perform the step of obtaining the target checkpoints contained in each of the target checkpoint lists.

[0169] The loading module 40 is further configured to obtain multiple active snapshot timestamps based on the snapshot table; obtain multiple data object information based on the target checkpoint; compare the creation timestamp and deletion timestamp of each data object information with each active snapshot timestamp to obtain a comparison result; determine deletable objects based on the comparison result; and add the deletable objects to the tombstone record.

[0170] The loading module 40 is further configured to obtain the total number of data objects, the data size of the data objects, and the percentage of deleted rows of the data objects; obtain a vacuum score based on the data size and the percentage of deleted rows; add the data objects to the queue to be merged when the vacuum score is greater than a preset score threshold; determine whether the data storage status meets the merge triggering conditions based on the total number of data objects, the data size of the data objects, and the vacuum score; and perform data merging based on the queue to be merged when the data storage status meets the merge triggering conditions.

[0171] This application provides a snapshot creation and reading device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the snapshot creation and reading method in the above embodiment 1.

[0172] The following is for reference. Figure 4 The diagram illustrates a structural schematic suitable for implementing the instantaneous snapshot creation and retrieval device in the embodiments of this application. The instantaneous snapshot creation and retrieval device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 4 The illustrated instantaneous snapshot creation and reading device is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0173] like Figure 4As shown, the snapshot creation and retrieval device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in ROM (Read Only Memory) 1002 or a program loaded from storage device 1003 into random access memory (RRAM) 1004. RAM 1004 also stores various programs and data required for the operation of the snapshot creation and retrieval device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via bus 1005. Input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to I / O interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the snapshot creation and retrieval device to communicate wirelessly or wiredly with other devices to exchange data. Although snapshot creation and retrieval devices with various systems are shown in the figures, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems can be implemented alternatively.

[0174] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0175] The instantaneous snapshot creation and reading device provided in this application, employing the instantaneous snapshot creation and reading method in the above embodiments, can solve the technical problem of how to improve the efficiency of snapshot creation and reading. Compared with the prior art, the beneficial effects of the instantaneous snapshot creation and reading device provided in this application are the same as those of the instantaneous snapshot creation and reading method provided in the above embodiments, and other technical features in this instantaneous snapshot creation and reading device are the same as those disclosed in the method of the previous embodiment, and will not be repeated here.

[0176] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0177] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0178] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, which are used to execute the instantaneous snapshot creation and reading method in the above embodiments.

[0179] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0180] The aforementioned computer-readable storage medium may be included in the instantaneous snapshot creation and reading device; or it may exist independently and not be assembled into the instantaneous snapshot creation and reading device.

[0181] The aforementioned computer-readable storage medium carries one or more programs that, when executed by a transient snapshot creation and retrieval device, cause the transient snapshot creation and retrieval device to: upon receiving a snapshot creation request, create a current snapshot based on the snapshot creation request and insert the current snapshot into a snapshot table; upon receiving a historical snapshot retrieval request, retrieve the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot retrieval request; perform a query based on the historical snapshot timestamp to obtain a target checkpoint list; and, based on the target checkpoint list, load data objects existing at the historical snapshot timestamps to construct a target snapshot partition.

[0182] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0183] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0184] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0185] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described instantaneous snapshot creation and reading method, thereby solving the technical problem of how to improve the efficiency of snapshot creation and reading. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the instantaneous snapshot creation and reading method provided in the above embodiments, and will not be repeated here.

[0186] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the instantaneous snapshot creation and reading method described above.

[0187] The computer program product provided in this application solves the technical problem of how to improve the efficiency of snapshot creation and reading. Compared with the prior art, the beneficial effects of the computer program product provided in this application are the same as those of the instantaneous snapshot creation and reading method provided in the above embodiments, and will not be repeated here.

[0188] The above description is only a part of the embodiments of this application and does not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for creating and reading instantaneous snapshots, characterized in that, The method includes: Upon receiving a snapshot creation request, a current snapshot is created based on the snapshot creation request, and the current snapshot is inserted into the snapshot table; Upon receiving a historical snapshot read request, the corresponding historical snapshot timestamp is obtained from the snapshot table based on the historical snapshot read request; Based on the historical snapshot timestamp, a target checkpoint list is obtained by querying. The target checkpoint list is an ordered set containing one or more checkpoints. A checkpoint is a persistent record of the state of all data objects in the database within a certain time period. Based on the target checkpoint list, load the data object that exists at the historical snapshot timestamp to construct the target snapshot partition; After constructing the target snapshot partition by loading the data object containing the historical snapshot timestamp based on the target checkpoint list, the method further includes: Get the total number of data objects, the data size of each data object, and the percentage of rows deleted from each data object; Based on the data size and the proportion of deleted rows, a vacuum score is obtained; When the vacuum score is greater than a preset score threshold, the data object is added to the queue to be merged. The data storage status is determined to meet the merge triggering conditions based on the total number of data objects, the data size of the data objects, and the vacuum degree score. When the data storage status meets the merge triggering condition, data merging is performed based on the queue to be merged; After constructing the target snapshot partition by loading the data object containing the historical snapshot timestamp based on the target checkpoint list, the method further includes: Based on the snapshot table, multiple active snapshot timestamps are obtained; Based on the target checkpoint, information on multiple data objects is obtained; The creation timestamp and deletion timestamp of each data object are compared with the timestamp of each active snapshot to obtain the comparison results; Based on the comparison results, determine the objects that can be deleted; Add the removable object to the tombstone record.

2. The method as described in claim 1, characterized in that, Upon receiving a snapshot creation request, the step of creating a current snapshot based on the snapshot creation request and inserting the current snapshot into the snapshot table includes: Upon receiving a snapshot creation request, a snapshot identifier and a snapshot timestamp are generated; The snapshot level and snapshot name are determined based on the snapshot creation request, wherein the snapshot level includes cluster level, account level, database level and table level; Extract the corresponding object identification information based on the snapshot level; The current snapshot is determined based on the snapshot identifier, the snapshot timestamp, the snapshot name, the snapshot level, and the object identifier information; Insert the current snapshot into the snapshot table.

3. The method as described in claim 1, characterized in that, Upon receiving a historical snapshot read request, retrieving the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot read request includes: Upon receiving a historical snapshot read request, determine the historical snapshot name or query timestamp based on the historical snapshot read request; Search for historical snapshots in the snapshot table based on the historical snapshot name or the query timestamp; Extract the historical snapshot timestamp from the historical snapshot.

4. The method as described in claim 1, characterized in that, The query based on the historical snapshot timestamp yields a list of target checkpoints, including: The historical snapshot timestamp is sent to the transaction node, and the checkpoint list returned by the transaction node based on the historical snapshot timestamp is obtained, wherein the checkpoint list includes multiple checkpoints; If the end time of the last checkpoint in the checkpoint list is greater than or equal to the historical snapshot timestamp, the checkpoint list is used as the target checkpoint list.

5. The method as described in claim 1, characterized in that, The step of loading data objects existing at the historical snapshot timestamps based on the target checkpoint list to construct target snapshot partitions includes: Obtain the target checkpoints contained in each of the target checkpoint lists, wherein each target checkpoint includes multiple data object information, and the data object information includes data object ID, creation timestamp, and deletion timestamp; When the creation timestamp of the data object information is less than or equal to the historical snapshot timestamp and the deletion timestamp of the data object information is greater than the historical snapshot timestamp, the data object corresponding to the data object ID of the data object information is obtained; Construct the target snapshot partition based on the data object.

6. The method as described in claim 5, characterized in that, Before obtaining the target checkpoints contained in each of the target checkpoint lists, the method further includes: Use the data object ID corresponding to the target checkpoint list and the historical snapshot timestamp as query conditions; Based on the query conditions, perform a query in the snapshot usage cache; When the corresponding snapshot partition is found in the snapshot usage cache, the snapshot partition is used as the target snapshot partition; If no corresponding snapshot partition is found in the snapshot usage cache, the step of obtaining the target checkpoints contained in each target checkpoint list is performed.

7. A device for creating and reading instantaneous snapshots, characterized in that, The device includes: A creation module is used to create a current snapshot based on a snapshot creation request when a snapshot creation request is received, and to insert the current snapshot into the snapshot table; The reading module is used to obtain the corresponding historical snapshot timestamp from the snapshot table based on the historical snapshot reading request when a historical snapshot reading request is received. The query module is used to query based on the historical snapshot timestamp to obtain a target checkpoint list, wherein the target checkpoint list is an ordered set containing one or more checkpoints, and a checkpoint is a persistent record of the state of all data objects in the database within a certain time period. The loading module is used to load data objects existing in the historical snapshot timestamps based on the target checkpoint list to construct the target snapshot partition; The loading module is further configured to obtain the total number of data objects, the data size of the data objects, and the percentage of deleted rows of the data objects; obtain a vacuum score based on the data size and the percentage of deleted rows; add the data objects to the queue to be merged when the vacuum score is greater than a preset score threshold; determine whether the data storage status meets the merge triggering conditions based on the total number of data objects, the data size of the data objects, and the vacuum score; and perform data merging based on the queue to be merged when the data storage status meets the merge triggering conditions. The loading module is further configured to obtain multiple active snapshot timestamps based on the snapshot table; obtain multiple data object information based on the target checkpoint; compare the creation timestamp and deletion timestamp of each data object information with each active snapshot timestamp to obtain a comparison result; determine deletable objects based on the comparison result; and add the deletable objects to the tombstone record.

8. A device for creating and reading instantaneous snapshots, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the instantaneous snapshot creation and reading method as described in any one of claims 1 to 6.

Images