Method and apparatus for managing multiple versions of data
By storing data from multiple versions of an LSM-Tree in the same key-value pair and using version information to distinguish between different versions, the problem of slow query response time is solved, and query performance is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HUAWEI TECH CO LTD
- Filing Date
- 2024-12-06
- Publication Date
- 2026-06-09
Smart Images

Figure CN122173544A_ABST
Abstract
Description
Technical Field
[0001] This application relates to computer technology, and more particularly to a method and device for managing multiple versions of data. Background Technology
[0002] In current database industry products, log structured merge tree (LSM-Tree) is one of the important data storage structure options adopted by storage engines. LSM-Tree is a hierarchical, ordered, and disk-oriented data structure that uses key-value pairs to store data.
[0003] When it is necessary to update the data corresponding to a certain key, the key-value pair of the previous version corresponding to the key is often not deleted directly. Instead, a new version of the key-value pair is generated. Therefore, there may be multiple versions of key-value pairs corresponding to multiple time points in an LSM-Tree. Each version of the key-value pair includes one version of the data. The multiple versions of the key-value pairs are independently distributed in different positions of the LSM-Tree.
[0004] Upon receiving a query command, the LSM-Tree is scanned layer by layer based on the first key in the query command. After finding a key-value pair containing the first key, it is determined whether the data of the version included in the key-value pair is the query result of the query command. If not, it is necessary to continue scanning the key-value pairs of the next version containing the first key, and continue to determine whether the data of the version included in the key-value pair of the next version is the query result of the query command, until the result of the query command is obtained.
[0005] However, in order to obtain the query result of the query command, it may be necessary to scan layer by layer to find key-value pairs containing multiple versions of the first key, which results in a slow response time for the query command and affects the performance when executing the query command. Summary of the Invention
[0006] This application provides a method and device for managing multiple versions of data. By placing multiple versions of data of the first key in the same key-value pair, the scanning can be stopped after the first key-value pair containing the first key is found, which greatly reduces the amount of data to be scanned, improves the response speed of query commands, and enhances the performance when executing query commands.
[0007] This application provides the following technical solution:
[0008] Firstly, this application provides a method for managing multiple versions of data. In this method, a storage engine receives a query instruction, which includes a first key. Based on the first key, a scanning operation is performed on multiple key-value pairs until a first key-value pair is obtained, at which point the scanning operation stops. The first key-value pair is the first key-value pair containing the first key found through the scanning operation. The first key-value pair includes a first key and a first value. The first value includes multiple versions of data for the first key; in other words, multiple versions of data for the first key are placed in the first value of the same first key-value pair. The query result of the query instruction is obtained based on the multiple versions of data for the first key, and the query result of the query instruction is sent. The query result may also include the first key.
[0009] In this embodiment, the data of multiple versions of the first key are placed in the same key-value pair. The scanning can be stopped after the first key-value pair containing the first key is found. The query result of the query instruction can be obtained based on the data of multiple versions included in the first key-value pair, which greatly reduces the amount of data scanned, improves the response speed of the query instruction, and improves the performance when executing the query instruction.
[0010] In one possible implementation, the method further includes: the storage engine receiving a write instruction, the write transaction including at least one first key and data of a first version corresponding to each first key; obtaining data of at least one second version corresponding to each first key, the second version being a historical version of the first version; obtaining a first value based on the data of the first version corresponding to each first key and the data of at least one second version, that is, obtaining a first key-value pair corresponding to each first key, each first key-value pair including a first key and a first value of the first key, the data of multiple versions in a first value including the data of the first version corresponding to a first key and at least one second version data; and writing the first key-value pair corresponding to each first key into an LSM-Tree.
[0011] Optionally, if the storage engine adopts an LSM-Tree storage structure, the storage engine's acquisition of at least one second version of data corresponding to a certain first key may include: the first key-value pair containing the first key found by the storage engine in the LSM-Tree (hereinafter referred to as the "third key-value pair" for convenience) is the key-value pair of the most recent historical version. The third key-value pair includes the first key and the third value, and the third value includes at least one second version of data for the first key. For example, the third value also includes version information corresponding to each second version in at least one second version. Optionally, the third value may also include a deletion marker.
[0012] For example, after obtaining the first version of the data and the third value, the storage engine can add the version information of the first version of the data and the first version of the data to the header of the third value to obtain the first value, that is, obtain the first key-value pair; optionally, if the write transaction is for performing a data deletion operation, the write transaction can also include a deletion flag, and the storage engine can add the version information of the first version of the data, the first version of the data and the deletion flag to the header of the third value to obtain the first value.
[0013] In this implementation, when a write command is received, after obtaining the data of the first key and the first version of the first key, at least one second version of the first key is also obtained. The second version is a historical version of the first version, thus obtaining the first key-value pair. The first value includes not only the data of the first version, but also the data of at least one historical version. This ensures that when executing a query task, multiple versions of the same key can be obtained from a key-value pair, thus ensuring the stability and reliability of the execution process of this solution.
[0014] In one possible implementation, the storage engine performs a scan operation on the merged tree (LSM-Tree) of the log structure based on a first key until a first key-value pair is obtained. This can include: the storage engine performs a scan operation on multiple key-value pairs in the LSM-Tree based on the first key until a first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key. For example, the storage engine can find the corresponding LSM-Tree based on the database ID + table ID in the first key, and then perform a scan operation on multiple key-value pairs in the aforementioned LSM-Tree based on the first key until a first key-value pair is obtained.
[0015] This implementation clarifies how to execute the scan operation corresponding to the query command based on the first key when using the LSM-Tree data storage structure, thus improving the integration of this solution with specific implementation scenarios.
[0016] In one possible implementation, the query result includes a target version of data visible to a read transaction generated based on the query command from multiple versions of data for the first key. The target version being visible to the read transaction means that its commit time is earlier than the start time of the read transaction, and its commit time is closest to the start time of the read transaction among at least one committed version included in the first value. Optionally, the multiple versions of data for the first key may include multiple versions of data arranged in ascending order of their creation time. The storage engine can sequentially determine the visibility of each version of data for the first key from front to back until a target version of data visible to the read transaction is obtained.
[0017] Furthermore, the rules by which the storage engine determines whether a certain version of data is visible can include: first, determining whether the data of that version has been committed at this time; if it has not been committed, then visibility determination can continue for the next version of data; if it has been committed, then determining whether the start time of the read transaction is later than the commit time of that version of data; if the start time of the read transaction is later than the commit time of that version of data, then that version of data is visible to the read transaction; if the start time of the read transaction is earlier than the commit time of that version of data, then that version of data is not visible to the read transaction, and visibility determination can continue for the next version of data.
[0018] In this implementation, it is clarified that the query result includes a target version of the data that is visible to the read transaction generated based on the query command among the multiple versions of the first key, and the meaning of the target version of the data being visible to the read transaction. This clarifies the specific implementation idea of obtaining the query result from the multiple versions of the first key and improves the feasibility of this solution.
[0019] In one possible implementation, the first value may include multiple sub-data items arranged sequentially. Optionally, the first value may include multiple sub-data items arranged in ascending order of their generation time, meaning that sub-data items generated later appear earlier in the first value, and sub-data items generated earlier appear later. In other words, when reading the content of the first value, sub-data items generated later will be read earlier. One of the sub-data items may include a version of the data for the first key and version information of the version of the data for the first key. The version information is used to distinguish different versions among the multiple versions of the data for the first key. Optionally, the version information of the version of the data for the first key is related to the generation time of the version of the data for the first key.
[0020] In this implementation, when a key-value pair contains multiple versions of data for the same key, version information can be introduced into the value of the key-value pair to distinguish between the multiple versions of data for that key. This provides a specific data structure for multiple versions of data for the same key-value pair, improving the feasibility of this solution.
[0021] In one possible implementation, the method further includes: during the merge sort compaction operation, the storage engine obtains at least one key-value pair corresponding to the second key; if the number of key-value pairs corresponding to the second key is at least two, deletes key-value pairs other than the second key-value pair among the at least one key-value pairs corresponding to the second key, wherein the second key-value pair is the last key-value pair written among the at least one key-value pairs corresponding to the second key; if the number of key-value pairs corresponding to the second key is one, the storage engine can use the key-value pair corresponding to the second key as the second key-value pair.
[0022] For example, the compaction operation is a merge sorting process that merges data from the current level to the next level. For example, in performing the compaction operation, data can be read into memory, merged, deleted, and sorted, and then rewritten to the next level to achieve orderly organization and deletion of data. The aforementioned sorting process is based on the keys in the key-value pairs.
[0023] Optionally, if at least one key-value pair corresponding to the second key is stored in the LSM-Tree, the key-value pair generated later will be stored in the lower level. Optionally, the second key-value pair can be the first key-value pair containing the second key when scanning the current level and the next level layer by layer based on the second key. That is, the first key-value pair containing the second key when scanning the current level and the next level layer by layer based on the second key is regarded as the latest generated key-value pair.
[0024] In this implementation, during the compaction operation, since the compaction operation itself merges multiple versions of key-value pairs corresponding to the same key, it utilizes the original characteristics of the compaction operation to delete redundant key-value pairs other than the second key-value pair in at least one key-value pair corresponding to the second key. This not only frees up more storage space and greatly improves the effective utilization of storage space, but also does not significantly affect the execution efficiency of the compaction operation. In addition, since centralized data cleanup will occupy a large amount of system resources for a long time, leading to read and write blocking during centralized data cleanup, compared with using a centralized method to clean up redundant key-value pairs, distributing the cleanup of redundant key-value pairs in various compaction operations avoids read and write blocking and improves the user experience of the database.
[0025] In one possible implementation, the second key-value pair includes a second key and a second value, the second value including multiple versions of data for the second key. The method further includes: the storage engine deleting a fourth version of the data in the second value, wherein the commit time of the fourth version is earlier than the commit time of the third version of the data in the second value, the third version being the version visible to the target active transaction at the start of the compaction operation, and the target active transaction being the earliest start time among all active transactions at the start of the compaction operation. For example, all active transactions at the start of the compaction operation refer to transactions that have started but not yet completed at the start of the compaction operation, and the target active transaction is the earliest start time among all active transactions at the start of the compaction operation.
[0026] Optionally, the second value may include multiple versions of data arranged in ascending order of their creation time. The storage engine sequentially evaluates each version of data in the second value from front to back. For example, the rules by which the storage engine determines whether a particular version of data is the third version may include: first, determining whether the data of that version has been committed; if not, it can be determined that the data of that version needs to be retained, and the evaluation continues for the next version; if it has been committed, it can be determined whether the commit time of the data of that version is earlier than the start time of the target active transaction; if the commit time of the data of that version is later than the start time of the target active transaction, it can be determined that the data of that version needs to be retained, and the evaluation continues for the next version; if the commit time of the data of that version is earlier than the start time of the target active transaction, it can be determined that the data of that version is the third version.
[0027] In this implementation, not only are redundant key-value pairs other than the second key-value pair deleted from at least one key-value pair corresponding to the second key, but also the fourth version of the data in the second value is deleted, that is, the expired version of the data in the second value is deleted, further freeing up more storage space and helping to improve the effective utilization of storage space. Moreover, the sorting step in the compaction operation is performed based on the key, and the modification of the value will not affect the execution of the compaction operation. In addition, since centralized data cleanup will occupy a lot of system resources for a long time, read and write blocking will occur during centralized data cleanup. Compared with the centralized method of cleaning up expired data, distributing the cleanup of expired data in various compaction operations avoids read and write blocking and improves the user experience of the database.
[0028] Secondly, this application provides a method for managing multiple versions of data. In this method, a storage engine receives a write instruction, which includes a first key and data of a first version of the first key; obtains data of at least one second version of the first key, wherein the second version is a historical version of the first version; and obtains a first key-value pair based on the data of the first version and the data of at least one second version, and writes the first key-value pair into an LSM-Tree. The first key-value pair includes a first key and a first value, wherein the first value includes data of multiple versions of the first key, and the data of multiple versions of the first key includes data of the first version and data of at least one second version.
[0029] In one possible implementation, the method further includes: the storage engine receiving a query instruction, the query instruction including a first key; performing a scan operation on multiple key-value pairs based on the first key until the first key-value pair is obtained, the first key-value pair being the first key-value pair containing the first key scanned by the scan operation; and sending the query result of the query instruction, the query result being obtained based on multiple versions of data of the first key.
[0030] In the second aspect of this application, the storage engine is also used to perform the steps performed by the storage engine in the first aspect and various possible implementations of the first aspect. The meanings of the terms in the second aspect of this application and various possible implementations of the second aspect, as well as the beneficial effects brought about by each possible implementation, can be referred to the descriptions in the various possible implementations of the first aspect, and will not be repeated here.
[0031] Thirdly, this application provides a management device for multiple versions of data, comprising: a receiving module for receiving a query instruction, the query instruction including a first key; a transaction and multi-version concurrency control (MVCC) module for performing a scanning operation on multiple key-value pairs based on the first key until a first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key, the first key-value pair is the first key-value pair containing the first key scanned by the scanning operation, the first key-value pair includes a first key and a first value, the first value including multiple versions of data corresponding to the first key; and a sending module for sending the query result of the query instruction, the query result being obtained based on the multiple versions of data of the first key.
[0032] In the third aspect of this application, the data management device of multiple versions is also used to perform the steps executed by the storage engine in the first aspect and various possible implementations of the first aspect. The meanings of the terms in the third aspect and various possible implementations of the third aspect, as well as the beneficial effects brought about by each possible implementation, can be referred to the descriptions in the various possible implementations of the first aspect, and will not be repeated here.
[0033] Fourthly, this application provides a management device for multiple versions of data. The management device for multiple versions of data includes: a receiving module for receiving a write instruction, wherein the write transaction includes a first key and data of a first version corresponding to the first key; a transaction and multi-version concurrency control (MVCC) module for obtaining data of at least one second version of the first key, wherein the second version is a historical version of the first version; and the transaction and MVCC module is further configured to obtain a first key-value pair based on the data of the first version and the data of at least one second version, wherein the first key-value pair includes a first key and a first value, the first value includes data of multiple versions of the first key, and the data of multiple versions of the first key includes data of the first version and data of at least one second version.
[0034] In the fourth aspect of this application, the data management device of multiple versions is also used to perform the steps executed by the storage engine in the first aspect and various possible implementations of the first aspect. The meanings of the terms in the fourth aspect and various possible implementations of the fourth aspect, as well as the beneficial effects brought about by each possible implementation, can be referred to the descriptions in the various possible implementations of the first aspect, and will not be repeated here.
[0035] Fifthly, embodiments of this application provide an apparatus including a processor and a memory, the processor being coupled to the memory, the memory being used to store a program; the processor being used to execute the program in the memory, causing the apparatus to perform the methods described in the first or second aspect above.
[0036] Sixthly, embodiments of this application provide a computer-readable storage medium storing a computer program that, when run on a computer, causes the computer to perform the methods described in the first or second aspect above.
[0037] In a seventh aspect, embodiments of this application provide a computer program product, which includes a program that, when run on a computer, causes the computer to perform the methods described in the first or second aspect above.
[0038] Eighthly, this application provides a chip system including a processor and a communication interface. The communication interface is used to communicate with modules outside the chip system, and the processor is used to support the functions involved in the foregoing aspects. In one possible design, the chip system further includes a memory for storing necessary program instructions and data. This chip system may be composed of chips or may include chips and other discrete devices.
[0039] The second to eighth aspects of this application correspond to the first aspect or multiple possible ways of the first aspect, and have corresponding beneficial effects. Attached Figure Description
[0040] Figure 1 A schematic diagram of the data management architecture for multiple versions provided in the embodiments of this application;
[0041] Figure 2 A flowchart illustrating a method for managing multiple versions of data provided in embodiments of this application;
[0042] Figure 3 A schematic diagram illustrating the distribution of multiple versions of key-value pairs for the first key provided in this application embodiment;
[0043] Figure 4Another flowchart illustrating the data management method for multiple versions provided in the embodiments of this application;
[0044] Figure 5 A flowchart illustrating a method for managing multiple versions of data provided in embodiments of this application;
[0045] Figure 6 A flowchart illustrating a scanning operation corresponding to an execution and query instruction provided in an embodiment of this application;
[0046] Figure 7 A schematic diagram illustrating a first key-value pair provided in an embodiment of this application and data for obtaining the target version from the first key-value pair;
[0047] Figure 8 Another flowchart illustrating the data management method for multiple versions provided in the embodiments of this application;
[0048] Figure 9 A schematic diagram illustrating the method for determining the third version of data provided in this application embodiment;
[0049] Figure 10 A schematic diagram illustrating the execution of a compaction operation provided in an embodiment of this application;
[0050] Figure 11 Another flowchart illustrating the data management method for multiple versions provided in the embodiments of this application;
[0051] Figure 12 A schematic diagram of a data management device for multiple versions provided in the embodiments of this application;
[0052] Figure 13 A schematic diagram of a data management device for multiple versions provided in the embodiments of this application;
[0053] Figure 14 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Detailed Implementation
[0054] The embodiments of this application will now be described with reference to the accompanying drawings. Those skilled in the art will recognize that, with technological advancements and the emergence of new scenarios, the technical solutions provided in the embodiments of this application are equally applicable to similar technical problems.
[0055] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such terms are interchangeable where appropriate; this is merely a way of distinguishing objects with the same attributes in the embodiments of this application. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion, so that a process, method, system, product, or apparatus that comprises a series of elements is not necessarily limited to those elements, but may include other elements not explicitly listed or inherent to those processes, methods, products, or apparatuses.
[0056] In the embodiments of this application, "instruction" can include direct and indirect instructions, as well as explicit and implicit instructions. The information indicated by a certain piece of information (hereinafter referred to as instruction information) is called the information to be instructed. In specific implementation, there are many ways to indicate the information to be instructed, such as, but not limited to, directly indicating the information to be instructed, such as the information to be instructed itself or its index. It can also indirectly indicate the information to be instructed by indicating other information, where there is an association between the other information and the information to be instructed; or it can indicate only a part of the information to be instructed, while the other parts are known or pre-agreed upon. For example, the instruction can be implemented by using a pre-agreed (e.g., protocol predefined) arrangement of various information, thereby reducing the instruction overhead to a certain extent. This application does not limit the specific method of instruction. It is understood that for the sender of the instruction information, the instruction information can be used to indicate the information to be instructed; for the receiver of the instruction information, the instruction information can be used to determine the information to be instructed.
[0057] The method provided in this application can be applied to scenarios using key-value pairs. For example, it can be applied to scenarios such as online transaction processing (OLTP) or online analytical processing (OLAP). Optionally, it can be applied to scenarios using a logstructured merge tree (LSM-Tree) data storage structure. For example, an LSM-Tree can store multiple key-value pairs, each of which can be understood as a record. An LSM-Tree may include multiple parts such as a memtable, an immutable memtable, and a sorted string table (SSTable). Data in the memtable is stored in memory, while data in the SSTable is stored in persistent storage (e.g., disk). When performing operations such as data insertion, modification, or deletion, data is first written to the memtable. When the memtable is full, it is converted to an immutable memtable, and the data in the immutable memtable is flushed to the SSTable.
[0058] The SSTable can include at least one level (L). For example, the SSTable can include three levels, L0, L1, and L2, in ascending order, or four levels, L0, L1, L2, and L3, in ascending order, etc., depending on the actual application scenario. For instance, data in the immutable memtable can be written to the lowest level in the SSTable. Once the amount of data in each level reaches a certain threshold, it can be merged with data in a higher level. Key-value pairs generated later will be stored in lower-level levels, and key-value pairs generated earlier will be stored in higher-level levels.
[0059] For example, the above data insertion, data modification, or data deletion operations can all be implemented by executing write transactions. For instance, data insertion can be understood as writing a new key and the value corresponding to the new key by executing a write transaction; data modification can be understood as writing a new version of the data corresponding to an existing key by executing a write transaction; and data deletion can be understood as writing a record that deletes an existing key and the existing version of the data corresponding to the existing key by executing a write transaction.
[0060] In the current approach, when it is necessary to update the data corresponding to an existing key, the key-value pair of the previous version of the key is not usually deleted directly. Instead, a new version of the key-value pair is generated. Therefore, there may be multiple versions of key-value pairs corresponding to multiple time points in the LSM-Tree. These multiple versions of key-value pairs are distributed independently in different positions of the LSM-Tree.
[0061] Upon receiving a query command, the LSM-Tree is scanned layer by layer based on the first key in the query command. After finding a key-value pair containing the first key, it is determined whether the data of the version included in the key-value pair is the query result of the query command. If not, it is necessary to continue scanning the key-value pairs of the next version containing the first key, and continue to determine whether the data of the version included in the key-value pair of the next version is the query result of the query command, until the result of the query command is obtained.
[0062] However, in order to obtain the query result of the query command, it may be necessary to scan layer by layer to find key-value pairs containing multiple versions of the first key, which results in a slow response time for the query command and affects the performance when executing the query command.
[0063] To address the aforementioned issues, this application discloses that when a query instruction is received, a scanning operation is performed on multiple key-value pairs based on the first key in the query instruction until a first key-value pair is obtained. The first key-value pair is the first key-value pair containing the first key obtained through the aforementioned scanning operation. The first key-value pair includes the first key and the first value. The first value includes data from multiple versions of the first key. The query result of the query instruction can then be sent, and the aforementioned query result is obtained based on the data from multiple versions of the first key.
[0064] In the method provided in this application, when it is necessary to maintain multiple versions of data for the same key, the multiple versions of data for that key can be maintained in a version chain within the same value. For example, the first value may include multiple sub-data items arranged sequentially. Optionally, the first value may include multiple sub-data items arranged sequentially according to their generation time from latest to earliest; that is, the sub-data items generated later are positioned earlier in the first value, and the sub-data items generated earlier are positioned later in the first value. In other words, when reading the content of the first value, the sub-data items generated later will be read earlier. One of the multiple sub-data items may include data for one version of the first key and version information of the data for that version of the first key. The version information is used to distinguish different versions among the multiple versions of data for the first key. Optionally, the version information of the data for one version of the first key is related to the generation time of the data for that version of the first key. When a key-value pair contains multiple versions of data for the same key, version information can be introduced into the value of the key-value pair to distinguish between the multiple versions of data for that key. This provides a specific data structure for multiple versions of data for the same key-value pair, improving the feasibility of this solution.
[0065] To further understand this scheme, the following is an example of the structure of Value:
[0066] Value: list[{version_value_deleteMaker},…]
[0067] Here, `{version_value_deleteMaker}` represents a sub-data item within the `Value`, where `value` represents a version of the data. For example, `value` could include an index, metadata, or a row of data from a table. `version` represents version information, such as a timestamp, a global transaction identifier (identity, ID) generated by the storage engine using an auto-incrementing method, or other information. Furthermore, version information can be a timestamp based on the creation time of a version of the data, or a global transaction ID based on the creation time of a version of the data. `deleteMaker` is a deletion marker. `deleteMaker` is optional; it can be included when performing a data deletion operation through a write transaction, but not when performing a deletion operation. When maintaining multiple versions of data for the same key, the same value can include multiple `{version_value_deleteMaker}` items. That is, the same value can maintain multiple sub-data items using a version chain, with each sub-data item containing a version of the data. This achieves the maintenance of multiple versions of data for the same key using a version chain, allowing the version information of the data to distinguish different versions.
[0068] To further understand this scheme, an example of a key-value pair in the method provided in this application is shown below:
[0069] Key: dbid_idxid_pk
[0070] Value: list[{xid_ctid2}, {csn_ctid0_deleteMaker}…]
[0071] Here, `dbid` represents the database ID, `idxid` represents the table ID within that database, and `pk` represents the primary key. After identifying a specific table within a specific database based on `dbid_idxid`, the primary key can be used to uniquely identify a row of data within that table. Taking the version information using a global transaction ID as an example, both `xid` and `csn` represent global transaction IDs. If `xid` is recorded, it indicates that the data for that version may still be in an uncommitted state; if `csn` is recorded, it indicates that the data for that version is definitely in a committed state. For example, the rules for `xid` and `csn` can be different. For instance, both `xid` and `csn` can be represented as strings, but the first two characters of `xid` and `csn` can be different. Or, for example, `xid` may have more characters than `csn`. Optionally, in the method provided in this application, when performing a merge sort operation, version information may be deleted. If the version information does not exist, it proves that the data of that version must be in a committed state and visible to all currently active transactions. The specific implementation of the compaction operation will be described in subsequent steps and will not be discussed here. ctid2 and ctid0 are both indexes, representing two different versions of data with the same key dbid_idxid_pk. It should be understood that the example here is only for the convenience of understanding the key-value pairs in the method provided in this application, and the specific content of the key-value pairs can be determined based on the actual application scenario.
[0072] In this embodiment, the data of multiple versions of the first key are placed in the same key-value pair. The scanning can be stopped after the first key-value pair containing the first key is found. The query result of the query instruction can be obtained based on the data of multiple versions included in the first key-value pair, which greatly reduces the amount of data scanned, improves the response speed of the query instruction, and improves the performance when executing the query instruction.
[0073] Before describing in detail the data management methods for the various versions provided in this application, we will first introduce the data management architecture for the various versions provided in this application. Please refer to [link / reference needed]. Figure 1 , Figure 1 A schematic diagram of the data management architecture for multiple versions provided in the embodiments of this application. Figure 1 Taking the LSM-Tree structure, where the SSTable includes three levels from L0 to L2, as an example, Figure 1The LSM-Tree structure stores various types of data, including indexes, real data, and other types such as metadata or logs. The specific data types stored are determined based on the application scenario. Data in the memtable is stored in memory, while data in the SSTable is stored on persistent storage devices (e.g., disk). The meanings of the various levels within the memtable and SSTable can be found in the description above and will not be elaborated upon here. The storage engine can be understood as the software or process used to manage the data in the LSM-Tree. The storage engine provides functions such as log management, index management, transaction and multi-version concurrency control, data cleanup, and fault recovery. The method provided in this application may involve transactions and multi-version concurrency control (MVCC). For example, MVCC refers to multiple versions of data with the same key maintained in an LSM-Tree. Transactions and MVCC represent the storage engine's ability to handle write and / or query commands related to the situation where multiple versions of data with the same key are maintained in an LSM-Tree. The processing methods for write and query commands in the method provided in this application will be described in detail below. Optionally, the method provided in this application may also involve data cleanup. For example, data cleanup can be performed during the compaction operation in the method provided in this application. Subsequent steps will describe this in detail. It should be understood that... Figure 1 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0074] Based on the above description, the following sections will introduce the detailed implementation processes of the write command processing, query command processing, and compaction operation processing.
[0075] I. Writing Instructions
[0076] Please see Figure 2 , Figure 2 This application provides a flowchart illustrating a method for managing multiple versions of data, which may include:
[0077] 201. Receive write command. The write command includes the first key and the first version of the first key's data.
[0078] For example, the storage engine receives write commands from other devices, such as user devices, and the write commands can be in the form of structured query language (SQL) statements. The storage engine can initiate a write transaction based on the received write command. The write transaction includes a first key and the first version of the data for that first key. Here, a transaction refers to a unit of program execution when reading or writing data from already stored data. Transactions can include two different types: read transactions and write transactions. A read transaction is initiated when data is read, and a write transaction is initiated when data is written.
[0079] Optionally, the storage engine adopts the read-modify-write model of pessimistic concurrency when executing write transactions to ensure the atomicity of write transactions. Pessimistic concurrency is one of the main technical means of concurrency control in databases. That is, when a transaction executes, it locks a row of data. Only when the transaction releases the lock can other transactions execute operations that conflict with the lock.
[0080] For example, a write transaction can correspond to the writing of one or more key-value pairs. In other words, a write transaction can include at least one first key and the first version of data corresponding to each of the at least one first key. After determining the write transaction, the storage engine can, based on the at least one first key included in the write transaction, complete the writing of the key-value pairs corresponding to each of the at least one first key one by one. For the implementation process of writing the key-value pairs corresponding to any one of the aforementioned at least one first key, the storage engine can first execute a read transaction before starting to execute the write transaction. The read transaction searches for the first key from the multiple stored key-value pairs. The write transaction in this application can also be replaced by a write transaction, etc. If the first key is found, a write lock can be acquired on the first key. Thus, when there are concurrent write transactions for the first key, if all the aforementioned concurrent write transactions are active transactions, that is, all the aforementioned concurrent write transactions want to modify the row of data where the first key is located, after acquiring the write lock on the first key, only the first write transaction executing on the first key can continue to execute, and other write transactions executing on the first key will be blocked to ensure the atomicity of the write transaction. If the first key cannot be found, a first key can be generated and a write lock can be acquired on the first key to obtain the first key-value pair. The first key-value pair includes the first key and the first value, and the first value includes the data of the first version.
[0081] For example, the storage engine can find the corresponding LSM-Tree based on the first key, such as the database ID and table ID in the first key. Based on the first key, a scan operation is performed on multiple key-value pairs in the aforementioned LSM-Tree. Further, the scan can start from the memtable in the LSM-Tree and proceed level by level until a key-value pair containing the first key is found, and a write lock is acquired on the first key. For example, the scan can start from the memtable, then scan the immutable memtable, and then scan the SSTable level by level from the lower level to the higher level. If the first key is not found in the LSM-Tree, it proves that there is no historical version of the data for the first key in the LSM-Tree. The first key can be generated and a write lock is acquired on the first key.
[0082] 202. Obtain at least one second version of the data for the first key, where the second version is a historical version of the first version.
[0083] For example, if the storage engine finds a first key, it can retrieve data for at least one second version of the first key, where each second version is a historical version of the first version.
[0084] For example, if the storage engine uses an LSM-Tree storage structure and the first key is found, as can be seen from the above introduction to LSM-Tree, the first key-value pair containing the first key found in the LSM-Tree (hereinafter referred to as the "third key-value pair" for ease of description) is the key-value pair of the most recent historical version. The third key-value pair includes the first key and the third value. For example, the third value includes data of at least one second version of the first key. Optionally, the third key also includes version information corresponding to each second version in at least one second version. Optionally, the third value also includes at least one deletion marker. The specific situation can be determined according to the actual application scenario.
[0085] To more intuitively understand the distribution of key-value pairs corresponding to the same first key in the LSM-Tree within the method provided in this application, please refer to [link to relevant documentation]. Figure 3 , Figure 3 This is a schematic diagram illustrating the distribution of key-value pairs across multiple versions of the first key provided in an embodiment of this application. Figure 3 Taking the storage of four versions of key-value pairs in the L0, L1, and L2 layers of an SSTable as an example, such as... Figure 3As shown, all four versions of the key-value pair use the same key. The difference lies in the value within each version. The key-value pair located in layer L0 is the most recently generated version, and its value includes the data from the four versions corresponding to the key. Figure 3 The key-value pairs in L1 (v4_v3_v2_v1) include data for the three versions corresponding to the key. Figure 3 In the v3_v2_v1), the two key-value pairs located in the L2 layer are: including the data of the two versions corresponding to the key (corresponding to...). Figure 3 (v2_v1) and data including one version corresponding to the key (corresponding to) Figure 3 v1 in the text, such as Figure 3 As shown, key-value pairs generated earlier are stored in higher-level layers. When the LSM-Tree maintains multiple versions of data corresponding to the same key, the values in the key-value pairs of this application will contain multiple versions of data for the same key. It should be understood that... Figure 3 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0086] In combination with the above Figure 3 As explained, when the storage engine searches for a key in the LSM-Tree, it performs a layer-by-layer scan of the LSM-Tree. It will first obtain the key / v4_v3_v2_v1 in the L0 layer, thereby obtaining the four second-version data corresponding to the key. It should be understood that this example is only for the convenience of understanding this solution.
[0087] 203. Based on the data of the first version and at least one data of the second version, obtain the first key-value pair and write the first key-value pair into the LSM-Tree. The first key-value pair includes a first key and a first value. The first value includes data of multiple versions of the first key. The data of multiple versions of the first key includes data of the first version and data of at least one data of the second version.
[0088] For example, after obtaining the first version of the data and the third value, the storage engine can add the version information of the first version of the data and the first version of the data to the header of the third value to obtain the first value, that is, the first key-value pair. Optionally, if the write transaction is for performing a data deletion operation, the write transaction may also include a deletion flag. Then, the storage engine can add the version information of the first version of the data, the first version of the data, and the deletion flag to the header of the third value to obtain the first value, that is, the first key-value pair. The first key-value pair includes a first key and a first value. The first value includes multiple versions of the data for the first key, and the multiple versions of the data for the first key include a version chain consisting of the first version of the data and at least one second version of the data.
[0089] To understand this solution more intuitively, please refer to [link / reference]. Figure 4 , Figure 4 Another flowchart illustrating the data management method for multiple versions provided in the embodiments of this application, such as... Figure 4 As shown, after receiving a write command, the system can retrieve the first key and the first version of the data from the write command; search for the first key, i.e., execute a read transaction based on the first key; if the first key is found, acquire a write lock on the first key, and retrieve the third value from the first third key-value pair containing the first key; append the first version of the data to the header of the third value to obtain the first value. If the first key is not found, acquire a write lock on the first key, and obtain the first key-value pair. The first key-value pair includes the first key and the first value, and the first value includes the first version of the data. It should be understood that... Figure 4 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0090] The storage engine may repeat steps 201 to 203 at least once to obtain at least one first key-value pair corresponding one-to-one with at least one first key included in the write transaction, and write the aforementioned at least one first key-value pair into the LSM-Tree; for example, the storage engine may write the aforementioned at least one first key-value pair into the memtable of the LSM-Tree.
[0091] In this embodiment of the application, when a write instruction is received, after obtaining the data of the first key and the first version of the first key, at least one second version of the first key is also obtained. The second version is a historical version of the first version, thereby obtaining the first key-value pair. The first value includes not only the data of the first version, but also the data of at least one historical version, thereby ensuring that when executing a query task, multiple versions of the same key can be obtained from a key-value pair, ensuring the stability and reliability of the execution process of this solution.
[0092] II. Query Command
[0093] Please see Figure 5 , Figure 5 This application provides a flowchart illustrating a method for managing multiple versions of data, which may include:
[0094] 501. Receive query command, which includes the first key.
[0095] After determining the query instruction, the storage engine can obtain the first key included in the query instruction and initiate a read transaction based on the received query instruction. The read transaction includes the first key. Optionally, the read transaction may also include the first global transaction ID, or it may also obtain the first timestamp corresponding to the start time of the read transaction.
[0096] 502. Perform a scanning operation on multiple key-value pairs based on the first key until the first key-value pair is obtained. The first key-value pair is the first key-value pair containing the first key that is scanned by the scanning operation. The first key-value pair includes the first key and the first value. The first value includes data from multiple versions of the first key.
[0097] The first key-value pair includes a first key and a first value. The first value includes multiple versions of data for the first key. The first value may also include version information for at least one version of the data among the aforementioned multiple versions. The version information is used to distinguish different versions among the multiple versions of data. For example, the version information may be a global transaction ID, a timestamp, or other information.
[0098] Optionally, if the storage engine uses an LSM-Tree data storage structure, the storage engine can perform a scan operation on multiple key-value pairs in the LSM-Tree based on the first key, stopping the scan operation when the first key-value pair containing the first key is obtained. For example, the storage engine can first determine the corresponding LSM-Tree based on the first key, such as determining the corresponding LSM-Tree based on the database ID and table ID in the first key; then, it can perform a scan operation on multiple key-value pairs in the aforementioned LSM-Tree based on the first key. For example, the storage engine can scan level by level from the memtable in the LSM-Tree based on the first key, stopping the scan operation when the first key-value pair containing the first key is obtained. It should be noted that when using other data storage structures besides LSM-Tree, a scan operation can also be performed on multiple key-value pairs in other data storage structures, which will not be described in detail in this application.
[0099] To understand this solution more intuitively, please refer to [link / reference]. Figure 6 , Figure 6 This is a flowchart illustrating a scanning operation corresponding to an execution and query instruction provided in an embodiment of this application. Figure 5 Taking the scanning of key-value pairs in an LSM-Tree as an example, Figure 5 The above can be combined with Figure 4 The description is as follows, and repeated parts will not be elaborated here. The scan starts from the memtable in the LSM-Tree and proceeds level by level until it reaches key / v4_v3_v2_v1 at level L0. Once the first key-value pair containing the key is found, the scan operation stops. This should be understood. Figure 6 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0100] In this embodiment, it is clarified how to perform a scanning operation corresponding to the query command based on the first key when using the LSM-Tree data storage structure, thereby improving the degree of integration between this solution and specific implementation scenarios.
[0101] 503. The query results of the query command are obtained based on multiple versions of data from the first key.
[0102] For example, after obtaining the first key-value pair, the storage engine can retrieve the query result of the query instruction from multiple versions of data for the first key, and then send the query result of the query instruction; for example, the query result may include data from a target version of the data for the first key that is visible to the read transaction generated based on the query instruction.
[0103] For example, the data of multiple versions of the first key may include at least one committed version of the first key. The data of the target version is visible to the read transaction if: the commit time of the target version is earlier than the start time of the read transaction, and the commit time of the target version is closest to the start time of the read transaction among the at least one committed version included in the first value. In other words, the target version is a version whose commit time is earlier than the start time of the read transaction and is the version most recently committed before the start time of the read transaction.
[0104] For example, the data of multiple versions of the first key may include multiple versions of data arranged in order of generation time from latest to earliest. The storage engine may, based on the first global transaction ID corresponding to the read transaction or the first timestamp corresponding to the start time of the read transaction and the version information of each version among the above multiple versions, perform visibility judgment on the data of each version of the data of the first key in a forward order until a target version of the data visible to the read transaction is obtained, thereby obtaining the query result corresponding to the read transaction, wherein the query result includes the data of the first key and the target version.
[0105] Furthermore, the rules by which the storage engine determines whether a certain version of data is visible can include: first, determining whether the data of that version has been committed at this time; if it has not been committed, then visibility determination can continue for the next version of data; if it has been committed, then determining whether the start time of the read transaction is later than the commit time of that version of data; if the start time of the read transaction is later than the commit time of that version of data, then that version of data is visible to the read transaction; if the start time of the read transaction is earlier than the commit time of that version of data, then that version of data is not visible to the read transaction, and visibility determination can continue for the next version of data.
[0106] Optionally, if the version information for each version is a global transaction ID, the rules for determining whether data for a certain version is visible may include:
[0107] In one scenario, if the global transaction ID corresponding to this version of the data is xid (i.e., an uncommitted transaction), the status information of the transaction can be queried based on xid. If the transaction status is still uncommitted, visibility determination can continue for the next version of the data. If the transaction status is committed, the corresponding CSN (i.e., committed sequence number) can be obtained. Then, based on the first global transaction ID and the transaction's CSN, it can be determined whether the start time of the read transaction is later than the commit time of the transaction. If the start time of the read transaction is later than the commit time of the transaction, this version of the data is visible to the read transaction. If the start time of the read transaction is earlier than the commit time of the transaction, this version of the data is not visible to the read transaction, and visibility determination can continue for the next version of the data.
[0108] In another scenario, if the global transaction ID corresponding to this version of the data is a csn, then based on the first global transaction ID and the csn of this transaction, it can be determined whether the start time of the read transaction is later than the commit time of the transaction. If the start time of the read transaction is later than the commit time of the transaction, then this version of the data is visible to the read transaction. If the start time of the read transaction is earlier than the commit time of the transaction, then this version of the data is not visible to the read transaction. Then, the visibility determination can continue to be performed on the next version of the data.
[0109] For a further understanding of this solution, please refer to [link / reference]. Figure 7 , Figure 7 This is a schematic diagram illustrating a first key-value pair provided in an embodiment of this application, and the data from which the target version is obtained. Figure 7 Taking the global transaction ID of the read transaction as the first global transaction ID and the first key as dbid_idxid_a as an example, the storage engine performs a scan operation level by level from the memtable in the LSM-Tree to find the key-value pair with the key dbid_idxid_a, thus obtaining the first key-value pair containing dbid_idxid_a. Figure 7 As shown, the first value in the first key-value pair is 1020_v3, 100_v2, and 98_v1. The storage engine can sequentially perform visibility checks on the three sub-data items (1020_v3, 100_v2, and 98_v1) included in the first value, from front to back. Once it is determined that the data of a certain version (i.e., the target version) is visible, the data of the target version can be directly obtained without further visibility checks on subsequent data. This should be understood. Figure 7 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0110] In this embodiment, the data of multiple versions of the first key are placed in the same key-value pair. The scanning can be stopped after the first key-value pair containing the first key is found. The query result of the query instruction can be obtained based on the data of multiple versions included in the first key-value pair, which greatly reduces the amount of data scanned, improves the response speed of the query instruction, and improves the performance when executing the query instruction.
[0111] Furthermore, it was clarified that the query results include data from multiple versions of the first key that are visible to read transactions generated based on the query command, as well as the meaning of the visibility of the target version of the data to read transactions. This clarified the specific implementation approach for obtaining query results from multiple versions of the first key and improved the feasibility of this solution.
[0112] III. Merge Sort Operation
[0113] Please see Figure 8 , Figure 8 This is another flowchart illustrating the method for managing multiple versions of data provided in this application embodiment. The method for managing multiple versions of data provided in this application embodiment may include:
[0114] 801. During the compaction operation, retrieve at least one key-value pair corresponding to the second key.
[0115] For example, the compaction operation is a merge sorting process that merges data from the current level to the next level. For example, in performing the compaction operation, data can be read into memory, merged, deleted, and sorted, and then rewritten to the next level to achieve orderly organization and deletion of data. The aforementioned sorting process is based on the keys in the key-value pairs.
[0116] For example, an LSM-Tree can include multiple levels. When the data in one of the levels exceeds a certain amount, a compaction operation can be triggered. Taking the Ln level as an example, during the compaction operation, some data in the Ln level is merged into the Ln+1 level, thereby freeing up some storage space in the Ln level. Optionally, when data in the memtable or immutable memtable of the LSM-Tree is written to the L0 level of the SSTable, a compaction operation can also be triggered. During the compaction operation, the storage engine can first determine which key-value pairs in the current level need to be merged into another level. Then, it can determine at least one second key corresponding to the key-value pairs to be merged into the next level, where the second key is the key in the key-value pair to be merged into the next level.
[0117] Optionally, the storage engine can consider key-value pairs in the current level that have a relationship with key-value pairs in the next level as key-value pairs that need to be merged into another level. For example, if the value range of the keys corresponding to multiple key-value pairs in the current level (hereinafter referred to as the "first key-value pair set") overlaps with the value range of the keys corresponding to some key-value pairs in the next level (hereinafter referred to as the "second key-value pair set"), then the first key-value pair set and the second key-value pair set are considered to have a relationship. Similarly, if the key of key-value pair 1 in the current level is the same as the key of key-value pair 2 in the next level, then key-value pair 1 and key-value pair 2 are considered to have a relationship. The specific meaning of "relationship" can be determined based on the actual application scenario.
[0118] Alternatively, the storage engine may determine all key-value pairs in the current level as key-value pairs that need to be merged into another level, or the storage engine may randomly select some key-value pairs from all key-value pairs in the current level as key-value pairs that need to be merged into another level, and so on. This is not an exhaustive list.
[0119] During the compaction operation, after determining the second key, the storage engine can scan the current level and the next level (e.g., Ln level and Ln+1 level, immutable memtable level and L0 level, memtable level and L0 level, etc.) in a forward-to-back order to obtain all key-value pairs corresponding to the second key from the multiple key-value pairs included in the current level and the next level. The key of each key-value pair corresponding to the second key is the second key.
[0120] 802. If the number of key-value pairs corresponding to the second key is at least two, delete the key-value pairs other than the second key-value pair in the at least one key-value pairs corresponding to the second key, wherein the second key-value pair is the last key-value pair written in the at least one key-value pairs corresponding to the second key.
[0121] Step 802 is optional. If there are at least two key-value pairs corresponding to the second key, delete the key-value pairs other than the second key-value pair among the at least one key-value pairs corresponding to the second key; if there is only one key-value pair corresponding to the second key, the storage engine does not need to execute step 802, that is, the key-value pair corresponding to the second key is regarded as the second key-value pair.
[0122] Optionally, referring to the above description of LSM-Tree, the later the key-value pair is generated, the lower the level it will be stored in. Optionally, the second key-value pair can be the first key-value pair containing the second key when scanning the current level and the next level layer by layer based on the second key. That is, the first key-value pair containing the second key when scanning the current level and the next level layer by layer based on the second key is considered the latest generated key-value pair. Alternatively, the storage engine can also determine the latest generated second key-value pair among the at least one key-value pairs corresponding to the second key based on the version information included in each key-value pair.
[0123] 803. Delete the fourth version of the data in the second value, where the second key-value pair includes the second key and the second value, the second value includes multiple versions of the data of the second key, the commit time of the fourth version of the data is earlier than the commit time of the third version of the data in the second value, the third version is the version that is visible to the target active transaction when the compaction operation starts, and the target active transaction is the active transaction with the earliest start time among all active transactions when the compaction operation starts.
[0124] Step 803 is optional. During the compaction operation, the storage engine can also scan the data of multiple versions of the second value in the second key-value pair in a sequential order to determine whether there is an expired version (i.e., the fourth version) in the data of the multiple versions of the second value. All fourth version data in the second value, along with the version information of each fourth version data (optionally, a deletion marker), are deleted to obtain the updated second value, which is the updated second key-value pair. The updated second key-value pair includes the second key and the updated second value.
[0125] The data that needs to be saved includes: data from uncommitted versions, data from versions whose commit time is later than the start time of the target active transaction, and data from the third version. The third version is the version that is visible to the target active transaction when the compaction operation begins. In other words, the version closest to the start time of the target active transaction among at least one version whose commit time is earlier than the start time of the target active transaction is the third version.
[0126] For example, "all active transactions at the start of the compaction operation" refers to transactions that have started but not yet completed at the start of the compaction operation, and the target active transaction is the active transaction with the earliest start time among all active transactions at the start of the compaction operation. For a more intuitive understanding of this scheme, please refer to [link to relevant documentation]. Figure 9 , Figure 9 A schematic diagram illustrating the method for determining the third version of data provided in this application embodiment, such as... Figure 9 As shown, the four versions of the second key are v4, v3, v2, and v1. v1 is visible to transactions whose start time is later than v1's commit time but earlier than v2's commit time; v2 is visible to transactions whose start time is later than v2's commit time but earlier than v3's commit time; v3 is visible to transactions whose start time is later than v3's commit time but earlier than v4's commit time; and v4 is visible to transactions whose start time is later than v4's commit time. Since the earliest active transaction among all active transactions at the start of the compaction operation has a start time between v3's and v4's commit times, any subsequent transactions will start later than v3's commit time. Therefore, versions prior to v3 are not visible to any active transactions at the start of the compaction operation, nor to any transactions that start afterward. Thus, versions v2 and v1 (previously v3) can be deleted to free up storage space. This should be understood. Figure 9 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0127] Optionally, the second value may include multiple versions of data arranged in ascending order of their creation time. The storage engine sequentially evaluates each version of data in the second value from front to back. For example, the rules by which the storage engine determines whether a particular version of data is the third version may include: first, determining whether the data of that version has been committed; if not, it can be determined that the data of that version needs to be retained, and the evaluation continues for the next version; if it has been committed, it can be determined whether the commit time of the data of that version is earlier than the start time of the target active transaction; if the commit time of the data of that version is later than the start time of the target active transaction, it can be determined that the data of that version needs to be retained, and the evaluation continues for the next version; if the commit time of the data of that version is earlier than the start time of the target active transaction, it can be determined that the data of that version is the third version.
[0128] Optionally, the storage engine can also determine whether there is a deletion mark corresponding to the data of the third version. If there is no deletion mark for the data of the third version, the data of the fourth version and the version information of the data of the fourth version (optionally, also including the corresponding deletion mark) can be deleted. Optionally, the version information of the data of the third version can also be deleted. If there is a deletion mark for the data of the third version, the data of the third version, the deletion mark corresponding to the data of the third version, the data of the fourth version and the version information of the data of the fourth version (optionally, also including the corresponding deletion mark) can be deleted.
[0129] Optionally, if the version information of each version is a global transaction ID, and the global transaction ID of the target active transaction is a second global transaction ID, the rule for determining whether the data of a certain version is the data of the third version may include:
[0130] In one scenario, if the global transaction ID corresponding to this version of the data is xid (i.e., an uncommitted transaction), the status information of the transaction can be queried based on xid. If the transaction status is still uncommitted, the next version of the data can be evaluated. If the transaction status is committed, the corresponding CSN (i.e., committed sequence number) can be obtained. Then, based on the second global transaction ID and the transaction's CSN, it can be determined whether the commit time of this version of the data is earlier than the start time of the target active transaction. If the commit time of this version of the data is later than the start time of the target active transaction, it can be determined that this version of the data needs to be retained, and the next version of the data can be evaluated. If the commit time of this version of the data is earlier than the start time of the target active transaction, it can be determined that this version of the data is the third version of the data. Data with a commit time earlier than this version can be determined as the fourth version of the data.
[0131] In another scenario, if the global transaction ID corresponding to this version of the data is a csn, then based on the second global transaction ID and the csn of that transaction, it can be determined whether the commit time of this version of the data is earlier than the start time of the target active transaction. If the commit time of this version of the data is later than the start time of the target active transaction, then it can be determined that this version of the data needs to be retained, and the next version of the data can be judged. If the commit time of this version of the data is earlier than the start time of the target active transaction, then it can be determined that this version of the data is the third version of the data, and data with a commit time earlier than this version can be determined as the fourth version of the data.
[0132] Optionally, during the compaction operation, while the storage engine is scanning the data of multiple versions of the second value in the second key-value pair in the order from front to back, if a certain version of the data has a deletion mark, then the data of that version, the version information of that version of the data, and the deletion mark corresponding to that version of the data will also be deleted. In other words, not only will the data of the fourth version be deleted, but also the data of the version with the deletion mark will be deleted.
[0133] Optionally, the storage engine can also delete the version information of the third version of the data. If the version information does not exist, it proves that the data of that version must be in a committed state and is visible to all currently active transactions.
[0134] To understand this solution more intuitively, please refer to [link / reference]. Figure 10 , Figure 10 A schematic diagram illustrating the execution of a compaction operation provided in an embodiment of this application, as shown below. Figure 10 As shown, the key-value pair is initially located in the Memtable. The value of this key-value pair records three versions of data corresponding to dbid_idxid_a. The version information corresponding to the three versions of data is 1020, 1010, and 1001, respectively. 1020, 1010, and 1001 are all xids. When writing the data in the Memtable to Level 0, the status information of each version of data in this key-value pair is checked. It is found that the status of the transactions corresponding to 1010 and 1001 is committed, and the corresponding CSNs are 50000 and 40000, respectively. Therefore, when writing this key-value pair to Level 0, 1010 is modified to 50000, and 1001 is modified to 40000.
[0135] When writing data from Level 0 to Level 1 by performing a compaction operation, v3 needs to be retained because it is an uncommitted version of the data. The target active transaction 1, which started earliest among all active transactions when the compaction operation was performed, is determined. Based on the global transaction ID of the target active transaction 1, 50000, and 40000, it is determined that the start time of the target active transaction 1 is later than the commit time of v2. Therefore, 40000_v1 is deleted.
[0136] When writing data from Level 1 to Level 2 via compaction, it was found that the transaction corresponding to 1020 had been committed. 1020 was changed to the corresponding CSN 65000. The target active transaction 2, which started earliest among all active transactions when the compaction operation was performed, was identified. Based on the global transaction ID, 65000, and 50000 of the target active transaction 2, it was determined that the start time of the target active transaction 2 was later than the committed time of v2 but earlier than the committed time of v3. Therefore, no version of the data was deleted.
[0137] When writing data from Level 2 to Level 3 via compaction, the target active transaction 3, which started earliest among all active transactions at the time of the compaction operation, is identified. Based on the global transaction ID, 65000, and 50000 of the target active transaction 3, it is determined that the start time of the target active transaction 3 is later than the commit time of v3. Therefore, 50000_v2 is deleted, and the CSN corresponding to v3 is also deleted. This should be understood. Figure 10 The examples in this document are for illustrative purposes only and are not intended to limit the scope of this solution.
[0138] During the compaction operation, the storage engine may repeat steps 801 to 803 at least once to clean up redundant key-value pairs and / or expired data corresponding to each of the at least one second key, sort the data based on each of the at least one second key to obtain sorted data, and write the sorted data to the next level.
[0139] In this embodiment of the application, when performing the compaction operation, since the compaction operation itself merges multiple versions of key-value pairs corresponding to the same key, it utilizes the original characteristics of the compaction operation to delete redundant key-value pairs other than the second key-value pair in at least one key-value pair corresponding to the second key. This not only frees up more storage space and greatly improves the effective utilization rate of storage space, but also the aforementioned operation does not significantly affect the execution efficiency of the compaction operation.
[0140] Not only were redundant key-value pairs other than the second key-value pair deleted from at least one key-value pair corresponding to the second key, but also the fourth version of the data in the second value was deleted, that is, the expired version of the data in the second value was deleted, which further freed up more storage space and helped to further improve the effective utilization of storage space. Moreover, the sorting step in the compaction operation is performed based on the key, and the modification of the value will not affect the execution of the compaction operation.
[0141] Furthermore, centralized data cleanup consumes a large amount of system resources for an extended period, which can lead to read / write blocking during the process. Compared to using a centralized approach to clean up redundant key-value pairs and / or expired data, distributing the cleanup of redundant key-value pairs and / or expired data across various compaction operations avoids read / write blocking and improves the user experience of the database.
[0142] Based on the above description, please refer to Figure 11 , Figure 11 This is another flowchart illustrating the method for managing multiple versions of data provided in this application embodiment. The method for managing multiple versions of data provided in this application embodiment may include:
[0143] 1101. Receive write command. The write command includes the first key and the first version of the first key's data.
[0144] 1102. Obtain at least one second version of the data for the first key, where the second version is a historical version of the first version.
[0145] 1103. Based on the data of the first version and at least one data of the second version, obtain the first key-value pair and write the first key-value pair into the LSM-Tree. The first key-value pair includes a first key and a first value. The first value includes data of multiple versions of the first key. The data of multiple versions of the first key includes data of the first version and data of at least one data of the second version.
[0146] 1104. Receive query command, which includes the first key.
[0147] 1105. Perform a scan operation on multiple key-value pairs based on the first key until the first key-value pair is obtained. The first key-value pair is the first key-value pair containing the first key that is scanned through the scan operation.
[0148] 1106. The query results of the query command are obtained based on multiple versions of the data of the first key.
[0149] 1107. During the compaction operation, retrieve at least one key-value pair corresponding to the second key.
[0150] 1108. If the number of key-value pairs corresponding to the second key is at least two, delete the key-value pairs other than the second key-value pair in the at least one key-value pairs corresponding to the second key, wherein the second key-value pair is the last key-value pair written in the at least one key-value pairs corresponding to the second key.
[0151] 1109. Delete the fourth version of the data in the second value, where the second key-value pair includes the second key and the second value, the second value includes multiple versions of the data of the second key, the commit time of the fourth version of the data is earlier than the commit time of the third version of the data in the second value, the third version is the version that is visible to the target active transaction when the compaction operation starts, and the target active transaction is the active transaction with the earliest start time among all active transactions when the compaction operation starts.
[0152] In this embodiment, steps 1107 to 1109 are optional. If steps 1107 to 1109 are not executed, the storage engine can also use other methods to clean up expired data. The specific implementation of steps 1101 to 1109 by the storage engine and the specific meanings of the terms in the steps can be found in the above. Figures 1 to 9 The descriptions in the corresponding method embodiments will not be repeated here.
[0153] exist Figures 1 to 11 Based on the corresponding embodiments, in order to better implement the above-described solutions of this application, related equipment for implementing the above solutions is also provided below. See details. Figure 12 , Figure 12 This is a schematic diagram of a data management device for multiple versions provided in an embodiment of this application. The data management device 1200 for multiple versions includes: a receiving module 1201, used to receive a query instruction, the query instruction including a first key; a transaction and multi-version concurrency control (MVCC) module 1202, used to perform a scanning operation on multiple key-value pairs based on the first key until a first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key, the first key-value pair is the first key-value pair containing the first key scanned by the scanning operation, the first key-value pair includes a first key and a first value, the first value includes data for multiple versions corresponding to the first key; and a sending module 1203, used to send the query result of the query instruction, the query result being obtained based on the data for multiple versions of the first key.
[0154] Optionally, the receiving module 1201 is further configured to receive a write instruction, the write instruction including a first key and data of a first version of the first key; the transaction and MVCC module 1202 is further configured to obtain data of at least one second version of the first key, the second version being a historical version of the first version; the transaction and MVCC module 1202 is further configured to obtain a first key-value pair based on the data of the first version and at least one second version, the first value including data of the first version and data of at least one second version; the transaction and MVCC module 1202 is further configured to write the first key-value pair into the merged tree (LSM-Tree) of the log structure.
[0155] Optionally, the transaction and MVCC module 1202 is specifically used to perform a scan operation on multiple key-value pairs in the LSM-Tree based on the first key until the first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key.
[0156] Optionally, the query results include data from multiple versions of the first key that are visible to a read transaction generated based on the query command. The visibility of the target version of the data to the read transaction means that the commit time of the target version of the data is earlier than the start time of the read transaction, and the commit time of the target version of the data is the closest to the start time of the read transaction among at least one committed version included in the first value.
[0157] Optionally, the first value includes multiple sub-data, one of which includes data for a version of the first key and version information of the data for the version of the first key. The version information is used to distinguish different versions among the multiple versions of the data for the first key.
[0158] Optionally, the data management device 1200 for multiple versions further includes a data cleaning module 1204, configured to: obtain at least one key-value pair corresponding to the second key during the execution of the merge sort compaction operation; and delete key-value pairs other than the second key-value pair among the at least one key-value pairs corresponding to the second key when the number of key-value pairs corresponding to the second key is at least two, wherein the second key-value pair is the last key-value pair written among the at least one key-value pairs corresponding to the second key.
[0159] Optionally, the second key-value pair includes a second key and a second value, and the second value includes multiple versions of data for the second key; the data cleanup module 1204 is also used to delete the fourth version of data in the second value, wherein the commit time of the fourth version of data is earlier than the commit time of the third version of data in the second value, the third version is the version visible to the target active transaction when the compaction operation starts, and the target active transaction is the active transaction with the earliest start time among all active transactions when the compaction operation starts.
[0160] It should be noted that the information interaction and execution process between the modules / units in the data management device 1200 of multiple versions are different from those in this application. Figures 1 to 11 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.
[0161] See Figure 13 , Figure 13 This is a schematic diagram of a data management device for multiple versions provided in an embodiment of this application. The data management device 1300 for multiple versions includes: a receiving module 1301, used to receive a write instruction, wherein the write transaction includes a first key and data of a first version corresponding to the first key; a transaction and multi-version concurrency control (MVCC) module 1302, used to obtain data of at least one second version of the first key, wherein the second version is a historical version of the first version; the transaction and MVCC module 1302 is also used to obtain a first key-value pair based on the data of the first version and the data of at least one second version, wherein the first key-value pair includes a first key and a first value, the first value includes data of multiple versions of the first key, and the data of multiple versions of the first key includes data of the first version and data of at least one second version.
[0162] Optionally, the receiving module 1301 is further configured to receive a query instruction, the query instruction including a first key; the transaction and MVCC module 1302 is further configured to perform a scan operation on the LSM-Tree based on the first key until a first key-value pair is obtained, wherein the first key-value pair is the first key-value pair containing the first key scanned by the scan operation; the data management device 1300 for multiple versions further includes: a sending module 1303, configured to send the query result of the query instruction, the query result being obtained based on multiple versions of data corresponding to the first key.
[0163] It should be noted that the information interaction and execution process between the modules / units in the data management device 1300 of multiple versions are different from those in this application. Figures 1 to 11 The various method embodiments are based on the same concept, and the details can be found in the descriptions of the method embodiments shown above in this application, which will not be repeated here.
[0164] This application also provides a device, see [link to relevant documentation] Figure 14 As shown, Figure 14 This is a schematic diagram of the structure of a device provided in an embodiment of this application. Optionally, device 1400 performs... Figures 1 to 11 The methods executed by the storage engine in the corresponding method embodiments.
[0165] Device 1400 includes at least one processor 1401, and optionally, device 1400 also includes a memory 1402. Processor 1401 implements the methods in the above embodiments by reading instructions stored in memory 1402, or processor 1401 may also implement the methods in the above embodiments by internally stored instructions. When processor 1401 implements the methods in the above embodiments by reading instructions stored in memory 1402, memory 1402 stores instructions for implementing the methods provided in the above embodiments of this application.
[0166] Optionally, at least one processor 1401 is one or more CPUs, including single-core and multi-core CPUs. The memory 1402 includes, but is not limited to, random access memory (RAM), read-only memory (ROM), flash memory, or optical memory. The memory 1402 stores the instructions of the memory engine. After the program instructions stored in the memory 1402 are read by the at least one processor 1401, the device 1400 executes the corresponding operations in the foregoing embodiments.
[0167] After the processor 1401 reads the program instructions from the memory 1402, other functions that the device 1400 can perform are described in the preceding method embodiments.
[0168] Optionally, device 1400 also includes a network interface 1403, which can be a wired interface or a wireless interface. It should be understood that network interface 1403 has the functions of receiving and sending data. The functions of "receiving data" and "sending data" can be integrated into the same transceiver interface, or the functions of "receiving data" and "sending data" can be implemented in different interfaces; this is not limited here. In other words, network interface 1403 can include one or more interfaces for implementing the functions of "receiving data" and "sending data".
[0169] Optionally, the device 1400 also includes a bus 1404, through which the processor 1401 and memory 1402 are typically interconnected, or in other ways.
[0170] The device 1400 provided in this application embodiment is used to execute the methods executed by the storage engine in the above-described method embodiments and to achieve the corresponding beneficial effects. Figure 14 The specific implementation of the device 1400 shown can be referred to the descriptions in the aforementioned method embodiments, and will not be repeated here.
[0171] This application also provides a computer-readable storage medium storing a program that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 1 to 11 The steps performed by the storage engine in the method described in the illustrated embodiment.
[0172] This application also provides a computer program product, which includes a program that, when run on a computer, causes the computer to perform the aforementioned actions. Figures 1 to 11 The steps performed by the storage engine in the method described in the illustrated embodiment.
[0173] This application embodiment also provides a circuit system, the circuit system including a processing circuit, the processing circuit being configured to perform the aforementioned... Figures 1 to 11 The steps performed by the storage engine in the method described in the illustrated embodiment.
[0174] This application also provides a chip system, which includes a processor and a communication interface. The communication interface is used to communicate with modules outside the chip system, and the processor is used to support the implementation as described above. Figures 1 to 11 The steps performed by the storage engine in the method described in the illustrated embodiment. In one possible design, the chip system further includes a memory for storing necessary program instructions and data. The chip system may consist of chips or may include chips and other discrete devices.
[0175] It should also be noted that the device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. In addition, in the device embodiment drawings provided in this application, the connection relationship between modules indicates that they have a communication connection, which can be implemented as one or more communication buses or signal lines.
[0176] Through the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware, or it can be implemented by special-purpose hardware including application-specific integrated circuits, special-purpose CLUs, special-purpose memory, special-purpose components, etc. Generally, any function performed by a computer program can be easily implemented by corresponding hardware, and the specific hardware structure used to implement the same function can also be diverse, such as analog circuits, digital circuits, or special-purpose circuits. However, for this application, software program implementation is more often the preferred implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a readable storage medium, such as a computer floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk, or optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.
[0177] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product.
[0178] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions may be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium may be any available medium that a computer can store or a data storage device such as a server or data center that integrates one or more available media. The available medium may be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).
Claims
1. A method for managing multiple versions of data, characterized in that, The method includes: Receive a query instruction, the query instruction including a first key; Based on the first key, a scanning operation is performed on multiple key-value pairs until the first key-value pair is obtained. The first key-value pair is the first key-value pair containing the first key that is scanned by the scanning operation. The first key-value pair includes the first key and a first value, and the first value includes data of multiple versions of the first key. The query result of the query command is obtained based on multiple versions of data of the first key.
2. The method according to claim 1, characterized in that, The method further includes: Receive a write instruction, the write instruction including the first key and data of a first version of the first key; Obtain at least one second version of the data for the first key, wherein the second version is a historical version of the first key; Based on the data of the first version and the data of at least one second version, the first key-value pair is obtained, wherein the data of multiple versions of the first key includes the data of the first version and the data of at least one second version; Write the first key-value pair into the merged tree LSM-Tree of the log structure.
3. The method according to claim 2, characterized in that, The step of performing a scan operation on the merged tree (LSM-Tree) of the log structure based on the first key until the first key-value pair is obtained includes: A scan operation is performed on multiple key-value pairs in the LSM-Tree based on the first key until the first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key.
4. The method according to any one of claims 1 to 3, characterized in that, The query result includes data from a target version of the data from multiple versions of the first key that is visible to a read transaction generated based on the query instruction. The visibility of the target version of the data to the read transaction means that the commit time of the target version of the data is earlier than the start time of the read transaction, and the commit time of the target version of the data is closest to the start time of the read transaction among at least one committed version included in the first value.
5. The method according to any one of claims 1 to 3, characterized in that, The first value includes multiple sub-data, one of which includes data of a version of the first key and version information of the data of a version of the first key. The version information is used to distinguish different versions among the multiple versions of the data of the first key.
6. The method according to any one of claims 1 to 3, characterized in that, The method further includes: During the merge sort compaction operation, at least one key-value pair corresponding to the second key is obtained; If the number of key-value pairs corresponding to the second key is at least two, delete the key-value pairs other than the second key-value pair among the at least key-value pairs corresponding to the second key, wherein the second key-value pair is the last key-value pair written among the at least key-value pairs corresponding to the second key.
7. The method according to claim 6, characterized in that, The second key-value pair includes a second key and a second value, wherein the second value includes data for multiple versions of the second key, and the method further includes: Delete the fourth version of the data in the second value, wherein the submission time of the fourth version of the data is earlier than the submission time of the third version of the data in the second value, the third version being the version visible to the target active transaction when the compaction operation begins, and the target active transaction being the active transaction with the earliest start time among all active transactions when the compaction operation begins.
8. A method for managing multiple versions of data, characterized in that, The method includes: Receive a write instruction, the write instruction including a first key and data of a first version of the first key; Obtain at least one second version of the data for the first key, wherein the second version is a historical version of the first key; Based on the data of the first version and the data of the at least one second version, a first key-value pair is obtained, wherein the first key-value pair includes the first key and the first value, the first value includes data of multiple versions of the first key, and the data of multiple versions of the first key includes the data of the first version and the data of the at least one second version; Write the first key-value pair into the merged tree LSM-Tree of the log structure.
9. The method according to claim 8, characterized in that, The method further includes: Receive a query instruction, the query instruction including the first key; A scan operation is performed on the LSM-Tree based on the first key until the first key-value pair is obtained, wherein the first key-value pair is the first key-value pair containing the first key scanned by the scan operation; The query result of the query command is obtained based on the data of multiple versions corresponding to the first key.
10. A device for managing multiple versions of data, characterized in that, The device includes: A receiving module is used to receive a query instruction, wherein the query instruction includes a first key. The Transaction and Multi-Version Concurrency Control (MVCC) module is used to perform a scanning operation on multiple key-value pairs based on the first key until the first key-value pair is obtained. The LSM-Tree is determined based on the first key. The first key-value pair is the first key-value pair containing the first key that is scanned by the scanning operation. The first key-value pair includes the first key and a first value. The first value includes data of multiple versions corresponding to the first key. The sending module is used to send the query result of the query instruction, the query result being obtained based on multiple versions of data of the first key.
11. The apparatus according to claim 10, characterized in that, The receiving module is further configured to receive a write instruction, the write instruction including the first key and data of a first version of the first key; The transaction and MVCC module is also used to obtain at least one second version of the data of the first key, wherein the second version is a historical version of the first version; The transaction and MVCC module is further configured to obtain the first key-value pair based on the data of the first version and the data of the at least one second version, wherein the first value includes the data of the first version and the data of the at least one second version; The transaction and MVCC module is also used to write the first key-value pair into the merged tree LSM-Tree of the log structure.
12. The apparatus according to claim 11, characterized in that, The transaction and MVCC module is specifically used to perform a scan operation on multiple key-value pairs in the LSM-Tree based on the first key until the first key-value pair is obtained, wherein the LSM-Tree is determined based on the first key.
13. The apparatus according to any one of claims 10 to 12, characterized in that, The query result includes data from a target version of the data from multiple versions of the first key that is visible to a read transaction generated based on the query instruction. The visibility of the target version of the data to the read transaction means that the commit time of the target version of the data is earlier than the start time of the read transaction, and the commit time of the target version of the data is closest to the start time of the read transaction among at least one committed version included in the first value.
14. The apparatus according to any one of claims 10 to 12, characterized in that, The first value includes multiple sub-data, one of which includes data of a version of the first key and version information of the data of a version of the first key. The version information is used to distinguish different versions among the multiple versions of the data of the first key.
15. The apparatus according to any one of claims 10 to 12, characterized in that, The device also includes a data cleaning module for: During the merge sort compaction operation, at least one key-value pair corresponding to the second key is obtained; If the number of key-value pairs corresponding to the second key is at least two, delete the key-value pairs other than the second key-value pair among the at least key-value pairs corresponding to the second key, wherein the second key-value pair is the last key-value pair written among the at least key-value pairs corresponding to the second key.
16. The apparatus according to claim 15, characterized in that, The second key-value pair includes a second key and a second value, wherein the second value includes data for multiple versions of the second key; The data cleaning module is further configured to delete the fourth version of the data in the second value, wherein the submission time of the fourth version of the data is earlier than the submission time of the third version of the data in the second value, the third version being the version visible to the target active transaction when the compaction operation is started, and the target active transaction being the active transaction with the earliest start time among all active transactions when the compaction operation is started.
17. A device for managing multiple versions of data, characterized in that, The receiving module is used to receive write instructions, wherein the write transaction includes a first key and data of a first version corresponding to the first key; The Transaction and Multi-Version Concurrency Control (MVCC) module is used to obtain at least one second version of the data for the first key, wherein the second version is a historical version of the first version; The transaction and MVCC module is further configured to obtain a first key-value pair based on the data of the first version and the data of the at least one second version, wherein the first key-value pair includes a first key and a first value, the first value includes data of multiple versions of the first key, and the data of multiple versions of the first key includes the data of the first version and the data of the at least one second version.
18. The apparatus according to claim 17, characterized in that, The receiving module is also configured to receive a query instruction, the query instruction including the first key; The transaction and MVCC module is further configured to perform a scan operation on the LSM-Tree based on the first key until the first key-value pair is obtained, wherein the first key-value pair is the first key-value pair containing the first key scanned by the scan operation; The device further includes a sending module, used to send the query result of the query instruction, the query result being obtained based on the data of multiple versions corresponding to the first key.
19. A device, characterized in that, The method includes a processor coupled to a memory storing program instructions, which, when executed by the processor, implement the method of any one of claims 1 to 9.
20. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 9.
21. A computer program product, characterized in that, The computer program product includes a program that, when run on a computer, causes the computer to perform the method as described in any one of claims 1 to 9.
22. A chip system, characterized in that, The chip system includes a processor and a communication interface, the communication interface being used to communicate with modules outside the chip system, and the processor being used to perform the method as described in any one of claims 1 to 9.