Transaction processing method of database, electronic device and medium

Abstract

The application provides a transaction processing method of a database, an electronic device and a medium, which can be applied to the technical field of databases. The method comprises the following steps: obtaining transaction information of a target transaction, wherein the transaction information comprises a modification operation indicated by the target transaction and directed to a data table, the modification operation comprises a deletion operation, an insertion operation and an update operation, the update operation is decomposed into a deletion operation directed to a to-be-updated row and an insertion operation for inserting updated data into the data table through a newly-added data row; generating a target visibility bitmap of the data table according to the modification operation, wherein each bit in the target visibility bitmap corresponds to a data row in the data table, and the value of the bit is used to mark whether the corresponding data row is deleted after the target transaction is committed; adding version information comprising the target visibility bitmap to a version chain table of the data table, and modifying the data table in the case where the modification operation involves the insertion operation, and committing the target transaction.

Description

Technical Field

[0001] This application relates to the field of database technology, and more specifically to a database transaction processing method, electronic device, and medium. Background Technology

[0002] In database management systems, multi-version concurrency control (MVCC) mechanisms are typically used to handle high-concurrency transactions, ensuring transaction isolation and consistency. Traditional MVCC schemes often require independent storage and management of the data tables modified by each transaction, leading to high memory consumption and potential performance bottlenecks when processing large amounts of data. Therefore, efficiently managing multiple transaction versions in memory and controlling memory usage is a critical issue in database systems. Summary of the Invention

[0003] In view of the above problems, this application provides a database transaction processing method, electronic device and medium.

[0004] According to a first aspect of this application, a database transaction processing method is provided, comprising: obtaining transaction information of a target transaction, the transaction information including modification operations on a data table indicated by the target transaction, the modification operations including deletion operations, insertion operations, and update operations, the update operations being decomposed into a deletion operation on a row to be updated and an insertion operation by adding new data rows to insert the updated data into the data table; generating a target visibility bitmap of the data table according to the modification operations, each bit in the target visibility bitmap corresponding to a data row in the data table, the value of the bit being used to mark whether the corresponding data row is deleted after the target transaction is committed; modifying the data table by adding version information including the target visibility bitmap to a version list of the data table, and when the modification operations involve insertion operations, and committing the target transaction, so as to obtain each version of the data table according to the version information in the version list.

[0005] According to an embodiment of this application, generating the target visibility bitmap of the data table based on the above modification operation includes: obtaining initial version information corresponding to the start time of the target transaction from the version list, wherein the initial version information includes an initial visibility bitmap; and modifying the value of the bit in the corresponding data row in the initial visibility bitmap according to the above modification operation to obtain the target visibility bitmap.

[0006] According to an embodiment of this application, modifying the value of the bit corresponding to the data row in the initial visibility bitmap includes: when the modification operation involves a deletion operation, modifying the value of the bit corresponding to the data row to be deleted in the initial visibility bitmap to a first preset value, wherein the first preset value is used to mark that the corresponding data row is deleted after the target transaction is committed; when the modification operation involves an insertion operation, adding a bit at the end of the initial visibility bitmap and setting the value of the added bit to a second preset value, wherein the second preset value is used to mark that the corresponding data row is not deleted after the target transaction is committed.

[0007] According to an embodiment of this application, the modification of the data table includes: when the modification operation involves a deletion operation, retaining the data in the data table corresponding to the data row to be deleted; when the modification operation involves an insertion operation, inserting a new data row in the data table, wherein the new data row corresponds to a bit newly added in the target visibility bitmap.

[0008] According to an embodiment of this application, the transaction information further includes the target transaction identifier of the target transaction, and the addition of the version information including the target visibility bitmap to the version list of the data table includes: obtaining the target transaction identifier of the target transaction from the transaction information; obtaining the version information based on the target transaction identifier and the target visibility bitmap; and adding the version information to the end of the version list.

[0009] According to an embodiment of this application, the transaction processing method further includes: determining an active transaction list based on the transaction identifier of the read transaction, wherein the active transaction list includes transaction identifiers of uncommitted transactions, and uncommitted transactions are transactions that have not yet been committed at the start time of the read transaction; searching backward from the end of the version list, and determining the version information of the first transaction identifier found that is not in the active transaction list as the target version information corresponding to the read transaction; and reading the data rows visible to the read transaction from the data table based on the visibility bitmap in the target version information.

[0010] According to an embodiment of this application, the transaction processing method further includes: when the length of the version list exceeds a preset length threshold, determining and deleting version information to be deleted from the version list, wherein the version information to be deleted is not depended on by active transactions.

[0011] According to an embodiment of this application, the transaction processing method further includes: detecting a plurality of consecutive bits with the same value in the target visibility bitmap, and storing the plurality of bits as the value and the number when the number of the plurality of bits is greater than a preset number threshold.

[0012] A second aspect of this application provides a database transaction processing apparatus, comprising: an acquisition module, configured to acquire transaction information of a target transaction, the transaction information including modification operations on a data table indicated by the target transaction, the modification operations including deletion operations, insertion operations, and update operations, the update operations being decomposed into a deletion operation on a row to be updated and an insertion operation by adding new data rows to insert the updated data into the data table; a generation module, configured to generate a target visibility bitmap of the data table according to the modification operations, each bit in the target visibility bitmap corresponding to a data row in the data table, the value of the bit being used to mark whether the corresponding data row is deleted after the target transaction is committed; and a commit module, configured to add version information including the target visibility bitmap to a version list of the data table, and modify the data table when the modification operations involve insertion operations, and commit the target transaction, so as to acquire each version of the data table according to the version information in the version list.

[0013] A third aspect of this application provides an electronic device comprising: one or more processors; and a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the method described above.

[0014] A fourth aspect of this application also provides a computer-readable storage medium having a computer program or instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0015] The fifth aspect of this application also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described method.

[0016] According to the embodiments of this application, by recording the visibility status of each data row based on a visibility bitmap, each data row only needs to occupy one bit to complete the recording of the deletion status in the version information. There is no need to store the complete version data table. Compared with the traditional multi-version storage scheme, it can significantly reduce redundant storage and effectively reduce the memory overhead caused by multi-version management. At the same time, by organizing the version information through the version linked list, the data table content of any historical version can be quickly located and obtained without adding extra time overhead for version reading. While ensuring the transaction isolation level, it significantly reduces memory consumption and improves concurrent access performance. Attached Figure Description

[0017] The above-mentioned contents, other objects, features and advantages of this application will become clearer from the following description of embodiments of this application with reference to the accompanying drawings.

[0018] Figure 1The diagram illustrates an application scenario of a database transaction processing method, electronic device, and medium according to embodiments of this application.

[0019] Figure 2 A flowchart of a database transaction processing method according to an embodiment of this application is shown.

[0020] Figure 3 A schematic diagram of generating a target visibility bitmap according to a specific embodiment of this application is shown.

[0021] Figure 4 A schematic diagram illustrating version selection during a data table access process according to an embodiment of this application is shown.

[0022] Figure 5 A structural block diagram of a database transaction processing apparatus according to an embodiment of this application is shown.

[0023] Figure 6 A block diagram of an electronic device suitable for implementing a database transaction processing method according to an embodiment of this application is shown. Detailed Implementation

[0024] The embodiments of this application will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of this application. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of this application for ease of explanation. However, it will be apparent that one or more embodiments may be implemented without these specific details. Furthermore, descriptions of well-known structures and technologies are omitted in the following description to avoid unnecessarily obscuring the concepts of this application.

[0025] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of this application. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0026] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0027] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0028] In the technical solution of this application, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, and displayed data) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, application, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0029] In traditional multi-version management schemes, each version of each data row is stored separately as a complete copy of the data content. When multiple transactions continuously modify the same data table, a large number of valid data rows are stored repeatedly, which greatly increases memory usage.

[0030] Therefore, embodiments of this application provide a database transaction processing method, including: obtaining transaction information of a target transaction, the transaction information including modification operations on a data table indicated by the target transaction, the modification operations including deletion operations, insertion operations, and update operations, the update operation being decomposed into a deletion operation on the row to be updated and an insertion operation to insert the updated data into the data table by adding new data rows; generating a target visibility bitmap of the data table according to the modification operations, each bit in the target visibility bitmap corresponding to a data row in the data table, the value of the bit being used to mark whether the corresponding data row is deleted after the target transaction is committed; adding version information including the target visibility bitmap to a version list of the data table, and modifying the data table when the modification operation involves an insertion operation, and committing the target transaction, so as to obtain each version of the data table according to the version information in the version list.

[0031] This application embodiment stores a visibility bitmap based on bits in the data table. Each data row only occupies one bit to record its visibility status, which greatly reduces memory usage compared to traditional solutions and can effectively reduce the memory overhead of multi-version management.

[0032] Figure 1 The diagram illustrates an application scenario of a database transaction processing method, electronic device, and medium according to embodiments of this application.

[0033] like Figure 1 As shown, the application scenario according to this embodiment may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing a communication link between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0034] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social media platform software, etc. (for example only).

[0035] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, laptops, and desktop computers.

[0036] Server 105 can be a server that provides various services, such as a backend management server that supports websites browsed by users using the first terminal device 101, the second terminal device 102, and the third terminal device 103 (this is just an example). The backend management server can analyze and process data such as received user requests, and feed back the processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal devices.

[0037] It should be noted that the database transaction processing method provided in this application embodiment can generally be executed by server 105. Correspondingly, the database transaction processing device provided in this application embodiment can generally be located in server 105. The database transaction processing method provided in this application embodiment can also be executed by a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105. Correspondingly, the database transaction processing device provided in this application embodiment can also be located in a server or server cluster that is different from server 105 and capable of communicating with the first terminal device 101, the second terminal device 102, the third terminal device 103, and / or server 105.

[0038] It should be understood that Figure 1The number of first terminal devices, second terminal devices, third terminal devices, networks, and servers shown in the diagram is merely illustrative. Depending on implementation needs, any number of first terminal devices, second terminal devices, third terminal devices, networks, and servers can be included.

[0039] Figure 2 A flowchart of a database transaction processing method according to an embodiment of this application is shown.

[0040] like Figure 2 As shown, the database transaction processing method of this embodiment includes operations S210 to S230.

[0041] In operation S210, the transaction information of the target transaction is obtained.

[0042] In operation S220, a target visibility bitmap of the data table is generated based on the modification operation.

[0043] In operation S230, version information including the target visibility bitmap is added to the version list of the data table, and the data table is modified when the modification operation involves an insert operation. The target transaction is committed to obtain each version of the data table based on the version information in the version list.

[0044] In the embodiments of this application, the transaction information includes modification operations on the data table indicated by the target transaction. The modification operations include deletion operations, insertion operations, and update operations. The update operation is decomposed into a deletion operation on the row to be updated and an insertion operation that inserts the updated data into the data table by adding new data rows.

[0045] Taking the target transaction as updating the data in the 3rd row of the data table to new content as an example, the update operation is broken down into a deletion operation for the 3rd row, and a new row needs to be added at the end of the data table to store the updated content.

[0046] During a write operation, each transaction maintains a private modified view. For example, if the target transaction includes both write and read operations, the write operation can be performed on the private modified view before the read operation.

[0047] Each bit in the target visibility bitmap corresponds to a data row in the data table. The value of the bit is used to indicate whether the corresponding data row has been deleted after the target transaction is committed. When generating the target visibility bitmap, the length of the target visibility bitmap can be determined first based on the total number of existing data rows in the current data table. Initially, the values ​​of all bits are consistent with the previous version of the visibility bitmap. Subsequently, for deletion operations in the current target transaction, the bits of the corresponding data rows are modified to the values ​​that mark deletion. If the current modification operation includes an insertion operation, the bit length will be extended accordingly for the new data rows. The corresponding bits will be allocated to the new data rows and set to the values ​​that indicate they have not been deleted, thus obtaining the target visibility bitmap corresponding to the current target transaction.

[0048] After generating the target visibility bitmap, version information carrying the target visibility bitmap and the current target transaction identifier can be added to the end of the version list of the data table. If the modification operation involves an insertion operation, the newly added data row is actually written to the data table to complete the data modification, and at this time the target transaction is committed.

[0049] When you need to retrieve any historical version of a data table, you only need to traverse the version linked list, obtain the visibility bitmap of the corresponding version, and then you can get the visibility status of all data rows under that version, and then read the complete data table content of the corresponding version.

[0050] According to the embodiments of this application, by recording the visibility status of each data row based on a visibility bitmap, each data row only needs to occupy one bit to complete the recording of the deletion status in the version information. There is no need to store the complete version data table. Compared with the traditional multi-version storage scheme, it can significantly reduce redundant storage and effectively reduce the memory overhead caused by multi-version management. At the same time, by organizing the version information through the version linked list, the data table content of any historical version can be quickly located and obtained without adding extra time overhead for version reading. While ensuring the transaction isolation level, it significantly reduces memory consumption and improves concurrent access performance.

[0051] According to an embodiment of this application, generating a target visibility bitmap for a data table according to a modification operation includes: obtaining initial version information corresponding to the start time of the target transaction from a version list, wherein the initial version information includes an initial visibility bitmap; and modifying the values ​​of the bits corresponding to the data rows in the initial visibility bitmap according to the modification operation to obtain the target visibility bitmap.

[0052] In scenarios where multiple transactions are executed concurrently, each target transaction starts by obtaining an initial visibility bitmap based on the latest version information of the current version linked list. Subsequently, the corresponding bits are modified only for the current transaction's own modification operations, without affecting the version information of other committed transactions. This ensures the independence of each transaction's modifications and meets the requirements of transaction isolation.

[0053] In the event of a target transaction being rolled back or terminated, since it has not been committed and has not affected other transactions globally, the target visibility bitmap corresponding to the target transaction can be deleted directly, along with its private modification view, to complete the rollback or termination of the target transaction.

[0054] In other embodiments, a basic visibility bitmap can be generated based on the number of data table rows corresponding to the start time of the target transaction, where each bit value indicates that the data row has not been deleted. After modifying the basic visibility bitmap according to the modification operation, an incremental visibility bitmap corresponding to the target transaction is obtained. The incremental visibility bitmap is then merged with the initial visibility bitmap to obtain the target visibility bitmap.

[0055] According to the embodiments of this application, by using the initial visibility bitmap corresponding to the start time of the target transaction as the basis for modification, each target transaction only needs to modify the bits corresponding to the data rows involved in its operation, without having to build a complete visibility bitmap from scratch. This can further reduce the computational overhead of generating version information and improve the processing efficiency of transaction commit.

[0056] According to an embodiment of this application, modifying the value of a bit corresponding to a data row in an initial visibility bitmap includes: when the modification operation involves a deletion operation, modifying the value of the bit corresponding to the data row to be deleted in the initial visibility bitmap to a first preset value; when the modification operation involves an insertion operation, adding a bit at the end of the initial visibility bitmap and setting the value of the added bit to a second preset value.

[0057] In embodiments of this application, a first preset value is used to mark that the corresponding data row was deleted after the target transaction was committed, and a second preset value is used to mark that the corresponding data row was not deleted after the target transaction was committed. In a specific example, the first preset value can be 1, and the second preset value can be 0.

[0058] Since update operations are broken down into delete and insert operations, a modification operation involving a delete operation can be understood as a modification operation that is either a delete or an update operation. Similarly, a modification operation involving an insert operation can be understood as a modification operation that is either an insert or an update operation.

[0059] When the modification operation is an update operation, the bits corresponding to the original data row to be updated need to be modified to the first preset value to mark the deletion, and then the bits with the second preset value are added to the newly added updated data row to complete the visibility bitmap modification.

[0060] For example, if the data table has 5 rows of data at the start of the target transaction, and the previous version's visibility bitmap was 00000, indicating that all data rows had not been deleted, the current target transaction needs to perform an update operation on the 3rd row of data. After breaking down the update operation into two steps—deleting the 3rd row of data and inserting the new updated data row—first, the 3rd bit in the initial visibility bitmap is modified to the first preset value 1, at which point the bitmap is 00100. Then, because a new data row needs to be inserted, a bit is added to the end of the bitmap and set to the second preset value 0. Finally, the target visibility bitmap is 001000, and its length is expanded from 5 bits to 6 bits, consistent with the total number of data rows in the data table.

[0061] According to embodiments of this disclosure, by decomposing and updating the operation, the generation of the target visibility bitmap can be completed simply by modifying the original row bits and adding the corresponding bits to the new row, without the need for additional complex calculations. This can further improve the efficiency of transaction processing and reduce the computational overhead of the modification process.

[0062] Figure 3 A schematic diagram of generating a target visibility bitmap according to a specific embodiment of this application is shown.

[0063] like Figure 3 As shown, taking the modification operation as updating the third data row as an example, none of the four data rows in the current data table have been deleted, and the values ​​of the four bits in the initial visibility bitmap are all 0. Modify the third bit in the initial visibility bitmap to 1, and add 0 to the end of the initial visibility bitmap (after the fourth bit) to obtain the target visibility bitmap.

[0064] According to an embodiment of this application, modifying a data table includes: when the modification operation involves a deletion operation, retaining the data in the data table corresponding to the data row to be deleted; when the modification operation involves an insertion operation, inserting a new data row into the data table, wherein the new data row corresponds to a newly added bit in the target visibility bitmap.

[0065] In the embodiments of this application, the modification operation involving the deletion operation can be understood as the case where the modification operation is a deletion operation or an update operation.

[0066] In the embodiments of this application, the original storage of the data row to be deleted is retained, and the corresponding data row is not immediately physically deleted. Physical cleanup is only triggered uniformly when all versions no longer need the data row. This ensures that any historical version can be correctly read by traversing the version linked list and combining it with the visibility bitmap, and there will be no problem of missing historical version data.

[0067] When the modification operation is an update operation, the data row to be updated can be treated as a data row to be deleted. Instead of deleting the data row to be updated, the updated data is added to the data table as a new data row.

[0068] According to the embodiments of this disclosure, by retaining the physical storage of deleted data rows and only deleting them logically by marking the bits of the visibility bitmap, it is possible to ensure that historical version data can be read and retrieved normally, while avoiding the additional performance overhead caused by frequent physical deletion. At the same time, the structure of the version linked list can efficiently locate the visibility information of the corresponding version without the need for additional space to store the complete historical data rows, thus further balancing storage overhead and access performance.

[0069] According to an embodiment of this application, adding version information, including a target visibility bitmap, to a version list of a data table includes: obtaining the target transaction identifier of the target transaction from transaction information; obtaining version information based on the target transaction identifier and the target visibility bitmap; and adding the version information to the tail of the version list.

[0070] In embodiments of this application, the transaction information further includes a target transaction identifier for the target transaction. The version information includes a target transaction representation of the target transaction and a target visibility bitmap corresponding to the target transaction.

[0071] In some embodiments, version information may also include the database identifier of the database where the data table is located, as well as the data table identifier of the data table itself, so as to distinguish and manage version information of different databases and different data tables, avoid confusion of version information in multi-database and multi-table scenarios, and improve the efficiency of version information search and location.

[0072] When you need to query a specific version of data in a specific data table, you can directly match the corresponding data table version list based on the database identifier and data table identifier, then locate the version information corresponding to the target transaction identifier in the version list, and read the target visibility bitmap to obtain the visibility status of all data rows of the corresponding version, thus quickly completing the data query.

[0073] The version list stores the version information corresponding to each write transaction in the order of transaction commit. When reading a data table based on the version list, if it is necessary to obtain the latest version of the data table, the target visibility bitmap of the latest version at the end of the version list can be read directly to obtain the visibility status of all data rows under the latest version. There is no need to traverse and merge layer by layer, which greatly improves the reading efficiency of the latest version data.

[0074] According to the embodiments of this application, different versions are distinguished by transaction identifiers. Combined with the storage structure of version linked lists in the order of commit, it is possible to quickly locate the latest version for efficient reading, and also to quickly retrieve any historical version based on the transaction identifier, thereby improving retrieval efficiency.

[0075] According to an embodiment of this application, the transaction processing method further includes: determining an active transaction list based on the transaction identifier of the read transaction; searching backwards from the end of the version list and determining the first version information whose transaction identifier is not in the active transaction list as the target version information corresponding to the read transaction; and reading the data rows visible to the read transaction from the data table based on the visibility bitmap in the target version information.

[0076] The list of active transactions includes transaction identifiers for uncommitted transactions. Uncommitted transactions are those that have not yet been committed at the start of a read transaction. Different transactions have different lists of active transactions. Each read transaction determines its target version information based on the status of active transactions at the start of its transaction. This ensures that read transactions read consistent data that meets the isolation level requirements and avoids reading modifications made by uncommitted transactions.

[0077] In some embodiments, the transaction state can be determined through a global transaction snapshot. The implementation of a global transaction snapshot relies on a global transaction identifier and a global snapshot. Each transaction obtains a unique transaction identifier and snapshot from the global transaction manager upon startup, used to identify the scope of the transaction's visibility. The snapshot contains the following information: the smallest transaction identifier currently visible to the transaction, the smallest transaction identifier not currently visible to the transaction, and a list of active transactions (including the transaction identifiers of each active transaction).

[0078] Since the version information includes the transaction identifier of each transaction, the list of active transactions corresponding to the read transaction can be determined based on the global transaction snapshot. The first version information that is not in the list of active transactions can be found by searching backward from the end of the version list. This version is the latest consistent version that the read transaction can access, and the visibility bitmap of this version can be read directly.

[0079] After obtaining the target version information, you only need to traverse the visibility bitmap in the target version information, filter out the data rows whose bit values ​​are the second preset value (i.e., those that have not been deleted), and you can directly read the data table content that meets the read transaction requirements.

[0080] According to the embodiments of this application, by combining the active transaction list to look up the target version from the version chain in reverse, it is possible to quickly locate the consistent version that meets the isolation requirements of the read transaction, which effectively improves the processing efficiency of the read transaction and adapts to the frequent read requirements in high-concurrency scenarios.

[0081] Figure 4 A schematic diagram illustrating version selection during a data table access process according to an embodiment of this application is shown.

[0082] like Figure 4As shown, the transaction begins execution by initiating a read request. The system retrieves the version list corresponding to the table, which contains multiple visibility bitmap versions arranged in commit order. Subsequently, the system retrieves a global transaction snapshot, returns a list of currently active transactions, and combines this with the version list to return the visibility bitmap corresponding to the transaction. Based on this visibility bitmap, visibility processing is performed on the data table to obtain the data rows visible to the transaction (rows marked "not deleted" in the bitmap). A read operation is performed on the visible data rows, and the read data rows are returned to the transaction, completing the read operation.

[0083] According to an embodiment of this application, the transaction processing method further includes: when the length of the version list exceeds a preset length threshold, determining and deleting version information to be deleted from the version list, wherein the version information to be deleted is not depended on by active transactions.

[0084] To prevent the version chain from growing indefinitely and causing memory overflow, embodiments of this application also design a version cleanup mechanism. When the length of the version chain exceeds a preset length threshold, version cleanup will be automatically triggered.

[0085] During version cleanup, all active transactions can be traversed to determine which version information is still in a dependent state. Only earlier versions that are no longer dependent on by any active transactions are included in the deletion scope. Cleanup does not directly delete all version information; at least the last version in the version list is retained, and subsequent versions continue to be organized based on that version.

[0086] In a specific example, each version can maintain its own reference count. If n transactions use this version, the reference count is n; if no transactions use it, the reference count is 0, and it can be cleared.

[0087] In other embodiments, version information to be deleted can also be periodically identified from the version list.

[0088] According to the embodiments of this application, by periodically cleaning up old version information that is no longer relied upon through the version cleanup mechanism, the problems of decreased traversal performance and excessive memory consumption caused by excessively long version lists can be avoided. Under the premise of ensuring the normal operation of transaction processing, stable memory overhead and access performance are maintained.

[0089] According to an embodiment of this application, the transaction processing method further includes: detecting multiple consecutive bits with the same value in the target visibility bitmap, and storing the multiple bits as a value and a number when the number of the multiple bits is greater than a preset number threshold.

[0090] In the embodiments of this application, when the number of data rows in the data table is large, the corresponding number of bits in the target visibility bitmap is also large. In order to save storage space and avoid memory overflow, the visibility bitmap can be further compressed for storage.

[0091] In real-world business scenarios, multiple consecutive bits will maintain the same value. By storing these multiple bits as a specific number and value, compressed storage can be achieved, which can significantly reduce the storage space occupied by the visibility bitmap without losing information.

[0092] For example, given a sequence of 100 consecutive bits all with a value of 0, instead of storing 100 zeros bit by bit, only "value 0, length 100" needs to be stored. What would have required 100 bits can now be stored in just a few dozen bits, effectively compressing storage space. When reading after compression, only the stored values ​​and number of bits need to be reconstructed, without affecting the accuracy of visibility checks. This results in significant storage gains with minimal computational overhead, further optimizing memory usage efficiency in multi-version scenarios.

[0093] According to the embodiments of this application, by compressing and storing consecutive bits of the same value, the memory footprint of the visibility bitmap can be further reduced without changing the visibility marking logic, thereby improving space utilization and adapting to transaction processing scenarios with large amounts of data.

[0094] Based on the above-described database transaction processing method, this application also provides a database transaction processing apparatus. The following will be combined with... Figure 5 The device is described in detail.

[0095] Figure 5 A structural block diagram of a database transaction processing apparatus according to an embodiment of this application is shown.

[0096] like Figure 5 As shown, the database transaction processing device 500 of this embodiment includes an acquisition module 510, a generation module 520, and a submission module 530.

[0097] The acquisition module 510 is used to acquire transaction information of the target transaction. The transaction information includes modification operations on the data table indicated by the target transaction. The modification operations include deletion, insertion, and update operations. The update operation is broken down into a deletion operation on the row to be updated and an insertion operation that inserts the updated data into the data table by adding new data rows. In one embodiment, the acquisition module 510 can be used to execute the operation S210 described above, which will not be repeated here.

[0098] The generation module 520 is used to generate a target visibility bitmap of the data table based on the modification operation. Each bit in the target visibility bitmap corresponds to a data row in the data table, and the value of the bit is used to indicate whether the corresponding data row is deleted after the target transaction is committed. In one embodiment, the generation module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0099] The commit module 530 is used to modify the data table by adding version information, including the target visibility bitmap, to the version list of the data table, and, when the modification operation involves an insertion operation, commit the target transaction to obtain each version of the data table based on the version information in the version list. In one embodiment, the commit module 530 can be used to perform the operation S230 described above, which will not be repeated here.

[0100] According to an embodiment of this application, the generation module 520 includes a bitmap determination submodule and a bitmap modification submodule.

[0101] The bitmap determination submodule is used to obtain the initial version information corresponding to the start time of the target transaction from the version list. The initial version information includes the initial visibility bitmap.

[0102] The bitmap modification submodule is used to modify the value of the corresponding bit in the initial visibility bitmap according to the modification operation, so as to obtain the target visibility bitmap.

[0103] According to an embodiment of this application, the bitmap modification submodule includes a bitmap deletion unit and a bitmap insertion unit.

[0104] The bitmap deletion unit is used to modify the value of the bit corresponding to the data row to be deleted in the initial visibility bitmap to a first preset value when the modification operation involves a deletion operation. The first preset value is used to mark that the corresponding data row is deleted after the target transaction is committed.

[0105] The bitmap insertion unit is used to add a bit to the end of the initial visibility bitmap when the modification operation involves an insertion operation, and sets the value of the added bit to a second preset value. The second preset value is used to mark that the corresponding data row has not been deleted after the target transaction is committed.

[0106] According to an embodiment of this application, the submission module 530 includes a data deletion submodule and a data insertion submodule.

[0107] The data deletion submodule is used to retain the data in the data table corresponding to the data row to be deleted when the modification operation involves a deletion operation.

[0108] The data insertion submodule is used to insert new data rows into the data table when the modification operation involves an insertion operation. The new data rows correspond to the newly added bits in the target visibility bitmap.

[0109] According to an embodiment of this application, the submission module 530 further includes an identifier acquisition submodule, a version generation submodule, and a version addition submodule.

[0110] The identifier acquisition submodule is used to obtain the target transaction identifier of the target transaction from the transaction information.

[0111] The version generation submodule is used to obtain version information based on the target transaction identifier and the target visibility bitmap.

[0112] The version addition submodule is used to add version information to the end of the version list.

[0113] According to an embodiment of this application, the database transaction processing apparatus 500 further includes a list determination module, a version query module, and a data reading module.

[0114] The list determination module is used to determine the list of active transactions based on the transaction identifier of the read transaction. The list of active transactions includes the transaction identifiers of uncommitted transactions, which are transactions that have not yet been committed at the start of the read transaction.

[0115] The version query module is used to search backwards from the end of the version list, identify the version information of the first transaction that is not in the active transaction list, and determine it as the target version information corresponding to the read transaction.

[0116] The data reading module is used to read data rows that are visible to the read transaction from the data table based on the visibility bitmap in the target version information.

[0117] According to an embodiment of this application, the database transaction processing apparatus 500 further includes a deletion determination module.

[0118] The deletion determination module is used to determine and delete version information to be deleted from the version list when the length of the version list exceeds a preset length threshold. The version information to be deleted is not depended on by active transactions.

[0119] According to an embodiment of this application, the database transaction processing apparatus 500 further includes a bitmap storage module.

[0120] The bitmap storage module is used to detect multiple consecutive bits with the same value in the target visibility bitmap. If the number of multiple bits is greater than a preset threshold, the multiple bits are stored as their value and number.

[0121] According to embodiments of this application, any multiple modules among the acquisition module 510, generation module 520, and submission module 530 can be merged into one module, or any one of these modules can be split into multiple modules. Alternatively, at least some of the functions of one or more of these modules can be combined with at least some of the functions of other modules and implemented in one module. According to embodiments of this application, at least one of the acquisition module 510, generation module 520, and submission module 530 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or implemented in hardware or firmware by any other reasonable means of integrating or packaging circuitry, or implemented in any one of software, hardware, and firmware methods, or in a suitable combination of any of these. Alternatively, at least one of the acquisition module 510, generation module 520, and submission module 530 can be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0122] Figure 6 A block diagram of an electronic device suitable for implementing a database transaction processing method according to an embodiment of this application is shown.

[0123] like Figure 6 As shown, an electronic device 600 according to an embodiment of this application includes a processor 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory ROM 602 or a program loaded from a storage portion 608 into a random access memory RAM 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of this application.

[0124] RAM 603 stores various programs and data required for the operation of electronic device 600. Processor 601, ROM 602, and RAM 603 are interconnected via bus 604. Processor 601 executes various operations of the method flow according to embodiments of this application by executing programs in ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than ROM 602 and RAM 603. Processor 601 may also execute various operations of the method flow according to embodiments of this application by executing programs stored in said one or more memories.

[0125] According to embodiments of this application, the electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to a bus 604. The electronic device 600 may also include one or more of the following components connected to the input / output (I / O) interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 608 including a hard disk, etc.; and a communication section 609 including a network interface card such as a LAN card, modem, etc. The communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to the input / output (I / O) interface 605 as needed. A removable medium 611, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 610 as needed so that computer programs read from it can be installed into the storage section 608 as needed.

[0126] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of this application.

[0127] According to embodiments of this application, the computer-readable storage medium can be a non-volatile computer-readable storage medium, such as including but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this application, the computer-readable storage medium may include ROM 602 and / or RAM 603 and / or one or more memories other than ROM 602 and RAM 603 described above.

[0128] Embodiments of this application also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code enables the computer system to implement the database transaction processing method provided in the embodiments of this application.

[0129] When the computer program is executed by the processor 601, it performs the functions defined in the system / apparatus of this application embodiment. According to the embodiments of this application, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0130] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 609, and / or installed from the removable medium 611. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0131] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from the removable medium 611. When the computer program is executed by the processor 601, it performs the functions defined in the system of this application embodiment. According to the embodiments of this application, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] According to embodiments of this application, program code for executing the computer programs provided in the embodiments of this application can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can be executed entirely on the user's computing device, partially on the user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0133] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0134] Those skilled in the art will understand that the features described in the various embodiments of this application can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this application. In particular, the features described in the various embodiments of this application can be combined and / or combined in various ways without departing from the spirit and teachings of this application. All such combinations and / or combinations fall within the scope of this application.

[0135] The embodiments of this application have been described above. However, these embodiments are merely illustrative and not intended to limit the scope of this application. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. Without departing from the scope of this application, those skilled in the art can make various substitutions and modifications, all of which should fall within the scope of this application.

Claims

1. A database transaction processing method, characterized in that, The transaction processing method includes: Obtain transaction information of the target transaction, the transaction information including the modification operations on the data table indicated by the target transaction, the modification operations including deletion operations, insertion operations and update operations, the update operations being decomposed into deletion operations on the rows to be updated and insertion operations that insert the updated data into the data table by adding new data rows; Based on the modification operation, a target visibility bitmap of the data table is generated. Each bit in the target visibility bitmap corresponds to a data row in the data table. The value of the bit is used to mark whether the corresponding data row is deleted after the target transaction is committed. By adding version information, including the target visibility bitmap, to the version list of the data table, and modifying the data table when the modification operation involves an insertion operation, and committing the target transaction, the versions of the data table can be obtained according to the version information in the version list.

2. The transaction processing method according to claim 1, characterized in that, The step of generating the target visibility bitmap of the data table based on the modification operation includes: Obtain the initial version information corresponding to the start time of the target transaction from the version list, wherein the initial version information includes an initial visibility bitmap; According to the modification operation, the value of the bit in the corresponding data row of the initial visibility bitmap is modified to obtain the target visibility bitmap.

3. The transaction processing method according to claim 2, characterized in that, Modifying the value of the bit in the corresponding data row of the initial visibility bitmap includes: In cases where the modification operation involves a deletion operation, the value of the bit corresponding to the data row to be deleted in the initial visibility bitmap is modified to a first preset value. The first preset value is used to mark that the corresponding data row is deleted after the target transaction is committed. In cases where the modification operation involves an insertion operation, a new bit is added to the end of the initial visibility bitmap, and the value of the new bit is set to a second preset value, which is used to mark that the corresponding data row has not been deleted after the target transaction is committed.

4. The transaction processing method according to claim 1, characterized in that, The modification of the data table includes: If the modification operation involves a deletion operation, the data in the data table corresponding to the data row to be deleted shall be retained; When the modification operation involves an insertion operation, a new data row is inserted into the data table, and the new data row corresponds to a newly added bit in the target visibility bitmap.

5. The transaction processing method according to claim 1, characterized in that, The transaction information also includes the target transaction identifier of the target transaction, and the step of adding the version information, including the target visibility bitmap, to the version list of the data table includes: Obtain the target transaction identifier of the target transaction from the transaction information; The version information is obtained based on the target transaction identifier and the target visibility bitmap; The version information is added to the end of the version list.

6. The transaction processing method according to claim 5, characterized in that, The transaction processing method further includes: An active transaction list is determined based on the transaction identifier of the read transaction. The active transaction list includes the transaction identifiers of uncommitted transactions, which are transactions that have not yet been committed at the start time of the read transaction. Starting from the tail of the version list, search backwards and identify the first transaction whose identifier is not in the active transaction list as the target version information corresponding to the read transaction; Based on the visibility bitmap in the target version information, read the data rows visible to the read transaction from the data table.

7. The method according to claim 1, characterized in that, The transaction processing method further includes: If the length of the version list exceeds a preset length threshold, the version information to be deleted is determined from the version list and deleted, and the version information to be deleted is not depended on by any active transaction.

8. The transaction processing method according to claim 1, characterized in that, The transaction processing method further includes: Detect multiple consecutive bits with the same value in the target visibility bitmap. If the number of these multiple bits is greater than a preset threshold, store the multiple bits as the value and the number.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more computer programs. The characteristic feature is that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 8.

10. A computer-readable storage medium having a computer program or instructions stored thereon, characterized in that, When the computer program or instructions are executed by a processor, they implement the steps of the method according to any one of claims 1 to 8.

Images