Database transaction system lock competition optimization method and device

Abstract

The invention relates to the technical field of databases, and provides a database transaction system lock competition optimization method and device. The method comprises the following steps: acquiring a current global active read-write transaction list and a transaction auxiliary index set; mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure; determining the number of Hash buckets and serial numbers of the Hash buckets according to the estimated concurrency of the current database, and splitting the global active read-write transaction into a plurality of Hash buckets based on a balanced splitting strategy according to the number of the Hash buckets and the serial numbers of the Hash buckets; wherein each hash bucket is provided with an independent mutual exclusion lock; and protecting critical resources of each Hash bucket by using the independent mutual exclusion lock corresponding to each Hash bucket. Through the embodiment of the invention, the performance of the database can be improved, and the problem that the global mutual exclusion lock of a transaction system becomes the performance bottleneck of the database in a high-concurrency scene is solved.

Description

Technical Field

[0001] The present invention relates to the field of database technology, and in particular to a method and device for optimizing lock contention in a database transaction system. Background Art

[0002] In the design of database systems, in order to ensure the ACID characteristics of transactions and multi-version concurrency control, a unified data structure (hereinafter referred to as the "transaction system") is often used to maintain relevant information of all transaction operations, such as active transaction sequence numbers, global read-write transaction lists, maximum transaction sequence numbers, etc., and a global mutex lock is used to achieve mutual exclusive access to transaction system resources. In various stages of transaction execution, such as transaction opening, committing, and snapshot reading at the RC isolation level, the global mutex lock needs to be used frequently to achieve exclusive access to the data structure maintained by the transaction system. In high-concurrency business scenarios, due to the frequent access of concurrent transactions to transaction system resources, the mutex lock competition used to protect the transaction system becomes a performance bottleneck, which limits the efficiency of the database's use of multi-core resources to a certain extent. Therefore, there is an urgent need for a database transaction system lock competition optimization method, which solves the problem that the global mutex lock of the transaction system becomes a performance bottleneck of the database in high-concurrency scenarios, while ensuring the ACID characteristics of transactions, multi-version concurrency control, and the correctness of the relevant functions and logic of the original transaction system. Summary of the invention

[0003] In view of the fact that in current high-concurrency business scenarios, due to the frequent access of concurrent transactions to transaction system resources, the competition for mutex locks used to protect the transaction system has become a performance bottleneck, which to a certain extent limits the efficiency of the database's utilization of multi-core resources. This solution is proposed to overcome the above problem or at least partially solve the above problem.

[0004] On the one hand, some embodiments of this specification aim to provide a method for optimizing lock contention in a database transaction system, the method comprising:

[0005] Get the current global active read-write transaction list and transaction auxiliary index set;

[0006] Mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure;

[0007] Determine the number of hash buckets and hash bucket sequence numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced splitting strategy according to the number of hash buckets and hash bucket sequence numbers;

[0008] The critical resources of each hash bucket are protected by an independent mutex lock corresponding to each hash bucket.

[0009] Furthermore, the isolation level of the database transaction includes at least a read committed level.

[0010] Furthermore, the global active read-write transaction list and the transaction auxiliary index set are mapped into a hash data structure, including:

[0011] The index relationship between the global active read-write transaction list and the transaction auxiliary index set is maintained unchanged, and the global active read-write transaction list and the transaction auxiliary index set are hashed to obtain a hash data structure with an index relationship.

[0012] Further, the number of hash buckets and the hash bucket sequence number are determined according to the estimated concurrency of the current database, and the global active read and write transactions are split into multiple hash buckets based on the balanced split strategy according to the number of hash buckets and the hash bucket sequence number, including:

[0013] Determine the number of hash buckets according to the interval of the estimated concurrency;

[0014] Determine the hash bucket sequence number according to the number of hash buckets;

[0015] Divide the global active read and write transactions by the number of hash buckets to perform a remainder operation to obtain a remainder result;

[0016] The hash bucket sequence number corresponding to each active read / write transaction is determined according to the modulo result, so as to distribute the global active read / write transactions to multiple hash buckets.

[0017] Further, determining the number of hash buckets according to the interval where the estimated concurrency is located includes:

[0018] Get the actual concurrency range of the current database;

[0019] According to the preset hash bucket quantity matching table, the number of hash buckets corresponding to the actual concurrency interval is determined.

[0020] Further, determining the number of hash buckets according to the interval where the estimated concurrency is located further includes:

[0021] Obtain the maximum load concurrency, historical actual concurrency, and historical network status and hardware configuration corresponding to the historical actual concurrency of a database with the same business scenario as the current database;

[0022] Establishing a regression analysis model between the historical actual concurrency, the historical network status and hardware configuration, and the maximum load concurrency;

[0023] Input the current database's network status, hardware configuration, and maximum concurrent load into the regression analysis model to obtain the predicted concurrent load of the current database, so as to determine the predicted concurrent load range.

[0024] Determine a target concurrency interval using the predicted concurrency interval and the actual concurrency interval; the actual concurrency interval is a subset of the target concurrency interval;

[0025] According to the preset hash bucket quantity matching table, the number of hash buckets corresponding to the target concurrency interval is determined.

[0026] Furthermore, after the critical resources of each hash bucket are protected by using the independent mutex lock corresponding to each hash bucket, the method further includes:

[0027] According to the changes of the global active read-write transaction list in the hash bucket, a log stream based on the time sequence number is constructed;

[0028] The latest log stream is obtained according to the preset time period to form a copy of the global active read-write transaction list at the current moment. The latest log stream is obtained in response to the read view creation operation to form a copy of the global active read-write transaction list at the time of read view creation, thereby realizing a global active read-write transaction snapshot.

[0029] On the other hand, some embodiments of this specification further provide a database transaction system lock contention optimization device, the device comprising:

[0030] The acquisition module is used to obtain the current global active read and write transaction list and transaction auxiliary index set;

[0031] A mapping module, used to map the global active read-write transaction list and the transaction auxiliary index set into a hash data structure;

[0032] A splitting module is used to determine the number of hash buckets and hash bucket serial numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced splitting strategy according to the number of hash buckets and hash bucket serial numbers; wherein each hash bucket has an independent mutex lock;

[0033] The optimization module is used to protect the critical resources of each hash bucket by using the independent mutex lock corresponding to each hash bucket.

[0034] On the other hand, some embodiments of the present specification further provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0035] On the other hand, some embodiments of the present specification further provide a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0036] On the other hand, some embodiments of the present specification further provide a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0037] One or more technical solutions provided by some embodiments of this specification have at least the following technical effects:

[0038] The embodiments of the present specification first automatically obtain the current global active read-write transaction list and transaction auxiliary index set, and perform hash processing on the global active read-write transaction list and transaction auxiliary index set to facilitate splitting of the global active read-write transactions. When splitting, the appropriate number of hash buckets and hash bucket sequence numbers are determined according to the estimated concurrency of the current database, and the global active read-write transactions are evenly split into multiple hash buckets according to the number of hash buckets and hash bucket sequence numbers, so that the independent mutex lock corresponding to each hash bucket is used to protect the critical resources of each hash bucket, thereby realizing fine-grained splitting of database transaction system locks and improving the processing throughput of the database transaction system.

[0039] The above description is only an overview of the technical solutions of some embodiments of this specification. In order to more clearly understand the technical means of some embodiments of this specification, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of some embodiments of this specification more obvious and easy to understand, the specific implementation methods of some embodiments of this specification are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate some embodiments of this specification or technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0041] Figure 1 A schematic diagram of an implementation system of a database transaction system lock contention optimization method in some embodiments of this specification is shown;

[0042] Figure 2 A flowchart of a database transaction system lock contention optimization method in some embodiments of this specification is shown;

[0043] Figure 3 A schematic diagram of the steps of splitting global active read and write transactions into multiple hash buckets in some embodiments of this specification;

[0044] Figure 4This is a schematic diagram of transaction distribution after global active read and write transactions are split in some embodiments of this specification;

[0045] Figure 5 A schematic diagram of the first step of determining the number of hash buckets in some embodiments of this specification;

[0046] Figure 6 A schematic diagram of the second step of determining the number of hash buckets in some embodiments of this specification;

[0047] Figure 7 A schematic diagram of steps for updating the current global active read-write transaction list in some embodiments of this specification;

[0048] Figure 8 This is a schematic diagram of the structure of a database transaction system lock contention optimization device in some embodiments of this specification;

[0049] Fig. 9 This is a schematic diagram of the computer device structure provided in some embodiments of this specification.

[0050] [Description of Reference Numerals]

[0051] 101. Terminal;

[0052] 102. Server;

[0053] 801, obtaining module;

[0054] 802, mapping module;

[0055] 803, split module;

[0056] 804. Optimization module;

[0057] 902. Computer equipment;

[0058] 904, processor;

[0059] 906. Memory;

[0060] 908, driving mechanism;

[0061] 910, input / output interface;

[0062] 912. Input device;

[0063] 914. Output device;

[0064] 916. Presentation equipment;

[0065] 918. Graphical user interface;

[0066] 920, network interface;

[0067] 922, communication link;

[0068] 924. Communication bus. DETAILED DESCRIPTION

[0069] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in some embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on some embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0070] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0071] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of relevant laws and regulations.

[0072] like Figure 1 The schematic diagram of the implementation system of a database transaction system lock contention optimization method according to an embodiment of the present invention is shown, which may include: a terminal 101 and a server 102, wherein the terminal 101 and the server 102 communicate with each other through a network, and the network may include a local area network (LAN), a wide area network (WAN), the Internet or a combination thereof, and is connected to a website, a user device (such as a computing device) and a back-end system. A staff member may send a database transaction system lock contention optimization request to the server 102 through the terminal 101, and after receiving the database transaction system lock contention optimization request, the server 102 optimizes the transaction system of the database, optimizes the original global mutex lock of the transaction system into a distributed multiple independent mutex locks, and obtains an optimized database, so that the staff member processes the business according to the optimized database.

[0073] In the embodiments of this specification, the server 102 can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDN, Content Delivery Network), and big data and artificial intelligence platforms.

[0074] In an optional embodiment, the terminal 101 may include but is not limited to electronic devices such as self-service terminal devices, desktop computers, tablet computers, and laptop computers. Optionally, the operating system running on the electronic device may include but is not limited to Android system, IOS system, Linux, Windows, etc. Of course, the terminal 101 is not limited to the above-mentioned electronic devices with a certain entity, and it can also be software running in the above-mentioned electronic devices.

[0075] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by the present disclosure. In actual application, multiple terminals 101 may be included, and this specification does not limit this.

[0076] Figure 2 It is a flowchart of a database transaction system lock contention optimization method provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, applied to the above-mentioned server side, the method may include:

[0077] S201: Obtain the current global active read-write transaction list and transaction auxiliary index set;

[0078] S202: Mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure;

[0079] S203: Determine the number of hash buckets and hash bucket sequence numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on a balanced splitting strategy according to the number of hash buckets and hash bucket sequence numbers; wherein each hash bucket has an independent mutex lock;

[0080] S204: Using the independent mutex lock corresponding to each hash bucket to protect the critical resources of each hash bucket.

[0081] The embodiments of the present specification first automatically obtain the current global active read-write transaction list and transaction auxiliary index set, and perform hash processing on the global active read-write transaction list and transaction auxiliary index set to facilitate splitting of the global active read-write transactions. When splitting, the appropriate number of hash buckets and hash bucket sequence numbers are determined according to the estimated concurrency of the current database, and the global active read-write transactions are evenly split into multiple hash buckets according to the number of hash buckets and hash bucket sequence numbers, so that the independent mutex lock corresponding to each hash bucket is used to protect the critical resources of each hash bucket, thereby realizing fine-grained splitting of database transaction system locks and improving the processing throughput of the database transaction system.

[0082] The critical resources of each hash bucket are protected by an independent mutex lock corresponding to each hash bucket, so that when a transaction in the database is opened and submitted, an independent mutex lock corresponding to the transaction can be obtained according to the transaction number and the balanced split strategy, and the active transaction list can be added or deleted based on the corresponding hash bucket. When querying whether a transaction in the database is active, an independent mutex lock corresponding to the transaction is obtained according to the transaction number and the balanced split strategy, and the transaction is checked based on the corresponding hash bucket to see whether it exists in the active transaction list, and the query result is returned.

[0083] Furthermore, in some embodiments, the isolation level of the database transaction includes at least a read committed level.

[0084] Specifically, in some embodiments, the existing technical solution is to protect the transaction system by a unified mutex lock, and the pseudo code can be expressed as:

[0085] struct trx_sys_t{ / / The structure of the information maintained by the transaction system, globally unique

[0086] TrxSysMutex mutex; / / Global unified mutex lock

[0087] MVCC*mvcc; / / Multi-version control system

[0088] trx_id_t max_trx_id; / / Transaction system number issuer

[0089] std::atomic<trx_id_t> min_active_id; / / Minimum active transaction number

[0090] trx_ut_list_t serialisation_list; / / transaction submission global ordered list

[0091] trx_ut_list_t rw_trx_list; / / global read and write transaction list

[0092] trx_ut_list_t mysql_trx_list; / / global read-write and read-only transaction list

[0093] trx_ids_t rw_trx_ids; / / Global active transaction view, used to provide snapshots for MVCC

[0094] TrxIdSet rw_trx_set; / / Transaction instance set, auxiliary structure for finding transactions

[0095] }

[0096] As shown in the above pseudo code, in the prior art, when the database is started, a globally unique transaction system object (i.e., trx_sys_t) is allocated to handle transaction-related work. The transaction needs to obtain the mutex lock of the object in the execution phases such as opening, committing, and snapshot reading to prevent other transactions from concurrently modifying the critical section. The protection of various resources of the transaction system is achieved through a global mutex lock (i.e., TrxSysMutex), so that the recognition of the active state of the transaction by each resource is unified, ensuring the ACID characteristics of the transaction and the correctness of multi-version concurrency control.

[0097] However, the existing technical solution only protects each critical resource of the transaction system through a coarse-grained global mutex lock, which still has shortcomings:

[0098] First, the coarse-grained global mutex lock has an impact on the performance of transaction opening and submission. The lock resource will be frequently acquired at each stage of transaction execution. While monopolizing all critical resources of the transaction system, it will cause the blocking of other concurrent transactions that would not read and write the same critical data. Especially in high-concurrency online transaction business scenarios, high-frequency transaction opening and submission will frequently compete for the global mutex lock of the transaction system, making it a performance bottleneck. The blocking of other concurrent transactions will affect the concurrency and response time of the database, and the blocking of concurrent transactions makes it impossible for the database to give full play to the multi-core advantages of the system, limiting the overall performance ceiling of the database. Take the following transaction submission process of the prior art solution as an example:

[0099] There are two main processes involving transaction system locks during transaction submission:

[0100] Process 1: Add the transaction to the commit list:

[0101] ①Get the transaction system lock TrxSysMutex;

[0102] ②Get the transaction submission sequence number;

[0103] ③Add the transaction to the commit serialization list;

[0104] ④Get the purge lock;

[0105] ⑤ Release the transaction system lock TrxSysMutex;

[0106] ⑥Add the transaction undo log to the purge queue;

[0107] ⑦Release the purge lock.

[0108] Process 2: Clean up the relevant data structures of the transaction in the transaction system:

[0109] ①Get the transaction system lock TrxSysMutex;

[0110] ②Remove the transaction from the commit serialization list;

[0111] ③Remove the transaction number from the global active transaction number list;

[0112] ④Remove the transaction entity from the global active transaction entity list;

[0113] ⑤Remove the transaction entity from the transaction auxiliary search structure (rw_trx_set);

[0114] ⑥Update the minimum active transaction number;

[0115] ⑦ Release the transaction system lock TrxSysMutex.

[0116] According to the above process 1 and process 2, once a transaction obtains the transaction system lock TrxSysMutex, it monopolizes the critical resources of the entire transaction system, avoiding concurrent modifications to the transaction system data structure. At this moment, when concurrent threads try to obtain the critical resources of the transaction system, even if they do not need to modify the transaction-related data structure, they will be blocked by the transaction system lock TrxSysMutex, such as determining whether the transaction is active, finding whether there is an implicit lock conversion in the transaction, and related operations such as Performance / Information Schema. Therefore, the existing technical solution has the defects of being too strict in the design of obtaining the transaction system lock TrxSysMutex, and there are too many data structures protected by the transaction system lock TrxSysMutex, and the lock granularity is too large.

[0117] Secondly, the coarse-grained global mutex lock will affect the performance of multi-version concurrent control snapshot reading. The isolation of the database transaction system can be divided into READ UNCOMMITTED, READ COMMITTED, REPEATABLE READ and SERIALIZABLE. For example, for the read committed (RC) isolation level, each query in the transaction will apply to create a read view to obtain the global active transaction sequence number list of the transaction system to ensure the isolation of the transaction. The existing technical solution to obtain the active transaction sequence number list will obtain the mutex lock of the entire transaction system in a rough manner to avoid concurrent modifications when obtaining the list. However, it also blocks the query request of the concurrent transaction to create the read view, which causes unnecessary performance loss to a certain extent and constrains the concurrency limit supported by the database. The following MVCC read view creation process is shown:

[0118] Schematic diagram of the process involving transaction system locks during read view creation:

[0119] ①Get the transaction system lock TrxSysMutex;

[0120] ②Copy the active transaction number list rw_trx_ids of the current transaction system;

[0121] ③ Release the transaction system lock TrxSysMutex.

[0122] The purpose of acquiring the transaction system lock TrxSysMutex is to prevent concurrent threads from modifying the list of active transaction numbers, such as the transaction start and submission process. However, acquiring the transaction system lock TrxSysMutex also blocks the process of creating a read view for concurrent transactions, even if the process does not modify the list of active transaction numbers. In addition, while the transaction system lock TrxSysMutex is held by other operations, the read view creation process is also blocked. Therefore, the use of global mutex locks in the prior art solution blocks unnecessary concurrent read operations, reflecting the defects of excessive lock granularity and low performance ceiling.

[0123] Furthermore, in some embodiments, mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure includes:

[0124] The index relationship between the global active read-write transaction list and the transaction auxiliary index set is maintained unchanged, and the global active read-write transaction list and the transaction auxiliary index set are hashed to obtain a hash data structure with an index relationship.

[0125] It can be understood that, in some embodiments, the global active read-write transaction list can be used to represent the global active transaction status, and the transaction auxiliary index set is used to quickly find transactions, similar to the table of contents of a book, in which the corresponding content can be immediately found according to a page number of the table of contents. Hash processing can be understood as a one-way function. By performing hash processing on the global active read-write transaction list and the transaction auxiliary index set, the amount of data can be reduced, and it has the advantage of a simple processing method, which facilitates the subsequent splitting of the global active read-write transactions.

[0126] See attached Figure 3 In some embodiments, determining the number of hash buckets and the sequence number of hash buckets according to the estimated concurrency of the current database, and splitting the global active read and write transactions into multiple hash buckets based on the balanced splitting strategy according to the number of hash buckets and the sequence number of hash buckets may include:

[0127] S301: Determine the number of hash buckets according to the interval where the estimated concurrency is located;

[0128] S302: Determine the hash bucket sequence number according to the number of hash buckets;

[0129] S303: Divide the global active read and write transactions by the number of hash buckets to perform a modulo operation to obtain a modulo result;

[0130] S304: Determine the hash bucket sequence number corresponding to each active read / write transaction according to the modulo result, so as to distribute the global active read / write transactions to multiple hash buckets.

[0131] It can be understood that, in some embodiments, the global active read-write transactions are split into different hash buckets based on the principle of balanced splitting. In a specified time interval, when the number of newly added global active read-write transactions to be split exceeds a preset threshold, the newly added global active read-write transactions to be split can be divided into two or more batches and then balanced split to improve the efficiency of balanced splitting. Furthermore, the number of hash buckets can be determined according to the concurrency level of the current database transactions, so that the corresponding hash bucket serial numbers are determined for multiple hash buckets according to the number of hash buckets, and then the modulus operation is performed on the number of the active read-write transactions, and the serial number of the active read-write transaction is determined according to the modulus result. For example, assuming that the ID number of the active read-write transaction is 18, the number of hash buckets is 4, 18%4=2, so the active read-write transaction is placed in the hash bucket with serial number 2, so that multiple active read-write transactions can be quickly split into different hash buckets. Refer to the attached Figure 4As shown, Bucket represents a hash bucket, Mutex represents an independent mutex, and the hash bucket stores paired key-value pairs TrxID KEY and TrxEntity Val. When confirming whether the transaction is active based on the transaction ID, the ID can be modulo, and the hash bucket sequence number is determined to be 2. The hash bucket with sequence number 2 is searched for the existence of the transaction, thereby greatly speeding up the search speed. It should be noted that in some embodiments, within a specified time interval, if the number of newly added global active read-write transactions to be split does not exceed the preset threshold, or there are other situations that have little impact on the overall performance of the database transaction system, the newly added global active read-write transactions to be split may not be divided into batches at this time, but the ID of the newly added global active read-write transaction to be split may be directly modulo-calculated to achieve fast and balanced splitting. The global active read-write transaction list maintained by multiple hash buckets after splitting avoids the monopoly of other critical resources that do not need to be accessed, thereby reducing the blocking of other concurrent read requests.

[0132] See attached Figure 5 In some embodiments, determining the number of hash buckets according to the interval where the estimated concurrency is located may include:

[0133] S501: Obtain the actual concurrency range of the current database;

[0134] S502: Determine the number of hash buckets corresponding to the actual concurrency interval according to a preset hash bucket number matching table.

[0135] It can be understood that, in some embodiments, when determining the number of hash buckets, the corresponding number of hash buckets can be matched from a preset hash bucket number matching table according to the actual concurrency interval, and the number of hash buckets corresponding to the actual concurrency interval can be used as the number of hash buckets corresponding to the estimated concurrency to achieve splitting of the hash buckets. Generally, the greater the actual concurrency, the greater the number of hash buckets.

[0136] See attached Figure 6 In some embodiments, determining the number of hash buckets according to the interval where the estimated concurrency is located may further include:

[0137] S601: Obtain the maximum carrying concurrency, historical actual concurrency, and historical network status and hardware configuration corresponding to the historical actual concurrency of a database having the same business scenario as the current database;

[0138] S602: Establishing a regression analysis model between the historical actual concurrency, the historical network status and hardware configuration, and the maximum load concurrency;

[0139] S603: Input the network status and hardware configuration of the current database and the maximum concurrent load into the regression analysis model to obtain the predicted concurrent load of the current database to determine the predicted concurrent load range;

[0140] S604: Determine a target concurrency interval using the predicted concurrency interval and the actual concurrency interval; the actual concurrency interval is a subset of the target concurrency interval;

[0141] S605: Determine the number of hash buckets corresponding to the target concurrency range according to a preset hash bucket number matching table.

[0142] It can be understood that, in some embodiments, when determining the number of hash buckets, in addition to determining the appropriate number of hash buckets based on the actual concurrency interval, the predicted concurrency interval can also be combined with the predicted concurrency interval. Specifically, the predicted concurrency interval is obtained based on the concurrency information of the database with the same business scenario as the current database. The concurrency information may include the maximum load concurrency, the historical actual concurrency, and the historical network status and hardware configuration corresponding to the historical actual concurrency. A regression analysis model between the historical actual concurrency and the historical network status and hardware configuration and the maximum load concurrency is established, so that the possible concurrency of the database with the same business scenario as the current database can be predicted to obtain a predicted concurrency interval. The concurrency data in the predicted concurrency interval may also be the concurrency that will appear in the current database. Therefore, the predicted concurrency interval can be used to expand the actual concurrency interval of the current database to obtain the target concurrency interval, and the number of hash buckets corresponding to the target concurrency interval is used as the number of hash buckets corresponding to the estimated concurrency, so as to ensure that when the current database experiences network configuration fluctuations, etc., in the face of sudden high concurrency scenarios, the number of split hash buckets is still sufficient to meet business needs well.

[0143] See attached Figure 7 In some embodiments, after protecting the critical resources of each hash bucket using the independent mutex lock corresponding to each hash bucket, the following may be further included:

[0144] S701: constructing a log stream based on a time sequence number according to changes in the global active read-write transaction list in the hash bucket;

[0145] S702: Obtain the latest log stream according to a preset time period to form a copy of the global active read-write transaction list at the current moment. In response to a read view creation operation, obtain the latest log stream to form a copy of the global active read-write transaction list at the read view creation moment, thereby achieving a global active read-write transaction snapshot.

[0146] It can be understood that in some embodiments, by modifying the global mutex in the prior art solution, the granularity requirements of the transaction system lock TrxSysMutex in the MVCC read view creation process can be reduced. Specifically, the changes in the active transaction number list of the transaction system are formed into a log stream based on the time sequence number, wherein each log entry in the log stream is designed as an atomic variable (the operation on the atomic variable is atomic, either completely successful, or the operation is not started at all, and there will be no simultaneous reading and writing of the atomic variable). When the computer is processing a read and write of the atomic variable, other concurrent reads and writes of the atomic variable are all blocked by the computer and the operating system's own mechanism, and there is no need to design additional locks to avoid concurrent reading and writing. Each log entry is a simple number that can represent the change record of the active transaction number list, so it can be designed as an atomic variable. Each log entry is a number that records the changes in the active transaction number list at that time. Then, by reading the log stream composed of log entries from front to back in sequence, the state of the active transaction number list at each moment can be traced back, so each log entry can also be regarded as a time sequence number.

[0147] Under normal circumstances, the design of log entries depends on what they record. Generally, they record what happened at a certain moment, so they contain at least two elements: timestamp and content. However, due to the particularity of the current solution and scenario, the changed content can be represented by only a number of the current change, without the need to specifically consider the two elements of timestamp and content. Since the log itself is recorded sequentially, the latest status of the active transaction number list can be obtained only through the latest log entry, without the need to first lock the current active transaction number list and then copy it to obtain it.

[0148] Furthermore, in some embodiments, when obtaining the data of the current active transaction number list, it is no longer necessary to obtain the transaction system lock and copy it, but only to obtain the latest log entry (atomic variable) of the log stream, and according to the state of the active transaction number list at the historical moment, apply the log stream through an asynchronous thread to obtain the current active transaction number list data, as shown in the following process, which replaces the "process involving transaction system locks in the process of creating a read view" in the prior art solution:

[0149] ① Get the transaction system lock TrxSysMutex (if other concurrent threads hold it, it will be blocked);

[0150] ②Copy the active transaction number list rw_trx_ids of the current transaction system;

[0151] ③ Release the transaction system lock TrxSysMutex;

[0152] Transformed into:

[0153] ① Get the log entry Tr of the current active transaction number list rw_trx_ids (log entry Tr is an atomic variable and does not require locking);

[0154] ② Get the rw_trx_ids status Th read-write lock of the active transaction number list at the historical moment (reading and reading are not mutually exclusive, the update frequency of Th at the historical moment is low, and there is almost no blocking);

[0155] ③Copy the active transaction number list rw_trx_ids data at the historical moment and release the read-write lock;

[0156] ④ Based on the active transaction number list rw_trx_ids data at the historical moment, apply the log stream to obtain the current rw_trx_ids data.

[0157] This makes the "MVCC global read snapshot creation process" lock-free. The MVCC read view no longer needs to obtain the system lock of the active transaction list for copying. Instead, it directly reads the copy of the active transaction list formed based on the log stream, avoiding the blocking of other parallel transactions caused by obtaining the global read-write lock or the blocking of the transaction itself by other transactions holding the global read-write lock, which greatly reduces the lock contention.

[0158] It should be noted that, although the operations of the method of the present invention are described in a specific order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0159] Corresponding to the above-mentioned database transaction system lock contention optimization method, some embodiments of this specification also provide a database transaction system lock contention optimization device, referring to Figure 8 As shown, in some embodiments, the apparatus may include:

[0160] The acquisition module 801 is used to acquire the current global active read-write transaction list and transaction auxiliary index set;

[0161] A mapping module 802, configured to map the global active read-write transaction list and the transaction auxiliary index set into a hash data structure;

[0162] The splitting module 803 is used to determine the number of hash buckets and the hash bucket sequence numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced splitting strategy according to the number of hash buckets and the hash bucket sequence numbers;

[0163] The optimization module 804 is used to protect the critical resources of each hash bucket by using the independent mutex lock corresponding to each hash bucket.

[0164] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0165] It should be noted that the computer program product described in this specification is a software product that mainly implements the method described in this specification through a computer program.

[0166] The embodiments of this specification also provide a computer device. Fig. 9 As shown, in some embodiments of the present specification, the computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 902 may also include any memory 906, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 906 and can be run on the processor 904, when the computer program is run by the processor 904, the instructions of the method described in any of the above embodiments may be executed. Non-limiting, for example, the memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes an associated instruction stored in any memory or a combination of memories, the computer device 902 may perform any operation of the associated instruction. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0167] The computer device 902 may also include an input / output interface 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output interface 910 (I / O), input device 912, and output device 914 may not be included, and the computer device 902 may be used as a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0168] The communication link 922 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0169] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0170] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0171] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0172] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0173] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0174] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0175] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0176] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0177] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0178] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0179] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0180] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A database transaction system lock contention optimization method, characterized in that: The method comprises: Get the current global active read-write transaction list and transaction auxiliary index set; Mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure; Determine the number of hash buckets and hash bucket serial numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced split strategy according to the number of hash buckets and hash bucket serial numbers; wherein each hash bucket has an independent mutex lock; The critical resources of each hash bucket are protected by an independent mutex lock corresponding to each hash bucket.

2. The method according to claim 1, characterized in that The isolation level of the database transaction includes at least a read committed level.

3. The method according to claim 1, characterized in that Mapping the global active read-write transaction list and the transaction auxiliary index set into a hash data structure includes: The index relationship between the global active read-write transaction list and the transaction auxiliary index set is maintained unchanged, and the global active read-write transaction list and the transaction auxiliary index set are hashed to obtain a hash data structure with an index relationship.

4. The method according to claim 1, characterized in that: Determine the number of hash buckets and hash bucket sequence numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced split strategy according to the number of hash buckets and hash bucket sequence numbers, including: Determine the number of hash buckets according to the interval of the estimated concurrency; Determine the hash bucket sequence number according to the number of hash buckets; Divide the global active read and write transactions by the number of hash buckets to perform a remainder operation to obtain a remainder result; The hash bucket sequence number corresponding to each active read / write transaction is determined according to the modulo result, so as to distribute the global active read / write transactions to multiple hash buckets.

5. The method according to claim 4, characterized in that The number of hash buckets is determined according to the estimated concurrency range, including: Get the actual concurrency range of the current database; According to the preset hash bucket quantity matching table, the number of hash buckets corresponding to the actual concurrency interval is determined.

6. The method according to claim 5, characterized in that Determining the number of hash buckets according to the interval of the estimated concurrency further includes: Obtain the maximum load concurrency, historical actual concurrency, and historical network status and hardware configuration corresponding to the historical actual concurrency of a database with the same business scenario as the current database; Establishing a regression analysis model between the historical actual concurrency, the historical network status and hardware configuration, and the maximum load concurrency; Input the current database's network status, hardware configuration, and maximum concurrent load into the regression analysis model to obtain the predicted concurrent load of the current database, so as to determine the predicted concurrent load range. Determine a target concurrency interval using the predicted concurrency interval and the actual concurrency interval; the actual concurrency interval is a subset of the target concurrency interval; According to the preset hash bucket quantity matching table, the number of hash buckets corresponding to the target concurrency interval is determined.

7. The method according to claim 1, characterized in that After using the independent mutex lock corresponding to each hash bucket to protect the critical resources of each hash bucket, it further includes: According to the changes of the global active read-write transaction list in the hash bucket, a log stream based on the time sequence number is constructed; The latest log stream is obtained according to the preset time period to form a copy of the global active read-write transaction list at the current moment. The latest log stream is obtained in response to the read view creation operation to form a copy of the global active read-write transaction list at the time of read view creation, thereby realizing a global active read-write transaction snapshot.

8. A database transaction system lock contention optimization device, characterized in that: The device comprises: The acquisition module is used to obtain the current global active read and write transaction list and transaction auxiliary index set; A mapping module, used to map the global active read-write transaction list and the transaction auxiliary index set into a hash data structure; A splitting module is used to determine the number of hash buckets and hash bucket serial numbers according to the estimated concurrency of the current database, and split the global active read and write transactions into multiple hash buckets based on the balanced splitting strategy according to the number of hash buckets and hash bucket serial numbers; wherein each hash bucket has an independent mutex lock; The optimization module is used to protect the critical resources of each hash bucket by using the independent mutex lock corresponding to each hash bucket.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 7.

10. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes instructions of the method according to any one of claims 1 to 7.

11. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program executes instructions of the method according to any one of claims 1 to 7.

Images