A lightweight real-time in-memory database transaction concurrency control method and device

Abstract

The application relates to the technical field of flexible direct current power transmission, and particularly provides a lightweight real-time memory database transaction concurrent control method and device, which comprises the following steps: S101. Classifying to-be-processed transactions through cluster analysis to determine the cluster to which the to-be-processed transactions belong; S102. Obtaining a transaction lock corresponding to the to-be-processed transaction in the cluster to which the to-be-processed transaction belongs and a data lock called by the to-be-processed transaction, and executing S103 under the condition that the acquisition state is successful; S103. Conflict verification is performed on the to-be-processed transaction, if there is a conflict, the to-be-processed transaction is prevented, otherwise, the to-be-processed transaction is executed. The technical scheme provided by the application meets the demand of high-efficiency access of Internet of Things data and improves the transaction execution performance of the real-time memory database.

Description

Technical Field

[0001] This invention relates to the field of flexible DC transmission technology, specifically to a lightweight real-time in-memory database transaction concurrency control method and apparatus. Background Technology

[0002] With the widespread adoption of computer applications in recent years, disk-based databases (DRDBs) have seen significant development and are widely used in business and transactional applications. However, over time, some problems with disk-based databases have become apparent, leading to unsatisfactory performance for applications with high real-time requirements. The main reason for these problems is that in real-time scenarios, a large number of transactions are established within the same timeframe, interacting with the disk and performing numerous input / output operations. However, buffer management related to transaction execution is unpredictable and lacks strong real-time performance. The same issues arise when handling page faults and other events, resulting in poor real-time performance and predictability. With the continuous development of modern technology, the integration of storage chips has increased, significantly improving computer main memory capacity. Current computer systems offer substantial main memory capacity while continuously reducing costs. Based on these conditions, performing data operations in memory becomes feasible, leading to the emergence of Main Memory Databases (MMDBs). Simultaneously, with the deep integration of industry and information technology, the Internet of Things (IoT) has become a new economic industry and application ecosystem, representing an economic and technological reform that uses internet information technology and advanced intelligent technologies to promote industrial upgrading and transformation. The IoT relies heavily on data. In industrial automated production processes, the data generated, collected, and processed by industrial equipment far exceeds the amount of data processed by computers. The data types are unstructured, and the high-speed operation of production lines demands strict real-time performance. Traditional relational in-memory databases alone cannot meet the demands for storing, processing, and analyzing the massive amounts of data generated during industrial production control, as well as the high timeliness of the data. Therefore, Real-Time Database (RTDB) technology, as a new branch of database system development, has been applied in the industrial field. RTDB is a new type of database resulting from the combination of real-time processing technology and database technology. It integrates real-time tasks and traditional database transactions, with real-time performance as its prominent feature, including data real-time and transaction real-time performance. However, most RTDBs still have shortcomings in transaction processing efficiency, which limits the use of related RTDB products. Against the backdrop of the booming development of the IoT, RTDs have gradually developed, but most only support the storage, processing, and analysis of data values ​​with relatively few data types, failing to fully meet the needs of current industrial process control systems, which have diverse data types, complex sources, and rapid update speeds. Therefore, in order to meet the business needs of efficient collection and storage of IoT data, this paper optimizes the high-concurrency processing efficiency of in-memory database transactions and implements a lightweight real-time in-memory database transaction concurrency control method to improve the reliability and efficiency of the database, which has broad prospects.

[0003] Currently, the patent CN109376165A, entitled "Implementation Method and Apparatus of Memory Database Lock and Computer-Readable Storage Medium," is applicable to the field of database technology. It provides an implementation method, apparatus, and computer-readable storage medium for memory database locks. The method includes: receiving a transaction processing request for a memory database; the transaction processing request includes information about a transaction to be processed; obtaining the value of the lock zone of the element corresponding to the transaction in the memory database based on the transaction information, and determining the lock holding status of the element according to the lock zone value; wherein the element includes an index node and row data; and adding a read lock or a write lock to the element based on the lock holding status and the operation type of the element contained in the transaction. Based on the above method, high-concurrency transaction processing in a memory database can be achieved, suitable for handling single-transaction latency and overall throughput. This solution is relatively simple to implement, but it does not consider the management relationship between various transactions, and the granularity of the locks is too large, which can easily lead to a decrease in concurrency. Summary of the Invention

[0004] To overcome the above-mentioned shortcomings, this invention proposes a lightweight real-time in-memory database transaction concurrency control method and apparatus.

[0005] Firstly, a lightweight real-time in-memory database transaction concurrency control method is provided, the method comprising:

[0006] Step S101. Classify the transactions to be processed by cluster analysis to determine the cluster to which the transactions belong;

[0007] Step S102. Obtain the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, proceed to step S103.

[0008] Step S103. Perform conflict verification on the pending transaction. If a conflict exists, block the pending transaction; otherwise, execute the pending transaction.

[0009] Preferably, step S101 includes:

[0010] Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed;

[0011] The hash value of the working set is determined using the MinHash algorithm;

[0012] The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table;

[0013] The target cluster is designated as the cluster to which the transaction to be processed belongs.

[0014] Furthermore, the step of obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes:

[0015] Perform static pre-compilation on the transaction to be processed;

[0016] Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access;

[0017] The working set corresponding to the transaction to be processed is generated based on all the tasks and the data records that each task needs to access.

[0018] Preferably, acquiring the transaction lock corresponding to the transaction to be processed in its cluster and the data lock for the data accessed by the transaction to be processed includes:

[0019] Determine whether the data corresponding to the pending transaction is locked;

[0020] If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful.

[0021] Furthermore, after determining whether the data corresponding to the transaction to be processed is locked, the method further includes:

[0022] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0023] If so, mark the acquisition status as failed.

[0024] Furthermore, after determining whether the data corresponding to the transaction to be processed is locked, the method further includes:

[0025] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0026] If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

[0027] Preferably, in step S103, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

[0028] Secondly, a lightweight real-time in-memory database transaction concurrency control device is provided, the lightweight real-time in-memory database transaction concurrency control device comprising:

[0029] The classification module is used to categorize the transactions to be processed through cluster analysis and determine the cluster to which the transactions belong.

[0030] The acquisition module is used to acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, it enters the verification module.

[0031] The verification module is used to perform conflict verification on the pending transaction. If a conflict exists, the pending transaction is blocked; otherwise, the pending transaction is executed.

[0032] Preferably, the classification module is specifically used for:

[0033] Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed;

[0034] The hash value of the working set is determined using the MinHash algorithm;

[0035] The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table;

[0036] The target cluster is designated as the cluster to which the transaction to be processed belongs.

[0037] Furthermore, the step of obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes:

[0038] Perform static pre-compilation on the transaction to be processed;

[0039] Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access;

[0040] The working set corresponding to the transaction to be processed is generated based on all the tasks and the data records that each task needs to access.

[0041] Preferably, the acquisition module is specifically used for:

[0042] Determine whether the data corresponding to the pending transaction is locked;

[0043] If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful.

[0044] Furthermore, after the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes:

[0045] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0046] If so, mark the acquisition status as failed.

[0047] Furthermore, after the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes:

[0048] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0049] If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

[0050] Preferably, in the verification module, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

[0051] Thirdly, a lightweight real-time memory database is provided, wherein the lightweight real-time memory database includes at least the aforementioned lightweight real-time memory database transaction concurrency control device.

[0052] Fourthly, a computer device is provided, comprising: one or more processors;

[0053] The processor is used to store one or more programs;

[0054] When the one or more programs are executed by the one or more processors, the lightweight real-time memory database transaction concurrency control method is implemented.

[0055] Fifthly, a computer-readable storage medium is provided, on which a computer program is stored, wherein when the computer program is executed, the lightweight real-time memory database transaction concurrency control method is implemented.

[0056] The above-described technical solutions of the present invention have at least one or more of the following beneficial effects:

[0057] This invention provides a lightweight real-time in-memory database transaction concurrency control method and apparatus, comprising: step S101. classifying transactions to be processed through cluster analysis to determine the cluster to which the transactions belong; step S102. acquiring the transaction lock corresponding to the transaction to be processed in its cluster and the data lock for the data called by the transaction to be processed; if the acquisition status is successful, proceeding to step S103; step S103. performing conflict verification on the transactions to be processed; if a conflict exists, blocking the transactions to be processed; otherwise, executing the transactions to be processed. This scheme first designs a transaction clustering analysis process, providing strong support for precise isolation and partitioning of database transactions, effectively reducing the probability of conflicts between transactions and improving the success rate of transaction execution. Secondly, it proposes a cluster-based transaction lock mechanism, optimizing the granularity of common transaction locks, reducing the waiting time required for transaction lock contention, and improving transaction execution efficiency while ensuring data consistency. Finally, based on the above algorithms and mechanisms, it proposes a concurrency control mechanism combined with an optimistic control algorithm to improve the real-time in-memory database transaction concurrency control capability and ensure the efficient and stable operation of the in-memory database. The concurrency control method is effectively applied in real-time memory databases for the Internet of Things (IoT) through specific application examples, which effectively improves the operating efficiency and reliability of IoT production systems. Attached Figure Description

[0058] Figure 1 This is a schematic diagram of the main steps of the lightweight real-time in-memory database transaction concurrency control method according to an embodiment of the present invention;

[0059] Figure 2 This is a flowchart of transaction clustering based on locality-sensitive hashing according to an embodiment of the present invention;

[0060] Figure 3 This is a diagram illustrating the lock acquisition and release process of the transaction lock mechanism based on cluster locking in an embodiment of the present invention.

[0061] Figure 4 This is a process diagram of the concurrency control technology based on transaction clustering according to an embodiment of the present invention;

[0062] Figure 5 This is a block diagram of the main structure of the lightweight real-time in-memory database transaction concurrency control device according to an embodiment of the present invention. Detailed Implementation

[0063] The specific embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.

[0064] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0065] Example 1

[0066] See appendix Figure 1 , Figure 1 This is a schematic flowchart illustrating the main steps of a lightweight real-time in-memory database transaction concurrency control method according to an embodiment of the present invention. Figure 1 As shown, the lightweight real-time in-memory database transaction concurrency control method in this embodiment of the invention mainly includes the following steps:

[0067] Step S101. Classify the transactions to be processed by cluster analysis to determine the cluster to which the transactions belong;

[0068] Step S102. Obtain the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, proceed to step S103.

[0069] Step S103. Perform conflict verification on the pending transaction. If a conflict exists, block the pending transaction; otherwise, execute the pending transaction.

[0070] In this embodiment, to establish the optimal isolation mechanism for any given transaction and reduce the probability of conflicts during real-time database transaction execution, a transaction clustering algorithm based on locality-sensitive hashing is proposed. This method uses the locality-sensitive hashing algorithm to measure the similarity of transactions, and combines it with a hash table mechanism to pre-cluster and partition the transaction set. This can significantly reduce conflicts between transactions under high contention load conditions, avoid transaction abortion, and improve transaction execution performance. Specifically, step S101 includes:

[0071] Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed;

[0072] The hash value of the working set is determined using the MinHash algorithm;

[0073] The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table;

[0074] The target cluster is designated as the cluster to which the transaction to be processed belongs. In one embodiment, obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes:

[0075] Perform static pre-compilation on the transaction to be processed;

[0076] Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access;

[0077] The working set corresponding to the transaction to be processed is generated based on all the tasks and the data records that each task needs to access.

[0078] In an optimal implementation, real-time in-memory databases involve rich data, transaction semantics, and complex structures, exhibiting many characteristics different from traditional applications. During transaction processing, data and resource conflicts and inter-transaction dependencies must be considered. If real-time databases can consciously cluster and isolate transactions based on their probability of conflict, it can support the rational arrangement of transaction execution sequences and management methods, improving the success rate of transaction execution and thus enhancing system efficiency. Since the probability of conflict increases when two transactions share access to more datasets, transaction clustering can be achieved through the Jaccard similarity between the working sets of two transactions. The working set of a transaction is defined as the set of all data records accessed by that transaction. The MinHash algorithm compares two hash values ​​to reveal the similarity between sets A and B; two sets with high similarity typically have the same hash value. The process is as follows... Figure 2 As shown, it specifically includes:

[0079] (1) Before the transaction arrives at the real-time in-memory database processing module, the transaction is statically pre-compiled to extract all task sets in the transaction and the data records that each task needs to access, and finally generate all working sets of the transaction.

[0080] (2) The hash value of the transaction working set is determined by using the MinHash algorithm. First, the extracted feature value vector and the working set are combined into a matrix. The feature value vector matrix is ​​operated by multiple hash functions to generate a "signature matrix". Then, the row numbers in the matrix are randomly transformed and arranged. The first 1 in the column corresponding to the working set is taken out. After multiple random hash transformations, multiple feature values ​​can be selected to represent the working set, thereby reducing the dimensionality of the working set. The high-dimensional working set is compressed into 16 integers, which is the minimum 16-bit hash value of the working set.

[0081] (3) Use a hash table to find the cluster for the incoming transaction. The hash table is implemented using an associative container. For any input key value, it can quickly locate a position in the hash table to access the data record, that is, obtain the cluster information to which the transaction belongs.

[0082] (4) Add cluster information to the incoming transaction to complete the isolation of different types and realize the transaction clustering task.

[0083] Furthermore, this embodiment addresses the problem of low concurrency caused by the overly coarse granularity of traditional real-time in-memory database transaction locks. It proposes a cluster-based transaction lock mechanism that automatically isolates the current transaction from other transactions in the same cluster. Based on a two-stage lock structure, a suitable transaction lock table is designed and implemented to add locks with optimal granularity to transactions. This improves the concurrency of the real-time in-memory database while ensuring normal transaction execution. In step S102, the transaction lock corresponding to the transaction to be processed in its cluster and the data lock for the data accessed by the transaction to be processed are obtained, including:

[0084] Determine whether the data corresponding to the pending transaction is locked;

[0085] If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful.

[0086] After determining whether the data corresponding to the transaction to be processed is locked, the method further includes:

[0087] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0088] If so, mark the acquisition status as failed.

[0089] After determining whether the data corresponding to the transaction to be processed is locked, the method further includes:

[0090] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0091] If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

[0092] In one implementation, a transaction is a program execution unit that queries and updates various data in a real-time in-memory database, and it is the basic unit of concurrency control. When handling concurrent transactions, lock-based concurrency control is commonly used. In lock-based management mechanisms, locks are categorized into row locks, table locks, and database-level locks, depending on their granularity. Currently, adding row locks to an in-memory database is complex to implement when handling concurrent transactions and cannot meet the needs of applications with stringent latency requirements. While adding database-level or table locks to an in-memory database is simpler to implement when handling concurrent transactions, it results in low concurrency and low system throughput. Therefore, based on transaction clustering, a simple, novel, and highly concurrent cluster-based transaction locking mechanism is proposed.

[0093] The lock table design of this transaction locking mechanism is similar to that of 2PL. The first layer indicates whether a data record is locked by a related transaction, and the second layer represents information about each transaction cluster. By searching the lock table, the current input transaction can be directly notified whether other transactions belonging to the same cluster and holding locks exist. If not, the lock can be granted to the requesting transaction. Otherwise, the lock will be granted if and only if the requested locking mode meets the current requirements. The lock compatibility matrix of this mechanism is exactly the same as that of the 2PL algorithm. The lock acquisition and release processes of this mechanism are as follows: Figure 3 As shown, it specifically includes:

[0094] (1) First, for the input real-time transactions in the memory database, obtain its working set, and use the transaction clustering algorithm based on locality-sensitive hashing mentioned above to divide the transactions into their respective clusters.

[0095] (2) Next, a transaction begins execution, requests to acquire the relevant data lock, and determines whether the data has been locked.

[0096] (3) If the data has been locked, proceed to step (4) to determine whether the data lock has been acquired by a transaction in the same cluster. Otherwise, the transaction successfully acquires the data lock and performs data operations. After the transaction is completed, release the data lock and the cluster lock.

[0097] (4) If the data lock is acquired by a transaction in the same cluster, i.e. the transaction cluster is already locked, the transaction fails to acquire the data lock and begins to queue for re-execution. Otherwise, the transaction successfully acquires the data lock and performs data operations. After the transaction is completed, it releases the data lock and the cluster lock.

[0098] In this embodiment, to address the problems of long transaction execution waiting time due to severe transaction lock contention in the pessimistic algorithm and high transaction abort rate leading to numerous repeated executions in the optimistic algorithm, a concurrency control mechanism combining pessimistic and optimistic control algorithms is proposed to alleviate the performance bottlenecks of these two algorithms. This mechanism utilizes the aforementioned transaction clustering algorithm and transaction lock mechanism, employing a pessimistic control algorithm for transactions within the same cluster and an optimistic control algorithm for transactions between different clusters. This effectively reduces the waiting time and abort rate required for transaction execution, improving the efficiency and practicality of real-time in-memory database concurrency control. In step S103, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

[0099] In one implementation, concurrency control is a mechanism provided by a database system that allows multiple transactions to execute simultaneously while maintaining data consistency, thereby improving database reliability and efficiency. Currently, mainstream concurrency control algorithms mainly fall into two categories: pessimistic algorithms and optimistic algorithms. Pessimistic algorithms require transactions to hold locks for their entire lifecycle, while optimistic algorithms only acquire locks upon transaction commit. Therefore, the lock holding time in pessimistic algorithms is typically much longer than in optimistic algorithms, leading to more severe lock contention. Research shows that lock contention is the main performance bottleneck of pessimistic algorithms, as transactions may be unable to perform any operations during lock contention due to long waiting times. In optimistic algorithms, transaction aborts primarily occur due to verification failures during transaction commit. For highly competitive workloads, almost all transactions will be aborted by optimistic control algorithms, resulting in a high abort rate. Therefore, based on the aforementioned transaction clustering algorithm based on locality-sensitive hashing and the transaction locking mechanism based on cluster locking, a concurrency control mechanism combining pessimistic and optimistic algorithms is proposed. This organically integrates the advantages of these two algorithms while overcoming their disadvantages. The transaction clustering-based concurrency control technology mainly includes three steps: transaction clustering, requesting and acquiring data locks within the cluster, and conflict verification. The process is as follows: Figure 4 As shown, it specifically includes:

[0100] (1) First, the transaction clustering algorithm based on locality-sensitive hashing described above is used to perform clustering analysis on the input transactions.

[0101] (2) Secondly, the data operation of a transaction is controlled by the above-mentioned cluster-based transaction lock mechanism. Before accessing any data, the transaction requests to acquire the data lock of the cluster. When other transactions in the cluster hold the data lock, the access request will be blocked until the lock is granted.

[0102] (3) Finally, even if transactions in the same cluster use lock isolation, access from transactions from different clusters is not controlled, which may lead to data inconsistency. This mechanism requires any transaction to perform conflict verification in optimistic concurrency control when it is committed, and if a conflict exists, the transaction must be stopped.

[0103] Example 2

[0104] Based on the same inventive concept, this invention also provides a lightweight real-time in-memory database transaction concurrency control device, such as... Figure 5 As shown, the lightweight real-time in-memory database transaction concurrency control device includes:

[0105] The classification module is used to categorize the transactions to be processed through cluster analysis and determine the cluster to which the transactions belong.

[0106] The acquisition module is used to acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, it enters the verification module.

[0107] The verification module is used to perform conflict verification on the pending transaction. If a conflict exists, the pending transaction is blocked; otherwise, the pending transaction is executed.

[0108] Preferably, the classification module is specifically used for:

[0109] Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed;

[0110] The hash value of the working set is determined using the MinHash algorithm;

[0111] The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table;

[0112] The target cluster is designated as the cluster to which the transaction to be processed belongs.

[0113] Furthermore, the step of obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes:

[0114] Perform static pre-compilation on the transaction to be processed;

[0115] Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access;

[0116] The working set corresponding to the transaction to be processed is generated based on all the tasks and the data records that each task needs to access.

[0117] Preferably, the acquisition module is specifically used for:

[0118] Determine whether the data corresponding to the pending transaction is locked;

[0119] If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful.

[0120] Furthermore, after the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes:

[0121] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0122] If so, mark the acquisition status as failed.

[0123] Furthermore, after the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes:

[0124] If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs;

[0125] If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

[0126] Preferably, in the verification module, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

[0127] Example 3

[0128] Based on the same inventive concept, the present invention also provides a lightweight real-time memory database, wherein the lightweight real-time memory database includes at least the aforementioned lightweight real-time memory database transaction concurrency control device.

[0129] Example 4

[0130] Based on the same inventive concept, this invention also provides a computer device, which includes a processor and a memory. The memory stores a computer program, which includes program instructions. The processor executes the program instructions stored in the computer storage medium. The processor may be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. It is the computing and control core of the terminal, suitable for implementing one or more instructions, specifically suitable for loading and executing one or more instructions from the computer storage medium to implement corresponding method flows or corresponding functions, thereby implementing the steps of a lightweight real-time memory database transaction concurrency control method in the above embodiments.

[0131] Example 5

[0132] Based on the same inventive concept, this invention also provides a storage medium, specifically a computer-readable storage medium (Memory), which is a memory device in a computer device used to store programs and data. It is understood that the computer-readable storage medium here can include both the built-in storage medium in the computer device and extended storage media supported by the computer device. The computer-readable storage medium provides storage space that stores the terminal's operating system. Furthermore, this storage space also stores one or more instructions suitable for loading and execution by a processor. These instructions can be one or more computer programs (including program code). It should be noted that the computer-readable storage medium here can be high-speed RAM or non-volatile memory, such as at least one disk storage device. The processor can load and execute one or more instructions stored in the computer-readable storage medium to implement the steps of the lightweight real-time memory database transaction concurrency control method in the above embodiments.

[0133] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0134] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0135] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0136] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0137] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. A lightweight real-time in-memory database transaction concurrency control method, characterized in that, The method includes: Step S101. Classify the transactions to be processed by cluster analysis to determine the cluster to which the transactions belong; Step S102. Obtain the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, proceed to step S103. Step S103. Perform conflict verification on the pending transaction. If a conflict exists, block the pending transaction; otherwise, execute the pending transaction. Step S101 includes: Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed; The hash value of the working set is determined using the MinHash algorithm; The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table; The target cluster is designated as the cluster to which the transaction to be processed belongs. The step of obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes: Perform static pre-compilation on the transaction to be processed; Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access; A working set corresponding to the pending transaction is generated based on all the tasks and the data records that each task needs to access. The step of acquiring the transaction lock corresponding to the transaction to be processed in its cluster and the data lock for the data called by the transaction to be processed includes: Determine whether the data corresponding to the pending transaction is locked; If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful; After determining whether the data corresponding to the transaction to be processed is locked, the method further includes: If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs; If so, mark the acquisition status as failed; After determining whether the data corresponding to the transaction to be processed is locked, the method further includes: If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs; If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

2. The method as described in claim 1, characterized in that, In step S103, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

3. A lightweight real-time in-memory database transaction concurrency control device, characterized in that, The device includes: The classification module is used to categorize the transactions to be processed through cluster analysis and determine the cluster to which the transactions belong. The acquisition module is used to acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed. If the acquisition status is successful, it enters the verification module. The verification module is used to perform conflict verification on the pending transaction. If a conflict exists, the pending transaction is blocked; otherwise, the pending transaction is executed. The classification module is specifically used for: Obtain the transactions to be processed and extract the working set corresponding to the transactions to be processed; The hash value of the working set is determined using the MinHash algorithm; The target cluster is obtained by looking up the hash value of the working set in a pre-built hash table; The target cluster is designated as the cluster to which the transaction to be processed belongs. The step of obtaining the transaction to be processed and extracting the working set corresponding to the transaction to be processed includes: Perform static pre-compilation on the transaction to be processed; Extract at least one task from the statically pre-compiled transaction to be processed, as well as the data records that each task needs to access; A working set corresponding to the pending transaction is generated based on all the tasks and the data records that each task needs to access. The acquisition module is specifically used for: Determine whether the data corresponding to the pending transaction is locked; If the data corresponding to the pending transaction is not locked, then acquire the transaction lock corresponding to the pending transaction in its cluster and the data lock of the data called by the pending transaction, and mark the acquisition status as successful; After the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes: If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs; If so, mark the acquisition status as failed; After the step of the acquisition module determining whether the data corresponding to the transaction to be processed is locked, the device further includes: If the data corresponding to the pending transaction is locked, then determine whether the locked data has been acquired by a similar transaction in the cluster to which the pending transaction belongs; If not, acquire the transaction lock corresponding to the transaction to be processed in its cluster and the data lock of the data called by the transaction to be processed, and mark the acquisition status as successful.

4. The apparatus as described in claim 3, characterized in that, In the verification module, the conflict verification is the conflict verification in the optimistic concurrency control algorithm.

5. A lightweight real-time in-memory database, characterized in that, The lightweight real-time memory database includes at least the lightweight real-time memory database transaction concurrency control device as described in any one of claims 3-4.

6. A computer device, characterized in that, include: One or more processors; The processor is used to store one or more programs; When the one or more programs are executed by the one or more processors, the lightweight real-time memory database transaction concurrency control method as described in any one of claims 1 to 2 is implemented.

7. A computer-readable storage medium, characterized in that, It contains a computer program, which, when executed, implements the lightweight real-time memory database transaction concurrency control method as described in any one of claims 1 to 2.

Images