Global distributed transactions across microservices
The global transaction system addresses the challenge of consistent and high-throughput distributed transactions across microservices by using an OCC protocol to capture and coordinate transactions across multiple databases, enhancing efficiency and reducing resource usage.
Patent Information
- Application Number
- CN202080008955.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-05-01
- Filing Date
- 2020-01-21
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-01-21
AI Technical Summary
Existing systems face challenges in implementing consistent and high-throughput distributed transactions across multiple microservices that use different languages and access various underlying databases, as traditional methods like two-phase commit lead to transaction conflicts and delays, while other methods require additional framework and business logic, increasing costs and affecting user experience.
A global transaction system utilizing an optimistic concurrency control protocol (OCC) captures read and write sets at each database service point, performs conflict checks, and coordinates global submission decisions, ensuring consistent and high-throughput transactions across multiple databases.
The system enables efficient, scalable, and serializable distributed transactions by reducing the need for locks and resource usage, minimizing delays and costs associated with traditional methods.
Smart Images

Figure CN113272790B_ABST
Abstract
Description
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS
[0002] This application claims priority to U.S. Application Serial No. 16 / 400,364, filed on May 1, 2019, which claims priority to U.S. Provisional Patent Application Serial No. 62 / 804,319, filed on February 12, 2019, each of which is incorporated herein by reference in its entirety. Background Art Technical Field
[0004] The subject matter disclosed herein generally relates to special-purpose machines including systems for consistent distributed transactions across microservices, including computerized variations of such special-purpose machines and improvements to such variations, and to techniques by which such special-purpose machines are improved over other special-purpose machines.
[0005] Microservice architecture is used in large-scale cloud data platform and application development. Microservice architecture provides flexibility for reuse of fine-grained services and application development. Microservices can be developed by different domain teams to support business applications. They can be implemented in various languages such as Java or Golang and can access multiple underlying databases. Applications in microservice architecture usually need to enable multiple microservices, which access multiple databases. When an application enables multiple microservices, the application relies on distributed transactions to perform consistent updates on the underlying database. BRIEF DESCRIPTION OF THE DRAWINGS
[0006] To easily identify the discussion of any particular element or act, the highest digit or digits in a reference number refer to the figure number in which the element is first introduced.
[0007] Figure 1 is a pictorial representation of a networked environment in which the present disclosure may be deployed, according to some example implementations.
[0008] Figure 2 is a block diagram illustrating an example of an application provided as part of a networked system in one example implementation.
[0009] Figure 3 is a block diagram illustrating an example of an architecture for distributed transactions according to one example implementation.
[0010] Figure 4 is a block diagram illustrating a global transaction system provided as part of a networked system in one example implementation.
[0011] Figure 5 is a block diagram illustrating an example of a database transaction log and a log player according to one embodiment.
[0012] Figure 6 is a block diagram illustrating example operations of a global transaction system according to one example implementation.
[0013] Figure 7 An example of various stages of a global transaction system according to one embodiment is shown.
[0014] Figure 8 A flow chart illustrating an example operation of a global transaction system according to one embodiment is shown.
[0015] Fig. 9 is a block diagram illustrating another example operation of the global transaction system according to one example implementation.
[0016] Fig.10 An example of the scalability of a global transaction system according to one embodiment is shown.
[0017] Fig.11 A flow chart of a method for operating a global transaction system according to one embodiment is shown.
[0018] Fig.12 A flow chart of a method for operating a global transaction system according to one embodiment is shown.
[0019] Fig.13 A process according to one embodiment is shown.
[0020] Fig.14 An example of transaction execution against multiple databases according to one embodiment is shown.
[0021] Fig.15 is a diagrammatic representation of a machine in the form of a computer system within which a set of instructions may be executed, causing the machine to perform any one or more of the methodologies discussed herein, according to an example embodiment. DETAILED DESCRIPTION
[0022] "Component" refers to a device, physical entity or logic with boundaries defined by function or subroutine calls, branch points, APIs or other technologies provided for partitioning or modularizing specific processing or control functions. Components can be combined with other components via their interfaces to perform machine processing. Components can be part of a program that encapsulates functional hardware units designed for use with other components and generally performs specific functions of related functions. Components can constitute software components (e.g., codes embodied on machine-readable media) or hardware components. "Hardware components" are tangible units that can perform certain operations and can be configured or arranged in a certain physical manner. In various example embodiments, one or more computer systems (e.g., independent computer systems, client computer systems or server computer systems) or one or more hardware components (e.g., processors or processor groups) of a computer system can be configured by software (e.g., applications or application parts) to operate to perform certain operations as described herein. Hardware components can also be implemented mechanically, electronically or in any suitable combination thereof. For example, a hardware component may include a dedicated circuit system or logic that is permanently configured to perform certain operations. The hardware component can be a special processor, such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC). The hardware component can also include a programmable logic or circuit system that is temporarily configured to perform certain operations by software. For example, the hardware component can include software executed by a general-purpose processor or other programmable processor. Once configured by such software, the hardware component becomes a specific machine (or a specific component of a machine) that is uniquely customized to perform the configured function, and is no longer a general-purpose processor. It should be understood that the decision to implement the hardware component mechanically, in a dedicated and permanently configured circuit system, or in a temporarily configured circuit system (e.g., configured by software) can be driven by cost and time considerations. Accordingly, the phrase "hardware component" (or "hardware-implemented component") should be understood to include a tangible entity, that is, an entity that is physically constructed, permanently configured (e.g., hardwired) or temporarily configured (e.g., programmed) to operate in some way or perform certain operations described herein. Considering the embodiment in which the hardware component is temporarily configured (e.g., programmed), each of the hardware components does not need to be configured or instantiated at any time. For example, where a hardware component includes a general purpose processor that is configured as a special purpose processor by software, the general purpose processor may be configured as different special purpose processors (e.g., including different hardware components) at different times. Software accordingly configures one or more specific processors to, for example, constitute a specific hardware component at one time and constitute different hardware components at different times. Hardware components may provide information to other hardware components and receive information from other hardware components. Accordingly, the described hardware components may be considered to be communicatively coupled.In the case of multiple hardware components being present at the same time, communication can be achieved by signal transmission (e.g., by appropriate circuits and buses) between or among two or more hardware components. In an embodiment in which multiple hardware components are configured or instantiated at different times, communication between such hardware components can be achieved, for example, by storing information in a memory structure accessed by multiple hardware components and retrieving information in the memory structure. For example, a hardware component can perform an operation, and the output of the operation is stored in a memory device to which it is communicatively coupled. Then, other hardware components can access the memory device at a subsequent time to retrieve and process the stored output. The hardware component can also initiate communication with an input device or an output device, and can operate on resources (e.g., the collection of information). The various operations of the example methods described herein can be performed at least in part by one or more processors that are temporarily configured (e.g., by software) or permanently configured to perform related operations. Whether it is temporarily configured or permanently configured, such a processor can constitute a processor-implemented component that operates to perform one or more operations or functions described herein. As used herein, "processor-implemented components" refer to hardware components implemented using one or more processors. Similarly, the method described herein can be implemented at least in part by a processor, wherein one or more specific processors are examples of hardware. For example, at least some of the operations in the operation of the method can be performed by one or more processors or components implemented by the processor. In addition, one or more processors can also operate to support the execution of related operations in a "cloud computing" environment or operate as "software as a service" (SaaS). For example, at least some of the operations in the operation can be performed by a group of computers (as an example of a machine including a processor), wherein these operations can be accessed via a network (e.g., the Internet) and via one or more appropriate interfaces (e.g., API). The execution of certain operations can be distributed between processors, not only residing in a single machine, but also deployed across multiple machines. In some example embodiments, a processor or a component implemented by a processor can be located in a single geographic location (e.g., in a home environment, an office environment, or a server cluster). In other example embodiments, a processor or a component implemented by a processor can be distributed across multiple geographic locations.
[0023] "Communications network" means one or more parts of a network, which may be an ad hoc network, an intranet, an extranet, a virtual private network (VPN), a local area network (LAN), a wireless LAN (WLAN), a wide area network (WAN), a wireless WAN (WWAN), a metropolitan area network (MAN), the Internet, a part of the Internet, a part of the public switched telephone network (PSTN), a plain old telephone service (POTS) network, a cellular telephone network, a wireless network, The coupling may be a network, another type of network, or a combination of two or more such networks. For example, the network or a portion of the network may include a wireless network or a cellular network, and the coupling may be a code division multiple access (CDMA) connection, a global system for mobile communications (GSM) connection, or other type of cellular or wireless coupling. In this example, the coupling may implement any of various types of data transmission technologies, such as single carrier radio transmission technology (1xRTT), evolution data optimized (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rates for GSM evolution (EDGE) technology, the third generation partnership project (3GPP) including 3G, fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high speed packet access (HSPA), world wide interoperability for microwave access (WiMAX), long term evolution (LTE) standards, other data transmission technologies defined by various standard setting organizations, other long distance protocols, or other data transmission technologies.
[0024] “Machine storage medium” refers to a single or multiple storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions, routines, and / or data. Thus, the term should be considered to include, but is not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include: non-volatile memory, including, for example, semiconductor memory devices such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGA, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage medium,” “device storage medium,” and “computer storage medium” mean the same thing and are used interchangeably in this disclosure. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are encompassed by the term “signal media.”
[0025] A "processor" refers to any circuit or virtual circuit (a physical circuit simulated by logic executed on an actual processor) that manipulates data values according to control signals (e.g., "commands," "opcodes," "machine codes," etc.) and produces corresponding output signals applied to operate a machine. For example, a processor may be a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an application specific integrated circuit (ASIC), a radio frequency integrated circuit (RFIC), or any combination thereof. A processor may also be a multi-core processor having two or more independent processors (sometimes referred to as "cores") that can execute instructions simultaneously.
[0026] "Carrier signal" refers to any intangible medium that can store, encode or carry instructions for execution by a machine and includes digital or analog communications signals or other intangible media to facilitate communication of such instructions. Instructions may be sent or received over a network using a transmission medium via a network interface device.
[0027] "Signal medium" refers to any intangible medium that is capable of storing, encoding or carrying instructions to be executed by a machine, and includes digital or analog communication signals or other intangible media to facilitate the communication of software or data. The term "signal medium" should be considered to include any form of modulated data signals, carrier waves, etc. The term "modulated data signal" means a signal that has one or more of its characteristics set or changed in such a manner as to encode the information in the signal. The terms "transmission medium" and "signal medium" mean the same thing and may be used interchangeably in this disclosure.
[0028] "Computer-readable media" refers to both machine storage media and transmission media. Therefore, these terms include both storage devices / media and carrier / modulated data signals. The terms "machine-readable medium", "computer-readable medium" and "device-readable medium" mean the same thing and can be used interchangeably in this disclosure.
[0029] The following description describes the system, method, technology, instruction sequence and computer program product of the example implementation of the present theme. In the following description, for the purpose of illustration, many specific details are set forth in order to provide an understanding of the various implementations of the present theme. However, it will be apparent to those skilled in the art that the implementation of the present theme can be put into practice without some of these specific details or other details. Examples represent possible variations only. Unless otherwise expressly stated, structures (e.g., structural components such as modules) are optional and can be combined or subdivided, and operations (e.g., in processes, algorithms or other functions) can be changed in sequence or combined or subdivided.
[0030] This application describes a new system that supports consistent distributed transactions across microservices that are implemented in various languages and use multiple underlying databases. The system is efficient and scalable, and provides serializability for transactions across multiple databases exposed by microservices.
[0031] The traditional technology is to use the two-phase commit (2PC) protocol to implement distributed transactions. Unfortunately, for applications with a certain number of transaction conflicts, 2PC does not work well in large-scale high-throughput systems. One reason is that the lock is held during the entire 2PC process, which significantly increases transaction conflicts and delays. Other methods include a persistent message queue pattern for loosely coupled distributed transactions. However, such a persistent message queue pattern requires some framework and application logic to compensate for failed transaction steps, or even business policies to remedy through business measures, which costs business funds and affects user experience. Other systems can implement distributed transactions on a single database. However, these systems cannot be applied across multiple microservices.
[0032] The present application describes a global transaction system that coordinates transactions and processing for distributed transactions across multiple databases. Microservice architectures for applications bring new challenges to consistent distributed transactions across multiple microservices, which use multiple underlying databases to implement their functions. These microservices can be implemented in different languages and access multiple underlying databases. The present application describes a system that solves this challenge by using deterministic technology and an optimistic concurrency control protocol (OCC). During the optimistic execution phase, the read set and write set of the transaction are captured at each point of the relevant database service point, and at the time of submission, conflict checks are performed at each database level, and global submission decisions are implemented by coordinating the databases involved. The logically committed transaction (write set) is first saved in the log and then deterministically applied asynchronously to the physical database. The present system can implement consistent, high-throughput and serializable distributed transactions for any microservice-enabled application.
[0033] In one example implementation, a global transaction system receives a transaction request for a database service of a microservice. The global transaction system also receives a local commit decision for a local commit request from a local transaction manager of the database service of the microservice. The local commit request corresponds to a transaction request. The global transaction system generates a physical commit request to the local transaction manager based on the local commit decision and the global commit decision. The local transaction manager submits the physical commit request to a database server of the database service corresponding to the transaction request.
[0034] Therefore, one or more methods in the method described herein help solve the technical problems of delay and throughput of conventional methods such as two-phase commit.As such, one or more methods in the method described herein can eliminate the need for certain work (effort) or computing resources, otherwise these work or computing resources will involve blocking transactions.For example, during the whole 2PC processing, locks are kept, which significantly increases transaction conflicts and delays.As a result, the resources used by (for example, in the environment) one or more machines, databases or devices can be reduced.The example of such computing resources includes processor cycles, network traffic, memory usage, data storage capacity, power consumption, network bandwidth and cooling capacity (coolingcapacity).
[0035] Figure 1 is a graphical representation of a network environment 100 in which some example implementations of the present disclosure may be implemented or deployed.
[0036] One or more application servers 104 provide server-side functionality to networked user devices in the form of client devices 110 via network 102. Web clients 110 (eg, browsers) and programmatic clients 108 (eg, "applications") are hosted and executed on web clients 110.
[0037] Application program interface (API) server 118 and web server 120 provide respective programmatic and web interfaces to application server 104. Specific application server 116 hosts application 122 and global transaction system 138, which includes components, modules and / or applications.
[0038] Applications 122 may provide a number of functions and services to users accessing application server 104. For example, applications 122 may include a publishing application that enables users to publish content (e.g., product item information) on a hosted web page. Figure 1 104, but it should be understood that in alternative embodiments, application 122 may be separate and distinct from application server 116.
[0039] The global transaction system 138 coordinates requests from applications 122 to access services provided by microservices 130. For example, the global transaction system 138 coordinates transaction requests from applications 122 across distributed database servers of microservices 130. In an example implementation, microservices 130 include entity services 132, database services 134, and database shard servers 136.
[0040] In addition, although Figure 1The illustrated network environment 100 employs a client-server architecture, but the embodiments are certainly not limited to such an architecture and may equally well find application in, for example, distributed or peer-to-peer architecture systems. The application 122 may also be implemented as a standalone software program that does not necessarily have networking capabilities.
[0041] Web client 110 accesses application 122 via a web interface supported by web server 120. Similarly, programmatic client 108 accesses various services and functions provided by application 122 via a programmatic interface provided by application program interface (API) server 118. In one example, programmatic client 108 may be, for example, a seller application (e.g., an eBay application developed by eBay, Inc. of San Jose, California) to enable sellers to author and manage listings on network environment 100 in an offline manner and to perform batch mode communications between programmatic client 108 and application server 104.
[0042] Figure 1 Also shown is a third-party application 114 executing on a third-party server 112, which is shown as having programmatic access to the application server 104 via a programmatic interface provided by an application program interface (API) server 118. For example, the third-party application 114 using information retrieved from the application server 116 can support one or more features or functions on a website hosted by a third party. For example, the third-party website can provide one or more promotional functions, marketing functions, or payment functions supported by the associated application of the application server 104.
[0043] Figure 1 Any system or machine shown or with Figure 1 Any associated system or machine (e.g., database, device, server) may be implemented with a special-purpose (e.g., specialized or other non-general purpose) computer that has been modified (e.g., configured or programmed by software such as one or more software modules in an application, operating system, firmware, middleware or other program) to perform one or more of the functions described herein for the system or machine. For example, the following description of Fig.11 A special purpose computer system capable of implementing any one or more of the methods described herein is discussed, and thus such a special purpose computer may be a device for performing any one or more of the methods discussed herein. In the technical field of such special purpose computers, a special purpose computer modified by the structure discussed herein to perform the functions discussed herein is technically improved over other special purpose computers that lack the structure discussed herein or are otherwise unable to perform the functions discussed herein. Therefore, a special purpose machine configured according to the systems and methods discussed herein provides an improvement in the art of similar special purpose machines.
[0044] also, Figure 1 Any two or more of the systems or machines shown may be combined into a single system or machine, and the functionality described herein for any single system or machine may be subdivided between multiple systems or machines. In addition, any number and type of client devices 106 may be implemented within the network environment 100. Furthermore, some components or functionality of the network environment 100 may be combined or located elsewhere in the network environment 100. For example, some of the functionality of the client device 106 may be implemented at the application server 116.
[0045] Figure 2 1 is a block diagram illustrating applications 122 provided as part of network environment 100 in one example implementation. Applications 122 may be hosted on dedicated server machines or shared server machines (not shown) that are communicatively coupled to enable communication between the server machines. Applications 122 themselves are communicatively coupled (e.g., via appropriate interfaces) to each other and to various data sources to allow information to be passed between applications 122 or to allow applications 122 to share and access common data. In addition, applications 122 may access one or more databases 126 via database server 124.
[0046] The application server 116 can provide a plurality of publishing, listing and price setting mechanisms, whereby sellers can list items or services for sale (or publish information about them), buyers can express interest in such items or services or indicate a willingness to purchase such items or services, and prices can be set for transactions related to items or services. To this end, the application 122 is shown to include at least one publishing application 202 and one or more order applications 204.
[0047] Figure 3 3 is a block diagram illustrating an example of an architecture 300 for distributed transactions according to an example implementation. The architecture 300 includes an application 122, a global transaction system 138, a microservice A 304, and a microservice B 306. The global transaction system 138 includes a global transaction manager 302, a global transaction log 324, a microservice A transaction manager 320, and a microservice B transaction manager 322. Microservice B 306 includes an entity service 308, a database service 310, and a database shard server 312. Microservice A 304 includes an entity service 314, a database service 316, and a database shard server 318.
[0048] Figure 3The architecture of horizontally scaled database services (e.g., database service 316, database service 310) and microservices (e.g., microservice A 304, microservice B 306) for application 122 is shown. The entire data set is divided (e.g., sharded) into non-overlapping shards (e.g., database shard server 318, database shard server 312). Each shard is presented in a replica set of identical replicas, distributed across multiple physical machines in a data center, supported by a replication protocol (e.g., multi-Paxos or Raft) or by playing the same log to achieve high availability and read throughput.
[0049] The DB server (e.g., database service 316, database service 310) includes a key-value storage model. The DB service (e.g., database service 316, database service 310) provides transparency of operation impact (e.g., segmentation of shards) and partitions to the client. The DB service also provides services at the data object level, such as table rows or JSON documents encoded as values. Entity services (e.g., entity services 314, database services 310), microservices for application 122 provide application 122 with a business entity interface with abstraction and mapping from business entities to data objects. Applications from application 122 enable multiple microservices to use the functions of microservices (e.g., microservice A 304, microservice B 306) to implement business functions. Microservices enable underlying DB services (e.g., database services 316, database services 310).
[0050] Each database access goes through one of the DB service instances (e.g., database service 316, database service 310). To ensure serializability correctness, all reads and writes go through the DB service, with no implicit reads hidden inside the DB server that would be part of the read set for the transaction. Transactions can be supported within each DB shard server.
[0051] The global transaction system 138 coordinates transactions across multiple databases (e.g., database shard servers 318, database shard servers 312) from microservices (e.g., microservice A 304 and microservice B 306). In an example implementation, the global transaction system 138 includes a global transaction manager 302 and a global transaction log 324.
[0052] The global transaction manager 302 (GTM) receives transaction requests and coordinates global transactions across multiple databases. In another example implementation, the global transaction system 138 includes multiple GTMs.
[0053] The global transaction log 324 (GTL) represents a queue of transaction requests for the global transaction manager 302. The order in the global transaction log 324 determines the relative serializability order between global transactions. The persistence of the GTL is optional.
[0054] The global transaction system 138 includes a microservice A transaction manager 320 and a microservice B transaction manager 322. Each microservice transaction manager (for each database service) performs conflict checking and resolution. For example, local commit decisions are performed at the DB transaction manager level.
[0055] Entity service 314 provides business-oriented services for applications to implement business logic. Each DB can support multiple microservices, and each microservice is independent of each other. Database service 316 (DB service) provides DB engine read / write interface and directly accesses DB server (e.g., database shard server 318). Database service 316 also caches the read / write results of each transaction during the execution phase and sends them to its local transaction manager for conflict resolution at submission. Database shard server 318 includes a deterministic database engine. Each server uses a deterministic concurrency control protocol to simultaneously materialize logically submitted transactions.
[0056] Figure 4 1 is a block diagram illustrating a global transaction system provided as part of a networked system in one example implementation. The global transaction system 138 includes a global transaction manager 302, a microservice A transaction manager 320, a microservice B transaction manager 322, and a global transaction log 324. The microservice A transaction manager 320 includes a database transaction manager 402, a database transaction log 404, and a log player 406. The microservice B transaction manager 322 includes a database transaction manager 408, a database transaction log 410, and a log player 412.
[0057] Each microservice transaction manager (e.g., microservice A transaction manager 320, microservice B transaction manager 322) includes a local database transaction manager DBTM (e.g., database transaction manager 402, database transaction manager 408). Each database-level DBTM performs conflict checking and resolution (e.g., local commit decisions are located here).
[0058] The database transaction log (DBTL) (e.g., database transaction log 404, database transaction log 410) records the logically committed transactions (including single-database transactions and multi-database transactions) related to its corresponding database at each database level. The transaction order in the DBTL determines the serializable order of the entire database system, and the global transaction from the GTL is reflected here.
[0059] In one example, for logically committed transactions, the database transaction manager 402 saves its transaction commit data to the database transaction log 404 strictly based on the order of transaction commit. The log player 406 tracks the database transaction log 404 log storage and sends the relevant W-set to the corresponding DB shard server to be deterministically applied to the DB shard. The log player 406 provides feedback of its latest physically committed DBTL LSN to the database transaction log 404 and the database transaction manager 402. This information is useful for the maintenance and conflict checking of the local cache of the database transaction manager 402.
[0060] Figure 5 An architecture using a log store (e.g., DB transaction log 502) for distributed transactions across shards using a deterministic approach is shown. A log player 504 reads data from the DB transaction log 502. Each log store entry includes a set of W for a committed transaction to be physically materialized to a DB shard server (e.g., DB shard server 518, DB shard server 522, DB shard server 520).
[0061] In an example implementation, the log player 504 includes a reader 506, a buffer 516, and a scheduler 514. The reader 506, the buffer 516, and the scheduler 514 read data from the database transaction log 502, analyze the W set, and divide it into multiple W sets based on the partitioning scheme. The scheduler 514 sends them to the corresponding DB shard servers (e.g., DB shard servers 518, DB shard servers 520, DB shard servers 522). For better performance, the queues (e.g., queues 508, queues 510, queues 512) batch multiple W sets from multiple log entries. Within each shard server, the shard server applies the W sets received from the log player 504 in sequence without coordination with other servers. No replication protocol is required between the same copies for the shard servers.
[0062] Figure 6is a block diagram illustrating an example operation of a global transaction system according to an example implementation. In step (1), application 122 initiates a global transaction by requesting a transaction ID (TXID) from global transaction manager 302. In step (2), application 122 enables entity service 314 (of microservice A 304) carrying TXID. At step (3), entity service 314 enables database service 316 carrying TXID. At step (4), database service 316 captures (read set, write set) for TXID. At step (5), application 122 requests global transaction manager 302 to commit TXID. At step (6), database service 316 sends a commit request for TXID with (read set, write set) to database transaction manager 402. At step (7), database transaction manager 402 sorts the commit requests and checks for conflicts for TXID. At step (8), database transaction manager 402 sends a local commit decision for TXID to global transaction manager 302. At step (9), the global transaction manager 302 sends a global commit decision for the TXID to all DBTMs after collecting all DBTM decisions. At step (10), the database transaction manager 402 receives the commit decision from the global transaction manager 302 and saves the w set for the TXID in the database transaction log 404. The database transaction manager 402 also puts the w set with the LSN into the w cache for conflict checking. The database transaction manager 402 responds to the global transaction manager 302. At step (11), the global transaction manager 302 sends a commit success to the application 122.
[0063] The three phases of transaction processing are described as follows:
[0064] Optimistic execution phase: Step 1 to Step 4. The transaction acquires and updates the database through a microservice (e.g., microservice A 304) and a database service (e.g., database service 316). The acquisition and update are captured by each database service as (r set, w set). In an example implementation, version information (i.e., log sequence number - LSN) for each data item in the r set is captured for conflict resolution.
[0065] Logical commit phase: Step 5 to Step 13. At commit time, the commit decision is made by the transaction manager (e.g., database transaction manager 402, global transaction manager 302). If there is a conflict, the transaction is aborted. Otherwise, the transaction is logically committed and put into the transaction log (Step 6 to Step 11).
[0066] Physical materialization phase: The database (eg, database shard server 318) materializes the transactions from the log (database transaction log 404) and physically commits the database (steps 12 to 13).
[0067] In one example implementation, the logical commit decision is done at two levels as follows:
[0068] At the DB level (e.g., microservice A304): Upon receiving a commit request, the DB service agent (e.g., database service 316) submits the request and the meta information for the transaction TXID and its local (r set, w set) to the DBTM (e.g., database transaction manager 402). The DBTM performs conflict checking based on its w set cache of recently submitted transactions. The logic for checking conflicts is similar to that in traditional OCC, except that the DBTM does not access the DB to find out the conflict, but uses a cache updated by past transactions. If there is no conflict, the transaction request can be submitted locally. And if the transaction involves only a single database, the transaction w set will be placed in the DBTL log (e.g., database transaction log 404) assigned with an LSN. Then, the transaction is logically submitted. If the transaction involves multiple databases, the DBTM (e.g., database transaction manager 402) sends its local submission decision for the TXID to the GTM (e.g., global transaction manager 302) and acts based on the response from the GTM. If there is a conflict during the conflict check, or the global submission decision from the GTM is to abort, the transaction is aborted.
[0069] At the global level: The GTM (e.g., the global transaction manager 302) coordinates commit requests for transactions involving multiple databases. Upon receiving a commit request for a transaction TXID, the GTM waits for the commit decision of the DBTM involved (e.g., the database transaction manager 402). If all commit decisions from the DBTM are commits, the transaction can be committed. If any DBTM reports an abort, the transaction is aborted. The GTM notifies the DBTM of the global commit decision by responding to the DBTM's commit. This interaction is simpler than the 2PC protocol and does not require locking in the database. For the commit concretization phase, a deterministic database engine is used to achieve this. Under normal circumstances, transactions are executed deterministically according to the transaction order in the DBTL (e.g., the database transaction log 404). In the abnormal case where the update cannot be performed (due to a hardware error or software crash), recovery is performed using a deterministic recovery algorithm by replaying the log entries to a snapshot of the DB shard.
[0070] Figure 7 An example of various stages of a global transaction system according to one embodiment is shown.
[0071] The consistency of distributed transactions is ensured through three phases: (A) optimistic execution phase 702, (B) logical commit phase 704, and (C) physical materialization phase 706. In the (A) optimistic execution phase 702, the application 122 submits a transaction request to the entity service 314 to update the database service 316. In one example, in the (A) optimistic execution phase 702, the transaction obtains and updates the database through the microservice (e.g., entity service 314) and the database service (e.g., database service 316), and the acquisition and update are captured by each database service instance (e.g., database shard server 318) as (r set, w set). In another example, the (A) optimistic execution phase 702 captures version information (i.e., log sequence number - LSN) for each data item in the r set for conflict resolution.
[0072] In (B) logical commit phase 704, global transaction manager 302 performs conflict resolution and coordination at the global level. Database transaction manager 402 performs conflict resolution and coordination at each database level. Once a transaction is committed, the transaction is first placed in a commit log (e.g., database transaction log 404).
[0073] In the (C) physical materialization phase 706 , the log player 406 reads the database transaction log 404 and materializes the committed transaction by applying the write to the database (eg, database shard server 31 ).
[0074] Figure 8 A flow chart showing an example operation of a global transaction system according to one embodiment is shown. One of the functions of a DBTM (e.g., database transaction manager 402, database transaction manager 408) is to check conflicts between transaction commit requests. Unlike traditional OCCs, DBTMs perform conflict checks based on their local cache to improve efficiency.
[0075] At box 802, a transaction request is received from an input transaction queue. At box 804, the DBTM performs a conflict check on the transaction request. At decision box 806, the DBTM determines whether the transaction request uses OCC. In response to the failure of the OCC test at decision box 806, at box 808, the DBTM aborts the transaction request. Then, at decision box 810, the DBTM determines whether the transaction request is a local transaction. For non-local transaction requests, at box 812, the DBTM provides a response to the coordinator (e.g., (global transaction manager 302). For local transaction requests or commit global transaction requests, the DBTM generates a w set (e.g., transaction w cache 816) to the local cache. After the conflict check, the DBTM sends a commit request to the DB storage via block 818 for physical commit.
[0076] The following shows a method that can be used to describe Figure 8 The notation of the algorithm is:
[0077] DB: {(key, value, lsn)}
[0078] Input Queue: [Tranx]
[0079] DBTL:[<LSN,Tranx,CommitState> ]TranxState:ID->(State,Commit_LSN)
[0080] State: In_Flight,Logically_Committed,Physically_Committed,AbortedCommitState: Logically_Committed,Physically_Committed
[0081] InFlight={ID|TranxState(ID)=In_Flight}
[0082] LogicallyCommitted={ID|TranxState(ID)=Logically_Committed}
[0083] PhysicallyCommitted={ID|TranxState(ID)=Physically_Committed}
[0084] Tranx: (ID, R-Set, W-Set, Read_LSN, Commit_LSN), where Commit_LSN is meaningful only after the transaction is committed.
[0085] R-Set: {(key, value, lsn)} W-Set: {(key, value, lsn)}
[0086] Tranx-W Cache: For all transactions belonging to Logically Committed, U<W-Set+LSN> .
[0087] For conflict checking, only the key and LSN are used, so the value does not have to be saved in the transaction data for the r set and does not have to be saved in the Tranx-W cache for the w set.
[0088] An example of an algorithm for ConflictChecking (Tranx) is shown below:
[0089] Input: Tranx(ID, R-Set, W-Set, Read_LSN)
[0090] Output: Commit decision(COMMIT / ABORT)
[0091] Algorithm:
[0092] For each entry(key, _, lsn) in R-Set do e1(key1, _, lsn1) = find_in_cache(key); If not found then continue; else if lsn1 > lsn then return ABORT;
[0093] return COMMIT;
[0094] End;
[0095] For the input transaction queue, the description of this algorithm is sequential because it assumes that there are no pending commit requests outside those already committed and that its W-set is already in the Tranx-W cache. When the DBTM needs to work with the global transaction manager, this ignores performance. Between the conflict check and the successful write to the commit log, there may be pending transactions whose w-sets are not in the cache. Logically, these transactions can be in the queue and included in these w-sets for checking.
[0096] For global transactions, due to waiting for the global commit decision, the DBTM blocks commit requests behind the input queue. One of the enhancements to the above algorithm is to have a lock manager to handle the checking of pending transactions. Transactions without conflicts can continue to commit, while transactions with conflicts need to wait behind the conflicting transactions.
[0097] The following shows additional notation for cache purge logic:
[0098] Last_Commit_LSN: For lsn <= Last_Commit_LSN, LOG(lsn) includes transactions with all states Physically_Committed
[0099] Last_Commit_LSN = max{Commit_LSN | For LSN < Commit_LSN, LOG(lsn) includes tranx with TranxState = Physically_Committed}
[0100] Read_LSN = The Last_Commit_LSN at the start (or first read) of the Tranx.
[0101] Oldest_Read_LSN: There exists a transaction ID such that TranxState(ID) = In_Flight and its Read_LSN >= Oldest_Read_LSN (= min{Read_LSN(ID)|ID in InFlight})
[0102] Tranx-W cache clearing: The cached w-set entries are only useful if there is a possible transaction that will conflict with it. The purpose of the conflict check is to see if any other transaction has changed the entry since the transaction read the entry. During the OCC execution phase, the transaction reads directly from the DB server, which includes all commits from the transaction until the Last_Commit_LSN (and possibly more). The transaction only needs to check those w-set entries in the cache that are after its Read_LSN (e.g., using Oldest_Read_LSN). Once Oldest_Read_LSN moves, the cache entries with LSN < Oldest_Read_LSN can be removed.
[0103] Fig. 9 FIG. is a block diagram showing another example operation of a global transaction system according to an example embodiment. A transaction can involve data across multiple DBs. Each database service is covered by a corresponding local DBTM (e.g., database transaction manager 902, database transaction manager 904, database transaction manager 906). The global transaction manager 302 coordinates among the multiple DBTMs involved during the transaction commit phase. For example, in step (1), the global transaction manager 302 submits a commit request to each local database transaction manager. In step (2), each local database transaction manager provides a response (e.g., a local commit response) based on local conflict checking using the local cache. In step (3), the global transaction manager 302 provides a global commit decision based on the local commit responses from all local database transaction managers.
[0104] If the participants spend time acquiring resources and performing the actual commit during processing, traditional 2PC cannot perform well. Here, the conflict checking and commit processing duration are short because the commit processing is divided into logical commit and physical commit. This coordination processing only needs to go through the logical commit.
[0105] The following shows Fig. 9 an example algorithm of the operation shown
[0106] Algorithm: commit_coordinate
[0107] Input:tranx(ID,{DBTMi})
[0108] Output:glocal commit decision(commit / abort)Algorithm:
[0109] commit=true;
[0110] For dbtm in{DBTMi}do
[0111] commit=commit&&
[0112] (dbtm.ConflictChecking(tranx)==COMMIT);
[0113] For dbtm in{DBTMi}do
[0114] dbtm.send(tranx,commit?COMMIT:ABORT);
[0115] Return commit;
[0116] End;
[0117] In another example implementation, request messages can be reduced by waiting for the local commit decision of the DBTM on the GTM side. This reduces the commit coordination to one round trip. Another problem with 2PC is handling failures. Timeouts can be used to detect failures to avoid blocking flows.
[0118] Fig.10 An example of scalability of a global transaction system according to one embodiment is shown. The abbreviated notation is as follows: CC stands for Commit Coordination, which is a GTM instance; CD stands for Commit Decision, which is a DBTM / DBTL instance pair; and CM is Commit Materialization, which is a log player and corresponding DB shard server. Fig.10 The diagram in shows possible deployment patterns:
[0119] (A) One CC with coordination among three CDs.
[0120] (B) There are two CCs coordinating three CDs, each CC coordinating between two CDs. One (middle) CD accepts commit requests from both CCs.
[0121] (C) Adds an (intermediate) CC to (B) that handles transactions involving three DBs, each covered by a CD.
[0122] (D) Split the traffic of the top CC in (A) into two CCs to support more traffic. Or it is a deployment that utilizes two CCs for transactions across three DBs.
[0123] Fig.11 1 is a flow chart illustrating a method 1100 for consistent distributed transactions across microservices utilizing multiple databases according to an example implementation. The operations in method 1100 may be performed by global transaction system 138 using the Figure 4 138. Therefore, the method 1100 is described by way of example with reference to the global transaction system 138. However, it should be understood that at least some operations of the method 1100 can be deployed on various other hardware configurations or performed by similar components residing elsewhere. For example, some operations can be performed at the third-party server 112.
[0124] At block 1102, the global transaction system 138, the global transaction manager 302 receives a transaction request (e.g., from the application 122). At block 1104, the global transaction manager 302 generates a transaction id for the transaction request. At block 1106, the global transaction manager 302 generates a local commit request to a first local transaction manager (e.g., the microservice A transaction manager 320). At block 1108, the global transaction manager 302 generates a local commit request to a second local transaction manager (e.g., the microservice B transaction manager 322). At block 1110, the global transaction manager 302 receives a first conflict check confirmation from the first local transaction manager. At block 1112, the global transaction manager 302 receives a second conflict check confirmation from the second local transaction manager. Once the global transaction manager 302 determines that there is no conflict based on the first conflict check and the second conflict check, at block 1114, the global transaction manager 302 generates a physical commit request to the first local transaction manager. At block 1116 , the global transaction manager 302 generates a physical commit request to the second local transaction manager.
[0125] Fig.12 1 is a flow chart illustrating a process 1200 for consistent distributed transactions across microservices utilizing multiple databases according to an example implementation. The operations in process 1200 may be performed by the global transaction system 138 using the Figure 4 138. Therefore, process 1200 is described by way of example with reference to global transaction system 138. However, it should be understood that at least some operations of process 1200 can be deployed on various other hardware configurations or performed by similar components residing elsewhere. For example, some operations can be performed at third-party server 112.
[0126] In block 1202, the process 1200 receives a transaction request. In block 1204, the process 1200 generates a local commit request to a local transaction manager corresponding to a database of a microservice associated with the transaction request in response to receiving the transaction request. In block 1206, the process 1200 confirms that there is no conflict based on the local commit request from the local transaction manager. In block 1208, the process 1200 generates a physical commit request to the local transaction manager in response to no conflict, wherein the local transaction manager is configured to submit the physical commit request to a database server of the database service corresponding to the transaction request.
[0127] Fig.13 1 is a flow chart illustrating a process 1300 for global transaction manager coordination across microservice operations utilizing multiple databases according to an example implementation. Figure 4 1300. Therefore, the process 1300 is described by way of example with reference to the global transaction system 138. However, it should be understood that at least some of the operations of the process 1300 can be deployed on various other hardware configurations or performed by similar components residing elsewhere. For example, some operations can be performed at the third-party server 112.
[0128] In block 1302, process 1300 receives a transaction request for a database service of a microservice. In block 1304, process 1300 receives a local commit decision of a local commit request from a local transaction manager of the database service of the microservice, the local commit request corresponding to the transaction request. In block 1306, process 1300 generates a physical commit request to the local transaction manager based on the local commit decision and the global commit decision, the local transaction manager being configured to submit the physical commit request to a database server of the database service corresponding to the transaction request.
[0129] Fig.14An example of transaction execution for multiple databases according to one embodiment is shown. The figure shows an example of transaction execution processing for a cross-DB transaction. We ignore the details of the individual DB transaction execution details because we have already described them in the previous figure. T1 and T2 are global distributed transactions (cross-DB transactions) that update records in both DB1 and DB2. T0 is a local transaction only for DB1, and T3 is a local transaction only for DB2. Only cross-DB transactions need to go through the GTM (global transaction manager), and local transactions only need to go through their local DB_TM, so in this example, T1 and T2 are sent to the GTM, and are pushed into the transaction input queue of the GTM for sorting after the OCC execution phase (note that the OCC execution phase is executed in each DB_TM), T0 is sent to DB1_TM and pushed into the transaction input queue of DB1_TM, and T3 is sent to DB2_TM and pushed into the transaction input queue of DB2_TM.
[0130] The execution process is described as follows:
[0131] (1) GTM will periodically send the decomposed transactions from its transaction input queue to the involved DBs in order. In this example, GTM sends T1 T2 to DB1_TM and DB2_TM, and DB1_TM and DB2_TM will put T1 T2 into their own transaction input queues.
[0132] (2) DB1_TM and DB2_TM will perform OCC verification and conflict resolution on the local read / write sets of T1 / T2.
[0133] (3) DB1_TM and DB2_TM will send their own commit / abort decisions to GTM. In this example, on the DB1 side, T1 can be committed and T2 will be aborted; on the DB2 side, both T1 and T2 can be committed.
[0134] (4) When the GTM receives all decisions from the involved DB_TMs, it can make a final decision on the cross-DB transaction. In this example, T1 can be committed because both DB1 and DB2 agree to commit, but T2 will be aborted because DB1 will abort the transaction. The GTM will save the committed transactions by putting them in the commit transaction log. The GTM then sends the decision to all involved DB_TMs.
[0135] (5) When DB_TM receives the final decision on the cross-DB transaction, DB_TM can commit the transaction by putting it in its own commit transaction log. The commit transaction logs in GTM and all DB_TMs determine the global serializability order. The transaction serializability order in this example is T0------>T1------>T3.
[0136] Fig.15 1500 in which instructions 1504 (e.g., software, programs, applications, applet, app, or other executable code) may be executed that cause the machine 1500 to perform any one or more of the methods discussed herein. For example, the instructions 1504 may cause the machine 1500 to perform any one or more of the methods described herein. The instructions 1504 convert a general purpose, unprogrammed machine 1500 into a specific machine 1500 that is programmed to perform the functions described and illustrated in the manner described. The machine 1500 may operate as a standalone device or may be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1500 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. The machine 1500 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a cellular phone, a smart phone, a mobile device, a wearable device (e.g., a smart watch), a smart home device (e.g., a smart home appliance), other smart devices, a web device, a network router, a network switch, a network bridge, or any machine capable of executing instructions 1504, sequentially or otherwise, specifying actions to be taken by the machine 1500. In addition, while only a single machine 1500 is shown, the term "machine" should also be taken to include a collection of machines that individually or jointly execute instructions 1504 to perform any one or more of the methodologies discussed herein.
[0137] The machine 1500 may include a processor 1506, a memory 1508, and an I / O component 1542, which may be configured to communicate with each other via a bus 1544. In an example embodiment, the processor 1506 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, a processor 1502 that executes instructions 1504 and a processor 1510. The term "processor" is intended to include a multi-core processor, which may include two or more independent processors (sometimes referred to as "cores") that may execute instructions simultaneously. Although Fig.15Multiple processors 1506 are shown, but the machine 1500 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.
[0138] The memory 1508 includes a main memory 1512, a static memory 1514, and a storage unit 1516, all of which are accessible to the processor 1506 via the bus 1544. The main memory 1508, the static memory 1514, and the storage unit 1516 store instructions 1504 that implement any one or more of the methods or functions described herein. During execution of the instructions 1504 by the machine 1500, the instructions 1504 may also reside, in whole or in part, within the main memory 1512, within the static memory 1514, within the machine-readable medium 1518, within the storage unit 1516, within at least one of the processors 1506 (e.g., within a cache memory of the processor), or within any suitable combination thereof.
[0139] I / O components 1542 may include various components for receiving input, providing output, generating output, sending information, exchanging information, capturing measurements, etc. The specific I / O components 1542 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone may include a touch input device or other such input mechanism, while a terminalless server machine may not include such a touch input device. It will be appreciated that I / O components 1542 may include Fig.15 1542 may include an output component 1528 and an input component 1530. In various example embodiments, the I / O component 1542 may include an output component 1528 and an input component 1530. The output component 1528 may include a visual component (e.g., a display such as a plasma display panel (PDP), a light emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), an acoustic component (e.g., a speaker), a tactile component (e.g., a vibration motor, a resistance mechanism), other signal generators, etc. The input component 1530 may include an alphanumeric input component (e.g., a keyboard, a touch screen configured to receive alphanumeric input, an optical keyboard, or other alphanumeric input component), a pointing-based input component (e.g., a mouse, a touch pad, a trackball, a joystick, a motion sensor, or another pointing instrument), a tactile input component (e.g., a physical button, a touch screen or other tactile input component that provides the location and / or force of a touch or touch gesture), an audio input component (e.g., a microphone), etc.
[0140] In other example embodiments, the I / O component 1542 may include: a biometric component 1532, a motion component 1534, an environmental component 1536, or a positioning component 1538, as well as a variety of other components. For example, the biometric component 1532 includes components for detecting expressions (e.g., hand expressions, facial expressions, voice expressions, body postures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition), etc. The motion component 1534 includes an acceleration sensor component (e.g., an accelerometer), a gravity sensor component, a rotation sensor component (e.g., a gyroscope), etc. The environment component 1536 includes, for example, an illumination sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects concentrations of hazardous gases for safety or measures pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. The positioning component 1538 includes a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects air pressure that can obtain altitude), an orientation sensor component (e.g., a magnetometer), etc.
[0141] Various technologies may be used to implement communications. I / O components 1542 also include communications components 1540 that are operable to couple machine 1500 to network 1520 or device 1522 via coupling 1524 and coupling 1526, respectively. For example, communications components 1540 may include a network interface component or another suitable device that interfaces with network 1520. In other examples, communications components 1540 may include wired communications components, wireless communications components, cellular communications components, near field communications (NFC) components, Components (e.g. Low power consumption), Device 1522 may be another machine or any of a variety of peripheral devices (eg, a peripheral device coupled via USB).
[0142] In addition, the communication component 1540 can detect an identifier or include a component operable to detect an identifier. For example, the communication component 1540 can include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting the following items: a one-dimensional barcode, such as a universal product code (UPC) barcode; a multi-dimensional barcode, such as a Quick Response (QR) code, Aztec code, Data Matrix, Dataglyph, MaxiCode, PDF417, Hypercode, UCC RSS-2D barcode, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying an audio signal of a tag). In addition, various information can be obtained via the communication component 1540, such as a location via Internet Protocol (IP) geolocation, a location via Internet Protocol (IP), ... The location of signal triangulation, the location of an NFC beacon signal via detection that can indicate a specific location, etc.
[0143] Various memories (e.g., memory 1508, main memory 1512, static memory 1514, and / or memory of processor 1506) and / or storage unit 1516 may store one or more sets of instructions and data structures (e.g., software) implemented or used by any one or more of the methods or functions described herein. These instructions (e.g., instructions 1504) when executed by processor 1506 cause various operations to implement the disclosed embodiments.
[0144] Instructions 1504 may be transmitted or received over network 1520 via a network interface device (e.g., a network interface component included in communications component 1540) using a transmission medium and using any of a number of well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instructions 1504 may be sent or received to device 1522 via coupling 1526 (e.g., a peer-to-peer coupling) using a transmission medium.
[0145] This application describes a system for supporting distributed transactions across microservices involving multiple underlying databases in a scalable setting. The system employs a logical transaction commit log and leverages a deterministic underlying database engine to improve performance and scalability. For coordination across databases, a mechanism similar to 2PC is used in principle, but only at the logical commit level. After transactions are logically committed, they are materialized by a deterministic database that is typically deployed in a scale-out manner. The logical commit log is replicated and can replace physical log-based replication.
[0146] The key to scalability and performance is techniques that avoid relatively long execution phases and coordination during transaction materialization (physical commit). Coordination is at short and fast commit times for conflict resolution. 2PC-like protocols are only used for cross-database transactions, and unlike traditional 2PC, where the databases involved usually need to lock related data records during the protocol broadcast, this system does not require locking during the protocol. In addition, this protocol is only required when the transaction actually affects multiple databases, and a single database transaction only needs to go to its local database transaction manager. Deterministic DBs simplify concurrency control and speed up commit processing. This system is suitable for use with deterministic database engines that require known (r-set, w-set) to perform deterministic transaction scheduling.
[0147] Although the embodiments have been described with reference to specific example embodiments, it will be apparent that various modifications and variations may be made to these embodiments without departing from the broader scope of the present disclosure. Therefore, the specification and the drawings are to be regarded as illustrative rather than restrictive. The drawings forming part of this article illustrate, in a graphical and non-restrictive manner, specific embodiments in which this subject matter may be practiced. The illustrated embodiments are described in sufficient detail to enable those skilled in the art to practice the teachings disclosed herein. Other embodiments may be utilized and derived therefrom so that structural and logical substitutions and changes may be made without departing from the scope of the present disclosure. Therefore, this detailed description is not adopted in a restrictive sense, and the scope of the various embodiments is limited only by the appended claims together with the full scope of the equivalents of such claims.
[0148] Such embodiments of the subject matter of the present invention may be referred to herein as the term "invention" individually and / or collectively only for convenience, without intending to actively limit the scope of the present application to any single invention or inventive concept when more than one invention or inventive concept is actually disclosed. Therefore, although specific embodiments have been shown and described herein, it should be appreciated that any arrangement calculated to achieve the same purpose can be replaced with respect to the specific embodiments shown. The present disclosure is intended to cover any and all adjustments or variations of various embodiments. When the above description is reviewed, the combination of the above embodiments and other embodiments not specifically described herein will be obvious to those skilled in the art.
[0149] An abstract of the disclosure is provided to enable the reader to quickly determine the essence of the technical disclosure. It should be understood that the submission of the abstract is not used to interpret or limit the scope or meaning of the claims. In addition, in the aforementioned detailed description, it can be seen that various features are combined together in a single embodiment for the purpose of streamlining the disclosure. The method of the disclosure is not to be interpreted as reflecting the following intention: the claimed embodiment requires more features than the features explicitly listed in each claim. On the contrary, as reflected in the attached claims, the subject matter of the invention lies in less than all the features in a single disclosed embodiment. Therefore, the attached claims are thereby incorporated into the detailed description, wherein each claim itself independently serves as a separate embodiment.
[0150] Example
[0151] Example 1 is a computer-implemented method. The method includes: receiving a transaction request for multiple database services of a microservice; receiving multiple local commit decisions for local commit requests from local transaction managers of the multiple database services of the microservice, the local commit request corresponding to the transaction request for each database service; and generating a physical commit request to each of the local transaction managers based on the local commit decision and the global commit decision, each of the local transaction managers being configured to submit the physical commit request to each database server of the database service corresponding to the transaction request.
[0152] In Example 2, the subject matter of Example 1 may optionally include: receiving a transaction request from an application; generating a transaction identifier for the transaction request, the transaction identifier identifying a database service of a microservice associated with the transaction request; and providing the transaction identifier to the application, the application being configured to initiate a local commit request to the database service of the microservice using the transaction identifier.
[0153] In Example 3, the subject matter of Example 1 can optionally include: wherein the transaction request includes a read request and a write request to the database server, wherein receipt of the transaction request is captured during an optimistic execution phase.
[0154] In Example 4, the subject matter of Example 1 can optionally include confirming that there is no conflict for the transaction request based on a local cache of the local transaction manager and the transaction identifier, the local commit decision indicating no conflict.
[0155] In Example 5, the subject matter of Example 4 can optionally include: wherein the local cache stores a log of committed transactions executed on a database server of the database service.
[0156] In Example 6, the subject matter of Example 1 may optionally include: receiving a global commit request identifying a transaction identifier from an application; receiving a local commit request identifying a transaction identifier from a database service; accessing a local cache of a local transaction manager, the local cache indicating a committed transaction; and confirming that there is no conflict for the local commit request based on the local cache and the transaction identifier, the local commit decision indicating no conflict.
[0157] In Example 7, the subject matter of Example 1 may optionally include: receiving multiple local commit decisions from multiple local transaction managers associated with a transaction request; generating a global commit decision based on the multiple local commit decisions; and transmitting the global commit decision to the multiple local transaction managers, each local transaction manager being configured to submit a physical commit request to a corresponding database server of a corresponding database service based on the global commit decision.
[0158] In Example 8, the subject matter of Example 1 may optionally include: saving a log of committed transactions for a corresponding local transaction manager; and applying the committed transactions from the log to a database server corresponding to the local transaction manager.
[0159] In Example 9, the subject matter of Example 1 can optionally include: performing a conflict check based on a comparison of the local commit request with a local cache of a local transaction manager without accessing the database server, the local cache indicating recent transaction updates to the database server, wherein the local transaction manager is configured to submit a physical commit request to the database server without locking the database server on which the transaction request is pending, wherein the global commit decision is based on a plurality of local commit decisions from a plurality of local transaction managers that are serving the database for the transaction request.
[0160] In Example 10, the subject matter of Example 1 may optionally include: receiving a transaction request from an application; generating a transaction identifier for the transaction request, the transaction identifier identifying a first database service of a first microservice associated with the transaction request and a second database service of a second microservice associated with the transaction request; receiving, at a first local transaction manager corresponding to the first database service, a first local commit request from the first database service, the first local commit request including the transaction identifier; receiving, at a second local transaction manager corresponding to the second database service, a second local commit request from the second database service, the second local commit request including the transaction identifier; confirming a first non-conflict for the first local commit request from the first local transaction manager; confirming a second non-conflict for the second local commit request from the second local transaction manager; and receiving a first local commit decision from the first local transaction manager based on the first non-conflict; receiving a second local commit decision from the second local transaction manager based on the second non-conflict; providing a global commit decision to the first local transaction manager and the second local transaction manager in response to receiving the first local commit decision and the second local commit decision; generating a first physical commit request to a first database server of the first database service in response to the global commit decision; and generating a second physical commit request to a second database server of the second database service in response to the global commit decision.
Claims
1. A computer-implemented method for cross-microservices, comprising: receiving a transaction request for a plurality of database services of a plurality of microservices, each microservice including a respective database service associated with a respective local transaction manager; submitting, to each local transaction manager, a local commit request corresponding to the transaction request for the corresponding database service; receiving a local commit decision for the local commit request from each local transaction manager, wherein each local commit decision from the corresponding local transaction manager is based on a conflict check using an optimistic concurrency control protocol and a local cache of the local transaction manager storing a log of committed transactions executed on a database server of the corresponding database service; generating a physical commit request to each local transaction manager based on a corresponding local commit decision and a global commit decision generated based on a plurality of local commit decisions from the plurality of local transaction managers; transmitting the global commit decision to each local transaction manager, each local transaction manager being configured to submit the physical commit request to a corresponding database server; Maintaining, by each local transaction manager, a log of transactions committed by the local transaction manager based on transaction commit order; and Committed transactions from the log are deterministically applied by each local transaction manager to the corresponding database server in the order of the log.
2. The computer-implemented method of claim 1 , further comprising: receiving the transaction request from an application; Generate a transaction identifier for the transaction request, the transaction identifier identifying a first database service among a plurality of database services of the plurality of microservices associated with the transaction request; as well as The transaction identifier is provided to the application, and the application is configured to initiate the local commit request to the first database service using the transaction identifier.
3. The computer-implemented method of claim 1 , wherein: The transaction request includes a read request and a write request to a first database server of a first database service of the plurality of database services, wherein receipt of the transaction request is captured during an optimistic execution phase.
4. The computer-implemented method of claim 2, further comprising: receiving, from the application, a global commit request identifying the transaction identifier; receiving, from a first database service of the plurality of database services, a local commit request identifying the transaction identifier; accessing a local cache of a first local transaction manager associated with the first database service, the local cache indicating committed transactions; as well as An absence of conflict is confirmed for the local commit request based on the local cache and the transaction identifier, the local commit decision indicating the absence of conflict.
5. The computer-implemented method of claim 1 , further comprising: performing a conflict check based on a comparison of the local commit request with a local cache of a first local transaction manager associated with a first database service of the plurality of database services without accessing a first database server of the first database service, the local cache indicating a most recent transactional update to the first database server, The first local transaction manager is configured to submit the physical commit request to the first database server without locking the first database server on which the transaction request is suspended.
6. The computer-implemented method of claim 1, receiving the transaction request from an application; Generate a transaction identifier for the transaction request, where the transaction identifier identifies a first database service of a first microservice associated with the transaction request and a second database service of a second microservice associated with the transaction request; receiving, at a first local transaction manager corresponding to the first database service, a first local commit request from the first database service, the first local commit request including the transaction identifier; receiving, at a second local transaction manager corresponding to the second database service, a second local commit request from the second database service, the second local commit request including the transaction identifier; confirming a first non-conflict for the first local commit request from the first local transaction manager; confirming a second non-conflict for the second local commit request from the second local transaction manager; as well as receiving a first local commit decision from the first local transaction manager based on the first lack of conflict; receiving a second local commit decision from the second local transaction manager based on the second conflict-free state; in response to receiving the first local commit decision and the second local commit decision, providing the global commit decision to the first local transaction manager and the second local transaction manager; generating, in response to the global commit decision, a first physical commit request to a first database server of the first database service; as well as In response to the global commit decision, a second physical commit request is generated to a second database server of the second database service.
7. A computing device, comprising: processor; as well as A memory storing instructions which, when executed by the processor, configure the computing device to perform the method according to any one of claims 1 to 6.
8. A computer-readable storage medium comprising instructions which, when executed by a computer, cause the computer to perform the method according to any one of claims 1 to 6.
