Method for transaction isolation within microservices

Abstract

The present invention provides methods and an apparatus for a SAGA Transaction Controller system to avoid blocking of non-local resources during the processing of distributed transactions by microservices that follow the SAGA standard. An isolation manager algorithm is used by a transaction isolation mechanism to avoid blocking resources. The isolation manager algorithm uses the concept of read and write time windows for isolation of transaction operations.

Description

[0001] METHOD FOR TRANSACTION ISOLATION WITHIN MICROSERVICES

[0002] FIELD OF THE INVENTION

[0003] The present invention generally pertains to the field of database management and transaction processing . More speci fically, it addresses the challenge of maintaining isolation in distributed systems , especially when implementing the SAGA transaction model .

[0004] PRIOR ART

[0005] In the realm of distributed transaction management , ensuring data integrity and consistency remains a critical concern . Historically, mechanisms that address these concerns have often been categori zed under concurrency control methods . Among these methods , the two-phase locking ( 2PL ) protocol and the optimistic concurrency control ( OCC ) have been prominent .

[0006] The 2PL protocol works by acquiring locks for every data item a transaction wishes to access and holding them until the transaction commits or aborts . While 2PL guarantees conflict-seriali zability, it can result in performance bottlenecks , especially in highly concurrent scenarios due to possible deadlocks and the overhead of lock management .

[0007] On the other hand, the OCC method typically uses a validation phase to ensure that the read data items remain unchanged until the transaction ' s end . While OCC avoids the pitfalls of deadlocks , it may lead to higher abort rates in scenarios with high contention, as transactions that fail the validation phase are often aborted . Timestamp-based concurrency control methods emerged as an alternative , aiming to of fer a balance between the stringent locking protocols of 2PL and the optimistic checks of OCC . Transactions in such systems are ordered based on their timestamps , determining the sequence in which they are allowed to access data items . Yet , traditional timestampbased methods still grapple with challenges like ensuring global clock synchroni zation acros s distributed systems and managing stale reads in environments with variable network latencies .

[0008] Furthermore , prior art also reveals various attempts to use temporal markers for transaction ordering, but most of these implementations either relied heavily on centrali zed timestamp authorities or faced challenges in handling skewed workloads where some data items are more frequently accessed than others .

[0009] PROBLEM TO BE SOLVED

[0010] In today ' s digital era, distributed systems have become the backbone of modern appl ications , ranging from e- commerce platforms to banking solutions . As these applications scale and witness increased user concurrency, the challenge of ensuring data integrity and consistency in distributed transactions becomes ever more pronounced .

[0011] The heart of the issue lies in the conflicting demands of performance and reliability . While systems need to be swi ft to accommodate the vast numbers of transactions per second, they simultaneously need to maintain strict transactional consistency . Existing solutions , as discussed in the prior art , exhibit speci fic drawbacks :

[0012] Performance Bottlenecks with 2PL : The 2PL protocol , with its approach of acquiring and holding onto locks , often becomes a bottleneck . Especially in systems with high user concurrency, deadlocks and lock management overheads hamper performance , leading to user dissatis faction and potential revenue losses .

[0013] High Abort Rates with OCC : The OCC method, though eliminating deadlocks , comes at the cost of increased transaction aborts . In scenarios with high contention, the frequent aborting of transactions due to failed validation checks can be resource-intensive and counter-productive .

[0014] Challenges with Timestamp-Based Methods : Even though timestamp-based concurrency control attempts to strike a balance , it introduces its own set of problems . Managing global clock synchroni zation across myriad nodes in distributed environments is complex and error-prone . Moreover, the variable network latencies often lead to issues like stale reads , further compromising data integrity .

[0015] Read time windowLimitations with Centrali zed Timestamp Authorities : Prior solutions that utili zed centrali zed timestamp authorities to order transactions faced single points of failure . Any downtime or inef ficiency at this centrali zed node could cascade through the entire system, causing widespread disruptions .

[0016] In summary, while various mechanisms have been proposed and implemented in the area of distributed transaction management , none have ef fectively addressed the dual challenge of ensuring robust isolation in the face of high concurrency while also minimi zing the performance overhead .

[0017] Given these challenges , there is a palpable gap in the current technological landscape . The industry is in dire need of a solution that can seamlessly handle distributed transactions , ensuring both speed and consistency without the limitations of existing methods . The obj ective is to overcome these inherent problems , paving the way for robust , ef ficient , and reliable distributed transaction management . Figure 3 illustrates the "confidence level" heuristic paradigm applied to an online charging method . The reference signs represent :

[0018] SUMMARY OF THE INVENTION

[0019] It is a general obj ect of the present invention to provide means to manage transaction isolation within microservices with distributed transactions in distributed systems without signi ficantly af fecting the performance in processing distributed transactions by developing a new apparatus and respective methods of operation which is suitable to manage transaction isolation within microservices . These and other obj ects are achieved in accordance with the attached set of claims .

[0020] The invention introduces an algorithm, referred to as the " I solation Manager Algorithm, " which utili zes temporal markers , to ensure consistency and isolation for reading and writing operations within concurrent environments .

[0021] In accordance with one aspect of the present invention there is provided a SAGA Transactions Controller system to manage transaction isolation within microservices with distributed transactions in distributed systems .

[0022] According to another aspect , the present invention provides a method for a SAGA to initiate control of a sequence of transactions by providing temporal markers to control when transactions can perform read and write operations . In accordance with another aspect of the present invention there i s provided a method for a SAGA to terminate control of a sequence of transactions and when removing the SAGA control temporal markers are updated accordingly . DESCRIPTION OF FIGURES

[0023] Figure 1 illustrates the isolation manager algorithm paradigm applied to a distributed system. The reference signs represent:

[0024] 101 - SAGA Coordinator;

[0025] 102 - SAGA Coordinator;

[0026] 103 - SAGA Initialization Handler;

[0027] 104 - SAGA Termination Handler;

[0028] 105 - Data repository;

[0029] 106 - Temporal marker;

[0030] 107 - Message Broker;

[0031] 108 - Isolation Manager mechanism instance;

[0032] 109 - SAGA start of control event;

[0033] 110 - SAGA end of control event.

[0034] Figure 2 depicts the isolation manager algorithm concept. The reference signs represent:

[0035] 200 - Read Time Window;

[0036] 201 - TSR temporal marker;

[0037] 202 - Unsafe Time Window;

[0038] 203 - TSW temporal marker;

[0039] 204 - Write Time Window;

[0040] Figure 3 depicts the isolation manager algorithm extension for distributed. The reference signs represent:

[0041] 301 - Global TSR;

[0042] 302 - Constant temporal difference (A) ;

[0043] 303 - Global TSW;

[0044] 304 - Local TSW. Figure 4 depicts the SAGA Transactions Controller system high level block diagram according to the present invention . The reference signs represent :

[0045] 109 - SAGA start of control event ;

[0046] 110 - SAGA end of control event .

[0047] 203 - TSW temporal marker ;

[0048] 400 - SAGA Transactions Controller system;

[0049] 401 - I solation Manager Mechanism instance ;

[0050] 402 - Message Broker .

[0051] Figure 5 depicts a flow chart describing the SAGA starts control method according to the present invention . The reference signs represent :

[0052] 500 - Check the running mode ;

[0053] 501 - Uses instance TSW and TSR .

[0054] 502 - Check others IM instances running mode ;

[0055] 503 - TSW as the lowest among active IM instances ;

[0056] 504 - TSW as the highest among all IM instances ;

[0057] 505 - TSW is own TSW;

[0058] 506 - Calculates tsWrite and tsRead .

[0059] Figure 6 depicts a flow chart describing the SAGA terminates control method according to the present invention . The reference signs represent :

[0060] 600 - Check execution result ;

[0061] 601 - Check IM instances running mode ;

[0062] 602 - TSW as the last tsWrite ;

[0063] 603 - TSW and TSR are the lowest among other IM instances ;

[0064] 604 - TSW are the lowest among other IM instances and TSR is relative to TSW . DETAILED DESCRIPTION

[0065] The present invention pertains to the field of distributed data processing systems and provides , in particular, a system designed to manage transaction isolation within microservices with distributed transactions in distributed systems .

[0066] This invention intends to use a new paradigm, to define an innovative algorithm, that ensures robust isolation and minimi ze performance overhead in processing distributed transactions , by calculating time markers based on concepts related to temporal timestamps ordering like : Adaptative Timestamp Generation for generating timestamps based on the activity and contention level in the distributed environment ; Decentrali zed Timestamp Management to employ a decentrali zed mechanism for timestamp dissemination .

[0067] The innovative algorithm, hereinafter referred to as the isolation manager algorithm, serves as a mechanism to prevent locking on non-local resources during the processing of distributed transactions by microservices that follow the SAGA pattern . In particular, the isolation manager algorithm will be used to calculate read and write time windows to apply, on a transactions sequence of a SAGA, for safe access to an entity without locking it .

[0068] Fig . 1 illustrates the use of the isolation manager algorithm approach applied to an I solation Manager mechanism

[0069] ( 108 ) to provide read and write time windows . Whenever a new SAGA is required to manage a transactions sequence , the SAGA Coordinator ( 101 ) will send a SAGA start of control event

[0070] ( 109 ) indicating to the I solation Manager mechanism ( 108 ) that a new SAGA has been created and needs timestamp markers to define the read and write time windows to apply in the sequence of transactions managed by the SAGA. The I solation Manager mechanism (108) will use the isolation manager algorithm to calculate time markers (106) that are used to determine moments from which it will be possible to read or write without overlapping the two operations. Whenever the transactions sequence controlled by a SAGA terminates, the SAGA Coordinator (102) will send a SAGA end of control event (110) indicating to the Isolation Manager mechanism (108) the SAGA has been removed and time markers in the Isolation Manager mechanism (108) are updated accordingly.

[0071] Further, Fig. 1 depicts the "isolation manager algorithm" being used by the SAGA Initialization Handler (103) , of a Isolation Manager (IM) mechanism (108) , to use information from SAGA start of control event (109) and from temporal markers (106) persisted in the database (105) (temporal markers can be from own IM instance or other IM instances) , to calculate new temporal markers (106) . The new temporal markers (106) will be persisted in the repository (105) as well as information of a SAGA going on and will be used to determine timestamp markers to define the read and write time windows, which will be returned in the response to the SAGA start of control event (109) .

[0072] Moreover, Fig. 1 depicts the "isolation manager algorithm" being used by the SAGA Termination Handler (104) , of a Isolation Manager (IM) mechanism (108) , to use information from SAGA end of control event (110) to remove from database (105) the information of the SAGA going on and persists in database (105) new temporal markers (106) .

[0073] On top of that, Fig. 1, also illustrates the scenario of multiple Isolation Manager (IM) mechanisms (108) running simultaneously. In that scenario, the SAGA Termination Handler (104) when receives the SAGA end of control event (110) , in addition to the tasks it does for the single instance scenario, will also publish to the others IM mechanism (108) instances its temporal markers (106) and if it does not have SAGAs going on. On the other hand, in the scenario multiple instances of IM mechanism (108) running, the SAGA Initialization Handler (103) when receives information from the other instances persists that information in the database (105) .

[0074] As depicted in Fig. 2, the isolation manager algorithm uses the concept of time windows to determine instants in time where are allowed operations to read or write: a read time window (200) where SAGAs are allowed to perform read operations; a write time window (204) which defines a safe time period for SAGAS perform write operations; an unsafe time window (202) where no read nor write operations are allowed for SAGAs. The integrity and isolation between SAGAs are guaranteed by the isolation manager algorithm if the read time window (200) and write time window (204) never overlap in time, so that SAGAs can ensure no entity is written and read simultaneously, thus avoiding corruption of entity information.

[0075] According to Fig. 2, the base of the isolation manager algorithm lies in two temporal markers: the TSW (203) , or Timestamp Write, which determines the lower limit instant of the write time window (204) and the TSR (201) , or Timestamp Read, (TSR) which determines the upper limit instant of the read time window (200) . The TSR (201) and TSW

[0076] (203) can also be used to determine the limits of the unsafe time window (202) .

[0077] Additionally, Fig. 2 depicts that to prevent the overlap of the read time window (200) and write time window

[0078] (204) it is necessary that the TSW (203) is always higher (newer) than the TSR (201) .

[0079] Further, based in Fig. 2, the isolation manager algorithm to be effective and correctly applied requires some assumptions to be taken: • The TSR (201) is a temporal marker defined as having a constant temporal difference with respect to the TSW (203) ;

[0080] • TSW (203) is a temporal marker determined by system activity;

[0081] • The constant temporal difference is pre-defined and the value must determine an unsafe time window (202) that can ensure integrity and isolation between SAGAs ;

[0082] • In the instantiation of an isolation manager, algorithm, the TSR (201) is set to zero, to determine a stable starting point, and the TSW (203) is set with the predefined constant temporal difference.

[0083] The concept of isolation manager algorithm can be extended to be applied in distributed systems scenarios, as illustrated in Figure 1, so that, it can be instantiated as many Isolation Manager mechanisms instances (108) as necessary to meet system performance and scalability, thus ensuring optimal performance even under heavy loads.

[0084] The same principles of the isolation manager algorithm concept can be applied also globally for scenarios with multiple instances of the Isolation Manager mechanisms instances (108) running. This means that global read time and write time windows can be defined for use by any SAGA and to determine these time windows are defined a global TSW and TSR temporal markers .

[0085] As depicted in Fig. 3, for each Isolation Manager mechanisms instance running, a local TSW (304) is available and is used to determine the instance local write time window .

[0086] The isolation manager algorithm concept to be applied globally effectively and correctly defines that the global write time window corresponds to the local write time window with the minimum TSW (303) . Thus, based on the solution manager algorithm concept, the global TSW (303) is then defined as the minimum TSW (303) and the global TSR (301) is defined as the global TSW (303) affected by the constant temporal difference (A) (302) . Then, the global read time can be determined based on the global TSR (301) .

[0087] DESCRIPTION OF THE EMBODIMENTS

[0088] In a preferred embodiment of the present invention being related to a SAGA Transactions Controller apparatus will be described with respect to Fig. 4.

[0089] Fig. 4 shows a SAGA Transactions Controller system (400) for processing SAGA requests (100) according to the present invention and comprises, as shown in Fig. 4, an Isolation Manager (401) module and a Message Broker (402) module .

[0090] The Isolation Manager (401) module, shown in Fig. 4, receives requests to start SAGA control (109) and terminate SAGA control (110) to provide read and write time windows for transaction operations, at SAGAs start, and notifications of SAGA results at SAGAs completion. The Isolation Manager (401) module uses the isolation manager algorithm to determine read and write time windows that ensures robust isolation and minimize performance overhead in processing distributed transactions. The Isolation Manager (401) module also exchanges the temporal markers TSW (203) and its running mode, through the Message Broker (402) module, with others Isolation Manager (401) modules, to calculate a global TSW (303) and a global TSR (301) to be used by the isolation manager algorithm.

[0091] The Message Broker (402) module, shown in Fig. 4, is a communication bus with the publish-subscribe pattern, which is used to exchange the temporal marker TSW (203) and the Isolation Manager running mode of each Isolation Manager

[0092] (401) instance among the other instances.

[0093] In another embodiment of the present invention being related to SAGA starts control method will be described with respect to Figure 5, which is comprised by the following steps :

[0094] • Step S500 - The SAGA starts control is trigger by receiving a request for SAGA time markers and a SAGA profile is created. The returned time markers need to be calculated and their calculus depends on the running mode of the Isolation Manager instance. Thus, it is required to evaluate the instance running mode;

[0095] • Step S501 - In case the Isolation Manager instance is in the active mode (is controlling SAGAs) it will be used the own TSW and TSR to calculate the time markers ;

[0096] • Step S502 - In case the Isolation Manager instance is in the idle mode (is not controlling any SAGA) it is needed to use the TSW from other Isolation Manager instance. Therefore, it is required to evaluate the running mode of other instances;

[0097] • Step S503 - If there are other Isolation Manager instances in active mode, the new TSW will be the lowest TSW among the TSWs of the others Isolation Manager instances in the active mode, to avoid to push the new TSW too far in the future. The TSR will be calculated as the new TSW minus the constant temporal difference (A) and the Isolation Manager instance changes to active mode;

[0098] • Step S504 - If all other Isolation Manager instances are in idle mode, the new TSW will be calculated as the value of the highest TSW between its own TSW and the TSWs of other Isolation Manager instances, plus the constant temporal difference (A) . The TSR will be calculated as the new TSW minus the constant time difference (A) and the Isolation Manager instance changes to active mode;

[0099] • Step S505 - If there are no other Isolation Manager instances, the new TSW will be the own TSW plus the constant temporal difference (A) . The TSR will be calculated as the new TSW minus the constant temporal difference (A) and the Isolation Manager instance changes to active mode;

[0100] • Step S506 - Determines that the tsWrite time marker, to be returned to the SAGA, is the largest of the timestamps between the TSW and the event timestamp. Determines that the tsRead timestamp, to be returned to SAGA, is the lowest of the timestamps among the TSR and the event timestamp.

[0101] In another embodiment of the present invention being related to SAGA terminates control method will be described with respect to Figure 6, which is comprised by the following steps:

[0102] • Step S600 - The SAGA terminates control is trigger by receiving a notification of SAGA execution result and the SAGA profile is removed, if was the last one the Isolation Manager instance changes to idle mode . The procedures to apply depends on the execution result. Thus, it is required to evaluate the execution result;

[0103] • Step S601 - In case the Isolation Manager instance is in the active mode (is controlling a SAGA) will be used different procedures to calculate TSW and TSR. Therefore, it is required to evaluate the running mode of instances; • Step S602 - If the Isolation Manager instances is in idle mode, the new TSW will be the last tsWrite. The TSR will be calculated as the new TSW minus the constant temporal difference (A) . The TSW and the running mode are published for other Isolation Manager instances;

[0104] • Step S603 - If the other Isolation Manager instances are in idle mode, the new TSW will be calculated as the value of the lowest TSW between its own TSW and the TSWs of other Isolation Manager instances. The new TSR will be calculated, as the value of the lowest TSR between its own TSR and the TSRs of other Isolation Manager instances, minus the constant temporal difference (A) . The TSW and the running mode are published for other Isolation Manager instances;

[0105] • Step S604 - If the other Isolation Manager instances are in active mode, the new TSW will be calculated as the value of the lowest TSW between its own TSW and the TSWs of other Isolation Manager instances. The TSR will be calculated as the new TSW minus the constant temporal difference (A) . The TSW and the running mode are published for other Isolation Manager instances .

Claims

CLAIMS1- A SAGA Transactions Controller apparatus (400) configured to orchestrate transactions in distributed transaction scenarios using the SAGA pattern, comprising : multiple Isolation Manager modules (401) comprising processing means to provide read and write time windows and SAGAS execution results, and comprising communication means to receive SAGAs execution control start and end requests, exchange TSW temporal markers ( 203) and execution modes, and wherein the Isolation Manager module (401) uses an isolation manager algorithm to determine read and write time windows to ensure isolation and minimize performance overhead in distributed transaction processing; a Message Broker module (402) comprising processing means for providing a communication bus with the publish-subscribe pattern and used to exchange, between instances of the Isolation Manager module (401) , the respective execution modes and the marker temporal TSW (203) .2- The SAGA Transactions Controller apparatus (400) according to claim 1, wherein the Isolation Manager module (401) determines the TSW (203) based on the system activity.3- The SAGA Transactions Controller apparatus (400) according to claim 1, wherein the Isolation Manager module (401) is configured to have a pre-defined time window used to define a minimum unsafe time window (202) .4- The SAGA Transactions Controller apparatus (400) according to claim 3, wherein the Isolation Manager module (401) is configured to have a stable starting point with the TSR (201) set to zero and the TSW (203) set with the predefined time window difference.5- Method for controlling the start of SAGA executionsSAGA Transactions Controller apparatus (400) of claims 1 to 4, and comprising the following steps: i. requests for SAGA time markers are received, SAGA's information profile is created and is determined whether the local Isolation Manager instance is in running mode in order to determine the new local temporal markers TSW and TSR; ii. in event the Isolation Manager instance is in the active mode (is controlling SAGAs) it will be used the own TSW and TSR to calculate the time markers; iii. in the case the Isolation Manager instance is in the idle mode (is not controlling any SAGA) and other Isolation Manager instances are in active mode, the new TSW will be the lowest TSW among of the others Isolation Manager instances in the active mode; and the new TSR will be calculated as the new TSW minus the constant time difference (A) , and the Isolation Manager instance changes to active mode; iv. in the case the other Isolation Manager instances are in idle mode, the new TSW will be determined as the value of the highest TSW between own TSW and the TSWs of other Isolation Manager instances, plus the constant temporal difference (A) ; the new TSR will be determined as the new TSW minus theconstant time difference (A) and the Isolation Manager instance changes to active mode; v. in the event there are no other Isolation Manager instances running, the new TSW will be the own TSW plus the constant temporal difference (A) ; the new TSR will be calculated as the new TSW minus the constant time difference (A) and the Isolation Manager instance changes to active mode; vi . determines that the timestamp write time marker, to be returned to the SAGA, is the largest of the timestamps between the new local TSW and the event timestamp and determines that the timestamp read time marker, to be returned to SAGA, is the lowest of the timestamps between the new local TSR and the event timestamp.6- Method for controlling the terminus of SAGA executions, SAGA Transactions Controller apparatus (400) of claims 1 to 4, and comprising the following steps: i. requests for SAGA terminate control are received, SAGA's information profile is removed, in the case is the last SAGA the Isolation Manager instance is changed to idle mode and in the event there are more SAGAs running is determined whether the SAGA execution result had success in order to determine the new local temporal markers TSW and TSR; ii. in case the SAGA execution result had success and the Isolation Manager instance is in the idle mode, the new TSW is the last TSW and the new TSR will be calculated as the new TSW minus the constant time difference (A) ; iii. in the case the other Isolation Manager instances are in idle mode, the new TSW will be determined as the lowest TSW between own TSW and the TSWs ofother Isolation Manager instances; the new TSR will be determined as the lowest TSR between own TSR and the TSRs of other Isolation Manager instances ; iv. in the event there are other Isolation Manager instances running, the new TSW will be determined as the lowest TSW between own TSW and the TSWs of other Isolation Manager instances; the new TSR will be calculated as the new TSW minus the constant time difference (A) ; v. The new local temporal markers TSW and the running mode are published to other Isolation Manager instances .

Images