Distributed transaction optimization method based on compensation mechanism

Abstract

The invention relates to a distributed transaction optimization method based on a compensation mechanism. The method comprises four core links: data standardization: collecting a business event and converting the business event into a structured log, and recording an operation background and a field snapshot; intelligent strategy generation: based on the real-time log data, a dynamic compensation scheme is automatically generated through a strategy engine; task scheduling execution: automatically distributing compensation tasks according to priorities; and a closed-loop optimization mechanism: tracking an execution state in the whole process, and collecting result data for strategy optimization. According to the method, the dynamic compensation strategy engine is constructed, the compensation parameters are adjusted in real time, and non-intrusive architecture design is adopted, so that distributed transaction compensation can be accurately matched with complex business requirements, invalid retry operation is reduced, and execution efficiency is improved; through a visual strategy configuration interface and full-link state tracking, the operation and maintenance complexity is reduced, and rapid iteration of a compensation strategy and accurate fault positioning are realized; rapid access to a new scene / service is realized, and the method can adapt to dynamic environment changes.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of distributed system architecture, in particular to a distributed transaction optimization method based on a compensation mechanism. BACKGROUND

[0002] When designing and developing microservice architecture, distributed transaction management mainly relies on two technical solutions:

[0003] 1. Maximum effort compensation mechanism: fixed strategies such as preset retry count, timeout threshold, etc.

[0004] There is a problem of static compensation strategy, that is, the lack of dynamic adjustment capability due to the dependence on fixed strategies, the inability to intelligently optimize based on real-time data such as historical compensation success rate and network delay, low compensation efficiency, difficulty in meeting dynamic business needs, and inability to adapt to complex and changing business scenarios.

[0005] Taking the "cache context" mechanism as an example:

[0006] In the maximum effort compensation mechanism of distributed transactions, "cache context" is a temporary storage unit for storing transaction compensation strategy parameters (such as retry count, timeout threshold, compensation path, etc.), used to record the context state of the compensation operation. In the transaction process of the e-commerce order payment-inventory deduction scenario, it includes: user payment success → inventory service deduction failure (network timeout) → trigger compensation mechanism (rollback payment). Therefore, in the distributed transaction compensation mechanism, cache context is the carrier of static strategy, and the static compensation strategy leads to the following problems:

[0007] State and strategy coupling: binding compensation logic (such as retry count) and runtime state (such as network delay) in fixed configuration, lack of "perception-decision-adjustment" closed loop.

[0008] Passive response mode: only execute compensation according to preset rules, unable to actively optimize strategy using historical data (such as success rate, delay trend), leading to "strategy lagging behind problems".

[0009] 2. Database proxy layer interception solution: adopt Seata AT mode, implement transaction management through SQL operation interception;

[0010] There is a problem of technical invasiveness, that is, the need to intercept SQL operations through the database proxy layer, which is highly invasive to business code and requires integration of framework-specific logic (such as Seata's @GlobalTransactional annotation, data source proxy), increasing system complexity (multi-layer proxy architecture, SQL interception performance loss) and maintenance cost (code coupling, framework dependency upgrade difficulty, distributed transaction exception positioning complexity), causing unnecessary burden on developers and maintainers.

[0011] The information disclosed in this background section is intended only to enhance the understanding of the general background of the invention and should not be construed as an admission or in any way implying that the information constitutes prior art known to those skilled in the art. Summary of the Invention

[0012] To address the shortcomings of existing technologies, the present invention aims to provide a distributed transaction optimization method based on a compensation mechanism. By constructing a dynamic compensation strategy engine, it enables real-time adjustment of compensation parameters. Adopting a non-intrusive architecture design, it allows distributed transaction compensation to accurately match complex business needs, reduce invalid retries, and improve execution efficiency. Through a visual strategy configuration interface and end-to-end status tracking, it reduces operational complexity, enables rapid iteration of compensation strategies and accurate fault location, and facilitates rapid integration with new scenarios / businesses, adapting to dynamic environmental changes.

[0013] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0014] A distributed transaction optimization method based on a compensation mechanism is characterized by including:

[0015] Event collection and log structuring produce standardized event logs containing context and snapshots, building the data foundation for compensation decisions;

[0016] Real-time strategy decision-making and generation: Based on standardized event logs, a dynamic compensation strategy engine is invoked to generate dynamic compensation strategies, which replace static parameters. The dynamic compensation strategies are dynamically updated according to business scenarios.

[0017] Compensation execution and priority scheduling: After obtaining the dynamic compensation strategy, compensation tasks are scheduled according to priority to decouple the compensation logic from the business code;

[0018] End-to-end status tracking and feedback records compensation results and status data, which are then used as training data for machine learning models to obtain strategy optimization suggestions and visualization reports.

[0019] Based on the above technical solutions, a non-intrusive combination of event listening and aspect-oriented programming is adopted to achieve non-intrusive collection of compensation events, specifically including:

[0020] By listening to transaction messages in RocketMQ and capturing inter-service transaction events using non-intrusive Spring AOP aspects, standardized JSON logs containing xids, failure reasons, and transaction snapshots are generated.

[0021] Based on the above technical solution, when using RocketMQ transaction message listening, business services only need to send standard transaction messages without adding compensation framework annotations, thus achieving no intrusion of business code.

[0022] Based on the above technical solutions, Spring AOP aspects match based on the combined signature of the target service package name, class name, and method name, or by reusing existing annotations. They capture inter-service calls through pointcut expressions, eliminating the need for business parties to add new annotations.

[0023] Based on the above technical solutions, standardized event logs are stored in Elasticsearch for retrieval and in Kafka for real-time log streaming.

[0024] Based on the above technical solution, Flink consumes Kafka log streams and extracts failure rate and network latency as real-time metrics;

[0025] Based on Flink real-time computing and XGBoost machine learning models, the business type and historical retry count in event log data are analyzed to generate a dynamic compensation strategy. The dynamic compensation strategy includes retry information, timeout information, and priority information.

[0026] Based on the above technical solution, the dynamic compensation strategy is dynamically distributed through the Nacos configuration center and accurately matched by xid.

[0027] Based on the above technical solution, the strategy is obtained from Nacos, and the target service compensation logic is called through the standard compensation interface and Feign client. Tasks are scheduled based on priority queues, and the Hystrix circuit breaker mechanism is integrated to prevent compensation storms.

[0028] The standard compensation interface is a Compensable contract without annotations. The target service only needs to implement the interface method and does not need to introduce a compensation framework dependency, thus achieving zero coupling between the compensation logic and the business code.

[0029] Based on the above technical solution, record the compensation results and the number of retries, and update the MySQL status table;

[0030] The compensation results are fed back via Kafka, and Flink updates the model training data.

[0031] Based on the above technical solutions, MySQL stores the compensation status, including the xid, strategy version, and failure reason; SkyWalking generates a call chain graph.

[0032] Grafana provides a visual representation of real-time compensation success rate and reasons for hotspot failures.

[0033] When the failure rate exceeds the threshold, the model automatically adjusts its strategy, and the compensation results are fed back to Flink via Kafka to update the XGBoost model training data. The model then automatically optimizes its strategy parameters based on the latest data.

[0034] The optimized strategy was then reissued via Nacos, affecting subsequent compensation execution.

[0035] The distributed transaction optimization method based on the compensation mechanism described in this invention has the following beneficial effects:

[0036] 1. It adopts a dynamic compensation strategy engine, which dynamically adjusts compensation strategy parameters such as retry count and timeout threshold based on real-time data (compensation history, network status, and service load), and supports visual configuration and hot activation of the strategy.

[0037] The compensation strategy engine perceives business scenarios in real time (such as peak sales load and regional network differences), automatically optimizes compensation parameters, and significantly improves the compensation matching degree in complex scenarios (such as extending the timeout threshold in high latency scenarios to avoid false failures).

[0038] Based on historical compensation data (success rate, failure mode), the compensation strategy engine provides strategy optimization suggestions (such as increasing the number of retries for high-risk businesses), changing "passive compensation" to "proactive prevention" and reducing compensation failures caused by strategy lag.

[0039] The compensation strategy engine records the compensation context (strategy parameters, retry records, and status snapshots), supporting one-click backtracking of the compensation path (such as "the second retry failed due to inventory service timeout"), accurately locating the root cause of the problem.

[0040] 2. Adopt a non-intrusive architecture, with compensation logic pushed down to an independent service (such as the compensation strategy engine), and decoupled from business through standard interfaces (such as CompensationService.rollback(xid)) to eliminate the intrusion of the database proxy layer;

[0041] The compensation logic is decoupled from the business code and is called through the standard interface of the compensation strategy engine. No annotations (such as @GlobalTransactional) or proxy layer integration are required. Developers only need to focus on the business logic, reducing code maintenance costs and learning barriers.

[0042] Eliminating the database proxy layer (such as SQL interception in Seata AT mode), deploying the compensation strategy engine independently, and transforming fault location from "cross-framework troubleshooting" (proxy layer + business layer) to "single compensation service tracing" significantly improves operational efficiency.

[0043] 3. It adopts full-link state tracking, and uses a unique transaction ID (XID) to link the compensation context, record the strategy execution path and state snapshot, and support the observability and accurate rollback of the compensation process. Attached Figure Description

[0044] The present invention includes the following figures:

[0045] The accompanying drawings are provided to better understand the invention and are not intended to unduly limit the scope of the invention. Wherein:

[0046] Figure 1 A flowchart of Embodiment 1 of the distributed transaction optimization method based on the compensation mechanism described in this invention. Wherein:

[0047] Event Acquisition Layer: Captures inter-service transaction events through RocketMQ transaction message listening and non-intrusive Spring AOP aspects, generating standardized event logs.

[0048] Intelligent Decision Layer: Based on Flink real-time computing and XGBoost machine learning models, analyze log data to generate dynamic compensation strategies.

[0049] Compensation Execution Layer: Policies are dynamically issued through the Nacos configuration center, compensation tasks are scheduled according to priority queues, and the Hystrix circuit breaker mechanism is integrated.

[0050] State tracking layer: Records compensation results to MySQL, updates model training through Kafka backflow data, and generates visual reports. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the accompanying drawings. This detailed description is an illustration in conjunction with exemplary embodiments of the invention, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the invention. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0052] like Figure 1 As shown, the distributed transaction optimization method based on the compensation mechanism of the present invention includes:

[0053] Event collection and log structuring produce standardized event logs containing context and snapshots, building the data foundation for compensation decisions;

[0054] For example, by using RocketMQ transaction message listening and non-intrusive Spring AOP aspects, inter-service transaction events can be captured, and standardized JSON logs containing xid, failure reason and transaction snapshot can be generated.

[0055] Real-time strategy decision-making and generation, based on standardized event logs, calls a dynamic compensation strategy engine (hereinafter referred to as the strategy engine) to generate dynamic compensation strategies, replacing static parameters. The dynamic compensation strategies are dynamically updated according to business scenarios. As a preferred solution, only dynamic compensation strategies are available when they exist, and static parameters are used only as a fallback when no strategy is generated. To improve compatibility, after the dynamic compensation strategy is generated, the dynamic compensation strategy and static parameters can coexist, and the user can set how to select them as needed. Both can be adjusted through optional settings.

[0056] For example, based on standardized event logs, the Flink real-time computing platform consumes Kafka log streams to extract real-time metrics such as failure rate and network latency. Simultaneously, combined with the XGBoost machine learning model, features such as business type and historical retries in the event logs are analyzed to dynamically generate compensation strategies that include retry information, timeout information, and priority information. This strategy uses the Nacos configuration center to accurately match the transaction ID (XID), replacing traditional static parameter configuration and enabling the strategy to be dynamically updated according to business scenarios.

[0057] Compensation execution and priority scheduling: After obtaining the dynamic compensation strategy, compensation tasks are scheduled according to priority to decouple the compensation logic from the business code;

[0058] For example, policies are dynamically distributed through the Nacos configuration center, and compensation tasks are executed by calling the standard compensation interface (Compensable contract) according to the priority queue, while integrating the Hystrix circuit breaker mechanism to prevent compensation storms.

[0059] End-to-end status tracking and feedback records compensation results and status data, which are then used as training data for machine learning models to obtain strategy optimization suggestions and visualization reports.

[0060] For example, the compensation results are recorded to a MySQL status table, data is fed back to Flink via Kafka to update the XGBoost model training parameters, and strategy optimization suggestions and Grafana visualization reports are generated.

[0061] In this embodiment, the technical problems caused by the static nature of compensation strategies in the prior art are solved by "dynamic compensation strategy engine → real-time parameter adjustment → accurate matching of business → reduction of invalid operations". By building a dynamic compensation strategy engine, the business scenario and running status are perceived in real time, and the compensation parameters (number of retries, timeout threshold, etc.) are intelligently adjusted.

[0062] In this embodiment, the technical problems caused by the strong technical intrusion in the prior art are solved by “non-intrusive architecture → decoupled design → visual configuration (no code required)”. The compensation logic is made independent as a service and standard interface calls are used to form a non-intrusive architecture, so that the distributed transaction compensation can accurately match complex business needs, such as the load of large-scale promotions and regional network differences.

[0063] This embodiment includes four core components:

[0064] Data standardization: Collect business events and convert them into structured logs, fully recording the operational background and on-site snapshots to form a basis for decision-making.

[0065] Intelligent strategy generation: Based on real-time log data, the strategy engine automatically generates dynamic compensation schemes, replacing fixed parameter configurations and enabling strategies to be adjusted in real time according to business needs.

[0066] Task scheduling and execution: Compensation tasks are automatically assigned according to priority, making the compensation mechanism run independently of the core business system.

[0067] Closed-loop optimization mechanism: Track the execution status throughout the process, collect result data for strategy optimization and generate visual analysis reports, forming a continuous improvement closed loop;

[0068] Dynamic policy generation and model optimization specifically include:

[0069] Flink consumes Kafka log streams in real time and extracts failure rate, network latency, and business type (such as inventory service, payment service) as features;

[0070] The XGBoost model generates policy parameter tuning rules based on historical compensation data (such as the number of retries and reasons for failure), for example:

[0071] When the inventory service failure rate is greater than 20%, the number of retries is increased by 1, and the timeout threshold is extended to 8 seconds.

[0072] When network latency is less than 100ms, shorten the timeout threshold to 60% of the base value;

[0073] The optimized strategy is precisely matched and distributed by xid through the Nacos configuration center. For example, the strategy for xid=TX-001 is retry=4, timeout=8s, priority=3.

[0074] The model is updated every 5 minutes, and parameters are optimized in real time by compensating for the results through Kafka backflow.

[0075] Based on the above technical solutions, a non-intrusive combination of Event Listening (MQ) and Aspect-Oriented Programming (AOP) is adopted to achieve non-intrusive collection of compensation events, specifically including:

[0076] By listening to transactional messages in RocketMQ and capturing inter-service transaction events using non-intrusive Spring AOP aspects, standardized JSON logs containing xids, failure reasons, and transaction snapshots (such as inventory data before deduction) are generated.

[0077] When using RocketMQ transaction message listening, business services only need to send standard transaction messages without adding compensation framework annotations, achieving no intrusion of business code.

[0078] Spring AOP aspects match based on a combination of the target service package name, class name, and method name, or by reusing existing annotations (such as @Transactional). They capture inter-service calls using pointcut expressions (such as execution(**..*Service.*(..))) without requiring the business side to add new annotations.

[0079] This embodiment differs from intrusive proxies. The message listening mechanism itself is non-intrusive, and the AOP implementation is non-intrusive, effectively avoiding intrusion into business code and improving the maintainability and scalability of the system.

[0080] Based on the above technical solutions, standardized event logs are stored in Elasticsearch for retrieval and in Kafka for real-time log streaming.

[0081] Based on the above technical solution, Flink consumes Kafka log streams and extracts failure rate and network latency as real-time metrics;

[0082] Based on Flink real-time computing and XGBoost machine learning models, the business type and historical retry count in event log data are analyzed to generate a dynamic compensation strategy. The dynamic compensation strategy includes retry information, timeout information, and priority information. For example, when the inventory timeout rate is >20%, the number of retry counts is increased by 1, the timeout threshold is increased by 3 seconds, and the priority is 3. Or, if the business type is inventory and the failure type is timeout, and the historical number of retry counts is <3, the number of retry counts is increased by 1, the timeout threshold is increased by 60% of the base value, and the priority is increased by 1 level.

[0083] Based on the above technical solution, the dynamic compensation strategy is dynamically distributed through the Nacos configuration center and accurately matched by xid, such as TX-001:retry=4,timeout=8s,priority=3.

[0084] Based on the above technical solution, the strategy is obtained from Nacos, and the target service compensation logic is called through the standard compensation interface and Feign client. Tasks are scheduled based on priority queues, and the Hystrix circuit breaker mechanism is integrated to prevent compensation storms.

[0085] The standard compensation interface is a Compensable contract without annotations; the target service only needs to implement the interface method (e.g., boolean compensate(Map...)).<String,Object> Context)) eliminates the need to introduce a compensation framework dependency, achieving zero coupling between compensation logic and business code.

[0086] Based on the above technical solution, record the compensation result (success / failure) and the number of retries, and update the MySQL status table, such as xid = TX-001: retries 3 / 4 times;

[0087] The compensation results are fed back via Kafka, and Flink updates the model training data.

[0088] Based on the above technical solutions, MySQL stores the compensation status, including the xid, strategy version, and failure reason; SkyWalking generates a call chain graph.

[0089] Grafana provides a visual representation of real-time compensation success rate and reasons for hotspot failures.

[0090] When the failure rate exceeds the threshold, the model automatically adjusts its strategy, and the compensation result (success / failure) is fed back to Flink via Kafka to update the XGBoost model training data. The model automatically optimizes the strategy parameters based on the latest data (e.g., when the failure rate exceeds the historical average + 1σ, the number of retries is increased; when the network latency is less than the threshold, the timeout threshold is shortened).

[0091] The optimized strategy was then reissued via Nacos, affecting subsequent compensation execution.

[0092] The contents not described in detail in this specification are existing technologies known to those skilled in the art.

[0093] The above description is only a preferred embodiment of the present invention. The scope of protection of the present invention is not limited to the above embodiments. Any equivalent modifications or changes made by those skilled in the art based on the content disclosed in the present invention should be included in the scope of protection set forth in the claims.

Claims

1. A distributed transaction optimization method based on a compensation mechanism, characterized in that, include: Event collection and log structuring produce standardized event logs containing context and snapshots, building the data foundation for compensation decisions; Real-time strategy decision-making and generation: Based on standardized event logs, a dynamic compensation strategy engine is invoked to generate dynamic compensation strategies, which replace static parameters. The dynamic compensation strategies are dynamically updated according to business scenarios. Compensation execution and priority scheduling: After obtaining the dynamic compensation strategy, compensation tasks are scheduled according to priority to decouple the compensation logic from the business code; End-to-end status tracking and feedback records compensation results and status data, which are then used as training data for machine learning models to obtain strategy optimization suggestions and visualization reports.

2. The distributed transaction optimization method based on a compensation mechanism as described in claim 1, characterized in that, A non-intrusive combination of event listening and aspect-oriented programming is used to achieve non-intrusive acquisition of compensation events, specifically including: By listening to transaction messages in RocketMQ and capturing inter-service transaction events using non-intrusive Spring AOP aspects, standardized JSON logs containing xids, failure reasons, and transaction snapshots are generated.

3. The distributed transaction optimization method based on a compensation mechanism as described in claim 2, characterized in that, When using RocketMQ transaction message listening, business services only need to send standard transaction messages without adding compensation framework annotations, achieving no intrusion of business code.

4. The distributed transaction optimization method based on a compensation mechanism as described in claim 2, characterized in that, Spring AOP aspects are matched based on a combination of the target service package name, class name, and method name, or by reusing existing annotations. They capture inter-service calls through pointcut expressions, eliminating the need for business parties to add new annotations.

5. The distributed transaction optimization method based on a compensation mechanism as described in claim 1, characterized in that, Standardized event logs are stored in Elasticsearch for retrieval and in Kafka for real-time log streaming.

6. The distributed transaction optimization method based on a compensation mechanism as described in claim 5, characterized in that, Flink consumes Kafka log streams and extracts failure rate and network latency as real-time metrics; Based on Flink real-time computing and XGBoost machine learning models, the business type and historical retry count in event log data are analyzed to generate a dynamic compensation strategy. The dynamic compensation strategy includes retry information, timeout information, and priority information.

7. The distributed transaction optimization method based on a compensation mechanism as described in claim 1, characterized in that, The dynamic compensation strategy is dynamically distributed through the Nacos configuration center and accurately matched by xid.

8. The distributed transaction optimization method based on a compensation mechanism as described in claim 7, characterized in that, The strategy is obtained from Nacos, and the target service compensation logic is called through the standard compensation interface and Feign client. Tasks are scheduled based on priority queues, and Hystrix circuit breaker mechanism is integrated to prevent compensation storms. The standard compensation interface is a Compensable contract without annotations. The target service only needs to implement the interface method and does not need to introduce a compensation framework dependency, thus achieving zero coupling between the compensation logic and the business code.

9. The distributed transaction optimization method based on a compensation mechanism as described in claim 1, characterized in that, Record the compensation results and the number of retries, and update the MySQL status table; The compensation results are fed back via Kafka, and Flink updates the model training data.

10. The distributed transaction optimization method based on a compensation mechanism as described in claim 9, characterized in that, MySQL stores the compensation status, including the xid, strategy version, and failure reason; SkyWalking generates a call chain graph. Grafana provides a visual representation of real-time compensation success rate and reasons for hotspot failures. When the failure rate exceeds the threshold, the model automatically adjusts its strategy, and the compensation results are fed back to Flink via Kafka to update the XGBoost model training data. The model then automatically optimizes its strategy parameters based on the latest data. The optimized strategy was then reissued via Nacos, affecting subsequent compensation execution.

Images