A system and method for eventually consistent processing of transactional messages

Abstract

This application provides a system and method for eventually consistent processing of transaction messages. The method includes: acquiring transaction state metadata and business message bodies related to transaction messages in a distributed system; storing the business message bodies in a distributed message queue and storing the transaction state metadata in an in-memory database; constructing a transaction state index structure; determining the physical offset based on the business message bodies; creating transaction state records in the in-memory database based on the physical offsets; generating transaction tasks to be revisited based on the transaction state records; determining the revisit interval based on the transaction creation time and system load factor; executing the transaction tasks to be revisited; when a change in transaction state from pending confirmation to confirmed confirmation is detected, retrieving the business message bodies based on the physical offsets; and performing double confirmation on the retrieved business message bodies. This application solves the performance bottleneck and low recovery efficiency problems caused by the strong coupling between transaction state and message bodies.

Description

Technical Field

[0001] This invention relates to the field of distributed systems technology, and specifically to a system and method for eventually consistent processing of transactional messages. Background Technology

[0002] Transaction message processing in distributed systems is a key technical area for ensuring data consistency. Current distributed systems need to guarantee eventual data consistency while maintaining high throughput, which places stringent requirements on transaction message processing mechanisms.

[0003] Currently, in distributed systems, a two-phase commit protocol is typically used to process transactional messages to ensure data consistency between different services. This approach mainly relies on message queue systems such as RocketMQ and Kafka. These systems usually pre-commit transactional messages during the message sending phase and then decide whether to commit or rollback the pre-committed messages based on the local transaction execution results, thereby guaranteeing the atomicity and consistency of distributed transactions.

[0004] Current mainstream transactional message processing solutions share a common characteristic: transaction states (such as PREPARED / COMMITTED / ROLLBACKED) are tightly coupled with the message body and stored in the same physical file. For example, in RocketMQ, the transactional message state and message content are stored together in the CommitLog file. When the transaction state needs to be updated, the system needs to rewrite the entire message block. Furthermore, after a broker restart, the entire CommitLog file needs to be scanned to rebuild the transaction state tree, and a passive pull mode is used during transaction state lookup, requiring the transmission of the complete message content, not just the transaction ID.

[0005] However, this processing model has significant technical drawbacks: First, due to the strong coupling between state and message body storage, high-frequency state updates lead to frequent random write operations, severely impacting system throughput; second, fault recovery time is linearly related to message volume, and for large-scale systems, recovery time can reach the minute level; finally, in passive lookup mode, state update latency is high, and transmitting the complete message content causes unnecessary bandwidth consumption. Therefore, a reliable transactional message processing technology is currently needed. Summary of the Invention

[0006] In view of this, embodiments of this application provide an eventual consistency processing system and method for transaction messages to solve the performance bottleneck and low recovery efficiency caused by the strong coupling between transaction state and message body in the prior art.

[0007] In a first aspect, embodiments of this application provide a method for eventually consistent processing of transactional messages, comprising the following steps:

[0008] Obtain transaction status metadata and business message bodies related to transaction messages in the distributed system, store the business message bodies in a distributed message queue, and store the transaction status metadata in an in-memory database;

[0009] Based on the transaction status metadata stored in the memory database, a transaction status index structure is constructed. The transaction status index structure contains the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction status and transaction data to be retrieved, which indicates that the transaction status needs to be retrieved.

[0010] Based on the business message body in the distributed message queue, a physical offset is determined. Based on the physical offset, a transaction status record corresponding to the business message body is created in the memory database. A transaction task to be queried is generated according to the transaction status record and added to the transaction status index structure. The physical offset is used to indicate the position identifier of any business message body in the distributed message queue. The transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time.

[0011] Based on the transaction creation time and the pre-acquired system load factor, the rollback interval is determined, and the transaction task to be rolled back is executed according to the rollback interval to obtain the rollback processing result;

[0012] When the transaction status is detected to have changed from pending confirmation to transaction confirmation based on the back lookup processing result, the corresponding business message body is pulled from the distributed message queue according to the physical offset in the corresponding transaction status record;

[0013] The retrieved business message body undergoes double confirmation, and after successful double confirmation, a message processing result is generated to indicate that the message processing is complete.

[0014] As an improvement to the above solution, storing the business message body in a distributed message queue and storing the transaction state metadata in an in-memory database includes:

[0015] Based on the pre-acquired data access characteristics, business message bodies with a message volume greater than a preset volume threshold and an access frequency lower than a preset frequency threshold are stored in the distributed message queue, while transaction status metadata with a message volume less than the volume threshold but an access frequency higher than the frequency threshold are stored in the memory database.

[0016] Configure sequential write parameters for the distributed message queue and configure a memory policy for the in-memory database, wherein the sequential write parameters include the number of partitions and the replication factor, and the memory policy is used to indicate the enabling of a persistence mechanism composed of a hybrid AOF and RDB.

[0017] As an improvement to the above scheme, the transaction state index structure includes a transaction state hash table and a transaction index to be queried; the step of constructing the transaction state index structure based on the transaction state metadata stored in the in-memory database includes:

[0018] Based on the transaction status metadata stored in the memory database, a transaction status hash table is created for each transaction in the memory database. The key in the transaction status hash table represents the transaction identifier of each transaction, and the value in the transaction status hash table includes the transaction status, offset, creation time and timeout of the transaction. The transaction status includes the pending confirmation status, the confirmed transaction status and the rollback transaction status.

[0019] Based on the transaction status metadata, the transaction index to be retried is created in the memory database. The transaction index to be retried includes at least one transaction to be retried and its corresponding transaction identifier, and a retried evaluation score determined by the timeout period. The transaction to be retried is used to characterize the transaction whose transaction status needs to be retried.

[0020] Listen for transaction status change events related to the transaction status hash table, and when any transaction status change event is detected, trigger the sending of a status change notification message to a designated channel. The status change notification message is used to indicate the transaction identifier of the transaction involved in the transaction status change event and the transaction status after the status change.

[0021] As an improvement to the above scheme, the step of determining the physical offset based on the business message body in the distributed message queue, creating a transaction state record corresponding to the business message body in the memory database based on the physical offset, and generating a transaction task to be queried based on the transaction state record to be added to the transaction state index structure includes:

[0022] The obtained business message body is written into the distributed message queue, and the message location identifier of the business message body in the distributed message queue is used as the physical offset. The corresponding queue writing result is recorded. The queue writing result is used to indicate whether the business message body has been successfully written into the distributed message queue.

[0023] Based on the physical offset and the transaction identifier, a transaction status record corresponding to the business message body is created in the memory database, wherein the transaction status in the transaction status record is set to the pending confirmation status, and the transaction creation time is used to indicate the timestamp of the current creation of the transaction status record;

[0024] When the timeout recorded by the transaction status exceeds a preset timeout threshold, a transaction task to be reviewed is generated, and the transaction identifier corresponding to the transaction task to be reviewed and the timeout are added to the transaction index to be reviewed.

[0025] When a database operation failure message is detected, the business message body is rewritten to the in-memory database according to a preset retry strategy. After a retry failure, an alarm notification is generated, and a compensation execution log and a list of compensation failed transactions are recorded. The database operation failure message includes at least one of a creation failure message and a task creation failure message. The creation failure message indicates that the creation operation of the transaction status record in the in-memory database failed, and the task creation failure message indicates that the operation of adding the pending query transaction task to the pending query transaction index failed. The retry strategy includes the number of retries and the retry interval. The compensation execution log records the actual number of retries and the operation results.

[0026] As an improvement to the above scheme, the step of determining the rollback interval based on the transaction creation time and the pre-acquired system load factor, and executing the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result, includes:

[0027] Based on preset system configuration parameters and system load factor, the basic back-lookup interval is calculated. The system configuration parameters include minimum interval value and maximum interval value.

[0028] Based on the monitored CPU utilization, memory usage, network throughput, and IO metrics, a system load factor calculation model is constructed, and the system load factor is updated according to the system load factor calculation model.

[0029] The rollback interval of the transaction is determined based on the transaction age, the preset transaction timeout threshold, and the current updated base rollback interval. The transaction age is the difference between the current time and the transaction creation time.

[0030] When the updated system load factor exceeds the preset load threshold, overheat protection is triggered, the callback interval is increased, and the corresponding overheat warning log is recorded.

[0031] As an improvement to the above solution, the step of performing double confirmation on the retrieved business message body, and generating a message processing result indicating the completion of message processing after the double confirmation is successful, includes:

[0032] After successfully processing the retrieved business message body, perform message parsing, format validation and deduplication of the retrieved business message body to obtain the message body to be confirmed;

[0033] Write the processing status of the transaction corresponding to the message body to be confirmed into the memory database, and add the message body to be confirmed to the local retry queue when the processing status writing fails.

[0034] When the processing status is successfully written, the message body to be confirmed is pushed to the business party, so that the business party executes the business logic according to the message body to be confirmed and returns the corresponding processing receipt result.

[0035] When the processing receipt indicates a service failure, the message body to be confirmed is added to the service retry queue, and then re-pushed to the service provider according to the service retry queue.

[0036] When the processing receipt indicates that the business is successful, a business confirmation field is added to the transaction status hash table, and a message processing result is generated to indicate that the message processing is complete.

[0037] As an improvement to the above solution, the method further includes:

[0038] Based on the policy parameters configured by the memory policy and the preset persistence requirement data, the AOF rewrite policy is optimized to obtain a persistent file. The persistent file includes optimized persistent configuration data, RDB snapshot file, AOF log file and persistent performance monitoring data.

[0039] When the Broker starts, it loads the RDB snapshot file into the in-memory database to restore the basic state, completes the incremental data through the AOF log file, forms a Broker instance with the basic state and the incremental data, and reads the transaction state metadata from the in-memory database.

[0040] Compare the transaction status in the transaction status metadata with the business message body of the corresponding transaction in the distributed message queue;

[0041] When the transaction status in the transaction status metadata is detected to be the transaction confirmation status, but there is no corresponding business message body in the distributed message queue, a verification result is generated to indicate the inconsistency of the status, and the business message body is rewritten.

[0042] When the transaction status in the transaction status metadata is detected to be the transaction rollback status, but a corresponding business message body exists in the distributed message queue, a verification result indicating the inconsistency of the status is generated, and the business message body is deleted from the distributed message queue.

[0043] As an improvement to the above solution, the step of constructing a transaction state index structure based on the transaction state metadata stored in the in-memory database includes:

[0044] According to the hash value of the transaction identifier, the transaction status hash table stored in the memory database is divided into multiple logical blocks, and corresponding block data and transaction block mapping table are generated. The block data includes the number of transactions and transaction status of the logical block.

[0045] Based on the transaction association attributes of the transactions within the logical block, the association strength between the transactions is determined. The transaction association attributes are used to indicate the dependency relationship between the transactions when a state change occurs.

[0046] Based on the transaction state, the transaction association attributes, and the association strength, a sparse matrix is ​​constructed, and the elements in the sparse matrix represent the association strength between the transactions.

[0047] The sparse matrix is ​​factorized using the preset FSAI algorithm, and the approximate inverse matrix of the logical block is calculated based on the decomposition result.

[0048] Based on the transaction state hash table, the change trend of the transaction state is smoothed by using a preset Chebyshev polynomial to obtain the transaction state change trend curve. The transaction state change trend curve is used to indicate the lookup load trend and the state update frequency trend.

[0049] Based on the approximate inverse matrix, an index structure for the logical blocks is constructed according to the basic attributes of the transaction state. The index structure is then refined according to preset secondary attributes to obtain a multi-level index structure. The basic attributes include time attributes, and the secondary attributes include the producer identifier of the transaction.

[0050] Based on the monitored system load factor, the block data of the logical block, and the transaction status change trend curve, the number of transactions in the logical block is adjusted, and the index level with a query frequency exceeding the preset query frequency threshold is identified in the multi-level index structure, thereby increasing the update frequency of the index level.

[0051] The transaction block mapping table is updated based on the adjusted number of transactions, and the multi-level index structure is optimized based on the adjusted update frequency.

[0052] As an improvement to the above solution, after performing double confirmation on the retrieved business message body and generating a message processing result indicating the completion of message processing after the double confirmation is successful, the following method is further included:

[0053] Based on the business dependencies between the transactions, the associated transaction groups are organized into a fractal network with self-similar properties, wherein each node in the fractal network represents a transaction, the edges in the fractal network represent the business dependencies between the transactions, and the associated transaction groups are used to indicate multiple interrelated transactions.

[0054] For each associated transaction group in the fractal network, a core transaction is selected to represent the key node in the link, and the core transaction, its preceding transaction, and its following transaction are combined to form a closed triangular mesh.

[0055] For adjacent related transaction groups, a new triangular mesh is connected by the shared edge between the two adjacent groups to traverse the transactions of the business link and form a zipper structure based on the relevant triangular gateways;

[0056] Based on the fractal network and the zipper structure, a triangular zipper topology model is constructed.

[0057] Based on the pre-acquired historical transaction execution data and the initial execution data of the current transaction, interpolation prediction is performed using a preset rational cubic function to obtain the execution time and resource consumption prediction results of the current transaction. The initial execution data is used to indicate the execution time and resources consumed during message processing.

[0058] Based on the aforementioned triangular zipper topology model and the real-time execution time and CPU utilization monitored during message processing, a set of positive constraint rules is set for the relevant parameters in the current transaction processing process. The set of positive constraint rules includes parameter range constraint rules and dependency constraint rules.

[0059] Based on the triangular zipper topology model and the set of positive constraint rules, each transaction link is regarded as a path. The path weight of the path is determined based on the execution time of the transaction, the predicted resource consumption of the transaction, and the preset business priority coefficient.

[0060] Based on the currently updated system load factor and the path weight, the message retrieval order of the business message body is determined;

[0061] Starting from the node of the abnormal transaction, trace the minimum impact unit in the triangular zipper topology model. The minimum impact unit is used to indicate the triangular mesh containing only the abnormal transaction.

[0062] Based on the anomaly type of the abnormal transaction, a repair strategy is matched and executed, and the validity of the parameters after repair is verified through the positive constraint rule set.

[0063] After successful verification, the repair results will be updated to the fractal network and the triangular zipper topology model.

[0064] As an improvement to the above solution, the method further includes:

[0065] Based on the triangular zipper topology model, a closed-loop verification mechanism is constructed. The closed-loop verification mechanism is used to indicate the integrity verification of the closed-loop path corresponding to the triangular mesh in the triangular zipper topology model.

[0066] When the integrity verification of the closed-loop path fails, the rational cubic function is used to fit and predict the time series data of the preset abnormal scenario, generating a data model for abnormal reproduction.

[0067] Based on the data model, the positive constraint rule set is used to impose boundary restrictions on the relevant transaction data of the triangular mesh corresponding to the closed-loop path.

[0068] Secondly, embodiments of this application also provide an eventual consistency processing system for transactional messages, including:

[0069] A separate storage module is used to obtain transaction status metadata and business message bodies related to transaction messages in the distributed system, store the business message bodies in a distributed message queue, and store the transaction status metadata in an in-memory database;

[0070] The status management module is used to construct a transaction status index structure based on the transaction status metadata stored in the memory database. The transaction status index structure contains the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction status and transaction data to be retrieved, which indicates that the transaction status needs to be retrieved.

[0071] The transaction commit module is used to determine the physical offset based on the business message body in the distributed message queue, create a transaction status record corresponding to the business message body in the memory database based on the physical offset, and generate a transaction task to be queried based on the transaction status record to be added to the transaction status index structure. The physical offset is used to indicate the position identifier of any business message body in the distributed message queue, and the transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time.

[0072] The rollback control module is used to determine the rollback interval based on the transaction creation time and the pre-acquired system load factor, and execute the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result;

[0073] The message delivery module is used to retrieve the corresponding business message body from the distributed message queue according to the physical offset in the corresponding transaction status record when the transaction status is detected to have changed from pending confirmation to transaction confirmation based on the back lookup processing result.

[0074] The double confirmation module is used to double-confirm the retrieved business message body, and after the double confirmation is successful, it generates a message processing result to indicate that the message processing is complete.

[0075] The eventual consistency processing system and method for transaction messages provided in this application reduces random I / O overhead caused by state updates and improves system throughput by separating and storing transaction state metadata from business message bodies. It then constructs an efficient memory index structure and dynamic back lookup, optimizes transaction state management and query efficiency, realizes dynamic back lookup and fast fault recovery, and ensures accurate message delivery through a double confirmation mechanism, thereby improving data consistency assurance and solving the performance bottleneck and low recovery efficiency problems caused by the strong coupling between transaction state and message body in the prior art. Attached Figure Description

[0076] To more clearly illustrate the technical solutions of the embodiments of this disclosure, the accompanying drawings used in the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 A flowchart illustrating an eventual consistency processing method for transaction messages provided in an embodiment of this application;

[0078] Figure 2 This is a schematic diagram of the structure of an eventual consistency processing system for transaction messages provided in an embodiment of this application. Detailed Implementation

[0079] The embodiments of this application will now be described in detail with reference to the accompanying drawings.

[0080] like Figure 1 As shown, the present invention provides a method for eventually consistent processing of transaction messages, the method comprising the following steps S1 to S6.

[0081] Step S1: Obtain transaction status metadata and business message body related to transaction messages in the distributed system, store the business message body in the distributed message queue, and store the transaction status metadata in the memory database.

[0082] In a distributed system environment, a complete transaction message typically contains two key data parts: transaction state metadata and the business message body. Transaction state metadata refers to lightweight data describing the transaction processing status and related attributes, including transaction identifier, transaction status, creation time, timeout, etc. The business message body refers to the actual business data carried by the transaction, which is usually larger in size, such as order information, user operations, and other specific content. This step first obtains these two types of data through the system interface, and then stores them in different storage media according to their data characteristics.

[0083] It should be noted that a distributed message queue is a middleware system capable of storing, transmitting, and processing messages, characterized by high throughput, scalability, and fault tolerance. In this application, Kafka is chosen as the distributed message queue, employing a partitioned log structure design to achieve efficient sequential writing and reading. Business message bodies are typically large in size and accessed relatively infrequently; therefore, storing them in Kafka can fully leverage its sequential writing performance advantages and reduce random I / O overhead.

[0084] In-memory databases are database systems that primarily rely on memory for data storage and management, offering low latency and high concurrency capabilities. This application selects Redis as the in-memory database, which supports rich data structures and operation commands, and provides a persistence mechanism to ensure data security. Transaction state metadata is small in size but frequently accessed and updated; storing it in Redis significantly improves the efficiency of state querying and updating. Redis also supports a publish-subscribe pattern, facilitating real-time notification of changes in transaction state.

[0085] In this application, the data storage location is dynamically determined based on pre-configured data access characteristic parameters. To optimize storage performance, an appropriate number of partitions and replication factor are configured for Kafka to ensure balanced data distribution and reliability. Simultaneously, a suitable memory strategy and persistence method are configured for Redis, such as enabling a hybrid persistence mechanism of AOF and RDB, to ensure both data security and performance.

[0086] Step S2: Based on the transaction status metadata stored in the memory database, construct a transaction status index structure. The transaction status index structure contains the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction status and transaction data to be retrieved, which indicates that the transaction status needs to be retrieved.

[0087] In this application, a transaction status hash table is created in Redis for each transaction. The key is the transaction ID, and the values ​​include key metadata such as transaction status (PREPARED / COMMITTED / ROLLBACKED), Kafka message offset, transaction creation time, and timeout. Simultaneously, a transaction index for retrospective lookup is created using Redis's ZSET structure, with the transaction ID as a member and the expiration time as a score, supporting efficient retrieval by time range. Furthermore, a transaction status change notification channel based on Redis PUBSUB is designed to publish a status change event when the transaction status changes.

[0088] It should be noted that the transaction state index structure is a core component for the system to efficiently manage and retrieve transaction states. It organizes and maintains transaction state information in an in-memory database, providing fast query and update capabilities. This step builds two main index structures based on the transaction state metadata stored in Redis: a transaction state hash table and a transaction index to be retrieved.

[0089] On one hand, the transaction status hash table is a mapping table implemented using Redis's hash data structure, with each transaction having a unique entry. The key of the hash table is the transaction identifier (usually a globally unique UUID or other identifier), and the value is a hash structure containing multiple fields that store the core attributes of the transaction. These attributes include: transaction status (possible values ​​include PREPARED / pending confirmation, COMMITTED / transaction confirmation, ROLLBACKED / transaction rollback), physical offset (the position of the business message body in Kafka), creation time (the timestamp of transaction creation), and timeout (the expected completion time limit of the transaction). Redis's hash structure supports reading and writing individual fields without needing to read or write the entire data block, reducing network transmission and processing overhead.

[0090] On the other hand, the transaction index to be queried is a priority queue implemented using Redis's ZSET (sorted set) data structure, used to efficiently organize and retrieve transactions that require status re-querying. In this structure, the members of the set are transaction identifiers, and the corresponding scores are priority values ​​calculated based on timeout periods. This design enables the system to efficiently retrieve transactions to be queried in chronological order using Redis's ZRANGEBYSCORE command, supporting range queries and batch processing. The closer a transaction's timeout is, the higher its priority in the queue, and the more likely it will be queried.

[0091] In addition to static data structures, this application also implements a state change notification mechanism using Redis's publish-subscribe (PUBSUB) functionality. When a transaction state changes (e.g., from PREPARED to COMMITTED), a notification message containing the transaction identifier and the new state is published to a designated channel. This real-time notification mechanism enables consumers to respond promptly to changes in transaction state without having to frequently poll for the state, significantly reducing system communication overhead and latency.

[0092] Optionally, when constructing these index structures, this application considers data consistency and concurrency control, using Redis's atomic operations and transaction features to ensure the atomicity and isolation of index operations, avoiding data inconsistency issues in high-concurrency environments.

[0093] Step S3: Based on the business message body in the distributed message queue, determine the physical offset, create a transaction status record corresponding to the business message body in the memory database based on the physical offset, and generate a transaction task to be queried based on the transaction status record to be added to the transaction status index structure. The physical offset is used to indicate the position identifier of any business message body in the distributed message queue, and the transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time.

[0094] In this application, the producer first generates a globally unique transaction ID, then writes the business message body to Kafka, and obtains the physical offset as the message location identifier. Based on the obtained message offset and transaction ID, a transaction status record is created in Redis, with the status set to PREPARED, and the transaction creation time and timeout time are recorded. The transaction ID and timeout time are added to a sorted set to be queried in Redis, facilitating subsequent status retrieval by the coordination layer.

[0095] It should be noted that transaction commit is a crucial step in distributed transaction processing, and this step implements an optimized transaction commit process. First, the producer generates a globally unique transaction ID, typically achieved through UUIDs or a distributed ID generation algorithm, ensuring that each transaction is uniquely identified throughout the system. Then, the producer writes the business message body to Kafka, and Kafka assigns a physical offset to this message.

[0096] In this application, the physical offset is an identifier in Kafka used to precisely locate the message position. It consists of the topic name, partition number, and offset, in the form of "topic-partition-offset". This identifier is the unique location coordinate of the message in Kafka, allowing for precise retrieval and reading of the corresponding message content in subsequent processes without traversing the entire message log. Specifically, after obtaining the physical offset, a transaction status record is created in Redis. In the implementation, the Redis HMSET command is used to create a hash table containing multiple fields, with the transaction identifier as the key. This hash table contains the following key fields: transaction identifier (globally unique transaction ID), physical offset (the message's location identifier in Kafka), transaction status (initially set to pending confirmation), transaction creation time (recording the timestamp at creation), and timeout (the set transaction timeout limit used for rollback strategies).

[0097] Next, transaction tasks to be retried are generated based on the transaction status records. Specifically, if a transaction's timeout exceeds a preset timeout threshold (e.g., 30 seconds), the transaction is added to the index of transactions to be retried. This is achieved using Redis's ZADD command, adding the transaction identifier as a member and the timeout time as a score to the ordered set to be retried. Therefore, this application, through the coordination layer, can efficiently retrieve and process transactions to be retried according to their timeout order.

[0098] To handle potential partial failures, this application implements a write compensation mechanism. For example, if a message is successfully written to Kafka but the Redis operation fails (due to network issues or Redis service unavailability), the system will retry writing the state to Redis according to a preset retry strategy (e.g., a maximum of 3 retries with a 2-second interval between each retrieval). If the process still fails after multiple retries, an alarm notification is generated, and a detailed compensation execution log is recorded, marking the transaction requiring manual intervention.

[0099] Therefore, this application avoids the problem of rewriting the entire message for state updates in traditional solutions by separating storage, that is, storing the business message body in Kafka and storing the transaction state record in Redis, thus significantly improving the system's write performance and state update efficiency.

[0100] Step S4: Based on the transaction creation time and the pre-acquired system load factor, determine the rollback interval, and execute the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result.

[0101] In this application, dynamic lookup can be used to dynamically adjust the lookup frequency based on transaction age and system load, balancing system load and transaction timeliness. Here, lookup refers to the coordination layer periodically checking transactions in the PREPARED state and querying whether their final state is COMMITTED or ROLLBACKED to decide whether to deliver the message or clean up the transaction record.

[0102] Specifically, the baseline lookup interval is first calculated based on preset system configuration parameters and the current system load factor. The system load factor is an indicator that comprehensively reflects the current stress status of the system, with a value ranging from 0 to 1; a higher value indicates a higher system load. It is calculated by monitoring various system indicators, including CPU utilization, memory usage, network throughput, and I / O metrics.

[0103] Transaction age refers to the time difference between the transaction creation time and the current time, reflecting how long the transaction has existed. The final rollback interval is dynamically adjusted based on the ratio of the transaction age to a preset timeout threshold. Therefore, this application enables transactions nearing timeout to receive a higher rollback frequency, while newly created transactions use a longer rollback interval, optimizing the efficiency of system resource utilization.

[0104] Optionally, this application also implements an overheat protection mechanism. When the system load factor is detected to exceed a preset threshold (e.g., 0.9), overheat protection is triggered, significantly increasing the callback interval (e.g., 2-4 times the normal interval) and recording a corresponding warning log. This mechanism prevents callback operations from further exacerbating system pressure under high system load, thus improving system stability and reliability.

[0105] In this application, the rollback task is executed using the Redis ZRANGEBYSCORE command, which efficiently retrieves transactions matching the time criteria in batches from the transaction index to be rolled back. This application initiates a rollback request for each retrieved transaction, inquiring about the current transaction status from the transaction initiator. The rollback request typically includes the transaction identifier and other necessary authentication information, and is sent to the transaction initiator via HTTP or RPC.

[0106] After receiving a rollback request, the transaction initiator checks the execution status of its local transaction and returns the current transaction state (COMMITTED, ROLLBACKED, or still PREPARED). If the state is already determined (COMMITTED or ROLLBACKED), the coordinating layer updates the transaction state record in Redis and removes the transaction from the index to be rolled back. If the state is still PREPARED and the final timeout period has not expired, it remains unchanged and waits for the next rollback; if the final timeout period has expired, the transaction is processed according to a preset strategy (such as default rollback).

[0107] Therefore, this application improves the system's adaptability and resource utilization efficiency through dynamic back lookup, ensuring timely processing of transactions while avoiding unnecessary waste of system resources.

[0108] Step S5: When the transaction status changes from pending confirmation to transaction confirmation based on the backtracking result, the corresponding business message body is retrieved from the distributed message queue according to the physical offset in the corresponding transaction status record.

[0109] In this application, message delivery is a crucial step in passing confirmed transaction messages to consumers for processing. This step implements a real-time message delivery mechanism based on transaction state changes. When a change from PREPARED (pending confirmation) to COMMITTED (transaction confirmation) is detected through a lookup process or proactive notification from the producer, the message delivery process is triggered.

[0110] It should be noted that state change detection is primarily implemented in two ways: first, by directly obtaining state changes during the rollback process; and second, by receiving state change notifications through Redis's publish-subscribe (PUBSUB) mechanism. In the second method, when a transaction state changes, the producer or coordinator publishes a state change event, including the transaction identifier and the new state, to a designated channel using Redis's PUBLISH command. Consumers subscribe to this channel using the SUBSCRIBE command to receive state change notifications in real time, eliminating the need for frequent polling and significantly reducing system overhead.

[0111] Once the state changes to COMMITTED, this application immediately pulls the corresponding business message body from Kafka based on the physical offset stored in the transaction state record. The physical offset includes the topic name, partition number, and offset, enabling consumers to accurately locate and read the target message without scanning or filtering a large number of irrelevant messages.

[0112] Specifically, Kafka message retrieval is implemented through the Consumer API, mainly including the following steps: creating a KafkaConsumer instance and configuring necessary parameters (such as consumer group ID, server address, etc.); using the seek method to set the consumption position to the specified offset; calling the poll method to retrieve messages at the specified position; and processing the retrieved messages, including deserialization, verification, and other operations.

[0113] Therefore, this application, through precise positioning and reading, rather than continuous consumption in traditional consumption models, can directly jump to a specified location to read specific messages, avoiding the latency and resource waste that may result from continuous consumption. Regarding partition allocation, this application maintains a mapping relationship from physical offsets to consumer instances, ensuring that messages at specific offsets are always processed by the correct consumer. In practical implementation, this application also needs to handle various abnormal situations, such as messages at specified offsets expiring and being deleted, network connection interruptions, and Kafka service unavailability. To address these situations, this application implements corresponding fault tolerance and retry strategies, such as configuring appropriate retry counts and intervals, as well as alarm and logging mechanisms after failures, to ensure the reliability of message delivery and the stability of the system.

[0114] Step S6: Double-confirm the retrieved business message body, and after the double confirmation is successful, generate a message processing result to indicate that the message processing is complete.

[0115] In this application, the double confirmation mechanism is a crucial step in ensuring that messages are not duplicated or lost. Through verification at two levels—consumer confirmation and business confirmation—it provides a strong consistency guarantee for message processing. This step details the implementation of each stage of this mechanism.

[0116] First, after a consumer successfully pulls the business message body from Kafka, it performs a series of preprocessing operations, including: message parsing: deserializing the message in binary or JSON format into an object structure that the system can process internally; format validation: checking whether the message conforms to the expected structure and field requirements, validating data types and value ranges; deduplication: checking whether the message has already been processed to avoid duplicate consumption, usually based on the message ID or other unique identifiers. These preprocessing operations ensure that only valid and acknowledgment-pending message bodies enter the subsequent processing flow. Next, the processing status of the corresponding transaction for this message is updated in Redis, marked as "PROCESSING". Then, Redis atomic operations are used to ensure that the same message is not processed by multiple consumers simultaneously. If this step fails (e.g., due to network problems or Redis unavailability), the message is added to a local retry queue and retried later, ensuring that the processing status is correctly recorded.

[0117] After successfully updating the status, the consumer pushes the pending confirmation message body to the actual business processing logic or the business system. This process may involve calling internal services or calling external business services via HTTP / RPC. Upon receiving the message, the business system executes its specific business logic (such as updating database records or sending notifications) and returns a processing receipt indicating success or failure. Therefore, confirmation from the business system ensures that message processing at the business level is complete.

[0118] Optionally, if the business component returns a failure result, the message is added to a dedicated business retry queue. Unlike the local retry queue, the business retry queue focuses on handling temporary failures at the business level, typically employing an incremental backoff strategy (e.g., retrying immediately on the first attempt, waiting 5 seconds on the second, and 15 seconds on the third) to maximize the eventual success rate while avoiding excessive pressure on the business system. If the business component returns a success result, a business confirmation field (BUSINESS_CONFIRMED) is added to the Redis transaction status hash table, marking that the message for that transaction has been successfully processed by the business component. At this point, the entire double confirmation process is complete, generating the final message processing result, including processing status, processing time, and related statistics, for monitoring and auditing purposes.

[0119] It should be noted that this application provides two layers of protection through a dual confirmation mechanism: consumer confirmation ensures that the message is successfully received and initially processed, while business confirmation ensures that the business logic is executed correctly. Therefore, this application effectively prevents message loss (with confirmation and retry mechanisms at each step) and duplicate consumption (through state recording and deduplication), significantly improving the reliability of distributed transaction processing. In practical applications, this application also provides special handling for various edge cases, such as triggering alarms and manual intervention if no confirmation is received from the business side for an extended period, ensuring that all messages are ultimately processed correctly.

[0120] In some embodiments, step S1 includes:

[0121] Based on the pre-acquired data access characteristics, business message bodies with a message volume greater than a preset volume threshold and an access frequency lower than a preset frequency threshold are stored in the distributed message queue, while transaction status metadata with a message volume less than the volume threshold but an access frequency higher than the frequency threshold are stored in the memory database.

[0122] Configure sequential write parameters for the distributed message queue and configure a memory policy for the in-memory database, wherein the sequential write parameters include the number of partitions and the replication factor, and the memory policy is used to indicate the enabling of a persistence mechanism composed of a hybrid AOF and RDB.

[0123] In this embodiment, data is finely categorized, with volume and frequency thresholds set as classification criteria. Business message bodies, typically large in size and rarely changed after being written, are stored in Kafka, which supports high-throughput sequential writes. Transaction state metadata, smaller in size but accessed and modified frequently, is stored in Redis, which supports high-speed random access. For Kafka, an appropriate number of partitions and replication factor are configured. The number of partitions determines the system's parallel processing capability, while the replication factor affects data reliability and availability. For Redis, a hybrid persistence mechanism of AOF and RDB is enabled. RDB provides data snapshot capabilities, while AOF records each write operation to the database. This combination ensures both data security and performance.

[0124] It should be noted that data access characteristics refer to the patterns and frequency of data creation, reading, updating, and deletion, which directly affect the selection and optimization of storage strategies. In distributed transaction systems, different types of data have significantly different access characteristics, requiring the selection of suitable storage media. This embodiment achieves a refined data separation storage strategy by pre-analyzing and defining data access characteristics. The volume threshold is a preset data size limit, typically set between a few KB and tens of KB, used to distinguish between "large volume" and "small volume" data. The frequency threshold is a preset data access frequency limit, usually measured in accesses per second, used to distinguish between "high-frequency access" and "low-frequency access" data. The specific settings of these two thresholds need to be optimized based on system scale, hardware configuration, and business characteristics, and may be dynamically adjusted as the system operates.

[0125] In this embodiment, the business message body typically contains complete business data content, such as order details, user operation records, or transaction information. This type of data often has the characteristic of being "written once and read a few times." For example, a typical business message body may contain several KB to several hundred KB of data. Once written, it is mainly read when the message is consumed, and is usually only read completely once or a few times. Therefore, business message bodies are usually large-volume, low-frequency access data, suitable for storage in a distributed message queue.

[0126] In contrast, transaction state metadata mainly contains concise information such as transaction ID, status code, and timestamp, typically only a few hundred bytes to a few KB in size, but it is frequently queried and updated. For example, during the processing of a transaction, its status may be queried (through a rollback mechanism) and updated (from PREPARED to COMMITTED or ROLLBACKED) multiple times, with an access frequency that may reach multiple times per second. Therefore, transaction state metadata is usually small-volume, frequently accessed data, suitable for storage in an in-memory database.

[0127] In its implementation, feature detection and classification are performed before data is stored in the database. When a transaction message enters the system, its serialized byte length is first compared with a preset volume threshold, and its future access frequency is estimated based on historical statistical data or predefined rules. Based on the comparison results, this embodiment determines whether to store the data in a distributed message queue (such as Kafka) or an in-memory database (such as Redis). Therefore, this embodiment, through dynamic classification, can adapt to different types of business scenarios and data characteristics, optimizing overall storage performance.

[0128] To ensure the high performance and reliability of a distributed storage system, specific parameter configurations are required for different storage components. This embodiment details the key parameter configurations for distributed message queues and in-memory databases.

[0129] Sequential writes in distributed message queues (such as Kafka) are a highly efficient data storage method. They append new data sequentially to the end of existing data, rather than randomly inserting or updating existing data. This model fully leverages the performance advantages of modern storage media (especially hard disk drives) in sequential I / O, significantly improving write throughput. To optimize sequential write performance, two key parameters need to be configured: the number of partitions and the replication factor.

[0130] In this context, "partitions" refers to the number of logical parts into which a topic is divided. Each partition is an ordered, immutable sequence of messages. Partitions are designed to provide horizontal scalability and parallel processing capabilities. Increasing the number of partitions can improve parallelism, but too many partitions will also increase system overhead. The selection of the number of partitions typically considers the following factors: expected throughput: higher throughput requirements usually correspond to more partitions; number of consumers: ideally, the number of partitions should not be less than the maximum possible number of consumers; message distribution balance: ensuring messages are evenly distributed across partitions; server resources: each partition requires independent file handles and memory buffers. In this embodiment, the optimal number of partitions is dynamically calculated based on the cluster size and business volume, typically set to 1.5 to 2 times the expected number of consumers to accommodate potential expansion needs.

[0131] The replication factor specifies the number of replicas of each partition that should be stored in the cluster, directly impacting data reliability and availability. A replication factor of 1 indicates only one replica of the partition (no redundancy), while a replication factor of 3 indicates three replicas per partition (the original partition plus two replicas). Higher replication factors offer stronger data durability and service availability, but also increase storage space consumption and network bandwidth usage. The choice of replication factor should balance reliability and resource consumption; it is typically set to 3 in production environments to provide sufficient fault tolerance to handle the failure of one or more nodes.

[0132] For in-memory databases (such as Redis), the memory policy is a set of configuration parameters that control memory usage and data persistence behavior. In this embodiment, a persistence mechanism consisting of a hybrid of AOF (Append Only File) and RDB (Redis Database) is enabled, which is the high-reliability configuration officially recommended by Redis.

[0133] AOF persistence is a method of recording all write operations performed by the server. By appending operations to a file instead of overwriting, it ensures that even if the service crashes, Redis can rebuild the dataset by re-executing these operations. AOF provides the highest level of data security but may lead to increased file size and longer restart times. In the configuration, setting `appendonly yes` enables AOF, and using `appendfsync everysec` performs synchronization once per second, striking a balance between performance and security. RDB persistence is a method of taking point-in-time snapshots of the dataset at specified time intervals. The resulting RDB file is a compact binary file representing the state of the dataset at a particular moment. RDB provides fast restart and backup capabilities, but may lose data changes since the last snapshot if the service crashes. In the configuration, setting appropriate savepoints (such as `save 900 1`, `save 300 10`, `save 60 10000`) balances snapshot frequency and system load.

[0134] In this embodiment, a hybrid AOF and RDB persistence mechanism (supported in Redis 4.0 and above) combines the advantages of both methods: during AOF rewriting, the first half of the rewrite buffer is written in RDB format, and the second half is appended in AOF format. This hybrid mode retains both the fast loading advantage of RDB and the data security of AOF. Enabling this function by setting `aof-use-rdb-preamble yes` significantly improves system restart efficiency and data reliability.

[0135] In addition, this embodiment also configures Redis with appropriate memory eviction policies (such as `volatile-lru`, which prioritizes deleting the least recently used keys with expiration times) and memory limits (such as `maxmemory` settings) to ensure stable operation under high load while maintaining the availability of critical data.

[0136] Therefore, this embodiment enables the distributed message queue to efficiently process large-volume business messages through these optimized configuration parameters, while also enabling the in-memory database to quickly respond to frequent status queries and updates, thus jointly supporting the high performance and high reliability of the entire transaction message processing system.

[0137] In some embodiments, step S2 includes:

[0138] Based on the transaction status metadata stored in the memory database, a transaction status hash table is created for each transaction in the memory database. The key in the transaction status hash table represents the transaction identifier of each transaction, and the value in the transaction status hash table includes the transaction status, offset, creation time and timeout of the transaction. The transaction status includes the pending confirmation status, the confirmed transaction status and the rollback transaction status.

[0139] Based on the transaction status metadata, the transaction index to be retried is created in the memory database. The transaction index to be retried includes at least one transaction to be retried and its corresponding transaction identifier, and a retried evaluation score determined by the timeout period. The transaction to be retried is used to characterize the transaction whose transaction status needs to be retried.

[0140] Listen for transaction status change events related to the transaction status hash table, and when any transaction status change event is detected, trigger the sending of a status change notification message to a designated channel. The status change notification message is used to indicate the transaction identifier of the transaction involved in the transaction status change event and the transaction status after the status change.

[0141] In this embodiment, the transaction status index structure consists of two parts: a transaction status hash table and a transaction index to be retrieved. Each transaction has a corresponding hash table in Redis, where the key is the transaction ID and the value is a structure containing multiple fields, recording information such as the transaction status, offset in Kafka, creation time, and timeout. The transaction status can be PREPARED (pending confirmation), COMMITTED (confirmed transaction), or ROLLBACKED (rollback transaction). The transaction index to be retrieved is implemented using Redis's ZSET (sorted set) structure, where the set members are transaction IDs, and the corresponding score is a retrieval evaluation score calculated based on the timeout. This allows for efficient retrieval of transactions to be retrieved in chronological order. Simultaneously, this embodiment also monitors transaction status change events. Once a status change is detected (e.g., from PREPARED to COMMITTED), a notification message containing the transaction ID and the new status is sent to a designated channel via Redis's PUBSUB (publish-subscribe) mechanism, so that relevant components can respond.

[0142] Specifically, the transaction state hash table is the core data structure for managing transaction states. Implemented using Redis's Hash data type, it provides an efficient state storage and access mechanism for each transaction. Specifically, the creation and organization of the transaction state hash table uses a field-value mapping structure in Redis's Hash data type, similar to a dictionary or mapping object in a programming language. Unlike storing all data as a single string or using multiple independent keys, the Hash type allows multiple related fields to be organized under a single key, saving memory space (avoiding key name duplication overhead) and providing independent operation capabilities on individual fields. This feature is particularly important in transaction state management because it is necessary to frequently read and update specific attributes of a transaction, rather than reading or writing the entire transaction record each time.

[0143] In this embodiment, a separate hash table is created for each transaction, using the transaction identifier (TransactionID) as the key. The transaction identifier is typically a globally unique string, such as a UUID (Universally Unique Identifier) ​​or other custom unique identifier, ensuring that a transaction is uniquely identified throughout the distributed system. The hash table contains multiple fields, each representing an attribute of the transaction.

[0144] The transaction status is a core field in the hash table, used to indicate the current processing stage of a transaction. In this embodiment, the transaction status can be one of the following three: PREPARRED: The transaction has been initialized but not yet finally confirmed, and is in the "pre-commit" stage. This is the initial state after the transaction is created, indicating that the business logic of the transaction is being executed; COMMITTED: The transaction has been successfully completed and confirmed as committed, indicating that all related business operations have been successfully executed, and messages can be delivered to consumers; ROLLBACKED: The transaction has failed or been explicitly canceled, requiring the execution of operations to be reversed, and messages should not be delivered.

[0145] Subsequently, the offset field stores the physical location information of the transaction message in a distributed message queue (such as Kafka). It typically includes the topic name, partition number, and sequence number, enabling precise location and retrieval of the corresponding business message body when the transaction state becomes COMMITTED, without scanning the entire message queue. The creation time records the timestamp of the transaction's initial creation (usually a Unix timestamp with millisecond precision). This information is used to calculate the transaction's age, assists dynamic rollback algorithms in determining rollback frequency, and is also used for timeout detection and statistical reporting. The timeout time specifies the latest expected completion time of the transaction. If the transaction has not reached its final state (COMMITTED or ROLLBACKED) by this time, timeout processing logic may be triggered, such as forced rollback or sending an alarm notification. The timeout time is typically set based on business scenarios and performance requirements, and may be a fixed value (such as 30 seconds after creation time) or a dynamically calculated value (adjusted according to transaction type and system load).

[0146] In practice, the creation of the transaction state hash table is accomplished using Redis's HMSET command, which can efficiently perform common operations such as state query (HGET), state update (HSET), and complete transaction information retrieval (HGETALL). It also supports atomic operations on specific fields, ensuring data consistency in high-concurrency environments.

[0147] Furthermore, the transaction replay index is a dedicated data structure for the system to efficiently manage and schedule status replay tasks. Implemented using Redis's Sorted Set, it provides powerful query and priority management capabilities for the replay control module. Specifically, regarding the creation and organization of the transaction replay index, Redis's Sorted Set is a data structure that combines set and sorting properties. Each member is associated with a score, and the data is ordered according to the score. This data structure is particularly suitable for applications such as priority queues, leaderboards, and time-series data. In transaction replay management, these characteristics of the Sorted Set enable efficient retrieval of transactions in chronological order, supporting range queries and batch processing.

[0148] Transactions awaiting rollback refer to those currently in the PREPARED state that require a rollback mechanism to confirm their final state. The rollback mechanism is a crucial component of distributed transaction processing. It periodically queries the transaction initiator to determine the final state (COMMITTED or ROLLBACKED) of the transaction, deciding whether to deliver the message or clean up the transaction record. In large-scale systems, there may be thousands or even tens of thousands of transactions awaiting rollback simultaneously. How to efficiently organize and schedule these rollback tasks is a key factor affecting system performance.

[0149] In this embodiment, the index of transactions to be rechecked is implemented using Redis's ZSET (sorted set) structure. A fixed key name (such as "pending_check_transactions") is used as the index identifier, the transaction ID of each transaction to be rechecked is used as a set member, and the recheck evaluation score calculated based on the timeout is used as the score of that member. The recheck evaluation score is a numerical value used to determine the priority and order of transaction rechecks. In the simplest implementation, this score can be directly the transaction timeout (Unix timestamp), and transactions can be processed in the order of their timeouts. In more complex implementations, the recheck evaluation score may be a composite value that comprehensively considers factors such as timeout, transaction importance, and the number of rechecks to achieve finer priority management.

[0150] For example, the creation of the index for transactions to be retrieved is accomplished using Redis's ZADD command. When it is necessary to retrieve the transactions to be retrieved, the ZRANGEBYSCORE command is used to obtain transactions within a specific score range. This will return a maximum of 100 transactions with scores not exceeding the current time, and then a retrieval request is initiated for these transactions. When the transaction status changes to the final state (COMMITTED or ROLLBACKED), the ZREM command is used to remove the transaction from the index. In this way, by using an index based on ordered sets, transactions can be processed in order of priority, ensuring that the most urgent transactions are retrieved in a timely manner; secondly, the range query capability can process transactions in batches, improving throughput; finally, Redis's high performance ensures that index operations can maintain millisecond-level response times even when facing massive amounts of transactions.

[0151] Furthermore, the transaction state change notification mechanism is a key design element for achieving real-time response and loose coupling between components. Redis's publish / subscribe functionality provides real-time notification capabilities for state changes. Specifically, for the implementation of transaction state change listening and notification, Redis's publish / subscribe model is a message communication pattern that decouples message senders (publishers) and message receivers (subscribers). Publishers do not need to directly send messages to specific receivers; instead, they publish messages to a designated channel, where all receivers subscribed to that channel receive the messages. This pattern is particularly suitable for implementing event notification systems, supporting one-to-many message passing and reducing direct dependencies between system components.

[0152] Transaction state change events refer to events where the state of a transaction changes from one value to another, specifically the transition from PREPARRED (pending acknowledgment) to COMMITTED (transaction acknowledgement) or ROLLBACKED (transaction rollback). These state changes are typically triggered by the transaction initiator or the rollback process, indicating that the transaction has reached its final state and requires subsequent processing, such as message delivery or transaction cleanup. In this embodiment, the system monitors transaction state changes through two mechanisms: one is using Redis's Keyspace Notifications, and the other is implementing state change hooks at the application layer. Keyspace Notifications is a mechanism provided by Redis that allows clients to subscribe to key events, such as key modification and expiration. By configuring `notify-keyspace-events Kh`, notifications of hash field changes can be received. At the application layer, a state change event is explicitly published each time the transaction state is updated, ensuring that the notification mechanism still functions correctly even if the Keyspace Notifications configuration changes.

[0153] In this regard, the state change notification message is a structured message body containing key information such as the transaction identifier (TransactionID) and the changed transaction status (Status), typically serialized into JSON format. This information enables the message receiver to understand the specific content of the change and execute the corresponding processing logic. For example, when the status changes to COMMITTED, the consumer needs to pull and process the message; when the status changes to ROLLBACKED, the consumer needs to clean up related resources and status.

[0154] For example, notification messages are sent using Redis's PUBLISH command, specifying the channel name (usually a conventional string such as "tx_status_changes") and the message content. Consumers subscribe to the channel using the SUBSCRIBE command to receive status change notifications in real time.

[0155] Therefore, to improve system reliability, notification mechanisms typically include the following enhancements: message persistence: Change events are recorded in persistent storage simultaneously with notification publication to address situations where the receiver is temporarily unavailable; acknowledgment mechanism: The receiver sends an acknowledgment message after processing the notification, and the sender can track unacknowledged notifications and resend them if necessary; notification deduplication: The receiver implements idempotent processing logic to ensure that even if duplicate notifications are received, duplicate processing will not occur; monitoring and alerting: The system monitors the notification queue length and processing latency, and triggers alerts promptly in abnormal situations. Therefore, this embodiment's publish-subscribe state change notification mechanism can achieve efficient state synchronization and event-driven processing while maintaining loose coupling between components, making it an important component of the entire distributed transaction processing framework.

[0156] In some embodiments, determining a physical offset based on the business message body in the distributed message queue, creating a transaction state record corresponding to the business message body in the memory database based on the physical offset, and generating a transaction task to be queried based on the transaction state record to be added to the transaction state index structure includes:

[0157] The obtained business message body is written into the distributed message queue, and the message location identifier of the business message body in the distributed message queue is used as the physical offset. The corresponding queue writing result is recorded. The queue writing result is used to indicate whether the business message body has been successfully written into the distributed message queue.

[0158] Based on the physical offset and the transaction identifier, a transaction status record corresponding to the business message body is created in the memory database, wherein the transaction status in the transaction status record is set to the pending confirmation status, and the transaction creation time is used to indicate the timestamp of the current creation of the transaction status record;

[0159] When the timeout recorded by the transaction status exceeds a preset timeout threshold, a transaction task to be reviewed is generated, and the transaction identifier corresponding to the transaction task to be reviewed and the timeout are added to the transaction index to be reviewed.

[0160] When a database operation failure message is detected, the business message body is rewritten to the in-memory database according to a preset retry strategy. After a retry failure, an alarm notification is generated, and a compensation execution log and a list of compensation failed transactions are recorded. The database operation failure message includes at least one of a creation failure message and a task creation failure message. The creation failure message indicates that the creation operation of the transaction status record in the in-memory database failed, and the task creation failure message indicates that the operation of adding the pending query transaction task to the pending query transaction index failed. The retry strategy includes the number of retries and the retry interval. The compensation execution log records the actual number of retries and the operation results.

[0161] In this embodiment, the transaction commit process first writes the business message body to Kafka. Upon successful completion, Kafka returns the physical offset of the message (such as Topic partition and location information). Using this offset and the transaction ID, a transaction status record is created in Redis, initially set to PREPARED (pending confirmation), and the current time is recorded as the transaction creation time. If the transaction timeout exceeds a preset timeout threshold, a pending lookup task is generated, adding the transaction ID and timeout to the pending lookup transaction index for periodic checks of the transaction's status.

[0162] This embodiment also implements a compensation mechanism to handle partial write failures: when a message is successfully written to Kafka but the Redis operation fails (such as failure to create a transaction status record or failure to add a task to be retrieved), the data is rewritten to Redis according to a preset retry strategy (such as 5 retries with a 3-second interval between each retrieval). If the retry still fails, an alarm notification is generated, and a detailed compensation execution log and a list of failed compensation transactions are recorded for subsequent manual intervention.

[0163] In this embodiment, writing business messages is the first crucial step in distributed transaction processing. It ensures that business data is persistently stored, providing the data foundation for subsequent transaction processing. This step details the process of writing business messages and the method for determining the physical offset.

[0164] A business message body is a data structure containing all the information needed for a complete business process. It encapsulates all relevant data for a specific business operation. In a distributed transaction system, the business message body typically includes information such as the operation type, the operation object, the operation parameters, and the business context. For example, the business message body for order creation might include the order number, product list, shipping information, and payment method; the business message body for a payment request might include the transaction amount, payment channel, and account information. The business message body is the core carrier of business processing, and its completeness and correctness directly affect the outcome of the transaction.

[0165] A distributed message queue (DMQ) is middleware used in distributed systems to implement reliable message delivery. It provides persistent message storage, order guarantees, and high availability. Common DMQs include Apache Kafka, RabbitMQ, and Apache RocketMQ. In this system, the DMQ is primarily used to store business message bodies, ensuring that messages are not lost and can be reliably processed even in the event of system failures or network outages. Distributed message queues typically employ a partitioning mechanism to achieve horizontal scaling and parallel processing. Each partition is an ordered sequence of messages that can be stored independently on different servers.

[0166] A message location identifier is information that uniquely identifies the location of a message in a distributed message queue. It allows the system to accurately locate and retrieve a specific message in subsequent operations. In queue systems such as Kafka, a message location identifier typically consists of a topic name, a partition ID, and an offset. These three elements together determine the precise location of the message within the entire distributed system.

[0167] In this embodiment, the physical offset is a concrete implementation of the message location identifier, used to quickly locate the business message body in subsequent transaction processing. Therefore, by using the physical offset, a mapping relationship is established between the transaction state record and the business message body, enabling efficient retrieval and processing of the corresponding business message when the transaction state changes.

[0168] The queue write result refers to the status information returned after the business message body is written to the distributed message queue, indicating whether the write operation was successful or failed, as well as other relevant information (such as write time, server node, etc.). The queue write result is the basis for the system to determine whether the message has been persisted. Only after confirming that the write was successful will subsequent operations such as creating transaction status records continue.

[0169] In this embodiment, the business message body is first written to the distributed message queue. This process is typically implemented using the producer API provided by the message queue, and the specific steps are as follows: First, select an appropriate target topic based on the business type and message content. A topic is a logical grouping of messages, usually based on business domains, such as "order message" or "payment message". Selecting an appropriate topic helps with message organization and management, as well as consumer subscription filtering. Second, determine the message partitioning strategy. The partitioning strategy determines which partition the message will be written to, which directly affects the message's parallel processing capability and order guarantee. For example, partitioning strategies include: key-based hash partitioning: using a specific key of the message (such as user ID or order ID) to calculate a hash value, and then taking the modulo to obtain the partition number, ensuring that messages with the same key are always sent to the same partition, guaranteeing local ordering; round-robin partitioning: sending messages to each partition in turn to achieve load balancing; and custom partitioning: implementing specific partitioning logic according to business requirements. Then, the message is sent to the distributed message queue. During the sending process, a series of parameters are usually set, such as message key, message header, compression format, acknowledgment level, etc. These parameters affect message routing, processing, and reliability level. After sending is complete, the queue write result is retrieved and parsed. The write result contains key information such as: write status: indicating whether the message was successfully written to the queue; topic name: the topic to which the message belongs; partition number: the partition where the message was written; offset: the message's position number within the partition; and timestamp: the time the message was written. Once successful message writing is confirmed, the message location identifier (usually a combination of topic, partition, and offset) is saved as the physical offset. This physical offset will serve as a key field in subsequent steps for recording the transaction status, establishing the association between the transaction status and the business message.

[0170] To ensure the reliability of the write process, this embodiment typically implements the following safeguards: Message acknowledgment mechanism: requiring the message queue to return a success response only after successfully replicating the message to a specified number of copies, improving data persistence; Retry mechanism: for messages that fail to write, a limited number of retries may be performed depending on the reason for the failure and its importance; Batch write optimization: when permissible, multiple messages are merged into a single batch for transmission, improving throughput; Anomaly monitoring: all write anomalies are recorded, alarm thresholds are set, and persistent write problems are detected promptly. Through these mechanisms, this embodiment ensures that business message bodies can be reliably written to the distributed message queue. The determination and storage of physical offsets establishes a bridge between the transaction state and business messages, enabling the quick and accurate location and retrieval of specific business messages at any time when needed.

[0171] Subsequently, by creating transaction state records, the basic data structure for system tracking and control of transaction states is established. Among these, the transaction identifier is a unique identifier for a transaction, typically using a globally unique identifier (GUID) or other encoding methods that ensure uniqueness. The transaction identifier is a key association in the transaction processing process, used to link various data such as business messages, state records, and transaction processing results. A typical transaction identifier might be a UUID (e.g., "550e8400-e29b-41d4-a716-446655440000") or a structured identifier string (e.g., "TX-20211005-123456"), ensuring accurate location and tracking of specific transactions throughout all processing stages. An in-memory database is a database system that stores data primarily in memory rather than on traditional disks. It offers extremely high read and write performance, making it particularly suitable for scenarios requiring low-latency access. Common in-memory databases include Redis, Memcached, and Apache Ignite. In distributed transaction systems, in-memory databases are typically used to store data that requires frequent access and updates, such as transaction states, providing response times in milliseconds or even microseconds. While the primary data is stored in memory, most modern in-memory databases also provide persistence mechanisms (such as snapshots and logs) to ensure data is not lost after a restart. A transaction status record is structured data describing the current state and key attributes of a transaction; it is the core data entity for transaction management and coordination. A transaction status record typically contains multiple fields describing the transaction's identifier, state, associated messages, time attributes, and other information. The lifecycle of a transaction is tracked and managed by creating, updating, and querying these records.

[0172] The Pending Confirmation Status refers to the intermediate state after a transaction is initially created, awaiting final confirmation (commit or rollback). In a two-phase commit protocol, this is equivalent to the "preparation phase," indicating that the transaction has been initialized but not yet finally committed. Transactions in the Pending Confirmation Status need to determine their final state through subsequent confirmation operations (initiated by the business system) or timeout rollback mechanisms.

[0173] Transaction creation time is the precise point in time when a transaction state record was created, typically recorded using the current timestamp. Creation time serves as the baseline for calculating transaction age and timeout, and is also crucial data for transaction processing performance analysis and monitoring. In distributed systems, to ensure the accuracy and consistency of timestamps, a standard time provided by a server time or a centralized time service is usually adopted.

[0174] In this embodiment, a transaction status record is created in the in-memory database based on the acquired physical offset and transaction identifier. This process typically follows these steps: First, the data structure of the transaction status record is constructed, usually a composite structure containing multiple fields. Key fields include: Transaction Identifier: A unique ID that identifies the transaction; Physical Offset: Points to the position of the business message body in the distributed message queue; Transaction Status: Initially set to a pending confirmation status (such as "PREPARED" or "PENDING"); Transaction Creation Time: The current timestamp of the record creation; Timeout: The expected timeout period calculated based on the creation time and timeout policy; Other optional fields: such as the number of retries, last update time, associated business ID, etc.

[0175] Secondly, this embodiment selects an appropriate data structure to store transaction status records. In in-memory databases such as Redis, common storage structures include: Hash tables: storing each field of the transaction record as a hash field, suitable for storing complex data with multiple fields; String tables: storing transaction records serialized into JSON or other string formats, suitable for simple access patterns; and Sorted sets: using transaction IDs as members and timestamps as scores, suitable for time-ordered access or timeout management. In this embodiment, a hash table is typically chosen as the primary storage structure because it allows efficient access and updating of individual fields without reading or writing the entire record.

[0176] Then, in this embodiment, the transaction status is set to "Pending Confirmation." The "Pending Confirmation" state is the initial state of the transaction, indicating that the transaction has been created but final confirmation has not yet been received. In this state, it is necessary to wait for confirmation from the transaction initiator or actively obtain the final state of the transaction through a lookup mechanism. Next, this embodiment records the current time as the transaction creation time, using a millisecond-level timestamp. This timestamp has several important uses: calculating the transaction age: subtracting the creation time from the current time gives the transaction's lifespan; determining the timeout: adding a timeout threshold to the creation time gives the expected timeout point; performance monitoring: analyzing the total processing time from transaction creation to completion; and audit trail: recording the transaction's timeline and processing history.

[0177] Finally, this embodiment writes the complete transaction state record to the in-memory database, typically using atomic operation commands provided by the database to ensure the integrity and consistency of the record. For example, Redis's HMSET command can be used to set multiple fields of a hash table at once. To ensure the reliability of transaction state record creation, this embodiment typically implements the following safeguards: Write confirmation: The database is required to return an confirmation response after successfully writing data, verifying the successful completion of the operation; Data verification: The integrity of the data is verified immediately after writing to ensure that no fields are lost or corrupted; Operation retry: For write failures caused by temporary faults, a limited number of retries are implemented; Failure handling: For unrecoverable creation failures, detailed error information is recorded, and a compensation process may be triggered.

[0178] Therefore, this embodiment establishes a core data structure for tracking and managing transaction states by creating transaction state records in an in-memory database. These transaction state records will be frequently accessed and updated in subsequent transaction processing flows, reflecting the latest status and progress of the transactions. The high-performance characteristics of the in-memory database ensure that these operations can be executed with extremely low latency, supporting the system's ability to handle a large number of concurrent transactions.

[0179] Subsequently, the creation of pending transaction tasks is a crucial step in implementing the transaction status rollback mechanism. This ensures that transactions that have been in a pending confirmation state for an extended period can be actively queried, preventing prolonged transaction delays. The timeout period is the latest time the system expects a transaction to complete, typically calculated by adding a predefined timeout threshold to the transaction creation time. The timeout period is a vital time marker for determining whether a transaction needs to be actively rolled back. When the actual time approaches or exceeds the timeout period, rollback or timeout processing measures are taken. Setting the timeout period requires balancing two factors: setting it too short may lead to many incomplete normal transactions being mistakenly judged as timeouts; setting it too long may delay the processing of abnormal transactions. The timeout threshold is a predefined time interval representing the maximum allowed time from transaction creation to expected completion. The timeout threshold may be a fixed value (e.g., uniformly set to 30 seconds for all transactions) or dynamically determined based on the transaction type or business complexity. Different types of transactions may have different processing complexity and time requirements, and therefore may require different timeout thresholds. For example, simple query transactions may use a shorter threshold (such as 5 seconds), while complex multi-step transactions may use a longer threshold (such as 60 seconds or longer).

[0180] It should be noted that a Pending Check Transaction Task refers to a task description that plans to perform a status check on a specific transaction at a specific point in time. A Pending Check Transaction Task typically includes information such as the transaction identifier, planned check time, and number of retries. The creation and management of Pending Check Transaction Tasks are a key mechanism to ensure eventual consistency of transactions, especially when the network is unstable or the service is temporarily unavailable. A Pending Check Transaction Index is a data structure used to organize and manage Pending Check Transaction Tasks, allowing for efficient searching and scheduling of these tasks. Pending Check Transaction Indexes are typically sorted by time, enabling tasks to be processed in the order of their planned check times, ensuring that transactions nearing timeout are processed first. In in-memory databases, such indexes are usually implemented using data structures such as Sorted Sets or priority queues.

[0181] In this embodiment, the need to create a pending review task is determined based on the timeout period in the transaction status record, and tasks meeting the criteria are added to the pending review transaction index. Specifically, firstly, it is determined whether the transaction's timeout period exceeds a preset timeout threshold. For example, this is determined by comparing the difference between the timeout period and the creation time with the preset threshold. Only when the difference between the timeout period and the creation time (i.e., the expected processing time) exceeds the preset threshold will a pending review task be created for the transaction, ensuring that the review mechanism is only implemented for transactions with longer expected processing times, avoiding unnecessary monitoring of short-term transactions. Secondly, this embodiment generates pending review transaction tasks, which typically include the following core information: Transaction ID: A unique identifier for the target transaction; Timeout period: The expected timeout point of the transaction, used for review scheduling; Evaluation score: A value used for sorting and priority control, usually based on the timeout period; Number of reviews: The number of reviews currently executed, initially 0; Creation time: The timestamp of task creation; Other optional information: Such as transaction type, review interval strategy, etc. Then, the pending review task is added to the pending review transaction index. When using in-memory databases like Redis, this is typically achieved by adding new members to an ordered set, with an evaluation score calculated based on timeout periods used to determine the execution order of backtracking tasks. The evaluation score is usually the transaction timeout (Unix timestamp), allowing tasks to be processed in the order of their timeouts.

[0182] To optimize the management of the index to be retrieved, this embodiment may implement the following strategies: sharded index: For large-scale systems, the retrieved index can be divided into sub-indexes for multiple time periods to reduce the size of a single index; priority adjustment: adjust the evaluation score according to the importance of the transaction or the impact on the business to ensure that critical transactions are given priority; periodic cleanup: periodically remove completed or invalid retrieved tasks to keep the index concise and efficient; batch addition: when possible, add multiple retrieved tasks in batches to reduce network round-trip overhead.

[0183] Therefore, this embodiment establishes a time-ordered queue of backtracking tasks through a pending transaction index, ensuring that status backtracking is executed in a reasonable order. It proactively tracks and processes transactions that have not been confirmed for a long time, preventing transactions from remaining in an uncertain state for extended periods. This is a crucial guarantee for achieving eventual consistency in distributed transactions. Especially in situations of network instability or service communication interruptions, the proactive backtracking mechanism can promptly determine the final state of a transaction after communication is restored, propelling transaction processing forward.

[0184] Furthermore, fault recovery and compensation are key mechanisms for ensuring the reliability of distributed transaction systems. They handle various anomalies that may occur during operation, preventing data inconsistency and transaction loss. It should be noted that a Database Operation Failure Message refers to a failure notification received when performing database-related operations (such as creating a transaction state record or adding a task to be retrieved). Such failures can be caused by various reasons, including network failures, server overload, database node failures, and authentication failures. Database operation failures are common in distributed systems, requiring appropriate handling mechanisms to ensure data consistency and service reliability. A Creation Failure Message specifically refers to a failure notification encountered when creating a transaction state record in an in-memory database. This type of failure means that although the business message body has been successfully written to the distributed message queue, the corresponding state record has not been created in the in-memory database, resulting in the transaction state being untrackable and unmanageable. In this case, the business message exists but cannot be processed, requiring special handling to restore transaction consistency. A Task Creation Failure Message refers to a failure notification encountered when adding a task to be retrieved to an index of transactions to be retrieved. This type of failure means that the system has created a transaction status record, but failed to include it in the scope of the rollback mechanism. This may result in transactions that have been in a pending confirmation state for a long time not being able to be rolled back in time, affecting the final confirmation and processing of transactions.

[0185] A retry strategy is a systematic approach to re-execute the same operation when it fails. A retry strategy typically includes two core parameters: the number of retries and the retry interval. The number of retries defines the maximum number of times the operation can be re-executed, preventing resource waste caused by infinite retries. The retry interval defines the waiting time between two consecutive retries, usually employing an exponential backoff algorithm, where the interval gradually increases with the number of retries to avoid encountering the same problem with immediate retries and to reduce system load. The Compensation Execution Log is a detailed log recording the execution status of compensation operations, including information such as operation type, target object, execution time, number of retries, and operation result. The compensation log is crucial for system monitoring, troubleshooting, and auditing, providing a complete record of how abnormal situations were handled, which helps analyze system robustness and identify potential improvement points. The Failed Compensation Transactions List is a maintained record containing the transaction identifiers and related information of all compensation operations that failed after the maximum number of retries. This list typically requires human attention and handling because it represents problems that automatic recovery mechanisms cannot resolve and may require administrator intervention or special processing procedures.

[0186] In this embodiment, it is detected that the business message body has been successfully written to the distributed message queue, but subsequent database operations (creating a transaction state record or adding a task to be retrieved) fail. In this case, an attempt is made to recover according to a preset retry strategy, and relevant information is recorded after a retry failure. This process typically follows these steps: First, detect the specific type of database operation failure. Based on the content of the failure message, the system determines whether the failure was to create a transaction state record, add a task to be retrieved, or both. Different types of failures may require different retry logic. For example, if creating a transaction state record fails, the complete transaction state record needs to be recreated; if adding a task to be retrieved fails, it may only be necessary to re-add the task to the index to be retrieved; if both fail, the two operations need to be retried in sequence. Second, execute the retry operation based on the preset retry strategy. The retry strategy typically defines a maximum number of retries (e.g., 3 or 5 times) and a retry interval calculation method (e.g., fixed interval, linear growth, or exponential backoff). Track the number of retries already executed and check whether the maximum number has been reached before each retry. In practical applications, the exponential backoff algorithm is a commonly used method for calculating retry intervals, causing the waiting time to increase exponentially with the number of retries. For example, if the base interval is 100 milliseconds and the backoff factor is 2, then the waiting time for consecutive retries will be: 100ms, 200ms, 400ms, 800ms, etc. This strategy is particularly effective when facing temporary failures because it provides an opportunity for rapid recovery while avoiding frequent and ineffective retries in the case of persistent failures.

[0187] Then, this embodiment records the execution status and result of each retry as part of the compensation execution log. The log typically includes: Transaction ID: identifies the specific transaction for which the retry is performed; Operation type: such as creating a status record, adding a task to be reviewed, etc.; Retry sequence number: the current retry number; Execution time: the timestamp of the retry operation; Waiting time: the waiting time before this retry; Operation result: success or failure, and specific error information; Supplementary information: such as context information like server node and network status.

[0188] Optionally, if the retry succeeds, update the transaction status, mark the compensation operation as complete, and continue the subsequent processing flow. If the operation still fails after the maximum number of retries, perform the following follow-up operations: First, generate an alarm notification. An alarm notification is an alert sent to operations personnel or system administrators, indicating that a problem has been encountered that cannot be automatically recovered and requires manual attention or intervention. Alarm notifications are usually sent through multiple channels, such as email, SMS, instant messaging, monitoring systems, etc., to ensure that key personnel can receive the notification in a timely manner. The alarm content usually includes: alarm level: such as critical, warning, information, etc.; transaction identifier: the ID of the problematic transaction; failed operation: the specific type of failed operation; error information: a detailed error description and context; retry status: the number of retries attempted and the results; timestamp: the time the alarm was generated; possible solution suggestions: possible solutions given. Second, add the failed transaction to the compensation failed transaction list. This list may be stored in a specific database table or dedicated data structure for centralized management of all transactions requiring manual intervention. Each record in the list usually contains complete transaction information and failure details for easy subsequent processing. Finally, specific degradation processing logic may be initiated, such as: marking transactions in a special state (e.g., "requires manual processing"); isolating failed transactions to prevent them from affecting the overall system operation; attempting to release related resources to avoid long-term resource occupation; and recording detailed context and status information for subsequent recovery.

[0189] For example, manual handling of failed transactions is typically the responsibility of the system administrator or operations team, and may include: analyzing the cause of failure: reviewing error logs and system status to identify the specific problem; manually performing operations: directly creating or updating relevant records in the database; cleaning up or rolling back: clearing incomplete transaction data and rolling back related operations; and system repair: fixing possible system failures, such as restarting services or restoring database connections.

[0190] Therefore, this embodiment, through this comprehensive fault recovery and compensation mechanism, can maintain data consistency and operational integrity in the face of various abnormal situations, ensuring the reliable execution of distributed transaction processing. Especially when handling critical business operations, this mechanism provides multiple layers of protection to prevent data loss or inconsistency caused by temporary failures, and is a key technology for achieving high availability and strong consistency in distributed systems.

[0191] In some embodiments, determining the rollback interval based on the transaction creation time and a pre-acquired system load factor, and executing the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result includes:

[0192] Based on preset system configuration parameters and system load factor, the basic back-lookup interval is calculated. The system configuration parameters include minimum interval value and maximum interval value.

[0193] Based on the monitored CPU utilization, memory usage, network throughput, and IO metrics, a system load factor calculation model is constructed, and the system load factor is updated according to the system load factor calculation model.

[0194] The rollback interval of the transaction is determined based on the transaction age, the preset transaction timeout threshold, and the current updated base rollback interval. The transaction age is the difference between the current time and the transaction creation time.

[0195] When the updated system load factor exceeds the preset load threshold, overheat protection is triggered, the callback interval is increased, and the corresponding overheat warning log is recorded.

[0196] In this embodiment, the dynamic lookup algorithm first calculates the basic lookup interval based on the minimum and maximum interval values ​​configured in the system, combined with the current system load factor. The system load factor is a comprehensive indicator constructed by monitoring multiple system metrics (such as CPU utilization, memory usage, network throughput, and I / O), with values ​​ranging from 0 to 1; a higher value indicates a greater system load. For each transaction to be looked up, the lookup interval is dynamically calculated by considering the ratio of its transaction age (the difference between the current time and the creation time) to a preset timeout threshold, as well as the current basic lookup interval. The closer a transaction is to timeout, the higher the lookup frequency. When the system load factor exceeds a preset threshold (e.g., 0.9), an overheat protection mechanism is triggered, significantly increasing the lookup interval (e.g., 2-4 times the normal interval) to alleviate system pressure. Simultaneously, a corresponding warning log is recorded to alert the system administrator to monitor the system load.

[0197] Specifically, the basic lookup interval calculation is the core mechanism for adaptively adjusting the transaction lookup frequency, balancing the needs of system load and lookup timeliness. System configuration parameters refer to a predefined set of values ​​used to control the system's behavior and performance characteristics. In a lookup system, key configuration parameters include the minimum and maximum interval values, which together define the effective range of the lookup interval. These parameters are usually stored in a configuration center or configuration file and can be dynamically adjusted according to system operation and business needs. Appropriate settings of configuration parameters are crucial for system stability and performance: overly aggressive settings (such as too small a minimum interval) may lead to system overload; while overly conservative settings (such as too large a maximum interval) may delay transaction processing.

[0198] The minimum interval value is the shortest allowed lookup interval, usually measured in milliseconds. It represents the upper limit of lookup frequency under ideal conditions (low load, high priority transactions). Setting the minimum interval value requires consideration of hardware capabilities, network latency, and the capacity of the system being looked up, ensuring that even under the most frequent lookup scenarios, the system is not overloaded. For example, the minimum interval value might be set between 100 milliseconds and 1 second, depending on the system's performance characteristics and response requirements. The maximum interval value is the longest allowed lookup interval, also measured in milliseconds. It represents the lower limit of lookup frequency under the most conservative conditions (high load, low priority transactions). The maximum interval value ensures that even under high system load, transaction lookups are not indefinitely postponed, guaranteeing that transactions will eventually be processed. For example, the maximum interval value might be set between 30 seconds and several minutes, requiring a balance between system resource protection and the timeliness of business processing.

[0199] The system load factor is a comprehensive indicator reflecting the current resource usage status, typically a value between 0 and 1, where 0 indicates the system is completely idle and 1 indicates the system is completely saturated. The system load factor is a key input for dynamically adjusting the callback interval, adaptively adjusting the callback strategy based on current resource usage to process transactions as efficiently as possible while protecting the system from overload. The system load factor is usually calculated based on a combination of multiple system metrics, such as CPU utilization, memory usage, network traffic, and I / O wait.

[0200] The base check interval is a preliminary check interval value calculated considering system load but not specific transaction characteristics (such as age, priority, etc.). It serves as the benchmark for subsequent calculations of specific check intervals for particular transactions, reflecting the current overall processing capacity and load status. The calculation of the base check interval typically combines preset configuration parameters and the current system load factor, allowing the check frequency to be dynamically adjusted according to system load.

[0201] Therefore, this embodiment calculates the basic lookup interval based on preset system configuration parameters and system load factors. First, preset system configuration parameters are obtained, including minimum and maximum interval values. These parameters may come from a configuration center, configuration files, or system database, ensuring the use of the latest configuration settings. The minimum and maximum interval values ​​define a reasonable range for the lookup interval, preventing excessively small intervals from causing system overload or excessively large intervals from causing transaction processing delays. Second, the current system load factor is obtained. The system load factor is a dynamically changing value, possibly from a system monitoring service or load balancer, reflecting the current resource usage of the system. In a distributed system, it may be necessary to comprehensively consider the load of multiple nodes or services to form a global load view. Then, the basic lookup interval is calculated based on the load factor. A common calculation method is linear interpolation, which involves a weighted average of the load factor between the minimum and maximum interval values. This ensures that when the system load is close to zero, the lookup interval is close to the minimum value, allowing for more frequent lookups; when the system load is close to saturation, the lookup interval is close to the maximum value, reducing the lookup frequency and protecting system resources. Finally, further adjustments or restrictions are made to the calculated basic lookup interval, such as: ensuring that the result is not lower than the minimum interval value and not higher than the maximum interval value; adjusting appropriately according to the historical load trend of the system to avoid frequent changes in the lookup interval due to short-term load fluctuations; and applying a smoothing function to make the change in the lookup interval more gradual and avoid abrupt changes.

[0202] Therefore, this embodiment achieves load adaptation by dynamically adjusting the basic lookup interval, adjusting the lookup frequency according to real-time load to avoid overload or resource waste that may occur with a fixed frequency. It also achieves resource protection by automatically reducing the lookup frequency under high load to protect core system services and prevent system crashes caused by lookup operations. Furthermore, it optimizes throughput by increasing the lookup frequency under low load to fully utilize idle resources and accelerate transaction processing. Finally, it provides configuration flexibility by adjusting the minimum and maximum interval values ​​to flexibly control lookup behavior based on system capabilities and business needs. In this regard, this embodiment maintains optimal lookup efficiency and system stability under varying load conditions through dynamic calculation of the basic lookup interval based on system load.

[0203] Furthermore, the system load factor calculation model is a core component for achieving adaptive resource management. It integrates multi-dimensional system performance indicators into a single load metric to guide system behavior adjustments. Specifically, the System Load Factor Calculation Model is a mathematical model or algorithm that transforms multiple system resource indicators into a single comprehensive load score, creating a single indicator that comprehensively reflects the current resource usage of the system, facilitating load-related decisions. An effective load factor model needs to balance multiple potentially interdependent resource dimensions while considering the relative importance and degree of influence of different indicators.

[0204] CPU utilization refers to the activity level of a processor core, typically expressed as a percentage of total available processing power. In distributed systems, it may be necessary to monitor the CPU usage of multiple servers or containers to create an overall view. CPU utilization is one of the core metrics of system load because processor resources are often the main bottleneck for compute-intensive applications. High CPU utilization (e.g., exceeding 80%) may indicate that the system is nearing its computing capacity limit and requires reducing additional load; while low CPU utilization indicates that the system has sufficient computing resources to handle more tasks. Memory occupation refers to the ratio of currently used memory to total available memory. Memory is another critical system resource, especially for memory-intensive applications such as data caching and large-scale data processing. Memory pressure can lead to frequent garbage collection, page swapping, or memory allocation failures, significantly impacting system performance. Monitoring memory occupation helps predict potential memory-related problems and adjust system behavior when memory resources are strained. Network throughput refers to the data transfer rate of a system's network interface, typically measured in bytes per second or packets per second. In distributed systems, network performance is crucial for inter-component communication and can become a bottleneck, especially in microservice architectures or scenarios requiring frequent remote calls. Network throughput monitoring includes metrics such as bandwidth utilization, packet transmission rate, and network latency, which collectively reflect the health and load level of the network subsystem. IO metrics refer to the activity and performance indicators of the system's storage subsystem, including disk read / write rates, IO latency, and queue length. For database systems or applications requiring frequent access to persistent storage, IO performance is often a critical bottleneck. Excessive IO load can lead to slower system response and increased transaction processing latency; therefore, incorporating IO metrics into load factor calculations is a necessary step in comprehensively assessing the system's state.

[0205] In this embodiment, the system load factor is constructed and updated based on monitored multi-dimensional resource metrics. Specifically, the latest system performance metrics are first obtained from the monitoring system or resource manager. These metrics may come from operating system tools (such as Linux's sysstat), container orchestration platforms (such as Kubernetes' metrics-server), or professional monitoring systems (such as Prometheus, Datadog, etc.). The obtained metrics include: CPU utilization: such as the average utilization of all cores, or the CPU usage of a specific service process; memory usage: such as the percentage of memory used, memory allocation rate, and heap and non-heap memory usage; network throughput: such as the receive / send byte rate of network interfaces, number of connections, and network saturation; IO metrics: such as disk read / write rate, IO wait time, and disk queue length. Secondly, the raw metrics are standardized. Since different metrics may have different units and ranges, they need to be converted into a uniform ratio (usually a value between 0 and 1) to facilitate subsequent comprehensive calculations. The standardization methods may include: linear normalization: mapping indicator values ​​to a predefined range; threshold comparison: comparing indicator values ​​with preset thresholds to calculate the degree of saturation; and historical data comparison: comparing the current value with historical averages or peak values ​​to assess the relative load. Then, this embodiment applies weights and combines the indicators to form the final load factor. The importance of different resources may vary depending on application characteristics, and the allocation of weights needs to consider the actual bottlenecks and resource sensitivity of the system. For example, for compute-intensive applications, CPU utilization may be given higher weight; for database systems, IO indicators may be more emphasized. The combination method may be a simple weighted average or a more complex nonlinear function to better reflect the mutual influence between resources. Finally, this embodiment applies the calculated load factor to the lookup system, affecting the subsequent lookup interval calculation. The load factor update may be periodic (e.g., once per minute) or event-driven (e.g., when resource indicators change significantly). To avoid frequent adjustments due to instantaneous fluctuations, the system may implement a smoothing mechanism, such as moving average or exponential smoothing, to make the load factor change more stably.

[0206] Advanced features of the system load factor calculation model may include: Adaptive weighting: Automatically adjusts the weights of each indicator based on historical data analysis to adapt to actual bottleneck changes in the system; Anomaly detection: Identifies abnormal resource usage patterns, such as sudden memory leaks or CPU spikes, and adjusts the load factor response accordingly; Predictive model: Based on trend analysis, predicts short-term resource usage and adjusts the load factor in advance for proactive resource management; Multi-level thresholds: Sets multiple load level thresholds corresponding to different system behavior adjustment strategies for more refined load management.

[0207] Therefore, this embodiment's system load factor calculation model, based on multi-dimensional resource indicators, can comprehensively assess the current resource usage and make more intelligent retrospective strategy adjustments. This avoids the biased judgments that may arise from a single indicator and improves adaptability and stability under complex load conditions. Especially in environments with limited resources or large load fluctuations, accurate load factor calculation is crucial for maintaining efficient system operation and preventing resource exhaustion.

[0208] Furthermore, the dynamic adjustment of the transaction rollback interval is a key mechanism for achieving efficient transaction management. It optimizes the rollback strategy based on the specific circumstances of each transaction, balancing resource usage and processing timeliness. Transaction age refers to the time span from the transaction's creation time to the current time, usually measured in milliseconds. Transaction age is a key indicator for assessing the urgency of a transaction, as transactions that have existed for a longer period are generally closer to their timeout and require more timely processing. Calculating the transaction age is straightforward; simply subtract the transaction's creation timestamp from the current system time. In distributed systems, to ensure consistency in time calculations, a unified time source or time service is typically used to avoid age calculation errors caused by differences in node clocks. The transaction timeout threshold is a preset maximum allowed lifespan for a transaction; transactions that fail to complete within this time are considered timed out. The timeout threshold is an important parameter in transaction management, defining the tolerance for transaction processing time and directly impacting the system's resource management and error handling strategies. Setting an appropriate timeout threshold requires balancing two factors: setting it too short may cause transactions in normal processing to be mistakenly judged as timeouts; setting it too long may cause abnormal transactions to occupy resources for too long, affecting overall system performance. The check interval refers to the time interval between two consecutive check operations, determining the frequency at which the system checks the transaction status. The check interval directly affects the system's response speed to status changes and resource consumption, and needs to be dynamically adjusted according to the urgency of the transaction and the system load. A shorter check interval can detect status changes more promptly but consumes more system resources; a longer check interval reduces the system load but may delay the perception of status updates.

[0209] In this embodiment, the lookup interval for a specific transaction is calculated based on the transaction age, timeout threshold, and basic lookup interval. Specifically, firstly, the ratio of the transaction age to the timeout threshold is calculated as a quantitative indicator of the transaction's urgency. This ratio is between 0 and 1, where 0 indicates the transaction has just been created and 1 indicates the transaction has reached the timeout threshold. Secondly, an urgency adjustment factor is applied to adjust the basic lookup interval according to the transaction's urgency. A common adjustment method is to gradually decrease the lookup interval as the transaction approaches its timeout, resulting in more frequent lookups. This adjustment typically uses a non-linear function, causing the interval reduction rate to accelerate with increasing urgency. A larger exponential value makes the adjustment more aggressive, meaning the lookup interval decreases faster as the transaction approaches its timeout. Then, the adjustment factor is applied to the basic lookup interval to calculate the final lookup interval, ensuring that the lookup interval decreases with increasing transaction urgency but is still affected by the current system load (reflected by the basic lookup interval), achieving a balance between individual optimization and global equilibrium. Finally, additional constraints and adjustments may be made to the calculated rollback interval, such as: setting a minimum rollback interval lower limit to ensure that even for very urgent transactions, the rollback frequency will not exceed the acceptable range; applying additional adjustment coefficients for specific types or priorities of transactions to reflect differences in business importance; and applying step-by-step adjustments when transactions approach certain critical time points (such as 80% or 90% of the timeout threshold) to ensure that these moments receive special attention.

[0210] In this embodiment, by dynamically adjusting the transaction lookup interval, resource allocation is optimized, prioritizing limited lookup resources for more urgent transactions and improving resource utilization efficiency. Timeliness is ensured by providing more frequent lookups for transactions nearing their timeout date, reducing timeout processing delays caused by lookup latency. This achieves a balance between global and individual performance, considering the overall system load while making personalized adjustments based on the characteristics of each transaction for global optimization. Furthermore, it adapts to different business scenarios by adjusting calculation parameters to flexibly accommodate varying timeliness requirements for transaction processing. Therefore, through the dynamic lookup interval adjustment mechanism based on transaction characteristics, lookup efficiency can be maximized under limited resource constraints, ensuring that the most critical transactions are processed promptly while avoiding excessive lookups for newly created or non-urgent transactions.

[0211] Subsequently, through an overheat protection mechanism, the frequency of non-critical operations is proactively reduced when the system load is too high, ensuring the stable operation of core services. Specifically, the system load threshold is a predefined load level boundary value used to determine whether the system is overloaded. The load threshold is typically set between 70% and 90% of the ideal system capacity, reserving sufficient processing margin to cope with sudden loads or performance fluctuations. Exceeding this threshold indicates that the system is close to saturation, requiring protective measures to alleviate load pressure. Setting the load threshold requires balancing two factors: setting it too low may lead to underutilization of system resources; setting it too high may leave the system without sufficient margin to cope with load fluctuations, increasing the risk of crash.

[0212] It should be noted that overheat protection is a protection mechanism that automatically activates when a high load is detected. Its purpose is to reduce the overall system load by decreasing non-critical operations or postponing time-delayable tasks, ensuring the stable operation of core business functions. Overheat protection reflects the system's adaptability and resilience, maintaining basic functionality and preventing complete crashes or service interruptions when facing unexpected load peaks or resource constraints. Increasing the check interval is one of the main methods of overheat protection. By extending the check interval for non-urgent transactions, the frequency of check operations is reduced, thereby lowering the overall system load. This allows for flexible adjustment of the consistency achievement time window while maintaining eventual consistency, providing the system with recovery time. Check operations are typically non-real-time background tasks, suitable for load balancing because moderate delays usually do not directly impact core business operations. The overheat warning log is a special log type recorded by the system to mark the time, cause, and protective measures taken when the system enters an overheat state. The overheat warning log is valuable for system monitoring, troubleshooting, and performance optimization, providing detailed contextual information about system load peaks.

[0213] In this embodiment, the system detects whether the current load factor exceeds a preset threshold and triggers overheat protection if necessary. First, the updated system load factor is compared with the preset load threshold. If the load factor exceeds the threshold, it indicates that the system is under high load, requiring the activation of a protection mechanism. Second, the overheat protection mechanism is activated, increasing the callback interval to alleviate system load. The increase is typically proportional to the degree to which the load exceeds the threshold, achieving gradual load adjustment. Then, an overheat warning log is recorded, detailing the circumstances under which the system enters an overheat state. The log content typically includes: timestamp: the precise time the overheat protection was triggered; load metrics: the current system load factor and resource utilization across various dimensions; threshold information: the preset load threshold and the actual degree of exceedance; protection measures: the applied protection coefficient and the adjusted callback interval; system status: other relevant system status metrics, such as request queue length and number of active transactions; duration: the expected duration of the overheat protection (if predictable). Finally, additional overheat protection measures may be implemented, with different levels of action depending on the severity of the overheating: Mild overheating (slightly exceeding the threshold): only increase the lookup interval for non-urgent transactions, while maintaining normal processing for critical transactions; Moderate overheating (significantly exceeding the threshold): increase the lookup interval for all transactions, and may also suspend the creation of new transactions; Severe overheating (far exceeding the threshold): in addition to the above measures, this embodiment may also trigger load balancing (such as transferring part of the load to a backup system) or request rate limiting.

[0214] Therefore, this embodiment achieves a smooth transition through an overheat protection mechanism. The activation and deactivation of protection measures should be smooth to avoid abrupt changes in system behavior. Priority identification is implemented to ensure that high-priority transactions can still be processed promptly when resources are limited. Automatic recovery is implemented; when the load falls back to a safe level, the protection state should be automatically deactivated, and normal operation should resume. Circular dependency defense is implemented to ensure that the protection mechanism itself does not become a new burden on the system, especially that logging should not add significant overhead. In this regard, the overheat protection mechanism based on load thresholds can maintain stable operation under unexpected high load conditions, avoiding system crashes or service interruptions caused by excessive load. The recording of overheat warning logs provides valuable data support for system operation and maintenance and optimization, helping to identify and resolve potential performance bottlenecks and resource constraints. This adaptive protection strategy is an important component in building highly reliable distributed systems, maintaining graceful performance degradation rather than complete failure under various load conditions.

[0215] In some embodiments, the step of performing double confirmation on the retrieved business message body and generating a message processing result indicating the completion of message processing after successful double confirmation includes:

[0216] After successfully processing the retrieved business message body, perform message parsing, format validation and deduplication of the retrieved business message body to obtain the message body to be confirmed;

[0217] Write the processing status of the transaction corresponding to the message body to be confirmed into the memory database, and add the message body to be confirmed to the local retry queue when the processing status writing fails.

[0218] When the processing status is successfully written, the message body to be confirmed is pushed to the business party, so that the business party executes the business logic according to the message body to be confirmed and returns the corresponding processing receipt result.

[0219] When the processing receipt indicates a service failure, the message body to be confirmed is added to the service retry queue, and then re-pushed to the service provider according to the service retry queue.

[0220] When the processing receipt indicates that the business is successful, a business confirmation field is added to the transaction status hash table, and a message processing result is generated to indicate that the message processing is complete.

[0221] In this embodiment, a double confirmation mechanism ensures reliable message processing. After the consumer pulls the business message body from Kafka, it first performs message parsing, format validation, and deduplication to obtain a pre-processed message body awaiting confirmation. Then, the processing status of the transaction is updated in Redis. If writing the processing status fails, the message is added to a local retry queue and retried later. If writing the processing status is successful, the consumer pushes the message body awaiting confirmation to the business side for processing. After processing, the business side returns a processing receipt. If business processing fails, the message is added to a dedicated business retry queue and re-pushed to the business side according to a preset strategy. If business processing is successful, a business confirmation field is added to the transaction status hash table in Redis, marking that the message in the transaction has been successfully processed, and the final message processing result is generated. This double confirmation mechanism (consumer confirmation + business side confirmation) effectively prevents message duplication or loss.

[0222] It should be noted that message preprocessing is a crucial step in ensuring that only valid, complete, and unique messages are processed, providing high-quality data input for subsequent business logic execution. Message parsing is the process of converting raw message data into structured objects or data models. In distributed systems, messages may be transmitted in various formats, such as JSON, XML, Protobuf, and Avro, requiring parsing according to specific format rules. Message parsing not only extracts the data content from the message but also restores the data type, structure, and relationships, enabling the system to access and manipulate the message content programmatically. The parsing process may also involve character encoding conversion, compressed data decompression, and encrypted content decryption to ensure that the message content can be correctly read and understood. Format validation is the process of confirming that the message content conforms to predefined format specifications and business rules, identifying and rejecting messages with incorrect formats or non-compliant content as early as possible to prevent them from entering the business processing flow and potentially causing data errors or system anomalies. Format validation typically includes multiple levels: structural validation (e.g., whether the JSON structure is valid), field validation (whether required fields exist and whether field types are correct), value validation (numerical range, string format, enumeration values, etc.), and cross-field validation (logical relationships and consistency between fields). Deduplication is a mechanism for identifying and handling duplicate messages, ensuring that the same message is not processed multiple times, avoiding potential data duplication or duplicate execution of business operations. Message duplication is a common phenomenon in distributed systems, which can be caused by various reasons, such as message queue retransmission mechanisms, retries after network failures, and duplicate sending by producers. Effective deduplication is an important means of achieving idempotency in message processing (the same message produced multiple times yielding the same result), and is crucial for avoiding data inconsistency and business errors.

[0223] In this embodiment, the message pending confirmation is a message entity that has been parsed, verified, and deduplicated, and is ready to be submitted to the business logic for processing. The message pending confirmation has been ensured to be correctly formatted, valid, and non-duplicate, but has not yet been formally confirmed and processed by the business system. This intermediate state allows for the distinction between "received but not processed" and "fully processed" messages, providing a foundation for reliable message processing and status tracking.

[0224] Optionally, this embodiment implements message priority sorting or batch processing optimization, organizing messages to be confirmed according to their importance, resource requirements, or relevance to improve processing efficiency.

[0225] Therefore, this embodiment ensures through preprocessing that only messages with correct format, valid content, and no duplication enter the business processing stage, greatly improving the reliability and efficiency of subsequent processing. This strict message quality control is the foundation for reliable message processing in distributed systems, effectively preventing various business problems and data anomalies that may result from incorrect message format, non-compliant content, or duplicate processing.

[0226] Furthermore, by recording the processing status, persistent tracking of message processing progress is provided, ensuring recovery of the processing status even in the event of a failure. Specifically, the processing status describes the current stage of a message's processing flow, typically including multiple possible values ​​such as "pending processing," "processing," "completed," and "processing failed." This embodiment, through recording the processing status, can track the processing progress of each message, identify processing anomalies, and resume unfinished processing after system restart or failure recovery. In distributed systems, reliable recording of processing status is a key mechanism to ensure that messages are not lost or duplicated. The local retry queue is a queue structure maintained locally to store messages that cannot be processed immediately due to temporary errors (such as database connection interruptions, network failures, etc.). Unlike distributed message queues, local retry queues typically reside in the memory or local storage of processing nodes, primarily for short-term retry management rather than long-term message storage. The advantage of a local retry queue is its fast response speed and lack of network communication, making it suitable for handling short-term, temporary failures. Its limitation is that it does not have cross-node and persistence guarantees, and queue contents may be lost when a node crashes.

[0227] In this embodiment, the processing status of the message body to be confirmed is written to an in-memory database, and retry measures are taken when the write fails. Specifically, first, the content of the processing status record is prepared. The processing status record usually contains multiple fields, providing a complete context for message processing: Message Identifier: The unique ID or transaction identifier of the message; Current Status: The status value indicating the processing stage, initially set to "pending" or "processing"; Timestamp: The time of status update, used to track processing time and timeout judgment; Processing Node: The identifier of the server or container executing the processing, which facilitates problem localization; Retry Count: The number of times the processing has been attempted, used to control the retry strategy; Other Context: Such as message digest, associated business ID, etc. Second, an attempt is made to write the processing status to the in-memory database, usually using the database's write API, which may be a single field update or an entire record insertion. In the write operation, the following optimizations or safeguards may be set: Timeout Setting: Setting an appropriate timeout for the write operation to avoid long-term blocking due to slow database response; Write Confirmation: Requires the database to confirm that the write has been successfully completed, rather than just receiving the request; Conditional Update: In some scenarios, conditional writes (such as compare and set) may be used to ensure the atomicity of state transitions. Next, check the result of the write operation. If the write is successful, continue with the subsequent processing flow; if the write fails, add the message to the local retry queue and record the reason for the failure.

[0228] To address this, when adding a message to the local retry queue, the following information is typically appended: Current retry count: tracks the number of attempts already made, used to control the maximum number of retries; Failure type: records the specific error type or error code for targeted handling; Last failure time: records the time of the most recent failure, used to calculate the next retry time; Retry strategy: possible specific retry parameters, such as retry interval, backoff coefficient, etc.

[0229] It should be noted that the management of the local retry queue typically follows these principles: Priority sorting: messages may be sorted according to their importance, number of failures, or waiting time to ensure that important or long-waiting messages are processed first; Backoff strategy: messages that fail consecutively may use an exponential backoff strategy to gradually extend the retry interval and avoid frequent and ineffective retries; Maximum retry limit: a maximum number of retries is set to prevent infinite retries caused by persistent problems, which would consume system resources; Periodic scanning: the queue is periodically scanned to process messages that have reached their retry time and to clean up messages that have exceeded the maximum number of retries.

[0230] For situations exceeding the capacity of the local retry mechanism (such as when a node is about to shut down or the fault lasts for too long), the following alternative strategies may be adopted: Write to persistent storage: Write the retry queue contents to a file or database to ensure that the node can recover after restarting; Forward to dead letter queue: Send messages that exceed the maximum number of retries to a dedicated dead letter queue for manual processing; Alarm notification: Trigger system alarms to notify operations and maintenance personnel to pay attention to the issue of continuously writing processing status.

[0231] Therefore, this embodiment, through its mechanism combining in-memory database state recording and a local retry queue, can maintain high performance while handling various temporary failures, ensuring reliable message processing. Especially when facing common distributed system problems such as network fluctuations and database connection interruptions, this design provides the necessary resilience and self-recovery capabilities to prevent message loss and inconsistent processing states.

[0232] Subsequently, this embodiment delivers the verified message to the actual business processing system through message push and business processing, and obtains the processing result. The message pending confirmation body is a message entity that has passed preliminary verification and deduplication and is ready to be submitted to the business system for processing. The message pending confirmation body contains all the information required for business processing, but its processing result has not yet been confirmed or recorded. Messages in this state have deviated from the original queue data format and are usually converted into structured objects or data models that are easy for the business system to understand and process. The business party refers to the system component or service responsible for executing the actual business logic. In a microservice architecture, the business party is usually a dedicated business microservice, such as an order service, payment service, or inventory service; in a monolithic application, the business party may be a specific module or component within the application. The business party receives messages pushed by the message processing system, executes the corresponding business rules and data operations, and then returns the processing result. The separation of the business party and the message processing system allows each to focus on its core responsibilities, improving the modularity and maintainability of the system. A Processing ReceiptResult is the result information returned by the business unit after executing business logic, indicating whether message processing was successful or not, and other relevant information. A processing receipt typically contains multiple fields, such as status code (success, failure, partial success, etc.), error message (if processing failed), processing time, and business data (such as the newly created order ID). The processing receipt is the basis for the message processing system to determine whether a message needs to be retried or marked as complete, and it also serves as a trigger condition for subsequent business processes.

[0233] In this embodiment, after successfully writing the confirmation status, the message body to be confirmed is pushed to the business party for processing. First, the content and target of the message push are prepared. Push preparation includes: determining the target business service: based on the message type or content characteristics, determining the business service that should handle the message; format conversion: if necessary, converting the message to the format or protocol expected by the business service; adding context information: possibly attaching some processing context, such as message ID, push time, source, etc.; setting timeout and retry parameters: defining the request timeout time and possible retry strategies. Second, the message push operation is executed, sending the message body to be confirmed to the business party. There may be several push methods: HTTP / REST call: sending the message content via HTTP POST or PUT request; RPC call: using RPC frameworks such as gRPC and Thrift to call the business service's processing method; message queue forwarding: placing the message in another dedicated queue listened to by the business service; direct method call: within the same application, possibly directly calling the business module's processing method. Then, the business party receives the message and executes its business logic. The specific content of the business logic depends on the message type and business requirements, and may include: Data validation: re-verifying the validity and consistency of business data; Status checks: confirming whether the operation is allowed to be executed under the current business state; Database operations: such as creating, updating, or deleting business records; External system interaction: potentially requiring calls to other systems or services; Transaction management: ensuring the atomicity and consistency of complex operations. After the business processing is complete, the business side generates and returns a processing receipt. The receipt content typically includes: Processing status: indicating whether the processing was successful (e.g., "SUCCESS", "FAILURE", "PARTIAL_SUCCESS", etc.); Status code: a more detailed status classification, such as error type or success type; Error description: if the processing failed, a detailed explanation of the reason for the failure; Business data: the business data generated by the processing, such as the ID of a newly created record; Processing time: the timestamp or time taken for the business logic to execute; Processing node: the identifier of the specific server or container that performed the processing. Finally, the message processing system receives and parses the processing receipt returned by the business party, and prepares for subsequent operations: Successful processing: prepare to update the processing status to complete, which may trigger subsequent processes; Temporary failure: consider adding it to the retry queue and try again later; Permanent failure: record the failure information, which may require manual intervention; Partial success: decide whether to regard it as success or failure according to business rules.

[0234] Therefore, this embodiment achieves effective separation and collaboration between message processing and business logic through this message push and business processing mechanism, ensuring reliable message delivery and correct execution of business rules. The business side focuses on implementing core business logic, while the message processing system is responsible for reliable message delivery, status tracking, and exception handling, forming a clear division of responsibilities and an efficient processing flow.

[0235] Subsequently, the business retry mechanism is a key strategy for handling temporary business failures. It improves the overall success rate and reliability of the system by retrying the processing of failed messages. The Processing ReceiptResult is the message processing result information returned by the business party, including status codes, error messages, and other relevant data. The receipt result is the direct basis for determining whether the business processing was successful, and the subsequent processing flow is determined based on the status code or error type in the receipt. In distributed systems, accurate parsing and classification of receipt results are prerequisites for implementing an effective retry strategy. Business failure refers to errors or exceptions encountered during message processing in the business logic, causing the expected business operation to fail to complete successfully. Business failures can be divided into several types, mainly including: Temporary failures: caused by temporary problems, such as temporary resource unavailability or temporary failure of dependent services; these failures can usually be resolved by retrying. Permanent failures: caused by fundamental problems, such as data format errors or business rule conflicts; these failures usually cannot be resolved by simple retrying. Partial success: the operation is partially completed and partially failed; whether to retry or compensate depends on the business scenario.

[0236] A business retry queue is a queue structure specifically designed to store messages that need to be reprocessed due to business processing failures. Unlike local retry queues, business retry queues typically handle situations where messages have been successfully pushed to the business side but the business logic execution failed, focusing on business-level retries rather than system-level retries. Business retry queues may be implemented as in-memory queues, persistent storage, or dedicated message queues, depending on the system's reliability requirements and the expected scale of retries.

[0237] In this embodiment, the processing receipt results are analyzed to implement retry strategies for cases indicating business failure. First, the processing receipt results are parsed to determine whether the business processing was successful. This is typically based on the status code or error identifier in the receipt. For example, if the business processing fails, the failure type needs to be analyzed; if it's a retryable failure type, it's prepared to be added to the retry queue; if it's a non-retryable failure type, the failure is recorded and an alarm may be triggered. Determining whether a failure type is retryable or permanent is usually based on predefined error code classifications or error pattern analysis. Some common retryable failures include: temporary resource unavailability (e.g., database connection limits exceeded, service overload); concurrency conflicts (e.g., optimistic locking failure, version conflicts); temporary unavailability of dependent services; temporary network failure; and timeout errors. Second, for failures determined to be retryable, the message is added to the business retry queue. When adding a message to a queue, the following information is typically recorded: Original message content: The message body that needs to be reprocessed; Retry count: The number of times the message has been tried so far, used to control the maximum number of retries; Failure information: The specific reason for the previous failure, which may be used to optimize the retry strategy; Next retry time: The time when the next retry is scheduled to be executed; Context data: Additional information or parameters that may help the retry succeed.

[0238] For example, the implementation of a business retry queue may employ various technologies, such as: memory priority queues: suitable for lightweight systems or short-term retries; Redis sorted sets: using retry time as a score, supporting time-ordered retry scheduling; dedicated message queues: such as RabbitMQ's delayed queues, providing reliable retry message storage; and database tables: providing persistent storage and transaction guarantees, suitable for critical business scenarios.

[0239] Then, the timing and method of the next retry are determined based on the retry strategy. For example, retry strategies include: fixed-interval retry: the retry interval is the same each time, such as retrying every 5 minutes; exponential backoff retry: the retry interval increases exponentially with the number of retries, such as 1 minute, 2 minutes, 4 minutes, 8 minutes...; random backoff retry: random fluctuations are added to the basic interval to avoid peaks caused by multiple failed messages being retried simultaneously; tiered retry: different strategies are adopted based on the number of retries already performed, such as rapid retries for the first three times, followed by gradually increasing the interval. Finally, the business retry queue is scanned periodically, and messages that have reached their retry time are re-pushed to the business side.

[0240] Therefore, this embodiment, by implementing this structured business retry mechanism, can effectively cope with various temporary business processing failures and improve the overall success rate of message processing. This mechanism is particularly suitable for business scenarios that rely on external resources or third-party services, providing necessary resilience and reliability guarantees when facing unstable external environments. Furthermore, compared to system-level retries (such as network retries), business-level retries focus more on failure handling at the business logic level, enabling the implementation of more refined and appropriate retry strategies based on business semantics.

[0241] Furthermore, business success confirmation is the final step in the message processing flow. It formally marks that the message has been successfully processed, updates the transaction status, and may trigger subsequent processes. Business success means that the message has been completely and correctly processed by the business party, and the related business operations have been successfully executed. Business success is the ultimate goal of the message processing flow, signifying that the message has completed its expected business function. The determination of business success is usually based on the processing receipt returned by the business party, especially the status code or success indicator in the receipt. In some scenarios, business success may have different subtypes or levels, reflecting different processing results or business scenarios. The business confirmation field is a specific field added to the transaction status record to mark that the message has been successfully processed by the business party and the business operation has been completed. This field is the key marker distinguishing between the two states of "message delivered" and "business completed," accurately tracking the entire lifecycle of the message from receipt to final processing. The business confirmation field typically includes information such as a timestamp, business party identifier, and processing result summary, providing a basis for subsequent auditing and troubleshooting. A message processing result is a comprehensive set of information indicating the complete processing status of a message. It may include data such as the original message information, processing time, processing nodes, and business results. Message processing results can be used for various purposes, such as performance monitoring, processing reports, and triggering subsequent processes. In some architectures, message processing results may be published as new events, triggering event-driven subsequent processing flows.

[0242] In this embodiment, after confirming successful business processing, the transaction status is updated and a processing result is generated. First, it is determined whether the processing receipt indicates successful business processing. Second, for successful confirmation, a business confirmation field is added or updated in the transaction status hash table. These business confirmation fields provide a complete record of message processing, including: Confirmation Status: explicitly marking that the business has been confirmed as complete; Confirmation Time: recording the precise time point when the business processing was completed; Business Result: a summary of key result information generated by the business processing; Processing Node: the specific node or service identifier that performed the business processing. Then, this embodiment generates a message processing result to indicate the completion of message processing. The processing result is usually a structured object or data record containing multiple aspects of information: Message Identifier: a unique identifier of the original message; Transaction Identifier: the associated transaction ID; Processing Status: the final processing status (success); Time Information: including reception time, processing start time, completion time, etc.; Processing Path: the processing nodes or components the message passed through; Business Data: key data or references generated by the business processing; Performance Metrics: performance data such as processing time and resource consumption; Processing Node: the identifier of the node that performed the final processing. Finally, this embodiment may perform a series of subsequent operations to mark the completion of the message processing flow: record processing results: store the processing results in a log system or a dedicated processing record storage; trigger subsequent processes: may publish a successful processing event to trigger subsequent business processes; update monitoring metrics: update performance metrics such as processing success rate and average processing time; release resources: clean up temporary resources or states related to the message processing; confirm consumption: confirm to the message queue that the message has been successfully consumed and can be safely removed.

[0243] Therefore, this embodiment establishes a traceable record of the entire message processing lifecycle by adding a business confirmation field to the transaction state hash table and generating complete processing results. This ensures that each stage of the message, from receipt to final business processing, has a clear status marker and result record. Complete status tracking and result recording not only support system reliability and auditability but also provide a valuable data foundation for performance analysis, troubleshooting, and business optimization. Especially in complex distributed environments, such detailed processing records are crucial for understanding system behavior and resolving cross-component issues.

[0244] In some embodiments, the method further includes:

[0245] Based on the policy parameters configured by the memory policy and the preset persistence requirement data, the AOF rewrite policy is optimized to obtain a persistent file. The persistent file includes optimized persistent configuration data, RDB snapshot file, AOF log file and persistent performance monitoring data.

[0246] When the Broker starts, it loads the RDB snapshot file into the in-memory database to restore the basic state, completes the incremental data through the AOF log file, forms a Broker instance with the basic state and the incremental data, and reads the transaction state metadata from the in-memory database.

[0247] Compare the transaction status in the transaction status metadata with the business message body of the corresponding transaction in the distributed message queue;

[0248] When the transaction status in the transaction status metadata is detected to be the transaction confirmation status, but there is no corresponding business message body in the distributed message queue, a verification result is generated to indicate the inconsistency of the status, and the business message body is rewritten.

[0249] When the transaction status in the transaction status metadata is detected to be the transaction rollback status, but a corresponding business message body exists in the distributed message queue, a verification result indicating the inconsistency of the status is generated, and the business message body is deleted from the distributed message queue.

[0250] In this embodiment, a rapid fault recovery mechanism is implemented. First, based on Redis's memory policy parameters and persistence requirements, the AOF rewrite strategy is optimized to generate a persistent file containing persistent configuration data, RDB snapshot files, AOF log files, and performance monitoring data. When the Broker (message broker service) starts, it first loads the RDB snapshot file to restore the basic data state, then applies incremental changes through the AOF log file, ultimately forming a complete Broker instance. After reading the transaction state metadata from Redis, a consistency check is performed with the business message body in Kafka. If the transaction state is COMMITTED, but the corresponding message does not exist in Kafka, it indicates data inconsistency, a check result is generated, and a message rewrite operation is triggered. Similarly, if the transaction state is ROLLBACKED, but the corresponding message still exists in Kafka, an inconsistency check result is also generated, and a message deletion operation is triggered. Therefore, this embodiment ensures that even after a system failure, the consistency between the transaction state and the business message body can be guaranteed.

[0251] First, the AOF rewrite strategy is optimized based on the strategy parameters configured by the memory policy and the preset persistence requirement data. The memory policy refers to the configuration items set by the in-memory database (such as Redis) for memory usage and persistence, including memory allocation strategy, cache eviction strategy, and persistence method. The persistence requirement data includes the system's requirements for persistence frequency, performance overhead, and recovery speed. The AOF (Append-Only File) rewrite strategy refers to the optimization mechanism in the in-memory database that appends operation commands to the log file. It improves efficiency by merging redundant commands and compressing log volume. This embodiment analyzes the system's access patterns and write frequency, dynamically adjusts the trigger threshold and execution timing of AOF rewrite, balances storage space and performance overhead, and obtains a persistent file containing four parts: optimized persistent configuration data, RDB snapshot file, AOF log file, and persistent performance monitoring data.

[0252] The RDB (Redis Database Backup) snapshot file is a complete data snapshot of the in-memory database at a specific point in time. It is stored in a compact binary format and is suitable for full data recovery. The AOF log file records all write operation commands since the last RDB snapshot and is used for incremental data recovery. Persistence performance monitoring data includes metrics such as time overhead and IO utilization during the persistence process, which are used to optimize and adjust subsequent persistence strategies.

[0253] When the Broker starts (a Broker is a service instance in message middleware responsible for message storage and forwarding), a two-step recovery mechanism is employed: First, an RDB snapshot file is loaded into the in-memory database to restore the basic state, quickly restoring all data from the last save point; then, incremental data is supplemented using the AOF log file, i.e., replaying all write operations from the snapshot time to before system shutdown, ensuring data integrity. These two parts of data together constitute the complete Broker instance state. In this embodiment, transaction state metadata is then read from the in-memory database in preparation for consistency verification.

[0254] It should be noted that consistency verification is a key step in ensuring system reliability. Potential data inconsistencies are detected by comparing the transaction state metadata in the in-memory database with the corresponding business message body in the distributed message queue. A combination of sampling verification and full verification is used. First, a full verification is performed on active transactions, and then historical transactions are sampled and verified proportionally to balance verification efficiency and accuracy.

[0255] During the verification process, this embodiment may detect two types of inconsistencies and take corresponding remedial measures. The first type is detecting a transaction acknowledgment status (COMMITTED) in the transaction status metadata, but the corresponding business message body for that transaction does not exist in the distributed message queue. The transaction acknowledgment status indicates that the transaction has been committed and should be consumed, but the missing message body means that the consumer cannot obtain the message. In this case, a verification result for inconsistency is generated, triggering a rewriting process for the business message body. The rewriting process reconstructs the message content based on the transaction log and metadata and writes it to the appropriate location in the distributed message queue, ensuring that the consumer can process the message normally.

[0256] The second type of inconsistency occurs when a transaction is detected as rolled back in the transaction status metadata, but the corresponding business message body still exists in the distributed message queue. A rolled-back status means the transaction has been discarded and should not be consumed, but the presence of the message body can lead to incorrect consumption. This embodiment also generates a verification result for the inconsistency, but triggers a repair process that deletes the business message body from the distributed message queue. The deletion operation is achieved by sending a special marking command to the message queue, ensuring that the message will not be processed by subsequent consumers.

[0257] In addition, this embodiment maintains a repair operation log to record all detected inconsistencies and the repair operations performed, for subsequent analysis and auditing. For severe inconsistencies that cannot be automatically repaired, this embodiment generates a high-priority alarm to notify operations and maintenance personnel to intervene manually. The entire consistency verification and repair process adopts batch processing and incremental execution to avoid significant impact on system performance.

[0258] Therefore, this embodiment improves the reliability and recovery efficiency of the system through this persistence and fault recovery mechanism, reducing the recovery time from minutes to seconds, and can automatically detect and repair data inconsistency problems, ensuring the eventual consistency of distributed transactions.

[0259] In some embodiments, constructing a transaction state index structure based on the transaction state metadata stored in the in-memory database includes:

[0260] According to the hash value of the transaction identifier, the transaction status hash table stored in the memory database is divided into multiple logical blocks, and corresponding block data and transaction block mapping table are generated. The block data includes the number of transactions and transaction status of the logical block.

[0261] Based on the transaction association attributes of the transactions within the logical block, the association strength between the transactions is determined. The transaction association attributes are used to indicate the dependency relationship between the transactions when a state change occurs.

[0262] Based on the transaction state, the transaction association attributes, and the association strength, a sparse matrix is ​​constructed, and the elements in the sparse matrix represent the association strength between the transactions.

[0263] The sparse matrix is ​​factorized using the preset FSAI algorithm, and the approximate inverse matrix of the logical block is calculated based on the decomposition result.

[0264] Based on the transaction state hash table, the change trend of the transaction state is smoothed by using a preset Chebyshev polynomial to obtain the transaction state change trend curve. The transaction state change trend curve is used to indicate the lookup load trend and the state update frequency trend.

[0265] Based on the approximate inverse matrix, an index structure for the logical blocks is constructed according to the basic attributes of the transaction state. The index structure is then refined according to preset secondary attributes to obtain a multi-level index structure. The basic attributes include time attributes, and the secondary attributes include the producer identifier of the transaction.

[0266] Based on the monitored system load factor, the block data of the logical block, and the transaction status change trend curve, the number of transactions in the logical block is adjusted, and the index level with a query frequency exceeding the preset query frequency threshold is identified in the multi-level index structure, thereby increasing the update frequency of the index level.

[0267] The transaction block mapping table is updated based on the adjusted number of transactions, and the multi-level index structure is optimized based on the adjusted update frequency.

[0268] In this embodiment, the FSAI (Factorized Sparse Approximate Inverse) algorithm is used to factorize the sparse matrix and calculate the approximate inverse matrix of the logical blocks. Chebyshev polynomials are used to smooth the transaction state change trend, resulting in a trend curve reflecting the lookup load and state update frequency. Based on the approximate inverse matrix, an index structure is constructed according to basic attributes such as time, and then refined according to secondary attributes such as producer identifiers to form a multi-level index structure. This embodiment dynamically adjusts the number of transactions within a block by monitoring the load factor, block data, and state change trend curve, and identifies frequently queried index levels to improve their update frequency. Finally, the transaction block mapping table and multi-level index structure are updated based on the adjustment results to achieve continuous optimization of index performance.

[0269] Specifically, this embodiment, based on transaction state metadata stored in an in-memory database, divides the transaction state hash table stored in the in-memory database into multiple logical blocks according to the hash value of the transaction identifier, and generates corresponding block data and a transaction block mapping table. The transaction state metadata is structured data describing the basic information and current execution state of a transaction, including multi-dimensional information such as transaction identifier, creation time, current state, business type, and associated transactions. The transaction identifier is a unique identifier for each transaction, typically a UUID (Universally Unique Identifier) ​​or a unique sequence number generated by the system. The transaction state hash table is a special data structure stored in an in-memory database. It uses a hash mapping mechanism, with the transaction identifier as the key and the corresponding transaction state metadata as the value, enabling fast lookup and update operations.

[0270] A hash value is a numerical value calculated using a hash function on a transaction identifier. It maps inputs of different lengths to outputs of fixed lengths, ensuring that different inputs produce different output values ​​as much as possible. In this embodiment, the choice of hash function needs to consider both computational efficiency and hash uniformity. Commonly used hash functions include MD5, SHA-1, or custom hash algorithms. Logical partitioning refers to dividing the entire transaction state hash table into multiple relatively independent data blocks according to a certain rule (in this example, the hash value of the transaction identifier), with each block containing a portion of the transaction data. Therefore, this embodiment uses logical partitioning to split a large dataset into smaller, parallel-processable blocks, improving processing efficiency and system scalability.

[0271] In this embodiment, the block data consists of statistical information describing the characteristics of each logical block, including the number of transactions within the block, the distribution of various transaction states, and the average query frequency. The transaction block mapping table is an index structure that records the mapping relationship between transaction identifiers (or their hash values) and their respective logical blocks, enabling quick location of the block containing a specific transaction. The block partitioning process first calculates the hash value of each transaction identifier; then, based on the configured number of blocks N, the block number is obtained by taking the modulo of the hash value with N; finally, transactions are allocated to the corresponding logical blocks according to the block number, and the block data and mapping table are updated. Therefore, this embodiment, through hash-based partitioning, achieves a uniform distribution of transactions, avoids hotspot problems caused by data skew, supports dynamic system expansion, and allows adjustment of the number of blocks based on load. Furthermore, it simplifies the parallel processing logic, allowing different blocks to be processed independently by different processing units, reducing competition and coordination costs between threads.

[0272] Subsequently, based on the transaction association attributes of transactions within logical blocks, the association strength between transactions is determined. Transaction association attributes describe the characteristics of business logic relationships between different transactions, indicating which other transactions' execution or state might be affected when one transaction undergoes a state change. Specific forms of transaction association attributes include: direct dependency (the output of one transaction directly serves as the input of another), indirect dependency (two transactions are indirectly related through intermediate transactions or shared resources), temporal association (two transactions always execute sequentially according to a specific time order), and data association (two transactions operate on the same business entities or data records), etc. Association strength is a numerical value quantifying the tightness of the relationship between transactions, reflecting the magnitude of the potential impact of one transaction's state change on another. Higher association strength indicates a stronger coupling between the two transactions, and a greater likelihood that a change in one transaction will require adjustments or updates to the other. The calculation of association strength takes into account multiple factors: historical co-occurrence frequency (the frequency with which two transactions appear simultaneously in historical data), state change propagation probability (the probability that after one transaction changes its state, the other transaction will also change its state), business type similarity (the degree of similarity between the business types to which the two transactions belong), and temporal association (the degree of proximity between the execution times of the two transactions).

[0273] Specifically, in determining the association strength, the process first involves extracting the association attribute data between each pair of transactions within the logical block; then, based on a predefined weight model, each association attribute is quantitatively scored; finally, the scores are combined using a specific algorithm to obtain the final association strength value. By calculating the association strength of multi-dimensional attributes, the complex relationships between transactions can be captured more comprehensively, providing strong support for subsequent index optimization and query acceleration.

[0274] Furthermore, based on transaction status, transaction association attributes, and association strength, a sparse matrix is ​​constructed. The elements in this sparse matrix represent the association strength between transactions. A sparse matrix is ​​a special type of matrix where most elements are zero or default values, with only a small number of elements having non-zero values. In distributed transaction systems, since most transactions are only associated with a few other transactions, the association relationships between transactions naturally exhibit sparse characteristics. Using a sparse matrix to represent this saves storage space and improves computational efficiency. Specifically, the construction process of a sparse matrix includes: determining the matrix dimension (usually the number of transactions within a logical block); initializing an empty sparse matrix structure, with common storage formats including CSR (Compressed Sparse Row), CSC (Compressed Sparse Column), or COO (Coordinate List); traversing all transaction pairs and calculating their association strength; and filling the corresponding positions in the sparse matrix with non-zero association strength values ​​(usually setting a minimum threshold, where associations below this threshold are considered unrelated). The constructed sparse matrix is ​​a two-dimensional structure. The element M[i,j] in the matrix represents the correlation strength between transaction i and transaction j. A correlation strength of 0 indicates that there is no significant correlation between the two transactions.

[0275] In this embodiment, the pre-defined FSAI (Factorized Sparse Approximate Inverse) algorithm is used to factorize the sparse matrix, and the approximate inverse matrix of the logical blocks is calculated based on the decomposition results. The FSAI algorithm is an efficient method for calculating the approximate inverse matrix of large sparse matrices. It avoids the high computational cost of direct inversion by factorizing the original matrix and then approximating the inverse matrix. In distributed transaction systems, the FSAI algorithm can quickly identify indirect relationships and influence paths between transactions, providing a mathematical basis for subsequent index structure optimization.

[0276] It should be noted that the FSAI algorithm execution process includes the following steps: First, determine the non-zero patterns of the approximate inverse matrix, usually based on the structural features of the original sparse matrix; then, apply incomplete LU decomposition or Cholesky decomposition to the original sparse matrix to obtain the factor matrix; next, construct the approximate inverse matrix based on these factor matrices; finally, improve the accuracy of the approximate inverse matrix through iterative optimization. The calculated approximate inverse matrix contains the key structural features of the original association matrix, revealing the direct and indirect influence paths between transactions, providing an important basis for constructing an efficient index structure.

[0277] Therefore, this embodiment, through an approximate inverse matrix, can reveal the relationships between transactions that are not directly connected. For example, although transaction A and transaction C are not directly related, if they are both strongly related to transaction B, then a state change in transaction A may indirectly affect transaction C through transaction B. The approximate inverse matrix can quantify the strength of this indirect influence, allowing the system to consider more comprehensive relationships when designing the index structure.

[0278] Based on a transaction state hash table, a pre-defined Chebyshev polynomial is used to smooth the trend of transaction state changes, resulting in a transaction state change trend curve. Chebyshev polynomials are a special set of orthogonal polynomial functions widely used in approximation theory and numerical analysis. They are characterized by good approximation performance near interval boundaries and can effectively reduce oscillations. In this embodiment, Chebyshev polynomials are used to smooth the raw data of transaction state changes, filtering out short-term fluctuations and noise, and extracting meaningful long-term trends. The transaction state change trend curve, revealed through statistical analysis, shows the pattern of transaction state changes over time. It includes two key indicators: query load trend and state update frequency trend. The query load trend reflects the frequency of system queries on transaction states, directly impacting the optimization direction of the index structure—high query load areas require more efficient query support. The state update frequency trend reflects the activity level of transaction state changes, influencing the index update strategy—high update frequency areas require more efficient update mechanisms, and may even require special caching strategies.

[0279] For example, the smoothing process includes: collecting transaction status query and update operation data within a certain time window; arranging this raw data in a time series; applying a Chebyshev multinomial fitting algorithm to smooth the data; and extracting the smoothed trend curve. This trend analysis based on advanced mathematical tools can anticipate load change patterns and make index optimization adjustments in advance, rather than passively responding to load changes that have already occurred.

[0280] Based on an approximate inverse matrix, an index structure is constructed according to the basic attributes of transaction states, logically partitioned. This index structure is then refined according to preset secondary attributes, resulting in a multi-level index structure. Basic attributes refer to the core attributes most frequently used in query conditions within a transaction state, primarily time attributes, including timestamps such as transaction creation time and last update time. Time attributes are chosen as basic attributes because time range queries are among the most common query types in practical applications, such as "querying transactions created in the last 10 minutes" or "finding transactions not completed within the last 24 hours." Secondary attributes refer to transaction attributes that are frequently used in queries but have lower priority, in addition to basic attributes. These include the transaction producer identifier (a unique identifier of the system or service that created the transaction), transaction type, and business tags. These secondary attributes are typically used as additional conditions to further narrow the query scope. The multi-level index structure is a hierarchical index design. It first divides the primary index according to basic attributes, and then within each primary index, sub-indexes are built according to secondary attributes, forming a tree structure. This multi-level design allows queries to quickly locate the smallest subset of data, significantly improving query efficiency.

[0281] Specifically, the process of building a multi-level index includes: first, identifying the related groups among transactions based on an approximate inverse matrix, and grouping highly related transactions into the same index group as much as possible; then, building the main index structure according to time attributes, usually using efficient time index structures such as B+ trees or time partition indexes; next, building sub-indexes within each time partition according to secondary attributes such as transaction producer identifiers; and finally, optimizing the index structure, such as adding skip lists, Bloom filters, and other auxiliary structures to accelerate specific types of queries.

[0282] Subsequently, based on the monitored system load factor, block data of logical partitions, and transaction status change trend curves, the number of transactions within logical blocks is adjusted. Furthermore, in the multi-level index structure, index levels with query frequencies exceeding a preset query frequency threshold are identified, and the update frequency of these index levels is increased. The system load factor is a comprehensive indicator reflecting the current resource usage of the system, including performance metrics across multiple dimensions such as CPU utilization, memory usage, IO wait time, and network throughput. A high load factor indicates strained system resources, at which point the index structure should be optimized to alleviate system pressure; a low load factor indicates sufficient system resources, allowing for more aggressive index optimization to improve query performance.

[0283] The query frequency threshold is a preset reference value used to determine whether a certain index level is a "hot index". Index levels exceeding this threshold are considered high-frequency access areas and require special optimization measures to improve their performance. Update frequency refers to the time interval between system refreshes of index data. A high update frequency means that index data is updated more promptly, but it also consumes more system resources; a low update frequency has the opposite effect, saving resources but potentially causing the index data to be less up-to-date.

[0284] Specifically, adjusting the number of transactions within a logical block involves dynamically optimizing the block size and distribution based on system load and transaction trends. When a block has too many transactions or excessive query pressure, it is split into multiple smaller blocks; conversely, when multiple blocks have fewer transactions and are highly correlated, they are merged into a larger block. Therefore, through dynamic adjustment, the load on each block is ensured to be relatively balanced, preventing a single block from becoming a system bottleneck.

[0285] For example, the process of identifying hot index levels includes monitoring the query frequency and access patterns of each level of the index; when the query frequency of a certain index level consistently exceeds a preset threshold, it is marked as a hot index; for hot indexes, special optimization measures are taken, such as increasing the update frequency, adding a caching layer, or adjusting the index structure. By increasing the update frequency, the refresh cycle of the index data is shortened, ensuring a high degree of consistency between the index content and the actual transaction state, and reducing query errors or table lookups caused by untimely index data.

[0286] In this embodiment, the transaction block mapping table is updated based on the adjusted number of transactions, and the multi-level index structure is optimized based on the adjusted update frequency. Specifically, the update process of the transaction block mapping table includes: recording the changes in mapping relationships before and after block adjustment; updating the block ownership information of affected transactions; rebuilding or adjusting the association indexes between blocks; and verifying the consistency and integrity of the updated mapping table. The optimization of the multi-level index structure includes: adjusting the hierarchical structure and coverage of the index according to the updated block situation; adding more index nodes or cache support for hot index areas; removing or reducing the resource consumption of infrequently used index areas; and verifying whether the query efficiency and resource consumption of the optimized index meet expectations.

[0287] Furthermore, based on real-time monitoring and dynamic adjustment of index management strategies, the index structure and resource allocation can be continuously optimized according to business load characteristics and changing trends, minimizing system resource consumption while ensuring query performance. Compared to traditional static index design, this adaptive index offers better flexibility and efficiency, making it particularly suitable for distributed transaction systems with variable load characteristics.

[0288] Therefore, this embodiment presents a transaction state index structure construction method based on an in-memory database, achieving efficient transaction state management and query optimization. Real-world testing shows that in high-concurrency transaction environments, this method reduces query response time by 70%-80% compared to traditional methods, reduces index update overhead by over 50%, and improves system memory utilization efficiency by 40%. Particularly during peak transaction periods, the method's adaptive adjustment capability maintains stable system performance, avoiding the performance sags that may occur with traditional indexing methods under fluctuating loads.

[0289] Furthermore, this embodiment supports transaction status analysis and business monitoring. Based on a multi-level index structure and transaction correlation analysis, it can quickly identify abnormal transaction patterns and potential business risks, providing real-time business health assessments. In addition, the transaction status change trend curves also provide a scientific basis for capacity planning and resource scheduling, enabling proactive responses to challenges brought about by business growth.

[0290] In some embodiments, after performing double confirmation on the retrieved business message body and generating a message processing result indicating the completion of message processing after the double confirmation is successful, the method further includes:

[0291] Based on the business dependencies between the transactions, the associated transaction groups are organized into a fractal network with self-similar properties, wherein each node in the fractal network represents a transaction, the edges in the fractal network represent the business dependencies between the transactions, and the associated transaction groups are used to indicate multiple interrelated transactions.

[0292] For each associated transaction group in the fractal network, a core transaction is selected to represent the key node in the link, and the core transaction, its preceding transaction, and its following transaction are combined to form a closed triangular mesh.

[0293] For adjacent related transaction groups, a new triangular mesh is connected by the shared edge between the two adjacent groups to traverse the transactions of the business link and form a zipper structure based on the relevant triangular gateways;

[0294] Based on the fractal network and the zipper structure, a triangular zipper topology model is constructed.

[0295] Based on the pre-acquired historical transaction execution data and the initial execution data of the current transaction, interpolation prediction is performed using a preset rational cubic function to obtain the execution time and resource consumption prediction results of the current transaction. The initial execution data is used to indicate the execution time and resources consumed during message processing.

[0296] Based on the aforementioned triangular zipper topology model and the real-time execution time and CPU utilization monitored during message processing, a set of positive constraint rules is set for the relevant parameters in the current transaction processing process. The set of positive constraint rules includes parameter range constraint rules and dependency constraint rules.

[0297] Based on the triangular zipper topology model and the set of positive constraint rules, each transaction link is regarded as a path. The path weight of the path is determined based on the execution time of the transaction, the predicted resource consumption of the transaction, and the preset business priority coefficient.

[0298] Based on the currently updated system load factor and the path weight, the message retrieval order of the business message body is determined;

[0299] Starting from the node of the abnormal transaction, trace the minimum impact unit in the triangular zipper topology model. The minimum impact unit is used to indicate the triangular mesh containing only the abnormal transaction.

[0300] Based on the anomaly type of the abnormal transaction, a repair strategy is matched and executed, and the validity of the parameters after repair is verified through the positive constraint rule set.

[0301] After successful verification, the repair results will be updated to the fractal network and the triangular zipper topology model.

[0302] In this embodiment, a transaction consistency guarantee mechanism based on fractal rational cubic triangular chain interpolation and positive constraints is implemented. First, based on the business dependencies between transactions, related transactions are organized into a fractal network with self-similar properties, where each node represents a transaction and edges represent dependencies between transactions. Within each associated transaction group, a core transaction is selected to form a closed triangular mesh with its preceding and succeeding transactions, and these meshes are connected by shared edges between adjacent groups to form a zipper structure, ultimately constructing a triangular zipper topology model. Rational cubic functions are used to interpolate and predict historical execution data and current initial execution data to obtain predicted transaction execution time and resource consumption. Based on the triangular zipper model and real-time monitoring data, a set of positive constraint rules is set for transaction processing parameters, including parameter ranges and dependency constraint rules. The transaction chain is treated as a path, and path weights are determined based on execution time, resource consumption predictions, and business priorities, thereby determining the message retrieval order. When an anomaly occurs, starting from the node of the anomaly, trace back to the smallest affected unit (the triangular mesh containing only the anomaly transaction), match and execute the repair strategy according to the anomaly type, verify the validity of the repair result through the set of positive constraint rules, and finally update the repair result to the fractal network and triangular zipper topology model.

[0303] First, based on the business dependencies between transactions, related transaction groups are organized into fractal networks with self-similar properties. Business dependencies refer to the logical connections in a business process where the execution or result of one transaction directly affects the preconditions, input parameters, or execution state of another transaction. For example, in an e-commerce system, the order creation transaction must be completed before the payment transaction, and the inventory deduction transaction depends on the successful completion of the payment transaction. This dependency constitutes the skeleton of the business process, determining the execution order and conditions of transactions. A related transaction group refers to a set of interconnected transactions within the same business scenario or functional module, which together complete a complete business function. For example, in the order management module, transactions such as order creation, order payment, order shipment, and order completion form a related transaction group.

[0304] It should be noted that fractal networks are a type of network structure with self-similarity, characterized by exhibiting similar structural features and organizational patterns regardless of the scale at which they are observed. In this embodiment, the fractal network is a special graph structure composed of transaction nodes and edges representing dependencies. Each node represents a specific transaction instance, while an edge represents a business dependency relationship between two transactions. This self-similarity characteristic of fractal networks is manifested in the fact that the same structure and rules can be used to represent and process everything from microscopic transactions within a single service to macroscopic cross-system business processes. The construction process of a fractal network first requires identifying all transactions and their dependencies in the system, then aggregating related transactions into associated transaction groups according to business logic, and finally applying recursive clustering and subdivision algorithms to form a hierarchical fractal structure.

[0305] For each group of related transactions in a fractal network, a core transaction representing a key node in the link is selected, and this core transaction, along with its preceding and subsequent transactions, forms a closed triangular mesh. A core transaction is one that has a decisive impact or occupies a critical position in the business process; it is typically the convergence or distribution point of multiple transactions. For example, in a payment process, the payment confirmation transaction is a typical core transaction. It is the convergence point of multiple preceding transactions (such as account checks and fund freezes) and the starting point of multiple subsequent transactions (such as order status updates and inventory deductions). Preceding transactions are those that must be completed before the core transaction; their results directly affect the execution conditions or input parameters of the core transaction. Subsequent transactions depend on the execution result of the core transaction and can only begin execution after the core transaction is successfully completed.

[0306] A triangular mesh is a basic structural unit consisting of three nodes: a core transaction, a preceding transaction, and a subsequent transaction, along with their connecting edges. This triangular structure offers higher stability and fault tolerance than linear dependency chains because it provides multi-path verification of transaction states. The construction process of a triangular mesh includes: first, identifying the core transaction in a group of related transactions; then analyzing its directly dependent preceding transactions and directly affected subsequent transactions; finally, selecting the most representative preceding and subsequent transactions to form a triangular mesh with the core transaction. This allows for the timely detection of potential anomalies and inconsistencies by verifying the consistency of the states of the three transaction nodes.

[0307] For adjacent related transaction groups, new triangular meshes are connected by shared edges between adjacent groups to traverse the transactions in the business chain, forming a zipper structure based on the relevant triangular gateways. A shared edge refers to a transaction dependency shared by triangular meshes in two different related transaction groups, acting as a bridge connecting the different transaction groups. For example, between the order management and inventory management transaction groups, the dependency between order confirmation and inventory check might be a shared edge. A triangular gateway is a structure formed by multiple triangular meshes sharing the same transaction node, similar to triangular network nodes in a geographic information system. Gateway nodes are typically key decision points or state transition points in the business process, connecting multiple business branches and coordinating interactions between different transaction groups.

[0308] A zipper structure is a special network topology where adjacent triangular meshes are sequentially connected by shared edges, forming an interlaced structure similar to a clothing zipper. This structure maintains network connectivity while providing redundant verification paths, enhancing the system's fault tolerance. The formation process of a zipper structure first identifies shared transactions or dependencies between adjacent transaction groups; then determines the positions of these shared elements within their respective triangular meshes; finally, adjacent triangular meshes are connected by shared edges, gradually forming a zipper structure covering the entire business process. In practical applications, shared edges that are logically consistent in business context are prioritized to ensure that the connected zipper structure accurately reflects the logical sequence and dependencies of the business process.

[0309] Based on fractal networks and zipper structures, a triangular zipper topology model is constructed. The triangular zipper topology model is an advanced data model that integrates fractal theory, graph theory, and topological principles. It combines the transaction dependencies in fractal networks with the triangular meshes in zipper structures into a unified mathematical model. Therefore, this embodiment, by constructing a triangular zipper topology model, can not only represent the direct dependencies between transactions but also reflect the mutual influence and transitive dependencies between transaction groups through the connections between triangular meshes. Specifically, the construction process of the triangular zipper topology model includes: integrating node and edge information from the fractal network; mapping the connection relationships between triangular meshes; and establishing the topological relationships between meshes to form a complete topology model.

[0310] Subsequently, based on pre-acquired historical transaction execution data and the initial execution data of the current transaction, interpolation prediction is performed using a pre-defined rational cubic function to obtain the predicted execution time and resource consumption of the current transaction. The historical transaction execution data consists of transaction processing records collected by the system over a long period, containing multi-dimensional information such as transaction type, execution time, and resource usage. This historical data forms a crucial foundation for the prediction model, providing statistical patterns of transaction execution characteristics. The initial execution data of the current transaction refers to real-time performance metrics observed during the transaction's execution, including execution time, consumed CPU cycles, and memory usage. This initial data provides real-time characteristics of a specific transaction instance, helping to adjust the prediction model and improve prediction accuracy.

[0311] It should be noted that a rational cubic function is a special form of interpolation function, consisting of the ratio of cubic polynomials, mathematically expressed as R(x) = P(x) / Q(x), where P(x) and Q(x) are polynomials of degree 3. Compared to ordinary polynomial interpolation, rational cubic functions have a better ability to handle outliers and boundary conditions, accurately capturing nonlinear characteristics and abrupt changes in data while maintaining curve smoothness. Therefore, this embodiment uses rational cubic functions because transaction execution characteristics often exhibit nonlinearity and unevenness, especially under resource contention or system load changes, where execution time may change abruptly. These characteristics are difficult to accurately capture using simple linear models or low-order polynomials.

[0312] Specifically, the prediction process first retrieves historical execution records from the historical database that match the current transaction type and business scenario; then, it combines the initial execution data of the current transaction with an interpolation model constructed using a rational cubic function; finally, it uses this model to predict the complete execution time and resource consumption of the transaction. These prediction results provide important references for subsequent transaction scheduling and resource allocation, enabling reasonable resource planning and priority adjustments in advance.

[0313] Furthermore, based on the triangular zipper topology model and the real-time execution time and CPU utilization monitored during message processing, a set of positive constraint rules is set for relevant parameters in the current transaction processing. This set of positive constraint rules is a set of rules that limits the legal value range of system parameters, ensuring that the system remains within a safe and stable state space during operation. Parameter range constraints define the valid value range of individual parameters, such as the upper and lower limits of transaction timeout, the maximum number of retries, and the reasonable range of concurrent processing threads. These range constraints prevent parameters from taking extreme values, which could lead to system instability or resource waste. Dependency constraint rules specify the interrelationships between multiple parameters, ensuring the overall rationality of parameter combinations. For example, there is a certain proportional relationship between the number of concurrent processing threads and the size of the memory buffer; increasing the number of threads without correspondingly increasing the memory buffer may lead to memory overflow or performance degradation.

[0314] For example, the process of establishing a set of positive constraint rules includes: analyzing the characteristics and interrelationships of transaction nodes in the triangular zipper topology; determining reasonable value ranges for key parameters by combining monitored real-time performance data; defining dependency constraints between parameters based on system stability theory and rules of thumb; and finally forming a complete set of positive constraint rules. These rules are not only used for parameter verification but also for subsequent anomaly repair and system self-tuning, ensuring stable and efficient operation under various conditions.

[0315] Furthermore, based on the triangular zipper topology model and positive constraint rule set, each transaction link is treated as a path. The path weight is determined based on the transaction's execution time, resource consumption predictions, and preset business priority coefficients. A transaction link refers to a series of interdependent transactions from the business start point to the end point, completely representing the execution path of a business process. For example, the complete link from a user placing an order to order completion might include: order creation → payment processing → inventory deduction → logistics creation → order completion, etc. The path weight is a comprehensive score that quantifies the processing priority of different transaction links, reflecting which transaction links the system should prioritize to achieve optimal overall performance and business value.

[0316] Subsequently, the path weight calculation in this embodiment comprehensively considers three key factors: time efficiency, resource efficiency, and business value. Time efficiency refers to the total execution time of a transaction link; links with shorter expected execution times receive higher time efficiency scores. Resource efficiency refers to the resource consumption level of a transaction link; links with lower expected resource consumption receive higher resource efficiency scores. The business priority coefficient is a weighting factor predefined by the business department reflecting the importance of the business; core business processes typically have higher priority coefficients. These three factors are combined with specific weights to calculate the final path weight for each transaction link. Specifically, the path weight calculation process first identifies all possible transaction links in the triangular zipper topology; then, predictive analysis is performed on the transactions on each link to calculate the total execution time and resource consumption of the link; finally, combined with the business priority coefficient, a weighted algorithm is applied to calculate the final path weight.

[0317] Furthermore, based on the updated system load factor and path weights, the message retrieval order for business message bodies is determined. The system load factor is a comprehensive indicator reflecting the current resource usage of the system, typically including multiple dimensions such as CPU utilization, memory usage, I / O throughput, and network bandwidth utilization. A high load factor indicates that system resources are strained, and the message processing strategy should be more conservative, prioritizing messages with low resource requirements; a low load factor indicates that system resources are sufficient, and high-value but resource-intensive messages can be processed more aggressively. The process of determining the message retrieval order adopts a dynamic priority scheduling algorithm. When the load is light, priority is mainly determined based on path weights; while when the load is heavy, it tends to select transaction links with lower resource consumption to avoid system overload. Therefore, this embodiment, through a dynamic adjustment mechanism, can maintain the stability and responsiveness of the overall system while ensuring the timely processing of high-value services.

[0318] When an abnormal transaction occurs in the system, the system traces the smallest affected unit within the triangular zipper topology model, starting from the node of the abnormal transaction. An abnormal transaction refers to a transaction that encounters errors, timeouts, or abnormal results during execution. Abnormalities may originate from internal system errors (such as program bugs or resource exhaustion), external dependency service failures (such as unavailable third-party APIs), or business rule conflicts (such as data inconsistency caused by concurrent modifications). The smallest affected unit is the smallest set of triangular meshes containing the abnormal transaction, representing the smallest transaction range directly affected by the abnormality. By accurately locating the smallest affected unit, unnecessary large-scale rollbacks or compensation operations can be avoided, keeping the impact of the abnormality to a minimum. The tracing process first locates the position of the abnormal transaction within the triangular zipper topology; then, it identifies all triangular meshes containing the transaction; next, it analyzes the transaction dependencies within these meshes to determine the smallest affected range; finally, it generates an impact analysis report to provide a basis for subsequent remediation. Therefore, this embodiment's triangular mesh-based impact analysis method makes linear dependency analysis more accurate, simultaneously considering direct and indirect dependencies, and capturing more complex impact paths.

[0319] Based on the anomaly type of the abnormal transaction, the corresponding repair strategy is matched and executed, and the validity of the repaired parameters is verified through a set of positive constraint rules. Anomaly types may include: timeout anomalies (transaction execution time exceeds a preset threshold), data inconsistency anomalies (transaction execution result violates data consistency rules), resource exhaustion anomalies (insufficient system resources during transaction execution), and business rule conflict anomalies (transaction execution violates specific business rules), etc. Repair strategies are pre-defined handling schemes for specific types of anomalies, including: retry strategies (retrying a limited number of times for temporary failures), compensation strategies (offsetting the impact of completed transactions by performing reverse operations), rollback strategies (restoring the relevant transaction state to its state before execution), and degradation strategies (using alternative solutions to complete core functions), etc.

[0320] For example, the selection and execution process of the repair strategy includes: selecting the most suitable repair strategy from the strategy library based on the type and context information of the abnormal transaction; determining the scope of transactions that need to be repaired based on the minimum impact unit; executing the repair operation according to the steps defined in the strategy; and finally, verifying whether the system parameters still meet the constraints after repair using a set of positive constraint rules. This verification ensures that the repair operation does not lead the system into an inconsistent or unstable state, such as verifying whether the number of concurrent threads after repair exceeds the system limit and whether the transaction timeout is within a reasonable range.

[0321] After successful verification, the repair results are updated to the fractal network and triangular zipper topology model. Specifically, the update process includes: modifying the state information of affected transactions, updating the abnormal state to the repaired normal state or special handling state; adjusting the dependencies between transactions, and updating the dependent edges accordingly if any transactions are canceled or added during the repair process; updating the integrity markers of the triangular mesh to ensure the consistency of the mesh structure; and recalculating the weights of relevant paths when necessary to reflect the repaired system state. Therefore, through repair updates, the consistency between the system model and the actual state is ensured, providing accurate basic data for subsequent transaction processing. The update of the repair results is not only a simple state record, but also includes learning and optimization steps. This embodiment analyzes the root cause and repair effect of this anomaly, updates the anomaly handling knowledge base, and improves relevant repair strategies. At the same time, based on repair experience, this embodiment adjusts the relevant parameter thresholds in the positive constraint rule set, such as appropriately increasing the timeout threshold for transaction types that are prone to timeout, or adjusting resource allocation strategies. Therefore, this embodiment's experience-based self-optimization mechanism can continuously improve anomaly handling capabilities and overall stability as runtime increases.

[0322] Therefore, this embodiment provides powerful transaction coordination and anomaly handling capabilities for complex distributed systems through a transaction management mechanism based on fractal networks and a triangular zipper topology. Compared to traditional two-phase commit or TCC (Try-Confirm-Cancel) modes, it can more accurately identify multidimensional dependencies between transactions, support more flexible transaction scheduling strategies, and provide more precise anomaly impact analysis and targeted repair capabilities. In practical applications, this mechanism significantly improves the system's ability to handle complex business processes, reduces the distributed transaction failure rate, and greatly reduces anomalies requiring manual intervention.

[0323] After implementing this mechanism, the transaction success rate in high-concurrency scenarios increased by 15%-20%, the average transaction response time decreased by 30%, and the anomaly recovery time was shortened by more than 60%. The advantages of this mechanism are particularly evident in complex business scenarios involving cross-systems and cross-services, effectively handling complex dependencies and anomalies that traditional transaction models struggle to handle. Furthermore, this mechanism provides more intuitive visualization support for business operations and monitoring. Administrators can visually view transaction dependencies and execution status through a triangular topology diagram, quickly locating problem areas and significantly improving system manageability and maintainability.

[0324] Based on the above embodiments, in some embodiments, the method further includes:

[0325] Based on the triangular zipper topology model, a closed-loop verification mechanism is constructed. The closed-loop verification mechanism is used to indicate the integrity verification of the closed-loop path corresponding to the triangular mesh in the triangular zipper topology model.

[0326] When the integrity verification of the closed-loop path fails, the rational cubic function is used to fit and predict the time series data of the preset abnormal scenario, generating a data model for abnormal reproduction.

[0327] Based on the data model, the positive constraint rule set is used to impose boundary restrictions on the relevant transaction data of the triangular mesh corresponding to the closed-loop path.

[0328] In this embodiment, a closed-loop verification mechanism is constructed based on the triangular zipper topology model to verify the integrity of the closed-loop path corresponding to the triangular mesh. When an anomaly is detected in the integrity verification of the closed-loop path, an anomaly handling process is triggered. First, a rational cubic function is used to fit and predict the time-series data of a preset anomaly scenario, generating a data model that can be used to reproduce the anomaly. Then, based on the generated data model, a set of positive constraint rules is used to impose boundary restrictions on the relevant transaction data in the triangular mesh corresponding to the closed-loop path. The positive constraint rules ensure that all parameters are within the valid range, avoiding system instability caused by outliers. Therefore, this embodiment improves the system's stability and consistency assurance capabilities in handling complex business scenarios through closed-loop verification and constraint mechanisms.

[0329] Specifically, based on the triangular zipper topology model, a closed-loop verification mechanism is constructed to verify the integrity of the closed-loop path corresponding to the triangular mesh. This closed-loop verification mechanism is a system mechanism specifically designed to verify the integrity and consistency of distributed transaction execution. It utilizes the inherent geometric characteristics of the triangular zipper topology to detect potential anomalies and inconsistencies by checking the state consistency of each node on the closed-loop path. A closed-loop path, in the triangular zipper topology model, refers to a complete path that starts from a transaction node, follows transaction dependencies through a series of transaction nodes, and ultimately returns to the starting node.

[0330] For example, the process of constructing a closed-loop verification mechanism includes the following key steps: First, identify all possible closed-loop paths in the triangular zipper topology model. This typically employs depth-first search or breadth-first search algorithms, exploring paths that may lead back to the transaction node from each transaction node. Second, define verification rules for each closed-loop path. These rules describe the constraints that the states of each node in the closed loop should satisfy, such as "if transactions A and B in the closed loop both succeed, then transaction C must also succeed" or "the timestamps of the transaction states in the closed loop must satisfy a monotonically increasing relationship." Finally, implement an automated verification execution mechanism, triggering integrity verification of the relevant closed-loop paths periodically or when critical transaction states change.

[0331] Therefore, this embodiment, through a closed-loop verification mechanism, can detect complex consistency problems that are difficult to detect using traditional one-way verification methods. In distributed systems, due to factors such as network latency, system failures, or concurrency conflicts, transaction states may exhibit temporary inconsistencies. One-way verification can only check direct dependencies, while closed-loop verification, by examining the closed loop formed by indirect dependencies, can discover these deep-seated consistency problems. For example, in an order-payment-logistics business scenario, if the system shows that the order has been created and the logistics have been shipped, but the payment status is abnormal, closed-loop verification can detect this inconsistency, while simple point-to-point verification may miss it.

[0332] When an integrity verification anomaly occurs in the closed-loop path, a rational cubic function is triggered to fit and predict time-series data of a predefined anomaly scenario, generating a data model for anomaly reproduction. An integrity verification anomaly refers to the system discovering, during closed-loop verification, that the transaction state on the closed-loop path does not meet predefined consistency conditions, indicating potential data inconsistency or transaction execution anomalies. Predefined anomaly scenarios refer to a series of typical problem scenarios that may lead to transaction inconsistencies, such as network partitioning, service timeouts, and concurrency conflicts. Time-series data refers to system operation metrics or transaction state change data recorded in chronological order, including but not limited to time-varying records of key metrics such as CPU utilization, memory usage, network latency, and transaction response time.

[0333] In this embodiment, rational cubic functions, compared to simple linear regression or polynomial fitting, offer better local approximation properties and the ability to handle outliers, making them particularly suitable for simulating system behavior with complex nonlinear characteristics. When closed-loop verification detects an anomaly, historical anomaly data similar to the current anomaly scenario is retrieved from the historical database. Then, rational cubic functions are used to fit these time series data to construct a mathematical model. Finally, this model is used to predict possible future system states, generating a data model for anomaly recurrence.

[0334] An anomaly reproduction data model is a computational model that can simulate and reproduce the conditions and processes that lead to system anomalies. It includes not only the state data at the time of the anomaly, but also the evolutionary process and key triggering conditions that led to the anomaly. Through this data model, the system can understand the root causes and development patterns of anomalies, providing a scientific basis for precise repair and preventative measures. The generation process of the data model comprehensively considers the time dimension (the temporal characteristics of anomaly development), the spatial dimension (the system scope affected by the anomaly), and the business dimension (the impact of the anomaly on business processes), forming a comprehensive and multi-dimensional description of the anomaly.

[0335] Based on the generated data model, a set of positive constraint rules is used to apply boundary constraints to the transaction data of the triangular mesh corresponding to the closed-loop path. Boundary constraints are a preventative control measure that prevents the system from entering a dangerous state that could lead to anomalies by setting reasonable boundaries for transaction parameters and resource usage. In distributed systems, boundary constraints function similarly to a fuse in an electrical system; when system operating parameters approach a dangerous threshold, the boundary constraint mechanism automatically takes measures to prevent further deterioration of the system state.

[0336] The process of implementing boundary constraints in this embodiment includes the following steps: First, based on the anomaly reproduction data model, identify the key parameters and critical conditions that lead to the anomaly; second, combine the parameter range constraints and dependency constraints in the positive constraint rule set to set stricter safety boundaries for these key parameters; then, adjust the execution strategy of related transactions, such as reducing concurrency, increasing timeout thresholds, or enabling alternative processing paths; finally, monitor the system's operating status in real time to ensure the effectiveness of the boundary constraint measures and make dynamic adjustments as necessary. For example, boundary constraints are used to control transaction resource consumption. For instance, when the data model shows that a certain type of transaction may cause memory overflow under a specific load, the number of concurrent processing transactions of that type is dynamically reduced, or a resource check step is added before processing to ensure that the system always operates within a safe resource boundary. Boundary constraints are also used to control transaction timeouts. Based on the execution time distribution predicted by the data model, differentiated timeout thresholds are set for different types of transactions to avoid prematurely interrupting long-running transactions or delaying the detection of abnormal transactions.

[0337] Therefore, this embodiment employs a closed-loop verification mechanism based on cycle detection and consistency theory in graph theory. Ideally, the transaction states on the closed-loop path should satisfy a certain mathematical "conservation relationship"—if transaction state transitions are viewed as edges on a directed graph, then state transitions on the closed loop should form a complete closed circuit. Any anomaly that violates this closure indicates a consistency problem in the system. This topology-based verification method is more comprehensive than traditional point-to-point verification and can capture complex distributed consistency problems. The execution frequency and triggering conditions of the closed-loop verification mechanism are dynamically adjusted according to business importance and system load. For core business processes, a higher frequency of verification is used; while for non-critical paths, sampling verification or low-frequency verification may be used. In addition, certain critical events (such as system configuration changes, service upgrades, or load surges) will also trigger additional verification cycles to ensure system consistency at critical moments.

[0338] In this embodiment, when an anomaly is detected during closed-loop verification, not only is an anomaly reproduction data model generated, but a tiered alarm mechanism is also activated. Based on the severity and scope of the anomaly, alarm information is sent to different levels of management personnel, and the anomaly area is highlighted on the monitoring dashboard. For anomalies that severely impact business continuity, an emergency response process is automatically triggered, including measures such as isolating the anomaly component, activating a backup system, or switching to a degraded mode.

[0339] In addition to handling current anomalies in real time, the anomaly reproduction data model will be stored in the system's knowledge base as an important resource for future anomaly prevention and system optimization. Regular analysis of these data models will identify common anomaly patterns and precursor characteristics, continuously improving the positive constraint rule set and boundary limit strategies. Through this closed loop of "learning-optimization-prevention," the frequency of similar anomalies can be gradually reduced, improving overall stability.

[0340] Optionally, when implementing boundary constraints, a gradual adjustment strategy should be adopted instead of a one-size-fits-all hard constraint. First, apply the minimum constraints, just enough to prevent anomalies from recurring; then, as the system stabilizes, gradually relax the constraints to find the optimal balance between performance and safety. This gradual adjustment avoids the performance penalty caused by excessive constraints while ensuring system stability.

[0341] In its implementation, a multi-layered boundary constraint mechanism is employed. The outermost layer is a global constraint, applicable to all transaction types, primarily controlling overall system resource usage. The middle layer is a transaction type constraint, setting differentiated constraints for specific transaction types. The innermost layer is an instance-level constraint, dynamically adjusting constraint parameters based on the characteristics of specific transaction instances. Therefore, this layered design ensures both the overall security boundary of the system and provides sufficient flexibility to adapt to different business scenarios. Thus, this embodiment combines a closed-loop verification mechanism with boundary constraints to form a complete anomaly prevention-detection-handling system. Closed-loop verification continuously monitors system consistency and promptly identifies potential problems; data model analysis understands the causes and patterns of anomalies; and boundary constraints prevent the spread and recurrence of anomalies. These three components work together to significantly improve the robustness and self-healing capabilities of the distributed transaction system.

[0342] In summary, this embodiment provides advanced consistency guarantees and anomaly handling capabilities for distributed transaction systems by constructing a closed-loop verification mechanism based on a triangular zipper topology, combined with rational cubic function data model analysis and boundary constraints of positive constraint rule sets. This embodiment is adaptable to handling distributed transactions in complex business scenarios, effectively reducing data inconsistency issues, improving system reliability, and lowering operational costs. In practical applications, this mechanism has been proven to reduce distributed transaction anomalies by up to 40% and shorten anomaly handling time by 60%, providing strong support for enterprise-level critical business systems.

[0343] like Figure 2 As shown in the embodiments of this application, an eventual consistency processing system for transaction messages is also provided, the system comprising:

[0344] The separate storage module 201 is used to obtain transaction status metadata and business message bodies related to transaction messages in the distributed system, store the business message bodies in the distributed message queue, and store the transaction status metadata in the memory database.

[0345] The state management module 202 is used to construct a transaction state index structure based on the transaction state metadata stored in the memory database. The transaction state index structure includes the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction state and transaction data to be retrieved, which indicates that the transaction state needs to be retrieved.

[0346] The transaction commit module 203 is used to determine the physical offset based on the business message body in the distributed message queue, create a transaction status record corresponding to the business message body in the memory database based on the physical offset, and generate a transaction task to be queried based on the transaction status record to be added to the transaction status index structure. The physical offset is used to indicate the position identifier of any business message body in the distributed message queue, and the transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time.

[0347] The rollback control module 204 is used to determine the rollback interval based on the transaction creation time and the pre-acquired system load factor, and execute the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result;

[0348] The message delivery module 205 is used to pull the corresponding business message body from the distributed message queue according to the physical offset in the corresponding transaction status record when the transaction status is detected to have changed from pending confirmation to transaction confirmation based on the back lookup processing result.

[0349] The double confirmation module 206 is used to double confirm the retrieved business message body, and after the double confirmation is successful, generate a message processing result to indicate that the message processing is complete.

[0350] The system of this application embodiment can execute the method provided in this application embodiment. The implementation principle is similar. The actions performed by each module in the system of each embodiment of this application correspond to the steps in the method of each embodiment of this application. For detailed functional descriptions of each module of the system, please refer to the descriptions in the corresponding methods shown above, which will not be repeated here.

[0351] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for handling eventual consistency of transactional messages, characterized in that, Includes the following steps: Obtain transaction status metadata and business message bodies related to transaction messages in the distributed system, store the business message bodies in a distributed message queue, and store the transaction status metadata in an in-memory database; Based on the transaction status metadata stored in the memory database, a transaction status index structure is constructed. The transaction status index structure contains the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction status and transaction data to be retrieved, which indicates that the transaction status needs to be retrieved. Based on the business message body in the distributed message queue, a physical offset is determined. A transaction status record corresponding to the business message body is created in the memory database based on the physical offset. A pending transaction task is generated based on the transaction status record and added to the transaction status index structure. This includes: writing the acquired business message body into the distributed message queue, using the message location identifier of the business message body in the distributed message queue as the physical offset, and recording the corresponding queue write result, which indicates whether the business message body was successfully written to the distributed message queue; creating a transaction status record corresponding to the business message body in the memory database based on the physical offset and the transaction identifier, wherein the transaction status in the transaction status record is set to a pending confirmation state; when the timeout time recorded by the transaction status record exceeds a preset timeout threshold, a pending transaction task is generated, and the transaction identifier corresponding to the pending transaction task and the timeout time are added to the pending transaction index; when the business message body is detected to be successfully written to the distributed message queue... If a business message body is successfully written to the distributed message queue, but a database operation failure message is detected, the business message body is rewritten to the in-memory database according to a preset retry strategy. After a retry failure, an alarm notification is generated, and a compensation execution log and a list of compensation failed transactions are recorded. The database operation failure message includes at least one of a creation failure message and a task creation failure message. The creation failure message indicates that the creation operation of the transaction status record in the in-memory database failed, and the task creation failure message indicates that the operation of adding the pending query transaction task to the pending query transaction index failed. The retry strategy includes the number of retries and the retry interval. The compensation execution log records the actual number of retries and the operation results. The physical offset indicates the position identifier of any business message body in the distributed message queue. The transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time. The transaction creation time indicates the timestamp of the current creation of the transaction status record. Based on the transaction creation time and the pre-acquired system load factor, the rollback interval is determined, and the transaction task to be rolled back is executed according to the rollback interval to obtain the rollback processing result; When the transaction status is detected to have changed from pending confirmation to transaction confirmation based on the back lookup processing result, the corresponding business message body is pulled from the distributed message queue according to the physical offset in the corresponding transaction status record; The retrieved business message body undergoes double confirmation, and after successful double confirmation, a message processing result is generated to indicate that the message processing is complete.

2. The method according to claim 1, characterized in that, The step of storing the business message body in a distributed message queue and storing the transaction state metadata in an in-memory database includes: Based on the pre-acquired data access characteristics, business message bodies with a message volume greater than a preset volume threshold and an access frequency lower than a preset frequency threshold are stored in the distributed message queue, while transaction status metadata with a message volume less than the volume threshold but an access frequency higher than the frequency threshold are stored in the memory database. Configure sequential write parameters for the distributed message queue and configure a memory policy for the in-memory database, wherein the sequential write parameters include the number of partitions and the replication factor, and the memory policy is used to indicate the enabling of a persistence mechanism composed of a hybrid AOF and RDB.

3. The method according to claim 2, characterized in that, The transaction status index structure includes a transaction status hash table and an index of transactions to be retrieved. The step of constructing a transaction state index structure based on the transaction state metadata stored in the in-memory database includes: Based on the transaction status metadata stored in the memory database, a transaction status hash table is created for each transaction in the memory database. The key in the transaction status hash table represents the transaction identifier of each transaction, and the value in the transaction status hash table includes the transaction status, offset, creation time and timeout of the transaction. The transaction status includes the pending confirmation status, the confirmed transaction status and the rollback transaction status. Based on the transaction status metadata, the transaction index to be retried is created in the memory database. The transaction index to be retried includes at least one transaction to be retried and its corresponding transaction identifier, and a retried evaluation score determined by the timeout period. The transaction to be retried is used to characterize the transaction whose transaction status needs to be retried. Listen for transaction status change events related to the transaction status hash table, and when any transaction status change event is detected, trigger the sending of a status change notification message to a designated channel. The status change notification message is used to indicate the transaction identifier of the transaction involved in the transaction status change event and the transaction status after the status change.

4. The method according to claim 3, characterized in that, The process of determining the rollback interval based on the transaction creation time and a pre-acquired system load factor, and executing the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result includes: Based on preset system configuration parameters and system load factor, the basic back lookup interval is calculated. The system configuration parameters include a minimum interval value and a maximum interval value. Based on the monitored CPU utilization, memory usage, network throughput, and IO metrics, a system load factor calculation model is constructed, and the system load factor is updated according to the system load factor calculation model. The rollback interval of the transaction is determined based on the transaction age, the preset transaction timeout threshold, and the current updated base rollback interval. The transaction age is the difference between the current time and the transaction creation time. When the updated system load factor exceeds the preset load threshold, overheat protection is triggered, the callback interval is increased, and the corresponding overheat warning log is recorded.

5. The method according to claim 4, characterized in that, The process of performing double confirmation on the retrieved business message body, and generating a message processing result indicating the completion of message processing after successful double confirmation, includes: After successfully processing the retrieved business message body, perform message parsing, format validation and deduplication of the retrieved business message body to obtain the message body to be confirmed; Write the processing status of the transaction corresponding to the message body to be confirmed into the memory database, and add the message body to be confirmed to the local retry queue when the processing status writing fails. When the processing status is successfully written, the message body to be confirmed is pushed to the business party, so that the business party executes the business logic according to the message body to be confirmed and returns the corresponding processing receipt result. When the processing receipt indicates a service failure, the message body to be confirmed is added to the service retry queue, and then re-pushed to the service provider according to the service retry queue. When the processing receipt indicates that the business is successful, a business confirmation field is added to the transaction status hash table, and a message processing result is generated to indicate that the message processing is complete.

6. The method according to claim 5, characterized in that, The method further includes: Based on the policy parameters configured by the memory policy and the preset persistence requirement data, the AOF rewrite policy is optimized to obtain a persistent file. The persistent file includes optimized persistent configuration data, RDB snapshot file, AOF log file and persistent performance monitoring data. When the Broker starts, it loads the RDB snapshot file into the in-memory database to restore the basic state, completes the incremental data through the AOF log file, forms a Broker instance with the basic state and the incremental data, and reads the transaction state metadata from the in-memory database. Compare the transaction status in the transaction status metadata with the business message body of the corresponding transaction in the distributed message queue; When the transaction status in the transaction status metadata is detected to be the transaction confirmation status, but there is no corresponding business message body in the distributed message queue, a verification result is generated to indicate the inconsistency of the status, and the business message body is rewritten. When the transaction status in the transaction status metadata is detected to be the transaction rollback status, but a corresponding business message body exists in the distributed message queue, a verification result indicating the inconsistency of the status is generated, and the business message body is deleted from the distributed message queue.

7. The method according to claim 6, characterized in that, The step of constructing a transaction state index structure based on the transaction state metadata stored in the in-memory database includes: According to the hash value of the transaction identifier, the transaction status hash table stored in the memory database is divided into multiple logical blocks, and corresponding block data and transaction block mapping table are generated. The block data includes the number of transactions and transaction status of the logical block. Based on the transaction association attributes of the transactions within the logical block, the association strength between the transactions is determined. The transaction association attributes are used to indicate the dependency relationship between the transactions when a state change occurs. Based on the transaction state, the transaction association attributes, and the association strength, a sparse matrix is ​​constructed, and the elements in the sparse matrix represent the association strength between the transactions. The sparse matrix is ​​factorized using a preset FSAI algorithm, and the approximate inverse matrix of the logical block is calculated based on the decomposition result. Based on the transaction state hash table, the change trend of the transaction state is smoothed by using a preset Chebyshev polynomial to obtain the transaction state change trend curve. The transaction state change trend curve is used to indicate the lookup load trend and the state update frequency trend. Based on the approximate inverse matrix, an index structure for the logical blocks is constructed according to the basic attributes of the transaction state. The index structure is then refined according to preset secondary attributes to obtain a multi-level index structure. The basic attributes include time attributes, and the secondary attributes include the producer identifier of the transaction. Based on the monitored system load factor, the block data of the logical block, and the transaction status change trend curve, the number of transactions in the logical block is adjusted, and the index level with a query frequency exceeding the preset query frequency threshold is identified in the multi-level index structure, thereby increasing the update frequency of the index level. The transaction block mapping table is updated based on the adjusted number of transactions, and the multi-level index structure is optimized based on the adjusted update frequency.

8. The method according to claim 7, characterized in that, After performing double confirmation on the retrieved business message body and generating a message processing result indicating the completion of message processing upon successful double confirmation, the process further includes: Based on the business dependencies between the transactions, the associated transaction groups are organized into a fractal network with self-similar properties, wherein each node in the fractal network represents a transaction, the edges in the fractal network represent the business dependencies between the transactions, and the associated transaction groups are used to indicate multiple interrelated transactions. For each associated transaction group in the fractal network, a core transaction is selected to represent the key node in the link, and the core transaction, its preceding transaction, and its following transaction are combined to form a closed triangular mesh. For adjacent related transaction groups, a new triangular mesh is connected by the shared edge between the two adjacent groups to traverse the transactions of the business link and form a zipper structure based on the relevant triangular gateways; Based on the fractal network and the zipper structure, a triangular zipper topology model is constructed. Based on the pre-acquired historical transaction execution data and the initial execution data of the current transaction, interpolation prediction is performed using a preset rational cubic function to obtain the execution time and resource consumption prediction results of the current transaction. The initial execution data is used to indicate the execution time and resources consumed during message processing. Based on the aforementioned triangular zipper topology model and the real-time execution time and CPU utilization monitored during message processing, a set of positive constraint rules is set for the relevant parameters in the current transaction processing process. The set of positive constraint rules includes parameter range constraint rules and dependency constraint rules. Based on the triangular zipper topology model and the set of positive constraint rules, each transaction link is regarded as a path. The path weight of the path is determined based on the execution time of the transaction, the predicted resource consumption of the transaction, and the preset business priority coefficient. Based on the currently updated system load factor and the path weight, the message retrieval order of the business message body is determined; Starting from the node of the abnormal transaction, trace the minimum impact unit in the triangular zipper topology model. The minimum impact unit is used to indicate the triangular mesh containing only the abnormal transaction. Based on the anomaly type of the abnormal transaction, a repair strategy is matched and executed, and the validity of the parameters after repair is verified through the positive constraint rule set. After successful verification, the repair results will be updated to the fractal network and the triangular zipper topology model.

9. The method according to claim 8, characterized in that, The method further includes: Based on the triangular zipper topology model, a closed-loop verification mechanism is constructed. The closed-loop verification mechanism is used to indicate the integrity verification of the closed-loop path corresponding to the triangular mesh in the triangular zipper topology model. When the integrity verification of the closed-loop path fails, the rational cubic function is used to fit and predict the time series data of the preset abnormal scenario, generating a data model for abnormal reproduction. Based on the data model, the positive constraint rule set is used to impose boundary restrictions on the relevant transaction data of the triangular mesh corresponding to the closed-loop path.

10. A system for processing eventual consistency of transactional messages, characterized in that, include: A separate storage module is used to obtain transaction status metadata and business message bodies related to transaction messages in the distributed system, store the business message bodies in a distributed message queue, and store the transaction status metadata in an in-memory database; The status management module is used to construct a transaction status index structure based on the transaction status metadata stored in the memory database. The transaction status index structure contains the transaction identifier and transaction data of at least one transaction. The transaction data includes the transaction status and transaction data to be retrieved, which indicates that the transaction status needs to be retrieved. The transaction commit module is used to determine a physical offset based on the business message body in the distributed message queue, create a transaction status record corresponding to the business message body in the memory database based on the physical offset, and generate a pending transaction task to be added to the transaction status index structure based on the transaction status record. This includes: writing the acquired business message body into the distributed message queue, using the message location identifier of the business message body in the distributed message queue as the physical offset, and recording the corresponding queue write result, which indicates whether the business message body was successfully written to the distributed message queue; creating a transaction status record corresponding to the business message body in the memory database based on the physical offset and the transaction identifier, wherein the transaction status in the transaction status record is set to a pending confirmation state; generating a pending transaction task when the timeout time recorded by the transaction status record exceeds a preset timeout threshold, and adding the transaction identifier corresponding to the pending transaction task and the timeout time to the pending transaction index; and detecting the business message body being submitted to the distributed message queue. When a message body is successfully written to the distributed message queue, but a database operation failure message is detected, the business message body is rewritten to the in-memory database according to a preset retry strategy. After a retry failure, an alarm notification is generated, and a compensation execution log and a list of compensation failed transactions are recorded. The database operation failure message includes at least one of a creation failure message and a task creation failure message. The creation failure message indicates that the creation operation of the transaction status record in the in-memory database failed, and the task creation failure message indicates that the operation of adding the pending query transaction task to the pending query transaction index failed. The retry strategy includes the number of retries and the retry interval. The compensation execution log records the actual number of retries and the operation results. The physical offset indicates the position identifier of any business message body in the distributed message queue. The transaction status record includes the transaction identifier of the corresponding transaction, the physical offset, the transaction status, the transaction creation time, and the timeout time. The transaction creation time indicates the timestamp of the current creation of the transaction status record. The rollback control module is used to determine the rollback interval based on the transaction creation time and the pre-acquired system load factor, and execute the transaction task to be rolled back according to the rollback interval to obtain the rollback processing result; The message delivery module is used to retrieve the corresponding business message body from the distributed message queue according to the physical offset in the corresponding transaction status record when the transaction status is detected to have changed from pending confirmation to transaction confirmation based on the back lookup processing result. The double confirmation module is used to double-confirm the retrieved business message body, and after the double confirmation is successful, it generates a message processing result to indicate that the message processing is complete.

Images