A data consistency guarantee method and system based on best-effort transactions and timed compensation
By introducing message state version numbers and dynamic locking mechanisms into the distributed microservice architecture, the mutual exclusion coordination problem between the best-effort retry and the timed compensation mechanism is solved, atomic updates and arbitration of message processing rights are realized, duplicate sending and resource waste are avoided, and the reliability of data consistency assurance is improved.
Patent Information
- Application Number
- CN202611228161.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-08-13
- Publication Date
- 2026-11-17
AI Technical Summary
In a distributed microservice architecture, the lack of mutual exclusion coordination between the best-effort retry and timed compensation mechanisms when they run independently can lead to duplicate message sending and wasted computing resources. This can cause message processing order disorder and inconsistent states, especially in high-concurrency scenarios.
By introducing message status version numbers as optimistic locks, and combining status version number condition updates and exponential backoff mechanisms, a dynamic lock validity period and message hiding window are designed to achieve atomic updates and arbitration of message processing rights, ensuring that only one recovery channel processes the same message at the same time.
It effectively avoids duplicate compensation and concurrent contention, reduces network bandwidth and computing resource consumption, and improves the determinism of the message recovery process and the guarantee of system data consistency.
Smart Images

Figure FT_1 
Figure SMS_2 
Figure SMS_3
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to a data consistency guarantee method and system based on best-effort transactions and timed compensation. Background Technology
[0002] Currently, in distributed microservice architectures, to ensure eventual consistency of data across services, a common approach is to use a local message table combined with multiple recovery mechanisms. Business operations and message persistence are completed within the same local transaction. For messages that fail to send initially, a best-effort retry module initiates several immediate retries within a short period based on an exponential backoff strategy. Simultaneously, a separate timed compensation task scans the message table at fixed intervals, performing supplementary delivery for messages that have been in an unacknowledged state for an extended period, forming a dual recovery channel with parallel immediate retries and timed compensation. These two recovery mechanisms operate independently, each determining whether processing is necessary based on the message record's status field. However, they lack a unified mutual exclusion coordination mechanism. When a message fails to send, both may simultaneously prepare to resend or process the same unacknowledged message within the same time window.
[0003] For example, when a message fails to be sent for the first time due to network jitter, the best-effort retry module immediately enters the retry process. During its retry waiting interval, the message's status remains "pending confirmation." Simultaneously, the timed compensation task scans and reads the message's "pending confirmation" status and prepares to send compensation. Because there is no distributed lock or strict state version identifier to mutually exclude processing rights, the two recovery channels will concurrently perform delivery operations on the same message, causing the message to be repeatedly sent to downstream business systems. This not only wastes network bandwidth but also causes the business side to repeatedly execute the same operation. Even if the consumer side implements idempotent control, the repeated transmission and redundant processing still consume considerable computing and communication resources, and in high-concurrency scenarios, it will further exacerbate lock contention, leading to derivative problems such as message processing order disorder and state inconsistency.
[0004] Therefore, there is an urgent need for a method that, even when best-effort retries and timed compensation are deployed simultaneously, can still ensure that only one recovery mechanism processes the same unacknowledged message at any given time by establishing a mutual exclusion coordination mechanism. This would completely eliminate concurrent contention and duplicate compensation between the two recovery channels, reduce network bandwidth waste and computing resource consumption, and improve the determinism of the message recovery process and the reliability of overall system data consistency assurance. Summary of the Invention
[0005] To address the aforementioned technical shortcomings, the present invention aims to propose a data consistency guarantee method based on best-effort transactions and timed compensation. This method addresses the technical problem that existing technologies lack mutual exclusion coordination between the independent operation of best-effort retries and timed compensation, especially under conditions where the retry and compensation scan cycles overlap and state version control is not introduced, thus failing to effectively avoid contention between the two recovery channels and duplicate compensation.
[0006] To solve the above-mentioned technical problems, the present invention adopts the following technical solution: The present invention provides a data consistency guarantee method based on best-effort transactions and timed compensation.
[0007] The data consistency guarantee method based on best-effort transactions and timed compensation includes: Step S10: Obtain the message to be sent, and based on the message to be sent, perform the message state initialization task using the local transaction message table persistence mechanism, and output the initial message record; Step S20: Based on the initial message record, a maximum effort retry mechanism combining status version number condition update and exponential backoff is used to perform the message retransmission task, and the retry result message record is output; Step S30: Based on the retry result message record, a timed compensation mechanism combining message hiding window and timeout status filtering is used to perform compensation preprocessing tasks and output compensation candidate message records; Step S40: Based on the compensation candidate message record, the compensation takeover task is executed using a processing right arbitration mechanism that combines status version number verification, condition update and message status review, and the processing right determination message record is output. Step S50: Based on the processing authority, determine the message record for compensation sending, message status convergence and message consumption deduplication, and output the completed message record.
[0008] Preferably, step S10, which involves obtaining the message to be sent, performing a message state initialization task based on the message to be sent using a local transaction message table persistence mechanism, and outputting an initial message record, specifically includes: Step S101: In the local business transaction, the message to be sent is written into the message table in the database. The message table is set with a unique message identifier, message status, status version number, number of retries, preset maximum number of retries, and last status change time. Step S102: Set the message status corresponding to the message to be sent to pending confirmation, set the status version number to 1, set the number of retries to 0, and set the last status change time to the creation time of the message to be sent, to obtain the initial message record; Step S103: Submit the initialization message record and the corresponding business data together in the business local transaction, and determine the initialization message record that has been submitted as the initial message record.
[0009] Preferably, step S20, which involves executing a message retransmission task based on the initial message record using a maximum effort retry mechanism combining state version number conditional update and exponential backoff, and outputting the retry result message record, specifically includes: Step S201: Read the message status and status version number of the initial message record, and update the message status from pending confirmation to retry processing through a first conditional update statement; wherein, the first conditional update statement takes the read message status as pending confirmation and the read status version number as the same as the current status version number in the message table as the update condition, and increments the status version number by 1 when the update is successful, thereby obtaining the retry processing right of the initial message record, and determines the incremented status version number as the expected version number for the first retry; Step S202: After obtaining the retry processing right, determine the first... The waiting interval corresponding to each retry is calculated using the first formula: The effective duration of the dynamic lock is determined based on the waiting interval corresponding to each retry, and the effective duration of the dynamic lock is calculated using the second formula: in, Indicates the first The waiting interval corresponding to each retry; Indicates the retry sequence number, and ; This indicates the preset maximum number of retries; Indicates the preset baseline backoff interval; This indicates a retreat from growth factors; Indicates the first The random jitter value corresponding to each retry; This indicates the effective duration of the dynamic lock; Indicates the preset maximum time for sending a single message; Indicates the preset redundancy protection duration; Step S203: In the first After the waiting interval corresponding to the retry is completed, the current message status and current status version number of the initial message record are read again; when the current message status is "retrying" and the current status version number is the same as the expected version number for this retry, the next step is executed. The message was resent; when the first message was resent. When the message is successfully resent, the message status is updated to "completed" using the status version number condition update, and the status version number is incremented by 1; when the message is resent successfully... The message resending failed and When the status version number is updated, the number of retries is incremented by 1 and the status version number is incremented by 1, and the incremented status version number is determined as the expected version number for the next retry; when the 1st The message resending failed and When the message status is updated from "Retrying" to "Pending Confirmation" by updating the status version number condition, the number of retries is incremented by 1 and the status version number is incremented by 1 to obtain the retry result message record; when the current message status is not "Retrying" or the current status version number is different from the expected version number for this retry, the corresponding message retransmission is terminated, and the reread message record is determined as the retry result message record.
[0010] Preferably, in step S202, the retreat growth factor The value is 2.0; the random jitter value The pseudo-random number generator generates random numbers from 0 to a preset maximum random jitter value. The samples were obtained between the two points, where, This represents the preset maximum random jitter value.
[0011] Preferably, step S30, which involves performing a compensation preprocessing task based on the retry result message record using a timed compensation mechanism combining message hiding window and timeout status filtering, and outputting compensation candidate message records, specifically includes: Step S301: Determine the message hiding window duration based on the effective duration of the dynamic lock, the preset compensation scan cycle, and the preset hysteresis buffer amount. The message hiding window duration is calculated using a third formula: in, This indicates the duration of the message hiding window; This indicates the preset compensation scan period; This indicates the preset delay buffer amount; Step S302: Scan the message table and filter the first timeout message record and the second timeout message record from the retry result message records; wherein, the message status of the first timeout message record is pending confirmation, and the difference between the current time and its last status change time is greater than the message hiding window duration; the message status of the second timeout message record is retrying, and the difference between the current time and its last status change time is greater than the dynamic lock validity period; record the status version number corresponding to the first timeout message record and the second timeout message record during the scan; Step S303: Reread the current message status and current status version number of the first timeout message record and the second timeout message record respectively, and determine the message record whose current message status is the same as the message status during scanning and whose current status version number is the same as the status version number during scanning as the compensation candidate message record.
[0012] Preferably, in step S40, the step of executing the compensation takeover task based on the compensation candidate message record using a processing right arbitration mechanism that combines status version number verification, condition update, and message status review, and outputting the processing right determination message record, specifically includes: Step S401: Read the message status, status version number, number of retries and last status change time of the compensation candidate message record, and divide the compensation candidate message record into candidate message records to be confirmed and candidate message records in retry processing according to the message status; Step S402: For the candidate message record to be confirmed, update its message status from pending confirmation to compensation processing through the second condition update statement; wherein, the second condition update statement takes the current message status as pending confirmation and the current status version number as the same as the status version number reread in step S303 as the update condition, and increments the status version number by 1 when the update is successful to obtain the corresponding compensation processing right; Step S403: For the candidate message record in the retry processing, when the number of retries reaches the preset maximum number of retries, and the difference between the current time and its last state change time is greater than the effective duration of the dynamic lock, the message status is atomically updated from "in retry processing" to "in compensation processing" through a third condition update statement; wherein, the third condition update statement uses the current message status as "in retry processing", the current status version number being the same as the status version number reread in step S303, and the number of retries reaching the preset maximum number of retries as update conditions, and increments the status version number by 1 when the update is successful to obtain the corresponding compensation processing right; the message record that obtains the compensation processing right through the second condition update statement or the third condition update statement is determined as the processing right determination message record.
[0013] Preferably, step S50, which involves determining the message record for compensation sending, message state convergence, and message consumption deduplication based on the processing authority, and outputting the completed message record, specifically includes: Step S501: Read the current message status and current status version number of the processing right determination message record. When the current message status is in compensation processing, perform compensation sending according to the message content carried by the processing right determination message record to obtain the compensation sending result. Step S502: When the compensation sending result is successful, the message status of the processing right determination message record is updated from "in the compensation processing" to "completed" using the fourth condition update statement, and the status version number is incremented by 1 to obtain the status convergence message record; when the compensation sending result is failed, the message status is updated from "in the compensation processing" to "pending confirmation" using the fifth condition update statement, the status version number is incremented by 1, and the last status change time is updated to the current time; wherein, both the fourth and fifth condition update statements use the message status "in the compensation processing" and the status version number being the same as the current status version number read in step S501 as the update condition; Step S503: When the message consumer receives the message corresponding to the state convergence message record, the message unique identifier is written into the deduplication table with a unique index for the message unique identifier; when the message unique identifier is successfully written, message consumption is performed, and the state convergence message record that has completed message consumption is determined as the completed message record; when the message unique identifier fails to be written due to the existence of the same message unique identifier in the unique index, the corresponding duplicate message consumption is stopped.
[0014] The present invention also provides a data consistency guarantee system based on best-effort transactions and timed compensation, comprising: The persistence initialization module is used to obtain the message to be sent, perform message state initialization tasks based on the message to be sent using the local transaction message table persistence mechanism, and output the initial message record; The maximum effort retry module is used to perform message retransmission tasks based on the initial message record using a maximum effort retry mechanism that combines status version number conditional update and exponential backoff, and outputs the retry result message record. The timed compensation preprocessing module is used to perform compensation preprocessing tasks based on the retry result message records using a timed compensation mechanism that combines message hiding window and timeout status filtering, and output compensation candidate message records. The processing right arbitration module is used to execute the compensation takeover task based on the processing right arbitration mechanism that combines status version number verification, condition update and message status review based on the compensation candidate message record, and output the processing right determination message record; The consistency confirmation module is used to determine the message record for compensation sending, message status convergence and message consumption deduplication based on the processing right, and output the completed message record.
[0015] The present invention also provides a data consistency guarantee device based on best-effort transactions and timed compensation. The data consistency guarantee device based on best-effort transactions and timed compensation includes: a memory, a processor, and a data consistency guarantee program based on best-effort transactions and timed compensation stored in the memory and executable on the processor. When the data consistency guarantee program based on best-effort transactions and timed compensation is executed by the processor, it implements the above-described method.
[0016] The present invention also provides a computer program product, the computer program product including a data consistency guarantee program based on best-effort transactions and timed compensation, the data consistency guarantee program based on best-effort transactions and timed compensation implementing the above method when executed by a processor.
[0017] The beneficial effects of this invention are as follows: This invention introduces a message status version number as an optimistic lock, encapsulating the acquisition of message processing rights and state transitions into atomic update operations. This ensures that both the best-effort retry module and the timed compensation task must preempt the processing rights through condition updates before operation, thereby achieving mutual exclusion access to the same message at the database level and eliminating the possibility of two recovery channels operating on the same message simultaneously due to parallel checks.
[0018] This invention further designs a message hiding window based on the effective duration of a dynamic lock and a zombie detection forced takeover mechanism. The compensation task actively avoids the window period during which the retry activity may continue, and only intervenes after the expected processing period has been exceeded. In the face of the "zombie" state caused by the abnormal crash of the retry module, the processing right is reclaimed through secondary verification and forced state transfer. While avoiding false takeover, it ensures the continuous recovery capability of messages in extreme cases and realizes the safe coordination between retry and compensation. Attached Figure Description
[0019] Figure 1 This is a flowchart illustrating the first embodiment of a data consistency guarantee method based on best-effort transactions and timed compensation according to the present invention. Detailed Implementation
[0020] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0021] Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0022] Example 1: As Figure 1The diagram shown is a flowchart of the first embodiment of a data consistency guarantee method based on best-effort transactions and timed compensation according to the present invention.
[0023] In the first embodiment, the data consistency guarantee method based on best-effort transactions and timed compensation includes: Step S10: Obtain the message to be sent, and based on the message to be sent, perform the message state initialization task using the local transaction message table persistence mechanism, and output the initial message record; The messages to be sent in this step refer to business messages that need to be asynchronously delivered to the message middleware or downstream services after the business system completes local business processing such as order creation, account modification, or inventory deduction. These messages must at least carry a unique message identifier, business type, and message content that can be jointly identified by the producer and consumer. The local transaction message table persistence mechanism refers to writing the message table within the same database transaction carrying business data changes, allowing business data and message records to be committed or rolled back simultaneously. The message table sets a unique message identifier, message status, status version number, number of retries, preset maximum number of retries, and last status change time. During writing, the message status is set to pending confirmation, the status version number is set to 1, the number of retries is set to 0, and the creation time of the message to be sent is used as the last status change time to obtain the initial message record. This initial message record, after successful submission of the corresponding business data, is determined as the initial message record and serves as the unified input for step S20 to read the message status, status version number, and retry control fields.
[0024] By persisting business data and initialization message records within the same local transaction, two types of inconsistencies—successful business operations with missing message records and business operations rolling back despite existing message records—are constrained by transaction atomicity. The pending confirmation status provides the initial state of the recovery process, the status version number provides the concurrency judgment criteria required for subsequent condition updates, the number of retries and the preset maximum number of retries limit the retry boundaries, and the final status change time provides a time base for timeout filtering. These fields form a traceable state snapshot in the initial message record, allowing step S20 to determine whether to allow the current retry instance to gain processing rights without relying on in-process temporary variables.
[0025] Traditional local message tables typically only record simple states such as pending delivery and successful delivery. Multiple recovery threads may read the same pending delivery record and still initiate delivery separately, making it impossible to undo the duplicate transmissions after subsequent status updates. This step synchronously sets the status version number and retry control field during the message generation phase, ensuring that subsequent recovery channels must simultaneously verify both the message status and version number before modifying the record; once any channel completes its condition update, the old version held by other channels becomes invalid. Therefore, concurrent conflicts can be identified by database condition updates before actual delivery, and no additional cross-node distributed locks are required.
[0026] Taking order creation in the order service as an example, the order record and the "Order Created" message with the unique message identifier MSG20260804001 are written to the database in the same local transaction. The initial status of this message is pending confirmation, the status version number is 1, the number of retries is 0, the preset maximum number of retries is 5, and the final status change time is the message creation time. After the transaction is committed, the initial message record is formed; if the business data writing fails, the message record is rolled back along with the transaction. When the transaction is successfully committed, the business result read in step S20 and the message status come from the same commit, which can be used to directly execute the first processing right competition.
[0027] Step S20: Based on the initial message record, a maximum effort retry mechanism combining status version number condition update and exponential backoff is used to perform the message retransmission task, and the retry result message record is output; The status version number update in this step refers to first reading the pending confirmation status and current status version number of the initial message record, and then using the condition that the message is still in the pending confirmation status and the status version number in the database is still equal to the read version number as the effective condition for the first conditional update statement. Upon successful update, the message status is atomically updated to "Retrying" and the status version number is incremented by 1. The incremented version number is used as the expected version number for the first retry. Instances with zero rows affected by the update do not obtain retry processing rights. After obtaining processing rights, the waiting interval is determined sequentially according to the retry sequence number: each waiting interval is determined by multiplying the preset baseline backoff interval by the corresponding power of the backoff growth factor, plus the current random jitter value, where the backoff growth factor is 2.0, and the random jitter value is sampled by a pseudo-random number generator between 0 and the preset maximum random jitter value. The effective duration of the dynamic lock is determined by adding the maximum value among all planned retry waiting intervals, the preset maximum time consumption for a single message transmission, and the preset redundancy protection duration. After each wait, the message status and version number are reread. Retransmission is only performed if the status is still "Retrying" and the version number equals the expected version number for this retry. If retransmission succeeds, the message status is updated to "Completed" and the version number is incremented by 1. If retransmission fails before the preset maximum number of retries is reached, both the number of retries and the version number are incremented by 1, and the incremented version number is used as the expected version number for the next retry. If the final retransmission also fails, the message status is rolled back to "Pending Confirmation," and both the number of retries and the version number are incremented by 1. If the reread status or version number does not match the expected version number, the corresponding retransmission is stopped. The above branches ultimately form a retry result message record, which is then filtered by step S30 according to status and time sequence.
[0028] The first conditional update statement connects the read state, contention for processing rights, and version advancement into a verifiable state transition. Even if multiple retry instances read the same initial message record simultaneously, only one instance can obtain the retry processing rights. Exponential backoff gradually widens the waiting period after consecutive failures, while random jitter disperses the wake-up times of different messages, reducing the instantaneous load caused by concentrated retransmissions when downstream services are temporarily unavailable. The effective duration of the dynamic lock covers the longest waiting time, the maximum sending time, and the redundancy protection period, setting an interpretable and effective boundary for retry activities and using this boundary as a direct input for determining the message hiding window in step S30.
[0029] Traditional retry processing often involves sending messages cyclically at fixed intervals, updating the message status only after the cycle ends. When multiple instances lack version information, they may all enter the sending phase based on the same old state. This step verifies the message status and the expected version number for this retry before each send, and synchronously advances the number of retries and the version number after each failure, ensuring that instances that have lost processing rights can stop before sending. Furthermore, the waiting interval and the effective duration of the dynamic lock are determined by explicit retry parameters, eliminating the need for compensation tasks to rely on fixed, empirical timeouts unrelated to the retry strategy to determine whether to take over.
[0030] Taking the message MSG20260804002 being read simultaneously by two retry instances as an example, both instances read the pending confirmation status and version number 1. One instance first changes the status to "retrying" and updates the version number to 2 using the first condition update statement. The other instance exits because the version number condition is no longer met. Assuming a preset baseline backoff interval of 5 seconds, a backoff growth factor of 2.0, and a preset maximum random jitter value of 1 second, the first wait is 5 seconds plus 0 to 1 second of random jitter, and the second wait is 10 seconds plus 0 to 1 second of random jitter. If the first retransmission fails, the number of retries changes from 0 to 1, and the version number changes from 2 to 3. Version number 3 becomes the expected version number for the second retransmission. After the second retransmission succeeds, the status is updated to "completed," and the version number is advanced again. If continuous retries continue until the preset maximum number of retries is reached without success, the record is rolled back to "pending confirmation," retaining the advanced version number and the number of retries. Step S30 identifies records that can enter the compensation preprocessing based on this.
[0031] Step S30: Based on the retry result message record, a timed compensation mechanism combining message hiding window and timeout status filtering is used to perform compensation preprocessing tasks and output compensation candidate message records; The message hiding window in this step refers to the time range within which the compensation scan may still be in the effective retry phase while actively avoiding interference. Its duration is determined by adding the effective duration of the dynamic lock obtained in step S20, the preset compensation scan cycle, and the preset delay buffer. When the timed compensation task scans the message table, it generates two types of timeout records from the retry result message records: the first type is a first timeout message record where the message status is pending confirmation and the difference between the current time and the last status change time is greater than the message hiding window duration; the second type is a second timeout message record where the message status is in retry processing and the difference between the current time and the last status change time is greater than the effective duration of the dynamic lock. During scanning, the status version number corresponding to each timeout record is saved, and then its current message status and current status version number are reread. Only records where the reread status is the same as the status during scanning and the reread version number is the same as the version number during scanning are identified as compensation candidate message records and submitted to step S40; records where a state transition has occurred between scanning and review are excluded, and compensation sending is not directly performed in this step.
[0032] The message hiding window links the compensation intervention time with the longest valid activity time of the retry strategy. Records awaiting confirmation only enter the candidate range if the silence time after a retry rollback exceeds this window. For records in retry processing, a dynamic lock validity period is used to identify states that may be delayed due to process exit, node disconnection, or execution interruption, allowing normal retries and abnormal delays to use different timeout benchmarks. After scanning, the status and version number are re-verified, effectively performing a concurrent validity check on the candidate set, preventing records updated by other threads during the scan from continuing to flow into step S40.
[0033] Traditional timed compensation typically filters records based solely on message creation time or a single fixed timeout. This fails to distinguish between messages already rolled back to pending confirmation and those still undergoing retry processing, and also cannot detect scan results that have become invalid before subsequent processing. This step uses a message hidden window and the validity period of a dynamic lock to filter for both states, and performs consistency verification on the reread results using the version number at the time of scanning. Therefore, the compensation preprocessing yields candidate records whose state and timing remain unchanged, rather than directly performing compensation on a single scan snapshot. This reduces the time gap between the retry and compensation channels during the candidate formation stage.
[0034] For example, if the dynamic lock duration for a certain type of message is 150 seconds, the preset compensation scan cycle is 30 seconds, and the preset latency buffer is 10 seconds, then the message hiding window duration is 190 seconds. Message MSG20260804003, after a failed retry, reverted to pending confirmation at 13:02:30. At 13:05:30, when scanned, only 180 seconds had passed since the last state change, so it did not enter the first timeout set. At 13:05:40, when scanned again, more than 190 seconds had passed, so the record entered the first timeout set and its version number (7) was saved. If, before verification, another retry instance had updated the record to "retrying" and changed the version number to 8, and the state and version number no longer matched, the timed compensation task would remove it. Another record in "retrying" that has been stalled for more than 150 seconds can enter the second timeout set, but it must also be scanned and verified before being output as a compensation candidate message record.
[0035] Step S40: Based on the compensation candidate message record, the compensation takeover task is executed using a processing right arbitration mechanism that combines status version number verification, condition update and message status review, and the processing right determination message record is output. The arbitration of processing rights in this step refers to reading the message status, status version number, number of retries, and last status change time of the compensation candidate message record, and dividing it into candidate message records awaiting confirmation and candidate message records in retry processing according to the current message status. For candidate message records awaiting confirmation, the second conditional update statement only atomically updates the message status to "in compensation processing" and increments the status version number by 1 if the current message status is still "awaiting confirmation" and the current status version number is equal to the status version number obtained in step S303. For candidate message records in retry processing, it is also required that the number of retries has reached the preset maximum number of retries, and the difference between the current time and the last status change time is greater than the effective duration of the dynamic lock; the third conditional update statement atomically updates the status to "in compensation processing" and increments the version number by 1, based on the common conditions that the status is still "in retry processing," the version number is still equal to the value verified in step S303, and the number of retries has reached the preset maximum number of retries. Only records that are successfully updated by the second or third condition update statement will be granted compensation processing rights and will be identified as processing rights confirmation message records; records that do not meet the conditions or whose update affects zero rows will not proceed to step S50.
[0036] The two types of candidate records employ branch arbitration, which can simultaneously cover compensation takeover after normal retry rollback and reclaiming processing rights after abnormal exit of the retry thread. Candidate records awaiting confirmation use status and version number as mutual exclusion conditions to prevent new retry instances from simultaneously acquiring processing rights with the compensation task. In addition to the above conditions, candidate records in retry processing are subject to a maximum retry count and a dynamic lock validity period check; takeover is only allowed when all retry opportunities have been exhausted and the valid processing period has expired. The successfully updated status and new version number in the compensation processing constitute the sole entry point for step S50 to execute compensation sending, ensuring continuous ownership proof between candidate selection and actual sending.
[0037] Traditional compensation mechanisms might directly reset all timeout records to pending confirmation or immediate sending, causing slow-running but not yet expired retry tasks to run in parallel with the compensation task. This step does not use timeout as the sole basis for forced takeover; instead, it simultaneously checks the status version number, number of retries, and last status change time for records in the retry process, and uses database condition updates to determine whether processing rights are truly transferred. Even if a candidate record is modified by another thread after being reviewed in step S30, the second or third condition update statement will refuse to update because the preconditions are not met, avoiding compensation based on expired snapshots.
[0038] For example, compensation candidate message record MSG20260804004 is in a pending confirmation state, and its version number is 7 during the review in step S303. The timed compensation task executes the second conditional update statement with the pending confirmation state and version number 7. After the update is successful, the status changes to compensation processing and the version number changes to 8, and this record acquires the right to process compensation. Another record, MSG20260804005, is in retry processing. Although it has been stalled for longer than the effective duration of the dynamic lock, it has already retried 2 times, while the preset maximum number of retry times is 5, so it is not allowed to take over. When this record has later retried 5 times, is still in retry processing, and the stall time continues to exceed the effective duration of the dynamic lock, the third conditional update statement still needs to check the version number obtained from the review in step S303; only when all three conditions are met and the update is successful will the record be changed to compensation processing. When the original retry instance is read again, it will be found that the status or version number has changed, and it no longer has the conditions to continue retransmission.
[0039] Step S50: Based on the processing authority, determine the message record for compensation sending, message status convergence and message consumption deduplication, and output the completed message record.
[0040] The processing right determination message record in this step refers to the record whose message status has been updated to "in compensation processing" and whose new status version number has been obtained in step S40. Before executing compensation sending, its current message status and current status version number are read first; only when the status is still "in compensation processing" is it sent to the message middleware or downstream service according to the message content carried by the record to obtain the compensation sending result. When sending is successful, the fourth condition update statement uses the condition that the message status is still "in compensation processing" and the status version number is still equal to the value read before sending to atomically update the status to "completed" and increment the version number by 1, forming a status convergence message record. When sending fails, the fifth condition update statement uses the same status and version conditions to roll back the status from "in compensation processing" to "pending confirmation", increment the version number by 1, and update the last status change time to the current time so that it can be re-filtered through the hidden window later. After the consumer receives the message corresponding to the status convergence message record, it first writes the message's unique identifier into the deduplication table with a unique index; when writing is successful, it performs one business consumption and forms a completed message record; when writing fails because the same identifier already exists, it stops duplicate consumption.
[0041] Before compensation is sent, the status during compensation processing is reviewed to ensure that only the executor still holding compensation processing rights can enter the sending phase. The fourth and fifth condition update statements after sending continue to use the version number read before sending, preventing older executors whose processing rights have changed during sending from overwriting the new status. If sending fails, updating the last status change time allows the record to re-enter the waiting process controlled by the message hiding window, preventing it from being continuously preempted in the next scan. The state convergence at the sending end and the unique index deduplication at the consumer end respectively constrain the delivery process and the business activation process, both jointly ensuring that the unique message identifier formed in step S10 is used throughout the final consumption result.
[0042] Traditional solutions, if they directly write the completion status after successful transmission without verifying the version number before transmission, may allow other recovery threads to overwrite the final state using an old snapshot. Similarly, querying before insertion may result in duplicate business logic due to concurrency gaps between queries and writes. This step incorporates both successful and failed branches after compensation transmission into the status version number condition update and uses a unique database index to reject writes with duplicate identifiers as the basis for atomic deduplication. This ensures that updates are rejected when the state transition does not match the pre-transmission snapshot, and that duplicate messages arriving at the consumer will not trigger further business processing, preventing occasional retransmissions at the sender from being transformed into duplicate data changes downstream.
[0043] Taking message MSG20260804006 as an example, step S40 updates its status to "compensation processing in progress" and advances the version number to 8. Step S50 executes compensation sending after reading the status and version number. If the sending is successful, the fourth conditional update statement updates the record to "completed" and version number 9 based on the "compensation processing in progress" status and version number 8. After the consumer successfully writes the unique identifier of the message for the first time, it executes business processing. When the message middleware delivers the same message again, the unique index of the deduplication table rejects duplicate writing, and the consumer stops the second business processing accordingly. If the compensation sending fails, the fifth conditional update statement rolls back the status to "pending confirmation," advances the version number, and refreshes the last status change time. The record must meet the message hidden window and review conditions of step S30 again before it can re-enter the compensation process.
[0044] Example 2: Furthermore, the present invention provides a data consistency guarantee system based on best-effort transactions and timed compensation, employing a data consistency guarantee method based on best-effort transactions and timed compensation from the above embodiments, which can solve the technical problem of data consistency guarantee based on best-effort transactions and timed compensation. The beneficial effects of the data consistency guarantee system based on best-effort transactions and timed compensation provided by the present invention are the same as those of the data consistency guarantee method based on best-effort transactions and timed compensation provided in the above embodiments, and other technical features of the data consistency guarantee system based on best-effort transactions and timed compensation are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.
[0045] Example 3: This invention provides a data consistency assurance device based on best-effort transactions and timing compensation. The device includes at least one processor and a memory communicatively connected to the processor. The memory stores instructions executable by the processor, which are then executed to enable the processor to perform the data consistency assurance method based on best-effort transactions and timing compensation described in Example 1. This data consistency assurance device based on best-effort transactions and timing compensation can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. This data consistency assurance device based on best-effort transactions and timing compensation is merely an example and should not limit the functionality or scope of the invention. A data consistency assurance device based on best-effort transactions and timing compensation may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in read-only memory or a program loaded from a storage device into random access memory. The random access memory also stores various programs and data required for the operation of the data consistency assurance device based on best-effort transactions and timing compensation. The processing unit, read-only memory, and random access memory are interconnected via a bus. An I / O interface is also connected to the bus. Typically, the following systems can be connected to the I / O interface: input devices including touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices including liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices including magnetic tapes, hard disks, etc.; and communication devices. The communication device allows the data consistency assurance device based on best-effort transactions and timing compensation to communicate wirelessly or wiredly with other devices to exchange data. While a data consistency assurance device based on best-effort transactions and timing compensation with various systems has been described, it should be understood that implementation or possession of all described systems is not required. It can be implemented alternatively or with more or fewer systems.
[0046] Example 4: This invention also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the data consistency guarantee method based on best-effort transactions and timed compensation as described above. The computer program product provided by this invention can solve the technical problem of data consistency guarantee based on best-effort transactions and timed compensation. Compared with the prior art, the beneficial effects of the computer program product provided by this invention are the same as the beneficial effects of the data consistency guarantee method based on best-effort transactions and timed compensation provided in the above embodiments, and will not be repeated here.
[0047] In particular, according to the embodiments disclosed in this invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this invention include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from a storage device, or installed from a read-only memory. When the computer program is executed by a processing device, it performs the functions defined in the methods of the embodiments disclosed in this invention.
[0048] It should be understood that the various parts disclosed in this invention can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics may be combined in any suitable manner in one or more embodiments or examples.
[0049] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the present invention and its equivalents, the present invention also intends to include these modifications and variations.
Claims
1. A data consistency guarantee method based on best-effort transactions and timed compensation, characterized in that, The method includes: Step S10: Obtain the message to be sent, and based on the message to be sent, perform the message state initialization task using the local transaction message table persistence mechanism, and output the initial message record; Step S20: Based on the initial message record, a maximum effort retry mechanism combining status version number condition update and exponential backoff is used to perform the message retransmission task, and the retry result message record is output; Step S30: Based on the retry result message record, a timed compensation mechanism combining message hiding window and timeout status filtering is used to perform compensation preprocessing tasks and output compensation candidate message records; Step S40: Based on the compensation candidate message record, the compensation takeover task is executed using a processing right arbitration mechanism that combines status version number verification, condition update and message status review, and the processing right determination message record is output. Step S50: Based on the processing authority, determine the message record for compensation sending, message status convergence and message consumption deduplication, and output the completed message record.
2. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 1, characterized in that, Step S10, which involves obtaining the message to be sent, performing a message state initialization task based on the message to be sent using a local transaction message table persistence mechanism, and outputting the initial message record, specifically includes: Step S101: In the local business transaction, the message to be sent is written into the message table in the database. The message table is set with a unique message identifier, message status, status version number, number of retries, preset maximum number of retries, and last status change time. Step S102: Set the message status corresponding to the message to be sent to pending confirmation, set the status version number to 1, set the number of retries to 0, and set the last status change time to the creation time of the message to be sent, to obtain the initial message record; Step S103: Submit the initialization message record and the corresponding business data together in the business local transaction, and determine the initialization message record that has been submitted as the initial message record.
3. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 2, characterized in that, Step S20, which involves executing a message retransmission task based on the initial message record using a maximum effort retry mechanism combining state version number conditional update and exponential backoff, and outputting the retry result message record, specifically includes: Step S201: Read the message status and status version number of the initial message record, and update the message status from pending confirmation to retry processing through a first conditional update statement; wherein, the first conditional update statement takes the read message status as pending confirmation and the read status version number as the same as the current status version number in the message table as the update condition, and increments the status version number by 1 when the update is successful, thereby obtaining the retry processing right of the initial message record, and determines the incremented status version number as the expected version number for the first retry; Step S202: After obtaining the retry processing right, determine the first... The waiting interval corresponding to each retry is calculated using the first formula: The effective duration of the dynamic lock is determined based on the waiting interval corresponding to each retry, and the effective duration of the dynamic lock is calculated using the second formula: in, Indicates the first The waiting interval corresponding to each retry; Indicates the retry sequence number, and ; This indicates the preset maximum number of retries; Indicates the preset baseline backoff interval; This indicates a retreat from growth factors; Indicates the first The random jitter value corresponding to each retry; This indicates the effective duration of the dynamic lock; Indicates the preset maximum time for sending a single message; Indicates the preset redundancy protection duration; Step S203: In the first After the waiting interval corresponding to the retry is completed, the current message status and current status version number of the initial message record are read again; when the current message status is "retrying" and the current status version number is the same as the expected version number for this retry, the next step is executed. The message was resent; when the first message was resent. When the message is successfully resent, the message status is updated to "completed" using the status version number condition update, and the status version number is incremented by 1; when the message is resent successfully... The message resending failed and When the status version number is updated, the number of retries is incremented by 1 and the status version number is incremented by 1, and the incremented status version number is determined as the expected version number for the next retry; when the 1st The message resending failed and When the message status is updated from "Retrying" to "Pending Confirmation" by updating the status version number condition, the number of retries is incremented by 1 and the status version number is incremented by 1 to obtain the retry result message record; when the current message status is not "Retrying" or the current status version number is different from the expected version number for this retry, the corresponding message retransmission is terminated, and the reread message record is determined as the retry result message record.
4. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 3, characterized in that, In step S202, the retreat growth factor The value is 2.0; the random jitter value The pseudo-random number generator generates random numbers from 0 to a preset maximum random jitter value. The samples were obtained between the two points, where, This represents the preset maximum random jitter value.
5. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 3, characterized in that, Step S30, which involves performing a compensation preprocessing task based on the retry result message record using a timed compensation mechanism combining message hiding window and timeout status filtering, and outputting compensation candidate message records, specifically includes: Step S301: Determine the message hiding window duration based on the effective duration of the dynamic lock, the preset compensation scan cycle, and the preset hysteresis buffer amount. The message hiding window duration is calculated using a third formula: in, This indicates the duration of the message hiding window; This indicates the preset compensation scan period; This indicates the preset delay buffer amount; Step S302: Scan the message table and filter the first timeout message record and the second timeout message record from the retry result message records; wherein, the message status of the first timeout message record is pending confirmation, and the difference between the current time and its last status change time is greater than the message hiding window duration; the message status of the second timeout message record is retrying, and the difference between the current time and its last status change time is greater than the dynamic lock validity period; record the status version number corresponding to the first timeout message record and the second timeout message record during the scan; Step S303: Reread the current message status and current status version number of the first timeout message record and the second timeout message record respectively, and determine the message record whose current message status is the same as the message status during scanning and whose current status version number is the same as the status version number during scanning as the compensation candidate message record.
6. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 5, characterized in that, In step S40, the compensation takeover task is executed based on the compensation candidate message record, using a processing right arbitration mechanism that combines status version number verification, condition update, and message status review. The step of outputting the processing right determination message record specifically includes: Step S401: Read the message status, status version number, number of retries and last status change time of the compensation candidate message record, and divide the compensation candidate message record into candidate message records to be confirmed and candidate message records in retry processing according to the message status; Step S402: For the candidate message record to be confirmed, update its message status from pending confirmation to compensation processing through the second condition update statement; wherein, the second condition update statement takes the current message status as pending confirmation and the current status version number as the same as the status version number reread in step S303 as the update condition, and increments the status version number by 1 when the update is successful to obtain the corresponding compensation processing right; Step S403: For the candidate message record in the retry processing, when the number of retries reaches the preset maximum number of retries, and the difference between the current time and its last state change time is greater than the effective duration of the dynamic lock, the message status is atomically updated from "in retry processing" to "in compensation processing" through a third condition update statement; wherein, the third condition update statement uses the current message status as "in retry processing", the current status version number being the same as the status version number reread in step S303, and the number of retries reaching the preset maximum number of retries as update conditions, and increments the status version number by 1 when the update is successful to obtain the corresponding compensation processing right; the message record that obtains the compensation processing right through the second condition update statement or the third condition update statement is determined as the processing right determination message record.
7. The data consistency guarantee method based on best-effort transactions and timed compensation as described in claim 6, characterized in that, In step S50, the steps of determining the message record for compensation sending, message state convergence, and message consumption deduplication based on the processing right, and outputting the completed message record, specifically include: Step S501: Read the current message status and current status version number of the processing right determination message record. When the current message status is in compensation processing, perform compensation sending according to the message content carried by the processing right determination message record to obtain the compensation sending result. Step S502: When the compensation sending result is successful, the message status of the processing right determination message record is updated from "in the compensation processing" to "completed" using the fourth condition update statement, and the status version number is incremented by 1 to obtain the status convergence message record; when the compensation sending result is failed, the message status is updated from "in the compensation processing" to "pending confirmation" using the fifth condition update statement, the status version number is incremented by 1, and the last status change time is updated to the current time; wherein, both the fourth and fifth condition update statements use the message status "in the compensation processing" and the status version number being the same as the current status version number read in step S501 as the update condition; Step S503: When the message consumer receives the message corresponding to the state convergence message record, the message unique identifier is written into the deduplication table with a unique index for the message unique identifier; when the message unique identifier is successfully written, message consumption is performed, and the state convergence message record that has completed message consumption is determined as the completed message record; when the message unique identifier fails to be written due to the existence of the same message unique identifier in the unique index, the corresponding duplicate message consumption is stopped.
8. A data consistency guarantee system based on best-effort transactions and timed compensation, applied to the data consistency guarantee method based on best-effort transactions and timed compensation as described in any one of claims 1 to 7, characterized in that, The data consistency guarantee system includes: The persistence initialization module is used to obtain the message to be sent, perform message state initialization tasks based on the message to be sent using the local transaction message table persistence mechanism, and output the initial message record; The maximum effort retry module is used to perform message retransmission tasks based on the initial message record using a maximum effort retry mechanism that combines status version number conditional update and exponential backoff, and outputs the retry result message record. The timed compensation preprocessing module is used to perform compensation preprocessing tasks based on the retry result message records using a timed compensation mechanism that combines message hiding window and timeout status filtering, and output compensation candidate message records. The processing right arbitration module is used to execute the compensation takeover task based on the processing right arbitration mechanism that combines status version number verification, condition update and message status review based on the compensation candidate message record, and output the processing right determination message record; The consistency confirmation module is used to determine the message record for compensation sending, message status convergence and message consumption deduplication based on the processing right, and output the completed message record.
9. A data consistency guarantee device based on best-effort transactions and timed compensation, characterized in that, The data consistency assurance device based on best-effort transactions and timed compensation includes: a memory, a processor, and a data consistency assurance program based on best-effort transactions and timed compensation stored in the memory and executable on the processor. When the data consistency assurance program based on best-effort transactions and timed compensation is executed by the processor, it implements a data consistency assurance method based on best-effort transactions and timed compensation as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes a data consistency guarantee program based on best-effort transactions and timed compensation. When the data consistency guarantee program based on best-effort transactions and timed compensation is executed by the processor, it implements a data consistency guarantee method based on best-effort transactions and timed compensation as described in any one of claims 1 to 7.
