Distributed cache consistency control method and system and computer readable medium
By establishing message topic subscription relationships and globally unique message identifier verification on the client side, combined with version number comparison and exponential backoff strategy, the problems of data inconsistency and cross-system monitoring in distributed cache consistency control are solved, and efficient consistency and reliability of distributed systems are achieved.
Patent Information
- Application Number
- CN202511820736.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-05
- Publication Date
- 2026-03-03
AI Technical Summary
Existing distributed local cache consistency control technologies are prone to data inconsistency with cache state in multi-instance environments, and traditional solutions are difficult to support unified monitoring and operation and maintenance across domains and systems, resulting in high migration costs and complex adaptation.
By registering business cache units on the client side, establishing message topic subscription relationships, recording data change contexts and constructing notification messages, performing idempotent verification of globally unique message identifiers and version number comparisons, and combining the automatic retry mechanism of exponential backoff strategy, the atomic alignment and eventual consistency of cache refresh and database commit are ensured.
It achieves global consistency convergence between cross-instance local cache and backend database, improves the system's transaction atomicity and data reliability, provides fine-grained and controllable monitoring and traceability capabilities, and avoids cache data rollback and overwriting issues.
Smart Images
Figure CN121597700A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of caching technology, and in particular to a distributed cache consistency control method, system, and computer-readable medium. Background Technology
[0002] With the widespread adoption of distributed systems and microservice architectures, more and more enterprise applications are choosing to achieve service elasticity and high availability through multi-instance deployment. In a multi-instance environment, application processes typically maintain local caches of frequently accessed data to reduce access latency, alleviate backend pressure, and improve overall system response speed. However, existing distributed local cache consistency control technologies have revealed a series of shortcomings during actual deployment and maintenance.
[0003] Traditional cache refresh notification mechanisms mostly rely on real-time message sending in business code, or directly trigger cache refresh under conditions lacking strict transaction constraints. This can easily lead to premature cache refresh or dirty data writing before business operations have been committed. If the database transaction fails to commit successfully, but the cache has already been updated asynchronously, inconsistencies between the data and the cache state will occur, severely impacting data accuracy and system reliability. Furthermore, existing solutions also have shortcomings in message notification and idempotent processing; duplicate notifications and out-of-order processing can easily cause erroneous cache refreshes and data overwriting.
[0004] As enterprise infrastructure continues to evolve, different teams and business lines may adopt different message middleware and caching engines, resulting in strong coupling between the notification chain and business logic, high migration costs, and complex adaptation. Traditional implementation methods struggle to support unified message topic naming and model abstraction, lack pluggable and centralized operation and maintenance methods, and are also difficult to efficiently achieve cross-domain and cross-system monitoring and remote management. Summary of the Invention
[0005] One objective of this invention is to propose a distributed cache consistency control method, system, and computer-readable medium. This invention achieves global consistency convergence between cross-instance local cache and backend database, improving the transaction atomicity and data reliability of the system.
[0006] A distributed cache consistency control method according to an embodiment of the present invention includes:
[0007] S1. Register multiple business cache units on the client side and establish a message topic subscription relationship for each business cache unit;
[0008] S2. When a business operation causes a change in the data associated with the business cache unit in the database, record the data change context. After the database transaction in which the business operation is located is successfully committed, construct a notification message and publish the notification message to the message topic corresponding to the business cache unit.
[0009] S3. After receiving the notification message, the client-side subscriber routes it to the target business cache unit based on the business domain identifier and model identifier. It performs idempotency verification and deduplication on the globally unique message identifier. If the message is not processed, it continues to execute the subsequent steps.
[0010] S4. Read the local version number in the target business cache unit and compare it with the version number to be released in the notification message. Only if the version number to be released is greater than the local version number will the refresh process continue. Otherwise, discard the notification message and record the version backtracking information.
[0011] S5. When the refresh is interrupted, perform exponential backoff retry and retain the globally unique message identifier; if the retry limit is exceeded, back off and report the failure status to the central side.
[0012] S6. The central side periodically receives the status information of the service cache unit reported by the client side;
[0013] S7. After receiving the remote control command, the client side verifies the legality and validity of the command, executes the corresponding operation, and sends the operation result and the latest local version number back to the central side after the execution is completed, thus completing the closed loop of distributed local cache consistency control.
[0014] Optionally, S1 further includes:
[0015] Establish a mapping relationship between business domain identifiers, model identifiers, and business cache units; initialize the lifecycle management capabilities of each business cache unit, including enabling, disabling, incremental refresh by key, full rebuild, and local version number maintenance.
[0016] The message topic consists of a unified prefix, a business domain identifier, and a model identifier. It maintains a continuous connection channel with the pluggable message middleware and persists the correspondence between message topics and business cache units locally.
[0017] Optionally, S3 further includes:
[0018] S31. After receiving the notification message, the client-side subscriber extracts the business domain identifier, model identifier, globally unique message identifier, and refresh type from the notification message. Based on the business domain identifier and model identifier, it routes to the target business cache unit and performs idempotency verification processing within the target business cache unit.
[0019] S32. Establish an idempotent processing tag set within the target business cache unit;
[0020] S33. If the globally unique message identifier of the current notification message already exists in the idempotent processing tag set, then the current notification message is determined to be a duplicate message, is immediately discarded, and the number of hits is recorded;
[0021] If the globally unique message identifier of the current notification message does not exist in the idempotent processing flag set, the refresh process continues and the globally unique message identifier of the current notification message is written into the idempotent processing flag set.
[0022] S34. Using the received timestamp when receiving the notification message and the sending timestamp contained in the notification message for joint verification, determine whether the difference between the received timestamp and the sending timestamp of the current notification message exceeds the maximum allowed message propagation delay. If it exceeds the maximum allowed message propagation delay, the corresponding notification message is determined to be a timeout invalid message. The timeout invalid message is no longer processed and is recorded as a delayed message, thus completing the time consistency processing of the idempotent processing mark set.
[0023] Optionally, S4 further includes:
[0024] S41. Compare the version number to be released with the local version number. If the version number to be released is less than or equal to the local version number, the version of the current notification message is deemed invalid, considered as a duplicate or outdated refresh request, the notification message is immediately discarded, and the version backtracking event of this notification message is recorded. At the same time, a version backtracking identifier is generated for the current notification message as a sign that the current notification message has a version conflict. The globally unique message identifier of the notification message is used to uniquely identify the notification message.
[0025] S42. If and only if the version number to be released is greater than the local version number, the notification message shall be regarded as a valid version update instruction, and the refresh process shall continue. After the refresh process is completed, the local version number of the current business cache unit shall be updated to the version number to be released.
[0026] S43. When the local version number fails to be updated correctly due to abnormal breakpoints or network interruptions during the refresh process of the client instance, it automatically restores to the refresh state of the last successful completion and continues to verify the version of newly received notification messages based on the largest version number that has been successfully completed in the current business cache unit.
[0027] Optionally, the refresh process includes:
[0028] When the refresh type is button incremental refresh, the target data is obtained from the data source according to the data identifier, the corresponding data item of the target business cache unit is updated, and if the target data does not exist, the deletion or invalidation operation is performed.
[0029] When the refresh type is full refresh, rebuild all the contents of the target business cache unit; after the refresh is completed, update the local version number to the version number to be released.
[0030] Optionally, S5 further includes:
[0031] S51. When the refresh process of the target service cache unit is interrupted due to network abnormality, execution failure or unavailability of dependent resources, record the globally unique message identifier of the current notification message and the failure context that caused the interruption, and start the automatic retry mechanism based on the exponential backoff strategy.
[0032] S52. Whenever the automatic retry mechanism initiates a retry, the current notification message is idempotent based on the globally unique message identifier;
[0033] S53. The automatic retry mechanism continues to execute until the number of retries reaches the preset retry threshold. When the current number of retries is greater than or equal to the retry threshold, the automatic retry mechanism stops executing, the current notification message is regarded as a refresh failure message, and the rollback process is initiated.
[0034] S54. When executing the rollback process, record the corresponding failure context and audit information based on the globally unique message identifier;
[0035] S55. After the rollback process is completed, the refresh failure status will be reported as failure status reporting information to the central side.
[0036] Optionally, S7 further includes:
[0037] S71. After receiving the remote control command issued by the central side, the client side extracts the command type, business domain identifier, model identifier, command validity period timestamp, and globally unique message identifier from the remote control command, and routes the remote control command to the corresponding target business cache unit based on the business domain identifier and model identifier.
[0038] S72. Read the current client local timestamp from the target business cache unit and perform a validity check on the instruction validity period timestamp:
[0039] If the current local timestamp of the client is less than or equal to the timestamp of the instruction's validity period, the remote control instruction will be considered a valid instruction and will continue to be executed.
[0040] If the current local timestamp of the client is greater than the expiration timestamp of the instruction, the remote control instruction is judged as a timeout instruction and discarded, and the instruction timeout event is recorded at the same time.
[0041] S73. Based on the instruction type, execute the corresponding control operation within the target service cache unit;
[0042] S74. After executing the control operation corresponding to the instruction type, generate a control operation result identifier, read the updated local version number of the target business cache unit, and combine the operation result identifier, the instruction globally unique message identifier, the latest local version number of the business cache unit, and the current client instance identifier into a receipt information;
[0043] S75. The receipt information is reported to the central side, so that the central side can know the execution result of the remote control command, the latest local version number of the target business cache unit, and the execution status of the client instance based on the receipt information, thus completing the closed loop of distributed local cache consistency control.
[0044] Optionally, performing the corresponding control operation within the target service cache unit includes:
[0045] When the instruction type is an enable operation instruction, the operation of enabling the business cache unit is executed;
[0046] When the instruction type is a close operation instruction, the operation of closing the business cache unit is performed;
[0047] When the instruction type is a full refresh instruction, the entire contents of the target business cache unit are rebuilt, and the local version number of the target business cache unit is updated after the reconstruction is completed, so that the local version number is consistent with the latest version.
[0048] Optional, including:
[0049] The business cache unit registration and subscription module is used to register multiple business cache units on the client side and establish a corresponding message topic subscription relationship for each business cache unit;
[0050] The data change context recording and notification construction module is used to record the corresponding data change context when a change is detected in the data associated with the business cache unit in the database. After the database transaction of the relevant business operation is successfully committed, a notification message is constructed and published to the message topic corresponding to the target business cache unit.
[0051] The notification receiving and idempotency verification module is used to route the notification message to the target business cache unit based on the business domain identifier and model identifier in the notification message after receiving the notification message on the client side, and to perform idempotency verification and deduplication on the globally unique message identifier of the notification message. The subsequent refresh process will only continue if it is determined that the message has not been processed.
[0052] The local version number comparison and refresh control module is used to read the local version number in the target business cache unit and compare it with the version number to be released carried in the notification message.
[0053] The abnormal retry and failure reporting module is used to automatically retries according to the exponential backoff strategy when the cache refresh process is interrupted due to network abnormalities, execution failures, or unavailability of dependent resources.
[0054] The status reporting module is used by the central side to periodically receive status information of each business cache unit reported by the client side.
[0055] The remote control command processing and acknowledgment module is used to verify the legality and validity of remote control commands sent by the central side after the client side receives them, and to execute the corresponding control operation in the target business cache unit according to the command type.
[0056] Optionally, when the program is executed by the processor, it implements a distributed cache consistency control method as described in any one of claims 1 to 8.
[0057] The beneficial effects of this invention are:
[0058] (1) On the client side, the present invention extracts the globally unique message identifier and the version number to be published for each notification message and compares it with the current local version number of the business cache unit. The refresh process is only allowed when the version number to be published is greater than the local version number. The uniqueness of the notification message is determined by establishing an idempotent processing mark set, ensuring that the same notification message is processed at most once. Combined with the monotonically increasing attribute of the version number, the problem of cache data rollback, overwriting and multiple updates caused by message out-of-order, network delay or repeated push is effectively avoided. Thus, the eventual consistency and data accuracy of the distributed local cache are guaranteed at the algorithm level.
[0059] (2) This invention proposes to strictly postpone the cache refresh notification until after the database transaction is successfully committed. It constructs a standardized notification message containing business domain identifier, model identifier, refresh type, data identifier, version number and globally unique message identifier, and publishes it to a pluggable message channel. Each client instance ensures that the cache refresh and database commit are atomically aligned in time through unified topic subscription and model routing. This effectively avoids the race condition problem in the traditional architecture where refresh is triggered by uncommitted data or the cache has changed after the commit fails. It achieves global consistency convergence between cross-instance local cache and backend database, and improves the transaction atomicity and data reliability of the system.
[0060] (3) In the process of cache refresh, this invention designs an automatic retry mechanism based on exponential backoff strategy to address refresh interruptions caused by network anomalies, unavailability of dependent resources, or execution failures. Each retry retains a globally unique message identifier to achieve idempotency control. Automatic retry continues until a preset threshold is reached, after which a rollback process is initiated. Complete failure context and audit information are recorded. All refresh failure statuses and operation receipts are uniformly summarized through the central side reporting and receipt mechanism, realizing comprehensive monitoring and high availability management of cache consistency, health, version continuity, and client execution status. This provides fine and controllable monitoring and traceability capabilities for the operation and maintenance of large-scale distributed systems. Attached Figure Description
[0061] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:
[0062] Figure 1 This is a flowchart of a distributed cache consistency control method, system, and computer-readable medium proposed in this invention. Detailed Implementation
[0063] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.
[0064] refer to Figure 1 As shown in Example 1: A distributed cache consistency control method, comprising:
[0065] S1. Register multiple business cache units on the client side and establish a message topic subscription relationship for each business cache unit;
[0066] In this embodiment, S1 further includes:
[0067] Establish a mapping relationship between business domain identifiers, model identifiers, and business cache units; initialize the lifecycle management capabilities of each business cache unit, including enabling, disabling, incremental refresh by key, full rebuild, and local version number maintenance.
[0068] The message topic consists of a unified prefix, a business domain identifier, and a model identifier. It maintains a continuous connection channel with the pluggable message middleware and persists the correspondence between message topics and business cache units locally.
[0069] S2. When a business operation causes a change in the data associated with the business cache unit in the database, record the data change context. After the database transaction in which the business operation is located is successfully committed, construct a notification message and publish the notification message to the message topic corresponding to the business cache unit.
[0070] S3. After receiving the notification message, the client-side subscriber routes it to the target business cache unit based on the business domain identifier and model identifier. It performs idempotency verification and deduplication on the globally unique message identifier. If the message is not processed, it continues to execute the subsequent steps.
[0071] In this embodiment, S3 further includes:
[0072] S31. After receiving the notification message, the client-side subscriber extracts the business domain identifier, model identifier, globally unique message identifier, and refresh type from the notification message. Based on the business domain identifier and model identifier, it routes to the target business cache unit and performs idempotency verification processing within the target business cache unit.
[0073] Idempotent verification refers to the determination of globally unique message identifiers to ensure that the same globally unique message identifier is processed at most once within the target business cache unit.
[0074] S32. Establish an idempotent processing tag set within the target business cache unit;
[0075] The idempotent processing flag set is used to record globally unique message identifiers that have been successfully processed. Each item in the idempotent processing flag set is a globally unique message identifier. The existence of the idempotent processing flag set is used to determine whether a notification message is a processed message.
[0076] S33. If the globally unique message identifier of the current notification message already exists in the idempotent processing tag set, then the current notification message is determined to be a duplicate message, is immediately discarded, and the number of hits is recorded;
[0077] If the globally unique message identifier of the current notification message does not exist in the idempotent processing flag set, the refresh process continues and the globally unique message identifier of the current notification message is written into the idempotent processing flag set.
[0078] For each business cache unit in the set of business cache units currently managed by the client node, an independent set of idempotent processing tags is maintained. The size of the idempotent processing tag set must not exceed the preset maximum capacity threshold of idempotent tags. After exceeding the maximum capacity threshold of idempotent tags, the oldest globally unique message identifier in the idempotent processing tag set is automatically replaced by the least recently used strategy, so that the capacity of the idempotent processing tag set remains constant within the preset threshold.
[0079] S34. Using the received timestamp when receiving the notification message and the sending timestamp contained in the notification message for joint verification, determine whether the difference between the received timestamp and the sending timestamp of the current notification message exceeds the maximum allowed message propagation delay. If it exceeds the maximum allowed message propagation delay, the corresponding notification message is determined to be a timeout invalid message. The timeout invalid message is no longer processed and is recorded as a delayed message, thus completing the time consistency processing of the idempotent processing mark set.
[0080] The maximum allowed message propagation latency is set by the client's business caching unit.
[0081] S4. Read the local version number in the target business cache unit and compare it with the version number to be released in the notification message. Only if the version number to be released is greater than the local version number will the refresh process continue. Otherwise, discard the notification message and record the version backtracking information.
[0082] In this embodiment, S4 further includes:
[0083] S41. Compare the version number to be released with the local version number. If the version number to be released is less than or equal to the local version number, the version of the current notification message is deemed invalid, considered a duplicate or outdated refresh request, the notification message is immediately discarded, and the version backtracking event of this notification message is recorded. At the same time, a version backtracking identifier is generated for the current notification message as a sign that a version conflict has occurred. The globally unique message identifier of the notification message is used to uniquely identify the notification message.
[0084] In Example 1, after receiving the notification message and completing the idempotency check on the client side, the version number to be released in the notification message is extracted, and the version number to be released is used as the version basis for this refresh request. The local version number currently maintained by the business cache unit is read from the target business cache unit and used as the local version basis for the current business cache unit.
[0085] S42. If and only if the version number to be released is greater than the local version number, the notification message shall be regarded as a valid version update instruction, and the refresh process shall continue. After the refresh process is completed, the local version number of the current business cache unit shall be updated to the version number to be released.
[0086] During arbitrary version number comparisons, the version number to be released and the local version number should be of the same data type, and both should be represented by a monotonically increasing timestamp type or an integer field. At the same time, it should be ensured that the two meet the same version number unit to prevent version judgment errors due to inconsistent version number units.
[0087] In this embodiment, the refresh process includes:
[0088] When the refresh type is button incremental refresh, the target data is obtained from the data source according to the data identifier, the corresponding data item of the target business cache unit is updated, and if the target data does not exist, the deletion or invalidation operation is performed.
[0089] When the refresh type is full refresh, rebuild all the contents of the target business cache unit; after the refresh is completed, update the local version number to the version number to be released.
[0090] S43. When the local version number fails to be updated correctly due to abnormal breakpoints or network interruptions during the refresh process of the client instance, it automatically restores to the refresh state of the last successful completion and continues to verify the version of newly received notification messages based on the largest version number that has been successfully completed in the current business cache unit.
[0091] The comparison is based on the largest version number to avoid state disorder caused by version update failure and to ensure the version continuity and consistency of business cache units.
[0092] S5. When the refresh is interrupted, perform exponential backoff retry and retain the globally unique message identifier; if the retry limit is exceeded, back off and report the failure status to the central side.
[0093] In this embodiment, S5 further includes:
[0094] S51. When the refresh process of the target service cache unit is interrupted due to network abnormality, execution failure or unavailability of dependent resources, record the globally unique message identifier of the current notification message and the failure context that caused the interruption, and start the automatic retry mechanism based on the exponential backoff strategy.
[0095] The automatic retry mechanism of the exponential backoff strategy refers to the waiting time before each retry being the base waiting time multiplied by two raised to the power of the current number of retry attempts. The base waiting time is the time constant preset by the automatic retry mechanism, and the current number of retry attempts is the number of retry attempts already attempted. The waiting time is used to control the delay interval before each automatic retry. A globally unique message identifier is continuously retained throughout the entire automatic retry process to ensure the idempotency of the automatic retry process.
[0096] S52. Whenever the automatic retry mechanism initiates a retry, the current notification message is idempotent based on the globally unique message identifier;
[0097] If it is determined that the current notification message has not yet completed the refresh process, the refresh process will be re-executed; if it is determined that the current notification message has completed the refresh process, subsequent retries will be stopped immediately, and the completion status of the refresh process will be recorded to avoid repeated execution of the refresh process.
[0098] S53. The automatic retry mechanism continues to execute until the number of retries reaches the preset retry threshold. When the current number of retries is greater than or equal to the retry threshold, the automatic retry mechanism stops executing, the current notification message is regarded as a refresh failure message, and the rollback process is initiated.
[0099] S54. When executing the rollback process, record the corresponding failure context and audit information based on the globally unique message identifier;
[0100] Audit information is used to identify the reason for the failure of the notification message, the timestamp of the failure, and the number of retries at the time of failure, so as to ensure that the processing link of the notification message can be traced in fault analysis or consistency verification.
[0101] S55. After the rollback process is completed, the refresh failure status will be reported as failure status reporting information to the central side.
[0102] The failure status reporting information includes a globally unique message identifier, failure context, audit information, and the current client instance identifier, which is used to enable the central side to obtain a complete failure record of this refresh process.
[0103] S6. The central side periodically receives the status information of the service cache unit reported by the client side;
[0104] S7. After receiving the remote control command, the client side verifies the legality and validity of the command, executes the corresponding operation, and sends the operation result and the latest local version number back to the central side after the execution is completed, thus completing the closed loop of distributed local cache consistency control.
[0105] In this embodiment, S7 further includes:
[0106] S71. After receiving the remote control command issued by the central side, the client side extracts the command type, business domain identifier, model identifier, command validity period timestamp, and globally unique message identifier from the remote control command, and routes the remote control command to the corresponding target business cache unit based on the business domain identifier and model identifier.
[0107] S72. Read the current client local timestamp from the target business cache unit and perform a validity check on the instruction validity period timestamp:
[0108] If the current local timestamp of the client is less than or equal to the timestamp of the instruction's validity period, the remote control instruction will be considered a valid instruction and will continue to be executed.
[0109] If the current local timestamp of the client is greater than the expiration timestamp of the instruction, the remote control instruction is judged as a timeout instruction and discarded, and the instruction timeout event is recorded at the same time.
[0110] S73. Based on the instruction type, execute the corresponding control operation within the target service cache unit;
[0111] In this embodiment, corresponding control operations are performed within the target service cache unit, including:
[0112] When the instruction type is an enable operation instruction, the operation of enabling the business cache unit is executed;
[0113] When the instruction type is a close operation instruction, the operation of closing the business cache unit is performed;
[0114] When the instruction type is a full refresh instruction, the entire contents of the target business cache unit are rebuilt, and the local version number of the target business cache unit is updated after the reconstruction is completed, so that the local version number is consistent with the latest version.
[0115] S74. After executing the control operation corresponding to the instruction type, generate a control operation result identifier, read the updated local version number of the target business cache unit, and combine the operation result identifier, the instruction globally unique message identifier, the latest local version number of the business cache unit, and the current client instance identifier into a receipt information;
[0116] The control operation result identifier is used to record the final state of the remote control command execution.
[0117] S75. The receipt information is reported to the central side, so that the central side can know the execution result of the remote control command, the latest local version number of the target business cache unit, and the execution status of the client instance based on the receipt information, thus completing the closed loop of distributed local cache consistency control.
[0118] This embodiment includes:
[0119] The business cache unit registration and subscription module is used to register multiple business cache units on the client side and establish a corresponding message topic subscription relationship for each business cache unit;
[0120] The data change context recording and notification construction module is used to record the corresponding data change context when a change is detected in the data associated with the business cache unit in the database. After the database transaction of the relevant business operation is successfully committed, a notification message is constructed and published to the message topic corresponding to the target business cache unit.
[0121] The notification receiving and idempotency verification module is used to route the notification message to the target business cache unit based on the business domain identifier and model identifier in the notification message after receiving the notification message on the client side, and to perform idempotency verification and deduplication on the globally unique message identifier of the notification message. The subsequent refresh process will only continue if it is determined that the message has not been processed.
[0122] The local version number comparison and refresh control module is used to read the local version number in the target business cache unit and compare it with the version number to be released carried in the notification message.
[0123] The abnormal retry and failure reporting module is used to automatically retries according to the exponential backoff strategy when the cache refresh process is interrupted due to network abnormalities, execution failures, or unavailability of dependent resources.
[0124] The status reporting module is used by the central side to periodically receive status information of each business cache unit reported by the client side.
[0125] The remote control command processing and acknowledgment module is used to verify the legality and validity of remote control commands sent by the central side after the client side receives them, and to execute the corresponding control operation in the target business cache unit according to the command type.
[0126] In this embodiment, when the program is executed by the processor, it implements a distributed cache consistency control method as described in any one of claims 1 to 8.
[0127] Example 2: In a distributed order service multi-instance system, each service instance maintains a cache of hot order data locally and performs consistent refresh through a unified message channel. Within a single end-to-end business cycle, the system undergoes a complete process including data change, transaction commit, notification push, message idempotency verification, version number comparison, refresh operation, exception retries, and central receipt.
[0128] During a batch deployment operation, the service system deployed 32 order service instances. Each instance automatically registered an "order cache unit" upon startup and was assigned a business domain identifier (e.g., "ORD") and a model identifier (e.g., "INFO"). All instances subscribed to the message topic "cache.ORD.INFO". During the initialization phase, each instance automatically reported its cache unit status, including a cache capacity of 16384 records, an initial local version number of "0", and 32 successfully established subscription relationships.
[0129] Simulating a peak order generation during a major promotional event, the system receives 8000 new order write requests per second. When processing the 321715th order write request, the instance receives the order data (order number: A1000321715, amount: 452.50), first marks it as "pending synchronization" in the local cache, and then commits it to the backend database. After the transaction is committed, a notification message containing the following is automatically generated:
[0130] Business domain identifier: ORD;
[0131] Model identifier: INFO;
[0132] Refresh type: Incremental refresh via button presses;
[0133] Data Identifier: A1000321715;
[0134] Release version number: V=1,876,120,016;
[0135] Timestamp: 1700108210589;
[0136] Globally unique message identifier: msg-7a839a4b69;
[0137] System log: A total of 8,000 notification messages were generated in this batch. According to Kafka message channel statistics, the average time for this batch of pushes was 8.6ms, and the push success rate was 100%.
[0138] Instances 15 and 28 received the notification message "msg-7a839a4b69" at system clock times of "1700108210604" and "1700108210607" respectively. Instance 15 did not find the identifier in its idempotency processing set, so it immediately added "msg-7a839a4b69" to the set and continued processing. Instance 28, due to the network retry mechanism, received the same message multiple times. Detecting that "msg-7a839a4b69" already existed in the idempotency processing set, it discarded the message and incremented the idempotency hit count. The idempotency deduplication statistics for this batch of 8,000 notification messages across all instances are as follows:
[0139] Total messages processed: 256,000 (8,000 messages × 32 instances);
[0140] Effective message processing: 8000 messages;
[0141] Idempotent message discarding: 248,000 messages;
[0142] Maximum idempotency hits per message: 31;
[0143] Upon receiving the notification message, instance 15 extracts the version number to be released, "1,876,120,016". The current version number in the local cache unit is "1,876,120,014". Detecting that the version number to be released is higher, it sets the corresponding data item "Order Number A1000321715" in the cache unit to "Updating". If another notification message, "msg-7a839a4b68", is encountered, with a version number to be released of "1,876,120,013" and a current version number in the local cache of "1,876,120,014", the instance immediately discards the notification message and inserts the following into the local "Version Backtracking Record" table: {Message Identifier: msg-7a839a4b68, Backtracking Type: Version Outdated, Timestamp: 1700108210612}. Version comparison results for this batch of 8000 messages:
[0144] Normal progress messages: 7,996;
[0145] Outdated or duplicate messages discarded: 4;
[0146] Total number of version rollback records: 4;
[0147] After successfully comparing versions in Instance 15, the system queries the data source for the latest status of order A1000321715 using the "Incremental Refresh" method and refreshes the data item into the local cache. The refresh takes 11.3ms, and the local version number of the cache unit is synchronously updated to "1,876,120,016". If there is a brief network failure during this process, the system automatically enters the exponential backoff retry process. The first retry waits for 2 seconds, the second for 4 seconds, and the third for 8 seconds; if the refresh still fails after the third retry, a failure context is generated.
[0148] Globally unique message identifier: msg-7a839a4b69;
[0149] Reason for failure: Data source connection timed out;
[0150] Number of retries: 3;
[0151] Last failure timestamp: 1700108210649;
[0152] The rollback process is executed, and the failure status of this notification message is reported to the central side. The central platform records this and pushes an anomaly warning.
[0153] One early morning, the central server issued a "full refresh" command to all instances. The command type was "full refresh," the expiration timestamp was "1700112000000," and the globally unique message identifier was "cmd-9ef08cdce8." Upon receiving the command, the instances verified that the current timestamp "1700111009999" had not expired, confirmed the command's validity, and immediately initiated a full cache rebuild. The rebuild took 2.23 seconds, and after the refresh, the local version number increased to "1,876,121,900." After the command was executed, the instances generated a receipt and reported it to the central server.
[0154] Operation result indicator: Full refresh completed;
[0155] The globally unique message identifier for this command is: cmd-9ef08cdce8;
[0156] Latest local version number: 1,876,121,900;
[0157] Client instance identifier: inst-15;
[0158] The central platform aggregates all instance receipts and detects that there are no timeouts or failures across the entire network, resulting in a consistent final status.
[0159] To compare the actual effects of the method of this invention with traditional solutions, cache consistency, idempotency processing, refresh latency, and fault recovery metrics were collected from all order service instances over a week:
[0160] Under the method of this invention, the cached data consistency rate is 100% in 8,000,000 order notifications, the maximum single message is idempotently discarded 32 times, there are 0 dirty write / erroneous refresh events, the cache refresh P99 delay is 14ms, the average number of automatic retries is 1.03, and all backtracking and failures are fully recorded and traceable for auditing.
[0161] Compared with traditional solutions, the cached data consistency rate is 98.81%, the number of idempotent discards per message is less than 3, the number of dirty write / erroneous refresh events is 138 per day, the P99 latency is 37ms, the average number of automatic retries is 2.7, and 90% of failure events lack traceable records.
[0162] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A distributed cache consistency control method, characterized in that, include: S1. Register multiple business cache units on the client side and establish a message topic subscription relationship for each business cache unit; S2. When a business operation causes a change in the data associated with the business cache unit in the database, record the data change context. After the database transaction in which the business operation is located is successfully committed, construct a notification message and publish the notification message to the message topic corresponding to the business cache unit. S3. After receiving the notification message, the client-side subscriber routes it to the target business cache unit based on the business domain identifier and model identifier. It performs idempotency verification and deduplication on the globally unique message identifier. If the message is not processed, it continues to execute the subsequent steps. S4. Read the local version number in the target business cache unit and compare it with the version number to be released in the notification message. Only if the version number to be released is greater than the local version number will the refresh process continue. Otherwise, discard the notification message and record the version backtracking information. S5. When the refresh is interrupted, perform exponential backoff retry and retain the globally unique message identifier; if the retry limit is exceeded, back off and report the failure status to the central side. S6. The central side periodically receives the status information of the service cache unit reported by the client side; S7. After receiving the remote control command, the client side verifies the legality and validity of the command, executes the corresponding operation, and sends the operation result and the latest local version number back to the central side after the execution is completed, thus completing the closed loop of distributed local cache consistency control.
2. The distributed cache consistency control method according to claim 1, characterized in that, S1 further includes: Establish a mapping relationship between business domain identifiers, model identifiers, and business cache units; initialize the lifecycle management capabilities of each business cache unit, including enabling, disabling, incremental refresh by key, full rebuild, and local version number maintenance. The message topic consists of a unified prefix, a business domain identifier, and a model identifier. It maintains a continuous connection channel with the pluggable message middleware and persists the correspondence between message topics and business cache units locally.
3. The distributed cache consistency control method according to claim 1, characterized in that, The S3 further includes: S31. After receiving the notification message, the client-side subscriber extracts the business domain identifier, model identifier, globally unique message identifier, and refresh type from the notification message. Based on the business domain identifier and model identifier, it routes to the target business cache unit and performs idempotency verification processing within the target business cache unit. S32. Establish an idempotent processing tag set within the target business cache unit; S33. If the globally unique message identifier of the current notification message already exists in the idempotent processing tag set, then the current notification message is determined to be a duplicate message, is immediately discarded, and the number of hits is recorded; If the globally unique message identifier of the current notification message does not exist in the idempotent processing flag set, the refresh process continues and the globally unique message identifier of the current notification message is written into the idempotent processing flag set. S34. Using the received timestamp when receiving the notification message and the sending timestamp contained in the notification message for joint verification, determine whether the difference between the received timestamp and the sending timestamp of the current notification message exceeds the maximum allowed message propagation delay. If it exceeds the maximum allowed message propagation delay, the corresponding notification message is determined to be a timeout invalid message. The timeout invalid message is no longer processed and is recorded as a delayed message, thus completing the time consistency processing of the idempotent processing mark set.
4. The distributed cache consistency control method according to claim 1, characterized in that, The S4 further includes: S41. Compare the version number to be released with the local version number. If the version number to be released is less than or equal to the local version number, the version of the current notification message is deemed invalid, considered as a duplicate or outdated refresh request, the notification message is immediately discarded, and the version backtracking event of this notification message is recorded. At the same time, a version backtracking identifier is generated for the current notification message as a sign that the current notification message has a version conflict. The globally unique message identifier of the notification message is used to uniquely identify the notification message. S42. If and only if the version number to be released is greater than the local version number, the notification message shall be regarded as a valid version update instruction, and the refresh process shall continue. After the refresh process is completed, the local version number of the current business cache unit shall be updated to the version number to be released. S43. When the local version number fails to be updated correctly due to abnormal breakpoints or network interruptions during the refresh process of the client instance, it automatically restores to the refresh state of the last successful completion and continues to verify the version of newly received notification messages based on the largest version number that has been successfully completed in the current business cache unit.
5. A distributed cache consistency control method according to claim 1, characterized in that, The refresh process includes: When the refresh type is button incremental refresh, the target data is obtained from the data source according to the data identifier, the corresponding data item of the target business cache unit is updated, and if the target data does not exist, the deletion or invalidation operation is performed. When the refresh type is full refresh, rebuild all the contents of the target business cache unit; after the refresh is completed, update the local version number to the version number to be released.
6. The distributed cache consistency control method according to claim 1, characterized in that, The S5 also includes: S51. When the refresh process of the target service cache unit is interrupted due to network abnormality, execution failure or unavailability of dependent resources, record the globally unique message identifier of the current notification message and the failure context that caused the interruption, and start the automatic retry mechanism based on the exponential backoff strategy. S52. Whenever the automatic retry mechanism initiates a retry, the current notification message is idempotent based on the globally unique message identifier; S53. The automatic retry mechanism continues to execute until the number of retries reaches the preset retry threshold. When the current number of retries is greater than or equal to the retry threshold, the automatic retry mechanism stops executing, the current notification message is regarded as a refresh failure message, and the rollback process is initiated. S54. When executing the rollback process, record the corresponding failure context and audit information based on the globally unique message identifier; S55. After the rollback process is completed, the refresh failure status will be reported as failure status reporting information to the central side.
7. A distributed cache consistency control method according to claim 1, characterized in that, The S7 further includes: S71. After receiving the remote control command issued by the central side, the client side extracts the command type, business domain identifier, model identifier, command validity period timestamp, and globally unique message identifier from the remote control command, and routes the remote control command to the corresponding target business cache unit based on the business domain identifier and model identifier. S72. Read the current client local timestamp from the target business cache unit and perform a validity check on the instruction validity period timestamp: If the current local timestamp of the client is less than or equal to the timestamp of the instruction's validity period, the remote control instruction will be considered a valid instruction and will continue to be executed. If the current local timestamp of the client is greater than the expiration timestamp of the instruction, the remote control instruction is judged as a timeout instruction and discarded, and the instruction timeout event is recorded at the same time. S73. Based on the instruction type, execute the corresponding control operation within the target service cache unit; S74. After executing the control operation corresponding to the instruction type, generate a control operation result identifier, read the updated local version number of the target business cache unit, and combine the operation result identifier, the instruction globally unique message identifier, the latest local version number of the business cache unit, and the current client instance identifier into a receipt information; S75. The receipt information is reported to the central side, so that the central side can know the execution result of the remote control command, the latest local version number of the target business cache unit, and the execution status of the client instance based on the receipt information, thus completing the closed loop of distributed local cache consistency control.
8. A distributed cache consistency control method, system, and computer-readable medium according to claim 7, characterized in that, The execution of corresponding control operations within the target service cache unit includes: When the instruction type is an enable operation instruction, the operation of enabling the business cache unit is executed; When the instruction type is a close operation instruction, the operation of closing the business cache unit is performed; When the instruction type is a full refresh instruction, the entire contents of the target business cache unit are rebuilt, and the local version number of the target business cache unit is updated after the reconstruction is completed, so that the local version number is consistent with the latest version.
9. A distributed cache consistency control system, characterized in that, include: The business cache unit registration and subscription module is used to register multiple business cache units on the client side and establish a corresponding message topic subscription relationship for each business cache unit; The data change context recording and notification construction module is used to record the corresponding data change context when a change is detected in the data associated with the business cache unit in the database. After the database transaction of the relevant business operation is successfully committed, a notification message is constructed and published to the message topic corresponding to the target business cache unit. The notification receiving and idempotency verification module is used to route the notification message to the target business cache unit based on the business domain identifier and model identifier in the notification message after receiving the notification message on the client side, and to perform idempotency verification and deduplication on the globally unique message identifier of the notification message. The subsequent refresh process will only continue if it is determined that the message has not been processed. The local version number comparison and refresh control module is used to read the local version number in the target business cache unit and compare it with the version number to be released carried in the notification message. The abnormal retry and failure reporting module is used to automatically retries according to the exponential backoff strategy when the cache refresh process is interrupted due to network abnormalities, execution failures, or unavailability of dependent resources. The status reporting module is used by the central side to periodically receive status information of each business cache unit reported by the client side. The remote control command processing and acknowledgment module is used to verify the legality and validity of remote control commands sent by the central side after the client side receives them, and to execute the corresponding control operation in the target business cache unit according to the command type.
10. A computer-readable medium having a computer program stored thereon, characterized in that... When the program is executed by the processor, it implements a distributed cache consistency control method as described in any one of claims 1 to 8.