Multi-terminal data synchronization and consistency verification method and system based on message queue

Abstract

The invention provides a multi-terminal data synchronization and consistency verification method and system based on a message queue, and relates to the technical field of data processing. According to the method, firstly, a service data change event sent from multi-terminal data is received, and the service data change event corresponds to core service data; secondly, releasing the business data change event to a RabbitMQ message queue and distributing the business data change event to a target terminal so as to realize data synchronization among multiple terminals; next, a preset lightweight consistency verification engine is called to execute periodic snapshot comparison on the core service data; and finally, when the data is detected to be inconsistent through the periodic snapshot comparison, triggering an abnormal automatic repair mechanism to maintain the consistency of the multi-terminal data. According to the technical scheme provided by the invention, high reliability and self-healing capability of a multi-terminal data synchronization process are realized, and final consistency of core service data is also ensured.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a method and system for multi-terminal data synchronization and consistency verification based on message queues. Background Technology

[0002] In business scenarios involving multi-terminal collaboration, such as e-commerce and local services, the user end, merchant end, and platform management end need to share and process core business data such as order status and stored value balance in real time. Any data change on any end needs to be synchronized to other ends in near real time and accurately to ensure business continuity and user experience, and there are strict requirements for data consistency.

[0003] Currently, the adopted technical solution is a master-slave synchronization architecture based on a central database. In this solution, all data changes are first written to a single master database, and then the changes are synchronized to each slave database through the database's own replication mechanism. Each business terminal connects to the corresponding slave database to read the data, thereby achieving the final synchronization of the data.

[0004] However, this solution has obvious limitations. First, it is heavily dependent on the availability of the master database. If the master database fails, the entire synchronization will be interrupted. Second, master-slave synchronization has inherent latency, which can easily lead to each terminal reading a temporarily inconsistent data state under high concurrency, causing business errors. More importantly, when inconsistency occurs, the system usually lacks automatic repair capabilities and requires manual intervention for verification and repair, which is inefficient and prone to errors. Summary of the Invention

[0005] This application provides a method and system for multi-terminal data synchronization and consistency verification based on message queues, which solves the technical problems of low reliability of synchronization links, reliance on manual repair when data is inconsistent, and low processing efficiency in the prior art.

[0006] Firstly, this application provides a method for multi-terminal data synchronization and consistency verification based on message queues, including:

[0007] Receive business data change events from user terminals, merchant terminals, and platform terminals. The business data change events correspond to core business data, which includes stored value balance, order status, and coupon usage records.

[0008] The business data change event is published to a RabbitMQ message queue based on an event-driven architecture, and then distributed to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals.

[0009] During the data synchronization process between the multiple terminals, a preset lightweight consistency verification engine is invoked to perform periodic snapshot comparisons on the core business data;

[0010] When the periodic snapshot comparison detects data inconsistency, an automatic anomaly repair mechanism is triggered to maintain data consistency across multiple devices.

[0011] Optionally, the system receives business data change events from the user end, merchant end, and platform end. These business data change events correspond to core business data, which includes stored value balance, order status, and coupon usage records, including:

[0012] Deploy data monitoring modules in the user interface, the merchant processing system, and the platform management backend;

[0013] The data monitoring module continuously captures interactive operations and internal system calls occurring on the corresponding terminal.

[0014] When interactive operations and internal system calls are captured, the data monitoring module identifies and filters out change operations related to core business data. These change operations include user-side prepaid account top-ups or consumption, merchant-side order status updates, and platform-side coupon rule activation or deactivation.

[0015] Based on the identified change operations, generate business data change events;

[0016] The corresponding core business data type is determined based on the business data change event.

[0017] Optionally, the business data change event is published to a RabbitMQ message queue based on an event-driven architecture, and the RabbitMQ message queue is used to distribute the business data change event to the target terminal to achieve data synchronization between multiple terminals, including:

[0018] Receive the business data change event, wherein the business data change event carries a type identifier;

[0019] Based on the type identifier, the business data change event is converted into a target message. The target message includes a message header and a message body. The message header records the sender information, the type identifier, and the event number.

[0020] In the RabbitMQ message queue, multiple initial message queues are established corresponding to the user end, the merchant end, and the platform end, respectively, and the routing key of each initial message queue is set according to the type identifier;

[0021] Send the target message to the RabbitMQ message queue exchange;

[0022] The exchange distributes the target message to the target message queue corresponding to the type identifier according to the type identifier in the message header and the routing key.

[0023] Each receiving end retrieves and parses the target messages sequentially from its associated target message queue to complete data synchronization.

[0024] Optionally, in the RabbitMQ message queue, multiple initial message queues corresponding to the user end, the merchant end, and the platform end are established respectively, and a routing key for each initial message queue is set according to the type identifier, including:

[0025] The terminal type is extracted from the received business data change event, and the terminal type is any one of user terminal, merchant terminal, and platform terminal;

[0026] Based on the extracted terminal type, create a first initial message queue, a second initial message queue, and a third initial message queue in the RabbitMQ message queue, corresponding to the user terminal, the merchant terminal, and the platform terminal, respectively.

[0027] Determine the type identifier carried by the business data change event;

[0028] Based on the type identifier, a routing key is assigned to each of the first initial message queue, the second initial message queue, and the third initial message queue.

[0029] Optionally, during the data synchronization process between the multiple terminals, a preset lightweight consistency verification engine is invoked to perform periodic snapshot comparisons of the core business data, including:

[0030] In response to a preset timed trigger command, the comparison process of the lightweight consistency verification engine is initiated;

[0031] The lightweight consistency verification engine obtains snapshots of the core business data at the same time from the user end, merchant end and platform end respectively, forming first snapshot data, second snapshot data and third snapshot data, wherein each snapshot data includes data identifier, data value and snapshot timestamp;

[0032] Within the lightweight consistency verification engine, the first snapshot data, the second snapshot data, and the third snapshot data are compared to target entries with the same data identifier to determine whether the data values ​​of the target entries are consistent.

[0033] When there are inconsistent data values, the corresponding difference information is recorded. The difference information includes the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamp corresponding to each data value.

[0034] Optionally, when the periodic snapshot comparison detects data inconsistency, an automatic anomaly repair mechanism is triggered to maintain data consistency across multiple endpoints, including:

[0035] When the aforementioned discrepancy information is recorded, the automatic anomaly repair mechanism is triggered;

[0036] The automatic anomaly repair mechanism extracts the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamps corresponding to each data value from the difference information.

[0037] Based on the inconsistent data values ​​and the snapshot timestamps corresponding to each data value, a baseline data value is determined;

[0038] Based on the data identifier and the baseline data value, a repair message instruction is generated;

[0039] The repair message instruction is published to the RabbitMQ message queue;

[0040] The repair message instruction is distributed through the RabbitMQ message queue to target terminals whose data values ​​are not the baseline data value among the user terminal, merchant terminal, and platform terminal, so as to update the core business data corresponding to the data identifier stored locally on the target terminal.

[0041] Optionally, a repair message instruction is generated based on the data identifier and the reference data value, including:

[0042] Based on the data identifier and the reference data value, a first operation instruction is generated;

[0043] Append an operation timestamp to the first operation instruction to generate a timestamp instruction;

[0044] The data identifier, the reference data value, and the timestamp instruction are combined to generate a first message instruction;

[0045] Based on the first message instruction, a message header containing an operation type identifier is generated;

[0046] The message header is combined with the first message to generate a repair message instruction.

[0047] Secondly, this application provides a multi-terminal data synchronization and consistency verification system based on a message queue, including:

[0048] The receiving module is used to receive business data change events sent from the user terminal, the merchant terminal and the platform terminal. The business data change events correspond to core business data, which includes stored value balance, order status and coupon usage records.

[0049] The distribution module is used to publish the business data change event to a RabbitMQ message queue based on an event-driven architecture, and distribute the business data change event to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals;

[0050] The comparison module is used to call a preset lightweight consistency verification engine to perform periodic snapshot comparisons of the core business data during the data synchronization process between the multiple terminals.

[0051] The triggering module is used to trigger an automatic anomaly repair mechanism when the periodic snapshot comparison detects data inconsistency, so as to maintain data consistency across multiple terminals.

[0052] Thirdly, this application provides a computing device, including a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are to be invoked and executed by the processing component to implement a message queue-based multi-terminal data synchronization and consistency verification method as described in the first aspect above.

[0053] Fourthly, this application provides a computer storage medium storing a computer program, which, when executed by a computer, implements a multi-terminal data synchronization and consistency verification method based on a message queue as described in the first aspect.

[0054] This application replaces the traditional master-slave database synchronization mechanism with an event-driven message queue. It publishes business data change events to the RabbitMQ message queue for distribution, realizing asynchronous and decoupled transmission of data changes between multiple terminals. This approach effectively avoids the problem of interruption of the entire synchronization link due to the failure of a single central database, enhancing availability and robustness. At the same time, the asynchronous message passing mechanism can better cope with the instantaneous data spikes in high-concurrency scenarios, enabling the data status between terminals to become consistent with lower latency, thereby improving the overall reliability of the multi-terminal data synchronization process.

[0055] Furthermore, by introducing periodic snapshot comparisons and automated repair mechanisms, a lightweight verification engine is used to periodically compare the status of core business data across multiple platforms, proactively identifying potential data inconsistencies. Once an inconsistency is detected, a repair process is automatically triggered, generating and issuing repair instructions based on predefined rules, eliminating the need for manual intervention in data verification and script execution. This not only significantly improves the efficiency of handling data inconsistency issues but also enables continuous and automated maintenance of data consistency across multiple platforms, reducing the risk of business errors caused by data inconsistencies.

[0056] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description

[0057] To more clearly illustrate the technical solutions in this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0058] Figure 1 This paper presents a flowchart of a multi-terminal data synchronization and consistency verification method based on a message queue, as provided in this application.

[0059] Figure 2 This paper presents a schematic diagram of the structure of a multi-terminal data synchronization and consistency verification system based on a message queue, as provided in this application.

[0060] Figure 3 A schematic diagram of the structure of a computing device provided in this application is shown. Detailed Implementation

[0061] To enable those skilled in the art to better understand the present application, the technical solution of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0062] In some of the processes described in the specification, claims, and accompanying drawings of this application, multiple operations appearing in a specific order are included. However, it should be clearly understood that these operations may not be executed in the order they appear herein, or may be executed in parallel. The operation numbers, such as 101, 102, etc., are merely used to distinguish different operations and do not themselves represent any execution order. Furthermore, these processes may include more or fewer operations, and these operations may be executed sequentially or in parallel. It should be noted that the descriptions such as "first," "second," etc., in this document are used to distinguish different messages, devices, modules, etc., and do not represent a chronological order, nor do they limit "first" and "second" to different types.

[0063] The technical solutions of this application will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0064] Figure 1 This application provides a flowchart of a multi-terminal data synchronization and consistency verification method based on message queues, as shown below. Figure 1 As shown, the method includes:

[0065] Step 101: Receive business data change events sent from the user terminal, the merchant terminal, and the platform terminal. The business data change events correspond to core business data, which includes stored value balance, order status, and coupon usage records.

[0066] Optionally, step 101 may specifically include:

[0067] Step 1011: Deploy the data monitoring module in the user interface, the merchant processing system, and the platform management backend.

[0068] Step 1012: The data monitoring module continuously captures the interactive operations and internal system calls occurring on the corresponding terminal.

[0069] Step 1013: When interactive operations and internal system calls are captured, the data monitoring module identifies and filters out change operations related to core business data. The change operations include user-side prepaid account top-ups or consumption, merchant-side order status updates, and platform-side coupon rule activation or deactivation.

[0070] Step 1014: Generate a business data change event based on the identified change operation.

[0071] Step 1015: Determine the corresponding core business data type based on the business data change event.

[0072] In this step, a business data change event refers to a data structure generated in a multi-terminal business system when the state of core business data changes. This structure is used to record the change and to notify and transmit data change information between various terminals.

[0073] Core business data refers to critical business information that must maintain a high degree of consistency in multi-terminal collaborative processing scenarios to support the correct operation of core business processes, including stored value balance, order status, and coupon usage records.

[0074] Stored value balance refers to the amount of currency pre-deposited in a user's account that can be used for consumption. It is used to represent the user's payment ability and is obtained by increasing or decreasing the account balance when the user makes a deposit or makes a purchase.

[0075] Order status refers to the different stages a transaction goes through from creation to completion, reflecting the current processing progress of the order. It is updated when the order is created, paid for, shipped, or completed.

[0076] Coupon usage records refer to data that records users' holding and use of coupons, as well as the coupon's validity status. This data is used to manage the execution of marketing activities and is generated when users configure coupon rules on the platform or when they redeem coupons.

[0077] A data monitoring module is a software functional unit deployed in the user interface, merchant processing system, and platform management backend. It is used to monitor and capture operation commands that may change core business data in real time.

[0078] Interactive operations and internal system calls refer to instructions directly triggered by users through actions such as clicking and inputting on the interface, as well as program execution instructions not directly triggered by users, such as system background services and scheduled tasks. Both are the root causes of changes in core business data and are automatically generated through user-initiated actions or system preset logic.

[0079] In this step, a dedicated monitoring code snippet is first embedded in the front-end code of the user interface, the back-end service logic of the merchant processing system, and the application service of the platform management backend through software integration technology, thereby completing the deployment of the data monitoring module and making it part of each terminal application.

[0080] Secondly, through event listening technology, the running data listening module can continuously intercept and obtain interactive operation instructions and API requests called internally by the terminal software it is located in. When the data listening module captures an instruction or request, it will trigger a preset rule matching algorithm. The rule matching algorithm compares the captured instruction content with a predefined rule base. The rule base describes all operation characteristics that can modify the stored value balance, order status and coupon usage records. If the instruction characteristics match any entry in the rule base, the rule matching algorithm determines that the instruction is a valid change operation and filters it out, while ignoring irrelevant operations that do not match.

[0081] Next, data serialization technology is used to invoke a formatting function in the data monitoring module. This function takes the specific information of the identified change operation as input, such as the operation action, target data ID, new data value, and timestamp. It encapsulates and encodes this information according to a predefined structured format such as JSON or Protocol Buffers, thereby generating a standardized business data change event data packet that can be transmitted over the network. Finally, using query matching logic, the key fields in the business data change event data packet, such as the operation object ID or event code, are parsed. By searching and matching this key field with a pre-configured data type mapping table, the specific category of the core business data associated with the business data change event is determined, such as mapping to the stored value balance type, and a corresponding type identifier is assigned to this event.

[0082] For example, in e-commerce platform A, firstly, when user A clicks the "Confirm Payment" button to complete the order payment in the user interface of the mobile application, the data monitoring module deployed on the user's end will immediately capture this click operation; secondly, the data monitoring module will analyze the built-in rules and find that this is an order payment operation, so it will identify it as a change operation; then, the monitoring module will call the event encapsulation function to generate a business data change event with the content "Order No. XXX Status changed to 'Paid'", and determine its order status type in the core business data based on the order number; then, when merchant B selects the same order from the order list and clicks the "Ship" button in the merchant processing system on the computer, the data monitoring module on the merchant's end will also capture this operation, identify it as an order shipment change operation, generate an event with "Order No. XXX Status changed to 'Shipped'", and classify it as an order status type.

[0083] The monitoring and event generation mechanism built in this step realizes the source capture and standardized transformation of business data changes. It not only ensures that all key data changes can be accurately and timely identified and transformed into structured synchronous events through automated rule filtering and event encapsulation, but also provides a precise and standardized input source for the subsequent highly reliable, asynchronous data synchronization process across terminals, eliminating the omission of data changes and format chaos from the source.

[0084] Step 102: Publish the business data change event to a RabbitMQ message queue based on an event-driven architecture, and distribute the business data change event to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals.

[0085] Optionally, step 102 may specifically include:

[0086] Step 1021: Receive the business data change event, the business data change event carries a type identifier.

[0087] Step 1022: Based on the type identifier, convert the business data change event into a target message. The target message includes a message header and a message body. The message header records the sender information, the type identifier, and the event number.

[0088] Step 1023: In the RabbitMQ message queue, establish multiple initial message queues corresponding to the user end, the merchant end, and the platform end respectively, and set the routing key of each initial message queue according to the type identifier.

[0089] Optionally, step 1023 may include the following steps: extracting the terminal type from the received business data change event, wherein the terminal type is any one of user terminal, merchant terminal, and platform terminal; creating a first initial message queue, a second initial message queue, and a third initial message queue in the RabbitMQ message queue corresponding to the user terminal, the merchant terminal, and the platform terminal, respectively, according to the extracted terminal type; determining the type identifier carried by the business data change event; and assigning a routing key to each of the first initial message queue, the second initial message queue, and the third initial message queue according to the type identifier.

[0090] Step 1024: Send the target message to the RabbitMQ message queue exchange.

[0091] Step 1025: Using the exchange, the target message is distributed to the target message queue corresponding to the type identifier according to the type identifier in the message header and the routing key.

[0092] Step 1026: Each receiving end retrieves and parses the target messages sequentially from its associated target message queue to complete data synchronization.

[0093] In this step, RabbitMQ message queue refers to an open-source message broker software based on the AMQP protocol, used to reliably and asynchronously deliver and store messages between different software applications or components. Messages are accessed and used by deploying them on the RabbitMQ server software and establishing client connections.

[0094] The target terminal refers to the terminal other than the sending terminal among the user terminal, merchant terminal, and platform terminal. It is used to update its own data status according to the synchronized message and is determined by identifying the routing information in the message.

[0095] A type identifier is a classification label attached to a business data change event to distinguish the specific type of core business data involved in the event, such as whether it is a change in stored value balance or an order status change. It is obtained by determining the corresponding core business data type based on the business data change event.

[0096] The target message refers to a data packet encapsulated to fit the message queue transmission format, used for routing and delivery in the RabbitMQ message queue system.

[0097] The message header is a component of the target message, used to carry the message's metadata and routing control information, and is generated by a formatter when the target message is created.

[0098] The message body refers to another component of the target message, which carries the specific content of the business data change event. It is directly formed by encapsulating the original business data change event.

[0099] The sender information refers to a field recorded in the message header, which indicates whether the original terminal that generated this business data change event is the user terminal, the merchant terminal, or the platform terminal. It is extracted from the original business data change event.

[0100] The event number is a unique sequence number recorded in the message header, used to uniquely identify a specific business data change event, which facilitates tracking and deduplication. It is usually generated by a sequence number generator or a combination of timestamps and random number algorithms.

[0101] The initial message queue refers to a buffer that is pre-created and declared in the RabbitMQ server for temporarily storing messages, used to receive and temporarily store messages that occur to different target terminals.

[0102] Terminal type refers to a classification of three types of systems: user terminal, merchant terminal, and platform terminal. It is used to distinguish between them when creating message queues and setting routing rules.

[0103] The first initial message queue, the second initial message queue, and the third initial message queue refer to buffers created in RabbitMQ specifically for temporarily storing messages that need to be sent to the user end, the merchant end, and the platform end, respectively. They are used to isolate and store messages according to terminal type and are created by calling the RabbitMQ client API based on the extracted terminal type.

[0104] The routing key is a string label bound to a queue in the RabbitMQ message queue system. It is used to indicate which messages the exchange should deliver to that queue. It is obtained by setting each initial message queue according to the type identifier and a preset mapping rule.

[0105] An exchange is a core component of the RabbitMQ message queuing system. It is used to receive messages sent by producers and route messages to one or more queues according to specific rules. Exchanges are obtained by configuring the RabbitMQ server or declaring an exchange instance on it.

[0106] The target message queue refers to the message queue associated with a terminal other than the sender, which is used by the target terminal to connect to and retrieve messages from it.

[0107] In this step, firstly, a unified event receiving interface is used to receive incoming business data change events with type identifiers attached. Secondly, a message construction function is used to create a target message that conforms to the message queue transmission protocol, taking the received business data change event and its type identifier as input. This process creates a message header, which contains the original terminal information that generated the event, i.e., the sender information, the type identifier, and a unique event number. At the same time, the specific content of the event is completely placed into the message body, together forming a complete target message.

[0108] Next, during the initialization of the RabbitMQ message queue system, three initial message queues will be established through its management interface or client declaration operation, corresponding to the user end, merchant end, and platform end respectively. These can be referred to as the first initial message queue, the second initial message queue, and the third initial message queue.

[0109] Simultaneously, based on the type identifier of the core business data, such as the routing key order.status corresponding to the order status type identifier, a routing key is bound to each of the three queues; then, by calling the publish method of the RabbitMQ client SDK, the encapsulated target message is sent to a predefined exchange;

[0110] After receiving the target message, the exchange reads the type identifier field in its message header. The routing logic inside the exchange matches the routing key bound to each initial message queue based on this type identifier. If the type identifier matches the routing key of a queue, the exchange will copy the target message and deliver it to that queue. This queue becomes the target message queue for this distribution. The target message may be delivered to one or more queues based on the type identifier.

[0111] Finally, the user end, merchant end, and platform end, as receiving ends, will run their respective RabbitMQ message consumption clients. These clients will continuously listen for or retrieve target messages from their respective associated target message queues in a first-in-first-out order. Then, they will parse the target messages, extract the business data change event content in the message body, and update their respective local data storage according to the business data change event content, thereby completing data synchronization.

[0112] For example, following the specific implementation of the previous step, firstly, the event receiving interface of e-commerce platform A receives an event from the user client indicating that the status of order number XXX has changed to "paid," with its type identifier being "order status." Secondly, the message construction function encapsulates this event into a target message, the header of which records that the sender is the user client, the type is "order status," the event number is event serial number 001, and the message body contains the specific order information. Next, the target message is published to the exchange in RabbitMQ responsible for handling order type events. Previously, RabbitMQ had created corresponding merchant-side message queues and platform-side message queues as initial message queues for the merchant client and platform client respectively, and both were bound to routing keys matching the order status type. Then, the exchange routes the message to these two queues according to the order status type identifier in the message header. Finally, the synchronization service program of merchant B's platform management backend, acting as the receiving end, retrieves this message from its respective associated queue, parses it, and updates the local status of order XXX to "paid," thus achieving data synchronization.

[0113] This step utilizes message queues to achieve reliable asynchronous distribution of business change events and decoupling between terminals. By encapsulating events into standardized messages and performing intelligent routing based on type, it ensures that change information can be delivered to all relevant terminals accurately and orderly. This mechanism effectively reduces direct dependencies between system components, improves the elasticity and scalability of the overall architecture, and ensures the stability and efficiency of the data synchronization process in high-concurrency scenarios.

[0114] Step 103: During the data synchronization process between the multiple terminals, a preset lightweight consistency verification engine is invoked to perform periodic snapshot comparisons on the core business data.

[0115] Optionally, step 103 may specifically include:

[0116] Step 1031: In response to a preset timed trigger command, start the comparison process of the lightweight consistency verification engine.

[0117] Step 1032: Using the lightweight consistency verification engine, obtain state snapshots of the core business data at the same time from the user end, merchant end and platform end respectively to form first snapshot data, second snapshot data and third snapshot data, wherein each snapshot data includes data identifier, data value and snapshot timestamp.

[0118] Step 1033: Inside the lightweight consistency verification engine, the target entries with the same data identifier in the first snapshot data, the second snapshot data, and the third snapshot data are compared to determine whether the data values ​​of the target entries are consistent.

[0119] Step 1034: When there is a data value inconsistency, record the corresponding difference information. The difference information includes the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamp corresponding to each data value.

[0120] In this step, the preset lightweight consistency verification engine refers to a pre-configured software module integrated into the platform that is specifically designed to check whether data is consistent between multiple terminals. It is used to automatically discover and record differences in the same core business data on different terminals, and is obtained through software development and system configuration.

[0121] Periodic snapshot comparison refers to a method that captures and compares the status of data distributed across multiple terminals at fixed time intervals, and is used to periodically and proactively detect data inconsistencies.

[0122] The preset timed trigger command refers to the start signal that is pre-configured by the platform and automatically issued at a specific time point or periodic time point. It is used to trigger the start of the consistency verification process and is generated by the timer component.

[0123] The first snapshot data, the second snapshot data, and the third snapshot data refer to the sets of records about the status of core business data obtained from the user end, the merchant end, and the platform end at the same point in time or within a very short time interval, respectively, and are used to represent the instantaneous status of the data on each terminal at the time of comparison.

[0124] Data identifiers refer to codes or strings that can uniquely identify a specific piece of core business data. They are used to match and associate the same piece of business data in snapshot data on different terminals and are extracted from core business data records.

[0125] Data values ​​refer to the specific content or status of core business data corresponding to a data identifier at a specific moment. They are used to determine whether the data of different terminals are consistent and are read from the snapshot data returned by each terminal.

[0126] A snapshot timestamp refers to the moment when snapshot data is acquired, used to identify the time point of data status. It is automatically recorded and appended by each terminal or verification engine when snapshot data is acquired from each terminal.

[0127] The target entry refers to the single data record with the same data identifier in the first snapshot data, the second snapshot data, and the third snapshot data. It is used for direct comparison operations across terminals and is obtained by matching and grouping multiple snapshot data according to their data identifiers.

[0128] In this step, a timer component is used to automatically issue a preset timed trigger command at a preset time point, such as every hour. This timed trigger command will directly call and start the lightweight consistency verification engine that is in standby state, so that it starts to execute the internally set comparison process.

[0129] Secondly, the lightweight consistency verification engine sends data status query requests to the user, merchant, and platform terminals simultaneously through its built-in communication interface. Upon receiving the request, each terminal immediately queries the latest status of the specified core business data from its own database or cache, packages the data identifier, current data value, and the precise time of the query (i.e., the snapshot timestamp) of each data item, and returns it to the lightweight consistency verification engine. The lightweight consistency verification engine collects the responses from the three terminals and organizes them into first snapshot data, second snapshot data, and third snapshot data, respectively.

[0130] Next, inside the lightweight consistency verification engine, a dedicated data comparison algorithm starts working. This data comparison algorithm will traverse all snapshot data, extract records with the same data identifier from the first snapshot data, the second snapshot data, and the third snapshot data, and merge them into a group. This group of records constitutes a target entry. For each target entry, the data comparison algorithm will extract and compare the data values ​​from the three snapshots one by one to determine whether the three data values ​​are completely the same.

[0131] Finally, if during the comparison process, the data comparison algorithm finds that the three data values ​​of a target item are not completely the same and there is an inconsistency, it will immediately trigger a recording action. This recording action will collect the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from snapshots on the user's side, merchant's side, and platform side, as well as the snapshot timestamp corresponding to each data value, combine this information into a complete difference information, and write it to the specified log file or database for storage.

[0132] For example, following the specific implementation of the previous step, firstly, in e-commerce platform A, when the data synchronization process is running continuously, a pre-set daily scheduled task at 2 AM is triggered, sending a start command to the lightweight consistency verification engine; secondly, the lightweight consistency verification engine simultaneously sends query requests to the user's App service, the merchant B's backend, and the platform management database, requesting to obtain the current status of order number XXX; then, the user returns the status: paid, time: T1, the merchant returns the status: shipped, time: T2, and the platform returns the status: paid, time: T3; then, the lightweight consistency verification engine organizes these responses into three snapshot data; its internal comparison algorithm finds that these three records have the same order number identifier, that is, it classifies them into one target entry; finally, the comparison algorithm compares the three data values ​​of paid, shipped, and paid, and finds that they are inconsistent, so it records the difference information: the data identifier is order number XXX, the inconsistent values ​​are paid, shipped, and paid respectively, and the corresponding timestamps are T1, T2, and T3.

[0133] This step establishes an automated data consistency monitoring mechanism through periodic proactive snapshot comparisons. This mechanism can promptly detect data state discrepancies between terminals caused by network latency, message delivery failures, or other anomalies. The mechanism transforms potential data inconsistency issues from passive discovery to proactive capture and generates structured difference records, providing a clear basis and target for subsequent automatic repairs, thereby adding an effective layer of consistency assurance during data synchronization.

[0134] Step 104: When the periodic snapshot comparison detects data inconsistency, an automatic anomaly repair mechanism is triggered to maintain data consistency across multiple devices.

[0135] Optionally, step 104 may specifically include:

[0136] Step 1041: When the difference information is recorded, the automatic anomaly repair mechanism is triggered.

[0137] Step 1042: Extract the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamps corresponding to each data value from the difference information through the automatic anomaly repair mechanism.

[0138] Step 1043: Determine the baseline data value based on the inconsistent data values ​​and the snapshot timestamps corresponding to each data value.

[0139] Step 1044: Generate a repair message instruction based on the data identifier and the baseline data value.

[0140] Optionally, step 1044 may specifically include the following steps: generating a first operation instruction based on the data identifier and the reference data value; attaching an operation timestamp to the first operation instruction to generate a timestamp instruction; combining the data identifier, the reference data value, and the timestamp instruction to generate a first message instruction; generating a message header containing an operation type identifier based on the first message instruction; and combining the message header with the first message to generate a repair message instruction.

[0141] Step 1045: Publish the repair message instruction to the RabbitMQ message queue.

[0142] Step 1046: Distribute the repair message instruction to the target terminals in the user terminal, merchant terminal, and platform terminal whose data values ​​are not the baseline data value through the RabbitMQ message queue, so as to update the core business data corresponding to the data identifier stored locally in the target terminal.

[0143] In this step, the automatic anomaly repair mechanism refers to a software process that can automatically start and execute a series of operations to restore the data to a consistent state after detecting inconsistencies in data across multiple terminals. It is used to solve data asynchrony problems without manual intervention and is automatically triggered by the system when it records discrepancies.

[0144] The baseline data value refers to a value selected from multiple inconsistent data values ​​according to specific rules, which is deemed correct or should be adopted and used as the target value for remediation to unify the data across various terminals.

[0145] Repair message instructions are a special type of formatted command message used to instruct target terminals to update their local data, accurately transmitting and applying baseline data values ​​to terminals that need repair.

[0146] The first operation command refers to the most crucial update operation command generated during the repair process, which instructs the data value corresponding to the data identifier to be updated to the baseline data value.

[0147] The operation timestamp refers to the current time recorded when the repair command is generated. It is used to identify the time point when the repair action was initiated and is obtained through the system clock.

[0148] A timestamp instruction is a composite instruction with a time stamp formed by appending an operation timestamp to the first operation instruction. It is used to ensure the timeliness and traceability of repair instructions.

[0149] The first message instruction refers to the data body containing complete repair operation content and timestamps, which has not yet been appended with a message header. It is used to form the payload of the final repair message and is obtained by combining and encapsulating data identifiers, baseline data values ​​and timestamp instructions.

[0150] In this step, when the lightweight consistency verification engine completes the comparison and writes a difference information to the storage module, a monitoring program will detect the generation of this new record in real time. The monitoring program will then send a signal to trigger the standby automatic anomaly repair mechanism.

[0151] Secondly, once the automatic anomaly repair mechanism is activated, it will immediately access the storage module, read the latest discrepancy information, and extract the data identifier that caused the inconsistency, multiple inconsistent data values ​​obtained from the user, merchant, and platform ends, and the snapshot timestamp corresponding to each value from the structured fields of the discrepancy information through an information parsing program.

[0152] Next, a decision module starts working, taking the extracted multiple data values ​​and their snapshot timestamps as input, and running a preset decision logic. For example, a common logic is to select the data value with the latest snapshot timestamp as the baseline data value. This process aims to select the value that is considered to be the most correct or final state from the conflicting values.

[0153] Then, the repair instruction generation module is invoked. First, based on the selected data identifier and baseline data value, it generates a clear first operation instruction, the content of which is similar to updating the data identifier to the baseline data value. Next, it obtains the current time as the operation timestamp and appends it to the end of the first operation instruction to form a timestamp instruction with a timestamp. Then, the generation module packages and combines the data identifier, baseline data value, and complete timestamp instruction into a structured first message instruction. In order for this first message instruction to be correctly routed by the message queue, the generation module creates a message header for the first message instruction. The header contains an operation type identifier to identify that this first message instruction is a repair type.

[0154] Finally, the generation module combines the message header with the first message instruction to generate the final repair message instruction that can be directly processed by the message queue. After that, it calls a similar message publishing interface to publish the repair message instruction as a new message to the designated exchange of the RabbitMQ message queue. The RabbitMQ message queue distributes this repair message instruction to the queues associated with the target terminals whose data values ​​recorded in the difference information do not match the baseline data values, based on the operation type identifier in the message header and the routing rules pre-set for each terminal queue. After these target terminals obtain the repair message instruction from their own queues, they parse the first operation instruction in it and update the core business data corresponding to the data identifier in their local storage to the baseline data value according to the instruction content, thereby completing the repair.

[0155] For example, following the specific implementation of the previous step, firstly, the system of e-commerce platform A detects a discrepancy record regarding order number XXX and triggers an automatic anomaly repair mechanism. Secondly, this automatic anomaly repair mechanism reads the discrepancy information, extracts the data identifier as order number XXX, and the inconsistent data values ​​as "paid," "shipped," and "paid," corresponding to timestamps T1, T2, and T3. Next, its decision module compares the timestamps T1, T2, and T3, assuming T2 is the latest, and determines "shipped" as the baseline data value. Then, the generation module creates a first operation instruction to update the status of order XXX to "shipped," appends the current time as the operation timestamp, forms a timestamp instruction, and then combines them to generate a first message instruction. Subsequently, a message header indicating data repair type is added to the first message instruction to form a complete repair message instruction and publish it to RabbitMQ. Finally, the message queue, according to the routing rules, distributes this repair message instruction only to the user client and platform client where the data value in the discrepancy information is "paid." After receiving the repair message instruction, the user client and platform client parse and execute it, ultimately updating the local order XXX status to "shipped," thus achieving consistency with the merchant's status.

[0156] This step achieves closed-loop processing of data inconsistency issues through an automated decision-making and instruction generation process. After detecting inconsistencies, it quickly determines the basis for repair and generates accurate repair instructions. Then, it reliably pushes the repair actions to the target terminal using the existing message queue infrastructure. This proactively restores the consistency of data across multiple terminals without human intervention, improving the self-healing capability and overall reliability of the data synchronization system.

[0157] Figure 2 This application provides a schematic diagram of the structure of a multi-terminal data synchronization and consistency verification system based on message queues, as shown below. Figure 2 As shown, the system includes:

[0158] The receiving module 21 is used to receive business data change events sent from the user terminal, the merchant terminal and the platform terminal. The business data change events correspond to core business data, which includes stored value balance, order status and coupon usage records.

[0159] The distribution module 22 is used to publish the business data change event to a RabbitMQ message queue based on an event-driven architecture, and distribute the business data change event to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals;

[0160] The comparison module 23 is used to call a preset lightweight consistency verification engine to perform periodic snapshot comparisons on the core business data during the data synchronization process between the multiple terminals.

[0161] Trigger module 24 is used to trigger an automatic anomaly repair mechanism when the periodic snapshot comparison detects data inconsistency, so as to maintain data consistency across multiple terminals.

[0162] Figure 2 The aforementioned multi-terminal data synchronization and consistency verification system based on message queues can perform... Figure 1 The implementation principle and technical effects of the message queue-based multi-terminal data synchronization and consistency verification method described in the illustrated embodiment will not be repeated here. The specific methods by which each module and unit performs operations in the message queue-based multi-terminal data synchronization and consistency verification system described above have been detailed in the embodiments related to this method, and will not be elaborated upon here.

[0163] In one possible design, Figure 2 The multi-terminal data synchronization and consistency verification system based on message queues shown in the embodiment can be implemented as a computing device, such as... Figure 3 As shown, the computing device may include a storage component 31 and a processing component 32;

[0164] The storage component 31 stores one or more computer instructions, wherein the one or more computer instructions are invoked and executed by the processing component 32.

[0165] The processing component 32 is used for the above Figure 1 The embodiment describes a method for multi-terminal data synchronization and consistency verification based on message queues.

[0166] The processing component 32 may include one or more processors to execute computer instructions to complete all or part of the steps in the above-described method. Alternatively, the processing component may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described method.

[0167] Storage component 31 is configured to store various types of data to support operations at the terminal. The storage component can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0168] Of course, computing devices may also include other components, such as input / output interfaces, display components, communication components, etc.

[0169] Input / output interfaces provide interfaces between processing components and peripheral interface modules, which can be output devices, input devices, etc.

[0170] The communication components are configured to facilitate wired or wireless communication between computing devices and other devices.

[0171] The computing device can be a physical device or an elastic computing host provided by a cloud computing platform. In this case, the computing device can refer to a cloud server, and the aforementioned processing components, storage components, etc., can be basic server resources rented or purchased from the cloud computing platform.

[0172] This application also provides a computer storage medium storing a computer program, which, when executed by a computer, can perform the above-described functions. Figure 1 The embodiment shown is a method for multi-terminal data synchronization and consistency verification based on message queues.

[0173] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0174] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0175] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0176] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A method for multi-terminal data synchronization and consistency verification based on message queues, characterized in that, include: Receive business data change events from user terminals, merchant terminals, and platform terminals. The business data change events correspond to core business data, which includes stored value balance, order status, and coupon usage records. The business data change event is published to a RabbitMQ message queue based on an event-driven architecture, and then distributed to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals. During the data synchronization process between the multiple terminals, a preset lightweight consistency verification engine is invoked to perform periodic snapshot comparisons on the core business data; When the periodic snapshot comparison detects data inconsistency, an automatic anomaly repair mechanism is triggered to maintain data consistency across multiple devices.

2. The method according to claim 1, characterized in that, The system receives business data change events from user terminals, merchant terminals, and the platform terminal. These business data change events correspond to core business data, which includes stored value balance, order status, and coupon usage records, including: Deploy data monitoring modules in the user interface, the merchant processing system, and the platform management backend; The data monitoring module continuously captures interactive operations and internal system calls occurring on the corresponding terminal. When interactive operations and internal system calls are captured, the data monitoring module identifies and filters out change operations related to core business data. These change operations include user-side prepaid account top-ups or consumption, merchant-side order status updates, and platform-side coupon rule activation or deactivation. Based on the identified change operations, generate business data change events; The corresponding core business data type is determined based on the business data change event.

3. The method according to claim 1, characterized in that, The business data change event is published to a RabbitMQ message queue based on an event-driven architecture. The RabbitMQ message queue then distributes the business data change event to the target terminal to achieve data synchronization between multiple terminals, including: Receive the business data change event, wherein the business data change event carries a type identifier; Based on the type identifier, the business data change event is converted into a target message. The target message includes a message header and a message body. The message header records the sender information, the type identifier, and the event number. In the RabbitMQ message queue, multiple initial message queues are established corresponding to the user end, the merchant end, and the platform end, respectively, and the routing key of each initial message queue is set according to the type identifier; Send the target message to the RabbitMQ message queue exchange; The exchange distributes the target message to the target message queue corresponding to the type identifier according to the type identifier in the message header and the routing key. Each receiving end retrieves and parses the target messages sequentially from its associated target message queue to complete data synchronization.

4. The method according to claim 3, characterized in that, In the RabbitMQ message queue, multiple initial message queues are established, corresponding to the user end, the merchant end, and the platform end respectively. A routing key for each initial message queue is set according to the type identifier, including: The terminal type is extracted from the received business data change event, and the terminal type is any one of user terminal, merchant terminal, and platform terminal; Based on the extracted terminal type, create a first initial message queue, a second initial message queue, and a third initial message queue in the RabbitMQ message queue, corresponding to the user terminal, the merchant terminal, and the platform terminal, respectively. Determine the type identifier carried by the business data change event; Based on the type identifier, a routing key is assigned to each of the first initial message queue, the second initial message queue, and the third initial message queue.

5. The method according to claim 1, characterized in that, During the data synchronization process across multiple devices, a preset lightweight consistency verification engine is invoked to perform periodic snapshot comparisons of the core business data, including: In response to a preset timed trigger command, the comparison process of the lightweight consistency verification engine is initiated; The lightweight consistency verification engine obtains snapshots of the core business data at the same time from the user end, merchant end and platform end respectively, forming first snapshot data, second snapshot data and third snapshot data, wherein each snapshot data includes data identifier, data value and snapshot timestamp; Within the lightweight consistency verification engine, the first snapshot data, the second snapshot data, and the third snapshot data are compared to target entries with the same data identifier to determine whether the data values ​​of the target entries are consistent. When there are inconsistent data values, the corresponding difference information is recorded. The difference information includes the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamp corresponding to each data value.

6. The method according to claim 1, characterized in that, When the periodic snapshot comparison detects data inconsistency, an automatic anomaly repair mechanism is triggered to maintain data consistency across multiple platforms, including: When the aforementioned discrepancy information is recorded, the automatic anomaly repair mechanism is triggered; The automatic anomaly repair mechanism extracts the data identifier that caused the inconsistency, the inconsistent data values ​​obtained from each terminal, and the snapshot timestamps corresponding to each data value from the difference information. Based on the inconsistent data values ​​and the snapshot timestamps corresponding to each data value, a baseline data value is determined; Based on the data identifier and the baseline data value, a repair message instruction is generated; The repair message instruction is published to the RabbitMQ message queue; The repair message instruction is distributed through the RabbitMQ message queue to target terminals whose data values ​​are not the baseline data value among the user terminal, merchant terminal, and platform terminal, so as to update the core business data corresponding to the data identifier stored locally on the target terminal.

7. The method according to claim 6, characterized in that, Based on the data identifier and the baseline data value, a repair message instruction is generated, including: Based on the data identifier and the reference data value, a first operation instruction is generated; Append an operation timestamp to the first operation instruction to generate a timestamp instruction; The data identifier, the reference data value, and the timestamp instruction are combined to generate a first message instruction; Based on the first message instruction, a message header containing an operation type identifier is generated; The message header is combined with the first message to generate a repair message instruction.

8. A multi-terminal data synchronization and consistency verification system based on message queues, characterized in that, include: The receiving module is used to receive business data change events sent from the user terminal, the merchant terminal and the platform terminal. The business data change events correspond to core business data, which includes stored value balance, order status and coupon usage records. The distribution module is used to publish the business data change event to a RabbitMQ message queue based on an event-driven architecture, and distribute the business data change event to the target terminal through the RabbitMQ message queue to achieve data synchronization between multiple terminals; The comparison module is used to call a preset lightweight consistency verification engine to perform periodic snapshot comparisons of the core business data during the data synchronization process between the multiple terminals. The triggering module is used to trigger an automatic anomaly repair mechanism when the periodic snapshot comparison detects data inconsistency, so as to maintain data consistency across multiple terminals.

9. A computing device, characterized in that, It includes a processing component and a storage component; the storage component stores one or more computer instructions; the one or more computer instructions are invoked and executed by the processing component to implement a message queue-based multi-terminal data synchronization and consistency verification method as described in any one of claims 1 to 7.

10. A computer storage medium, characterized in that, The system stores a computer program, which, when executed by a computer, implements a multi-terminal data synchronization and consistency verification method based on a message queue as described in any one of claims 1 to 7.

Images