Order data synchronization method, system and device

By chunking and hash mapping and distributed database construction of order data, combining logical vector clocks and multi-data voting consensus mechanism, the data consistency problem caused by distributed storage in e-commerce systems is solved, and the reliability of the system is improved and performance, adapt to the processing needs of high concurrency and large data volumes.

CN118520045BActive Publication Date: 2024-11-29SHENZHEN BENZHENLIXING TECH SERVICE CO LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
CN202410648831.3
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-05-23
Publication Date
2024-11-29
Estimated Expiration
2044-05-23

AI Technical Summary

Technical Problem

In e-commerce systems, distributed storage of order data leads to data consistency problems. Especially in high concurrency and complex network environments, existing message queues or event-driven synchronization methods are difficult to effectively solve data inconsistency, affecting transactions and logistics processing.

Method used

By blocking and hashing the order data, an ordered sequence of order data blocks and storage node cluster mapping tables are formed, a distributed database is built, and a global orderly maintenance time stamp sequence is recorded using logical vector clocks to perform data consistency detection. And through the multi-data voting consensus mechanism, data consistency decisions are achieved to ensure the consistency and reliability of data among multiple nodes.

Benefits of technology

It realizes efficient and reliable synchronization of order data, improves the fault tolerance and performance of the system, ensures data consistency and integrity, reduces management costs, and adapts to the processing needs of high concurrency and large data volumes.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118520045B_ABST
    Figure CN118520045B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of data synchronization technology, and in particular to a method, system and device for synchronizing order data. The method comprises the following steps: real-time order collection of an e-commerce system to obtain original order data; block and hash map the original order data to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one-to-one, and each order data block corresponds to a unique storage node; a distributed database is constructed based on the ordered order data block sequence and the storage node cluster mapping table to obtain a distributed database; and a logical vector clock record is maintained for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence. The present invention effectively solves the consistency problem in distributed order data synchronization, improves the reliability, performance and fault tolerance of the system, and simplifies data management.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data synchronization, and in particular to a method, system and device for synchronizing order data. Background Art

[0002] In modern e-commerce systems, order data is the most core and critical part of the data. Order data records the detailed information of the goods or services purchased by customers, such as product name, quantity, amount, and delivery address, and is the basis for processing transactions and logistics. Due to the high concurrency and high complexity of e-commerce business, the order system often needs to adopt a distributed architecture to store order data in different data centers or databases to improve the scalability and fault tolerance of the system.

[0003] However, distributed storage of order data brings challenges to data consistency. Due to network delays and hardware failures, order data on different nodes may differ, resulting in data inconsistency. Inconsistent data will have a serious impact on subsequent order processing and product delivery. Therefore, an efficient and reliable order data synchronization method is needed to synchronize data from each node in a timely manner to ensure the final consistency of the data.

[0004] In addition, some synchronization methods based on message queues or event-driven have also been tried. These methods use message passing mechanisms to transmit order data update messages between systems to achieve real-time synchronization. However, these methods face performance and latency challenges when processing large amounts of order data, especially under high load and complex network environments. Summary of the invention

[0005] Based on this, it is necessary for the present invention to provide a method, system and device for synchronizing order data to solve at least one of the above technical problems.

[0006] To achieve the above purpose, a method for synchronizing order data includes the following steps:

[0007] Step S1: collect orders from the e-commerce system in real time to obtain original order data; divide the original order data into blocks and perform hash mapping to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node;

[0008] Step S2: construct a distributed database based on the ordered order data block sequence and the storage node cluster mapping table to obtain a distributed database; maintain the logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result;

[0009] Step S3: Based on the data inconsistency detection results, a multi-data voting consensus is reached on the storage node cluster mapping table to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection results include the storage nodes with inconsistent order data and their corresponding event divergence point data.

[0010] The present invention accurately locates order data to specific storage nodes through order data block and hash mapping, forms an ordered order data block sequence and a storage node cluster mapping table, realizes accurate data management and allocation, and makes the allocation, storage and synchronization of data blocks more orderly and efficient. By maintaining the logical vector clock record and maintaining the global orderly maintenance timestamp sequence, the data consistency of each storage node is monitored and detected in real time, and the problem of data inconsistency is effectively identified and located to ensure timely correction. By adopting a distributed architecture, the order data is dispersed and stored in multiple nodes. Even if a node fails, the data of other nodes can still ensure the overall operation of the system, and improve the fault tolerance and stability of the system. Based on the data inconsistency detection result, a multi-data voting consensus mechanism is used to realize the consistency decision of the storage node cluster, ensure the consistency and reliability of the final data, and effectively operate even in a complex network environment. By adopting a distributed database construction, the method of allocating ordered data blocks and maintaining the timestamp sequence can effectively improve the scalability and performance of the system and adapt to the processing requirements of high concurrency and large data volume. By maintaining the use of logical vector clock records and the global orderly maintenance timestamp sequence, the detection and processing of data inconsistency are simpler and more intuitive, and the management and maintenance costs of the system are reduced. The application of the multi-data voting consensus mechanism ensures the generation and write-back of consistent order data through the collaborative work of multiple nodes and joint decision-making, which greatly improves the reliability and consistency of system data. The detailed analysis and processing of data inconsistency detection results can timely discover and correct data anomalies, avoid problems in subsequent transactions and logistics caused by data inconsistencies, and ensure the overall stable operation of the e-commerce system. In summary, the present invention effectively solves the consistency problem in distributed order data synchronization, improves the reliability, performance and fault tolerance of the system, and simplifies data management.

[0011] Preferably, the present invention further provides an order data synchronization system for executing the order data synchronization method as described above, the order data synchronization system comprising:

[0012] The order data collection module is used to collect orders from the e-commerce system in real time and obtain the original order data; the original order data is divided into blocks and hash mapped to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node;

[0013] A distributed database construction module is used to construct a distributed database based on an ordered order data block sequence and a storage node cluster mapping table to obtain a distributed database; maintain a logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result;

[0014] The consistency data processing module is used to reach a multi-data voting consensus on the storage node cluster mapping table based on the data inconsistency detection results to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection results include the storage nodes with inconsistent order data and their corresponding event divergence point data.

[0015] Preferably, the present invention also provides an order data synchronization device, comprising:

[0016] Memory for storing computer programs;

[0017] A processor is used to implement any of the above-mentioned order data synchronization methods when executing the computer program.

[0018] In the present invention, the order data acquisition module is used to realize the real-time collection of order data of the e-commerce system, ensuring the timely acquisition and processing of order data. By dividing the original order data into blocks and hash mapping, the data is accurately allocated to specific storage nodes, and the order data block sequence and the storage node cluster mapping table are managed in an orderly manner, thereby improving the systematicness and efficiency of data management. An efficient distributed database is constructed based on the ordered order data block sequence and the storage node cluster mapping table through the distributed database construction module, which can effectively cope with the high concurrency and large data volume processing requirements of the e-commerce system. By maintaining the logical vector clock record and the global orderly maintenance of the timestamp sequence, the consistency detection and maintenance of the data are realized, ensuring that the system still maintains high performance and high efficiency under a high load environment. By maintaining the timestamp sequence in a global orderly manner, the problem of data inconsistency can be discovered and located in time, ensuring the data consistency in the system. By adopting a distributed architecture, the order data is dispersedly stored in multiple nodes, and even if an individual node fails, it will not affect the overall operation of the system, thereby improving the fault tolerance and reliability of the system. By the consistency data processing module, the consistency order data is written back to the original storage node, ensuring the consistency and integrity of the data, and can effectively operate even in a complex network environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments thereof made with reference to the following drawings:

[0020] Figure 1 A schematic flow chart of the steps of a method for synchronizing order data according to an embodiment is shown.

[0021] Figure 2 A detailed flow chart of step S3 of an embodiment is shown.

[0022] Figure 3 A detailed flow chart of step S32 of an embodiment is shown. DETAILED DESCRIPTION

[0023] The following is a clear and complete description of the technical method of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0024] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0025] It should be understood that, although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another unit. For example, without departing from the scope of the exemplary embodiments, the first unit may be referred to as the second unit, and similarly the second unit may be referred to as the first unit. The term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0026] To achieve this, please refer to Figures 1 to 3 The present invention provides a method for synchronizing order data, comprising the following steps:

[0027] Step S1: collect orders from the e-commerce system in real time to obtain original order data; divide the original order data into blocks and perform hash mapping to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node;

[0028] Step S2: construct a distributed database based on the ordered order data block sequence and the storage node cluster mapping table to obtain a distributed database; maintain the logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result;

[0029] Step S3: Based on the data inconsistency detection results, a multi-data voting consensus is reached on the storage node cluster mapping table to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection results include the storage nodes with inconsistent order data and their corresponding event divergence point data.

[0030] The present invention accurately locates order data to specific storage nodes through order data block and hash mapping, forms an ordered order data block sequence and a storage node cluster mapping table, realizes accurate data management and allocation, and makes the allocation, storage and synchronization of data blocks more orderly and efficient. By maintaining the logical vector clock record and maintaining the global orderly maintenance timestamp sequence, the data consistency of each storage node is monitored and detected in real time, and the problem of data inconsistency is effectively identified and located to ensure timely correction. By adopting a distributed architecture, the order data is dispersed and stored in multiple nodes. Even if a node fails, the data of other nodes can still ensure the overall operation of the system, and improve the fault tolerance and stability of the system. Based on the data inconsistency detection result, a multi-data voting consensus mechanism is used to realize the consistency decision of the storage node cluster, ensure the consistency and reliability of the final data, and effectively operate even in a complex network environment. By adopting a distributed database construction, the method of allocating ordered data blocks and maintaining the timestamp sequence can effectively improve the scalability and performance of the system and adapt to the processing requirements of high concurrency and large data volume. By maintaining the use of logical vector clock records and the global orderly maintenance timestamp sequence, the detection and processing of data inconsistency are simpler and more intuitive, and the management and maintenance costs of the system are reduced. The application of the multi-data voting consensus mechanism ensures the generation and write-back of consistent order data through the collaborative work of multiple nodes and joint decision-making, which greatly improves the reliability and consistency of system data. The detailed analysis and processing of data inconsistency detection results can timely discover and correct data anomalies, avoid problems in subsequent transactions and logistics caused by data inconsistencies, and ensure the overall stable operation of the e-commerce system. In summary, the present invention effectively solves the consistency problem in distributed order data synchronization, improves the reliability, performance and fault tolerance of the system, and simplifies data management.

[0031] In this embodiment, the original order data is collected from the e-commerce system in real time, and is divided into blocks and hash mapped to create an ordered sequence of order data blocks and a storage node cluster mapping table. Each order data block will be uniquely mapped to a storage node, ensuring balanced distribution and efficient access to the data. For example, if there are five orders, after mapping through the hash algorithm, order 1 and order 2 are mapped to node A, order 3 is mapped to node B, and order 4 and order 5 are mapped to node C. A distributed database is constructed using an ordered sequence of order data blocks and a storage node cluster mapping table, and a logical vector clock is recorded for each storage node to generate a globally ordered maintenance timestamp sequence. These timestamps are used for data consistency detection to ensure that the order data on each storage node is globally ordered. For example, the timestamp sequence recorded by node A is [1, 2], node B is [3], and node C is [4, 5]. In this step, data inconsistency is detected, such as the order 2 data on node A conflicts with other nodes. According to the data inconsistency detection results, a multi-data voting consensus is reached on the storage node cluster mapping table to obtain consistent order data. For example, if the order 2 data on node A is found to be inconsistent, a vote consensus will be made to determine the correct data version. This version will be updated to all nodes to ensure that the order data on each node is consistent.

[0032] Preferably, step S1 comprises the following steps:

[0033] Step S11: deploying order event probes on the e-commerce system to obtain an order event monitoring agent;

[0034] Specifically, Apache Kafka is used as the order event probe software. Kafka is configured to set topics and subscribed event types to ensure that it can capture order events in the system, such as order creation, payment, and shipment. The configured Kafka is deployed to key nodes of the e-commerce system, such as order processing servers and payment servers, to obtain order event monitoring agents.

[0035] Step S12: using an order event monitoring agent to capture order transaction data streams of the e-commerce system in real time to obtain original order data;

[0036] Specifically, the original order data captured by the order event probe is transmitted to the configured Kafka monitoring agent through the network. This involves sending the data to a specified topic in Kafka. After receiving the order event data, the Kafka monitoring agent stores the original order data in an appropriate location, such as a database or file system.

[0037] Step S13: normalizing the format of the original order data to obtain standardized order data;

[0038] Specifically, the pandas library is used to process the original order data, including but not limited to removing duplicate data, filling missing values, and unifying date format operations, so as to obtain standardized order data.

[0039] Step S14: Divide the standard order data into blocks to obtain an ordered sequence of order data blocks;

[0040] Specifically, a strategy is selected to divide the order data into blocks by month according to the order creation time. According to the determined strategy, the normalized order data is divided into blocks by month to form order data blocks for multiple months. The order data blocks for each month are arranged in chronological order to form an ordered sequence of order data blocks.

[0041] Step S15: Based on the ordered order data block sequence, each order data block is mapped to a storage node through a ring virtual node mapping to obtain a storage node cluster mapping table, wherein the ordered order data block sequence corresponds one-to-one to the storage node cluster mapping table, and each order data block corresponds to a unique storage node.

[0042] Specifically, the consistent hashing algorithm is used to perform storage node mapping operations on each order data block, mapping it to the corresponding node in the ring virtual node space. According to the mapping relationship between the order data block and the storage node, a storage node cluster mapping table is generated to ensure that the mapping relationship between each order data block and its corresponding storage node is correct.

[0043] The present invention can monitor order events in an e-commerce system in real time through the deployment of order event probes, and ensure the real-time and accuracy of order data. The order transaction data stream is captured in real time by an order event monitoring agent, and the original order data is obtained in time to ensure the integrity and timeliness of the data, which helps to improve the order processing efficiency. The original order data is formatted and processed to ensure the consistency and standardization of the data, improve the data quality, reduce errors and inconsistencies in subsequent data processing, and improve the data processing efficiency and reliability of the entire system. The standardized order data is divided into blocks to generate an ordered sequence of order data blocks, making data management more systematic and orderly. Block processing helps to split large data sets into smaller parts for easy storage and management. Based on the ordered sequence of order data blocks, each order data block is mapped to a storage node through a ring virtual node mapping to form a storage node cluster mapping table. This mapping method optimizes data distribution and storage, and improves data retrieval and access efficiency.

[0044] Preferably, step S15 comprises the following steps:

[0045] Step S151: constructing an N-dimensional ring virtual node space for the e-commerce system to obtain an N-dimensional ring virtual node space, wherein the physical storage nodes in the N-dimensional ring virtual node space are mapped to the N-dimensional ring according to the consistent hashing algorithm, and each physical storage node has M virtual nodes evenly distributed on the ring;

[0046] Specifically, select the consistent hashing algorithm libconsistent, and determine the dimension of the N-dimensional ring according to the system requirements and performance requirements. The choice of dimension should take into account the data scale, load and performance requirements of the system. Assume that a 3-dimensional ring is selected as an example. Use the consistent hashing algorithm libconsistent to construct a 3-dimensional ring virtual node space. Map the physical storage nodes to the 3-dimensional ring according to the hash algorithm. Each physical storage node has M virtual nodes evenly distributed on the ring. For example, if there are 10 physical storage nodes and each node has 3 virtual nodes evenly distributed, there will be a total of 30 virtual nodes distributed on the 3-dimensional ring. Verify the uniformity of the distribution of nodes in the constructed 3-dimensional ring virtual node space. Ensure that the number of virtual nodes on the ring for each physical storage node is uniform and the distance distribution between nodes is reasonable. Configure and deploy the constructed 3-dimensional ring virtual node space into the e-commerce system. Ensure that the system can correctly use this virtual node space for data storage and access.

[0047] Step S152: using a fractal hash coding algorithm to encode each order data block in the ordered order data block sequence to obtain a number of order N-dimensional hash vectors;

[0048] Specifically, MurmurHash3 is selected as the implementation of the fractal hash coding algorithm. MurmurHash3 is a fast and efficient hash algorithm suitable for hash coding order data. The MurmurHash3 algorithm is applied to each order data block to obtain the corresponding order N-dimensional hash vector.

[0049] Step S153: performing distance measurement on each order's N-dimensional hash vector based on the N-dimensional ring virtual node space to obtain the nearest virtual node distance data;

[0050] Specifically, Euclidean distance is selected as the distance measurement method. Euclidean distance is a common distance measurement method, which is suitable for calculating the distance between vectors. Based on the N-dimensional ring virtual node space, the Euclidean distance measurement operation is performed on each order N-dimensional hash vector. The distance between each order N-dimensional hash vector and the virtual node on the ring is calculated to obtain the distance data of the nearest virtual node.

[0051] Step S154: Map the virtual node corresponding to the nearest virtual node distance data back to the physical storage node, and record the order data block and the corresponding physical storage node in the storage node cluster mapping table;

[0052] Specifically, the consistent hashing algorithm libconsistent is selected as a tool to map virtual nodes back to physical storage nodes. Based on the nearest virtual node distance data, the consistent hashing algorithm is used to map each virtual node back to the corresponding physical storage node. This step ensures the uniform distribution and load balancing of order data blocks in the storage node cluster. Each order data block and its corresponding physical storage node are recorded in the storage node cluster mapping table. This table records the mapping relationship between the ordered order data block sequence and the storage node.

[0053] Step S155: Repeat steps S153 and S154 until all order data blocks have completed node mapping, that is, a storage node cluster mapping table is obtained, in which the ordered order data block sequence corresponds one-to-one to the storage node cluster mapping table, and each order data block corresponds to a unique storage node.

[0054] Specifically, steps S153 and S154 are repeated until all order data blocks have completed the node mapping operation. This ensures that each order data block of the entire data set is correctly mapped to the corresponding physical storage node. When all order data blocks have completed the node mapping, the final storage node cluster mapping table is generated. This table records the mapping relationship between the ordered order data block sequence and the storage node, and each order data block corresponds to a unique storage node.

[0055] The present invention constructs an N-dimensional annular virtual node space, and uses a consistent hashing algorithm to map physical storage nodes to an N-dimensional ring, and each physical storage node has multiple virtual nodes evenly distributed on the ring. This uniform distribution method effectively avoids data hotspot problems, achieves load balancing, and improves system performance and reliability. The fractal hash coding algorithm is used to encode the ordered order data blocks, generate the order N-dimensional hash vector, ensure that the data block hash value has good distribution characteristics, and improve the accuracy and efficiency of data mapping. Through the N-dimensional annular virtual node space, the order N-dimensional hash vector is measured for distance, the nearest virtual node distance data is obtained, the virtual node closest to the data block is accurately found, the data block storage position is optimized, and the storage space utilization rate is improved. By mapping the virtual node corresponding to the nearest virtual node distance data back to the physical storage node, and recording it in the storage node cluster mapping table. The mapping relationship between virtual nodes and physical nodes enables the system to only adjust the mapping of a small number of data blocks when adding or removing physical storage nodes, greatly improving the system scalability and fault tolerance. Accurate virtual node mapping and distance measurement methods ensure that each order data block is mapped to the most appropriate physical storage node, reducing inconsistencies in data transmission and storage, and improving data consistency and reliability. Ensure that all order data blocks complete node mapping, build a complete storage node cluster mapping table, achieve a unique correspondence between data blocks and physical storage nodes, and ensure data consistency. By adopting N-dimensional ring virtual node space and consistent hashing algorithm, the mapping and storage process of data blocks is simplified, the computational complexity is reduced, and the system processing speed and efficiency are improved.

[0056] Preferably, step S153 includes the following steps:

[0057] Step S1531: Perform non-Euclidean geometric space mapping on the N-dimensional ring-shaped virtual node space to obtain an N-dimensional non-Euclidean space;

[0058] Specifically, a non-Euclidean geometric space mapping method is selected, such as the Locally Linear Embedding (LLE) algorithm in manifold learning, and the N-dimensional annular virtual node space is mapped to a non-Euclidean geometric space using the LLE algorithm. This step maps the nodes in the annular virtual node space to a low-dimensional non-Euclidean geometric space.

[0059] Step S1532: Perform self-similarity measurement analysis on the order N-dimensional hash vector to obtain the fractal dimension of the order hash vector;

[0060] Specifically, a fractal dimension calculation algorithm is used to perform a self-similarity measurement analysis on each order N-dimensional hash vector to obtain the fractal dimension of the order hash vector. This step calculates the fractal dimension of each order hash vector and describes its self-similarity or complexity.

[0061] Step S1533: performing fractal similarity measurement on each virtual node in the N-dimensional non-Euclidean space and the fractal dimension of the order hash vector to obtain a number of node-order fractal distance values;

[0062] Specifically, the similarity between each virtual node and the fractal dimension of the order hash vector is calculated in the N-dimensional non-Euclidean space using the Euclidean distance or correlation coefficient. This step calculates the fractal similarity between each virtual node and the fractal dimension of the order hash vector, and obtains several node-order fractal distance values.

[0063] Step S1534: Perform nearest matching on the fractal dimension of the order hash vector according to a number of node-order fractal distance values ​​to obtain the nearest virtual node distance data.

[0064] Specifically, for each order hash vector, find the virtual node closest to it according to its fractal dimension. Traverse all virtual nodes, calculate the distance between them and the fractal dimension of the order hash vector, and record the virtual node corresponding to the minimum distance. Record the distance value between each order hash vector and its nearest virtual node to form the nearest virtual node distance data.

[0065] The present invention can maintain efficient mapping and storage capabilities and improve the scalability of the system when facing complex data structures through non-Euclidean geometric space mapping and fractal similarity measurement. The fractal dimension is used to map and match order data blocks, which improves the fault tolerance of the system when storage nodes fail or increase, and enables the system to dynamically adjust the distribution of storage nodes and maintain efficient operation. Non-Euclidean geometric space mapping and self-similar measurement analysis methods improve the distribution uniformity and matching accuracy of data blocks, reduce data access conflicts and delays, and improve data storage and access efficiency. Fractal similarity measurement and nearest matching methods ensure that data blocks are stored in the most suitable nodes, optimize data storage structures, and improve the overall efficiency of the system. Through accurate mapping of order data blocks to storage nodes, data consistency and reliability are improved, and system errors and failures caused by data inconsistency are reduced. Ensure that data blocks are stored on the most suitable nodes, reduce inconsistencies in data migration and replication processes, and improve the reliability of data storage. Through automated non-Euclidean geometric space mapping and fractal similarity measurement methods, manual intervention and management costs are reduced. The establishment of the storage node cluster mapping table makes the correspondence between data blocks and physical storage nodes clearer, facilitates system monitoring and maintenance, and improves the convenience and effectiveness of management.

[0066] Preferably, step S2 comprises the following steps:

[0067] Step S21: Based on the ordered order data block sequence and the storage node cluster mapping table, a Gossip database instance is deployed on each storage node in the storage node cluster to obtain a distributed database;

[0068] Specifically, select a Gossip database suitable for a distributed environment, such as Apache Cassandra or ScyllaDB. Use the storage node cluster mapping table to determine the number and location of database instances on each storage node. For each storage node in the mapping table, deploy the corresponding number of Gossip database instances. Use automated deployment tools or manual configuration to deploy the selected Gossip database instance on each storage node. Ensure that each database instance is correctly configured and started and has joined the database cluster. Verify that all nodes in the database cluster have been successfully joined and that the cluster status is normal. Through the above steps, a distributed database is obtained.

[0069] Step S22: Initialize the vector clock of each Gossip database instance in the distributed database to obtain the node initial vector clock state data;

[0070] Specifically, in each Gossip database instance, the vector clock initialization operation is performed, and the initial value is set to an appropriate value, such as 0 or the current timestamp, so as to obtain the node initial vector clock state data.

[0071] Step S23: Based on the node initial vector clock state data, the vector clock coordinator in each Gossip database instance in the distributed database is used to exchange anti-entropy states with the storage nodes in the storage node cluster in a preset period to obtain node update vector clock state data, wherein, in the anti-entropy state exchange process, after receiving the incremental state of other storage nodes, the incremental state is merged according to the causal order and the node number to update the local vector clock;

[0072] Specifically, in each Gossip database instance, a scheduled task is set to trigger periodic state exchange. Determine the state exchange period and set the corresponding scheduled task. In each period, the vector clock coordinator of each node communicates with other nodes in the storage node cluster to perform state exchange operations. When receiving the incremental state of other nodes, the incremental state is merged in causal order and node number increment, and the local vector clock is updated. The vector clock state of the local node is updated according to the merged incremental state, so as to obtain the node update vector clock state data.

[0073] Step S24: periodically archiving the node update vector clock state data of each storage node according to a preset archiving period, generating an ordered vector clock tuple, and broadcasting it to the entire storage node cluster to obtain ordered vector clock tuple data;

[0074] Specifically, a preset archiving cycle is set to determine how often the node update vector clock status data of the storage node is archived. At the end of each archiving cycle, the node update vector clock status data of each storage node is triggered to archive. Each node in the storage node cluster is traversed, its updated vector clock status data is collected, and an ordered vector clock tuple is generated. The generated ordered vector clock tuple data is broadcast to the entire storage node cluster to ensure that all nodes can obtain the latest archived data.

[0075] Step S25: Summarize the ordered vector clock tuple data of each storage node in the storage node cluster to obtain a global ordered maintenance timestamp sequence, wherein the global ordered maintenance timestamp sequence includes a plurality of ordered vector clock tuple data;

[0076] Specifically, on a node in the storage node cluster or a dedicated management node, the ordered vector clock tuple data of each storage node in the storage node cluster is collected, and the collected data is aggregated to generate a global ordered maintenance timestamp sequence.

[0077] Step S26: Perform data consistency check on the order data blocks stored in each storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result.

[0078] Specifically, according to the global ordered maintenance timestamp sequence, the order data blocks stored in the storage node are checked for data consistency. The order relationship between the timestamp of each order data block and the global ordered maintenance timestamp sequence is checked to ensure data consistency. If data inconsistency is found, a data inconsistency detection result is generated and the corresponding data block is marked for subsequent repair or processing.

[0079] The present invention can dynamically expand and accommodate new storage nodes through the Gossip protocol and vector clock mechanism without affecting the data consistency and synchronization of existing nodes. The anti-entropy exchange and periodic archiving of node status enable the system to have a high fault tolerance capability and maintain data consistency in the event of node failure or network partition. The deployment of the Gossip database instance and the use of the vector clock improve the data propagation speed and processing efficiency of the system, reduce the delay of data processing, and improve the overall performance of the system. Periodic archiving and broadcasting of global timestamps help to update and maintain data status in a timely manner and optimize the efficiency and accuracy of data processing. Automated vector clock initialization, state exchange, archiving and consistency detection reduce manual intervention and management costs. Data synchronization and consistency maintenance can be completed autonomously, improving the convenience and effectiveness of management. The generation and use of the global orderly maintenance timestamp sequence can intuitively understand the state changes of each node and simplify the monitoring and maintenance process of the system.

[0080] Preferably, step S26 comprises the following steps:

[0081] Step S261: construct a topological model of the global order data event based on the global ordered maintenance timestamp sequence to obtain a global ordered event graph, in which each ordered vector clock tuple data corresponds to an event node, and the dependency between nodes is determined by the causal order and the node number;

[0082] Specifically, the topological modeling of global order data events is constructed based on the globally ordered maintenance timestamp sequence. Each ordered vector clock tuple data corresponds to an event node, and the dependency between nodes is determined by the causal order and the node number. For example, if the timestamp of event A is earlier than that of event B, then there is a directed edge from event A to event B. The constructed event nodes and the dependencies between them are integrated into a globally ordered event graph. Ensure that the globally ordered event graph can accurately reflect the order relationship and dependency relationship between order data events.

[0083] Step S262: extracting a local order data block sequence from each storage node based on the storage node cluster to obtain a local order data block sequence;

[0084] Specifically, on each storage node, the order data block sequence is extracted from the local storage. The order data block sequence should be extracted according to the order in the storage node cluster mapping table, so as to obtain the local order data block sequence.

[0085] Step S263: Perform local event topology modeling on the corresponding storage node according to the local order data block sequence to obtain a local ordered event graph;

[0086] Specifically, for example, assume that there are two storage nodes, node A and node B. Node A contains a sequence of local order data blocks [A1, A2, A3], and node B contains a sequence of local order data blocks [B1, B2, B3]. Each order data block is regarded as an event node: A1, A2, A3. According to the order relationship between the data blocks, the dependency relationship between the events is established: A1->A2->A3. This means that after the order data block A1 occurs, A2 can occur; after A2 occurs, A3 can occur. Construct a local ordered event graph: The local ordered event graph on node A contains event nodes A1, A2, A3, and the dependency relationship between them. Similarly, each order data block is regarded as an event node: B1, B2, B3. According to the order relationship between the data blocks, the dependency relationship between the events is established: B1->B2->B3. Construct a local ordered event graph: The local ordered event graph on node B contains event nodes B1, B2, B3, and the dependency relationship between them. Through local event topology modeling, each storage node generates a local ordered event graph that reflects the sequential relationship and dependency relationship of the order data.

[0087] Step S264: performing topological isomorphism detection on the local ordered event graph and the global ordered event graph to obtain consistency detection result data;

[0088] Specifically, for example, the local ordered event graphs on nodes A and B are compared with the global ordered event graphs to check whether their event nodes and dependencies match. If there is a consistent topological structure between the two, the consistency test result is consistent.

[0089] Step S265: If the consistency detection result data shows that there is an inequality relationship between the local ordered event graph and the global ordered event graph, then the vector clock state of the local ordered event graph is backtracked based on the global ordered maintenance timestamp sequence to obtain event divergence point data;

[0090] Specifically, for example, assume that there is an inequality between the local ordered event graphs on nodes A and B and the global ordered event graph, indicating that there are some events that exist in the local graph but not in the global graph, or vice versa. Perform vector clock state backtracking on these inequality events to find the event divergence point data.

[0091] Step S266: construct a storage node mapping for the event divergence point data to obtain a data inconsistency detection result, wherein the data inconsistency detection result includes the storage node with inconsistent order data and its corresponding event divergence point data.

[0092] Specifically, for example, if an event divergence point is found in the local event graph on node A, it means that the order data on node A is inconsistent with the global event graph. Therefore, node A is marked as a storage node containing inconsistent order data, and the event divergence point data is recorded. Through the operations of topological isomorphism detection, event graph inequivalence relationship detection, and storage node mapping construction, the data inconsistency detection result can be obtained, that is, which storage nodes contain inconsistent order data and their corresponding event divergence point data.

[0093] The present invention can clearly understand the event sequence and dependency of order data by establishing a global ordered event map. This helps the system to manage and maintain order data more effectively and improve the overall data management capability of the system. By performing local event topology modeling on the order data on each storage node, a local ordered event map is formed. This can monitor and grasp the order data status on each storage node in real time and promptly discover data inconsistencies. By performing topological isomorphism detection on the local ordered event map and the global ordered event map to discover data consistency problems. This can promptly identify and solve data inconsistencies and ensure the consistency and accuracy of order data in the system. If the consistency test result data shows that there is an inequality relationship between the local ordered event map and the global ordered event map, the event divergence point data is obtained by backtracking the vector clock state. This helps to determine the cause and occurrence time of data inconsistency and provides important clues for subsequent data consistency repair. By performing storage node mapping construction on the event divergence point data, the specific location and cause of data inconsistency are further determined. This helps to locate and repair data inconsistency problems more quickly and improve the data management efficiency and reliability of the system.

[0094] Preferably, step S3 comprises the following steps:

[0095] Step S31: Based on the data inconsistency detection result, a fluctuation map is constructed for the storage nodes with inconsistent order data to obtain a divergent fluctuation map, wherein the nodes in the divergent fluctuation map represent storage nodes, the edges represent causal dependencies, and the inconsistent storage nodes construct branches according to the corresponding event divergence point data;

[0096] Specifically, for example, assume that storage node A and storage node B contain inconsistent order data. In the fluctuation graph, nodes A and B are created, and branches are constructed based on the event divergence point data. The edge between node A and node B represents the causal dependency between them. For example, event A1 and event A2 represent two inconsistent events on storage node A, while event B1 and event B2 represent two corresponding events on storage node B. By connecting these events, a fluctuation graph is constructed to represent the data inconsistency between storage nodes.

[0097] Step S32: Calculate the expected entropy of each branch in the divergence fluctuation map to obtain a number of branch expected entropy data;

[0098] Specifically, for example, suppose there is a branch that contains an event sequence between storage node A and storage node B. The expected entropy can be calculated by analyzing the relationship and degree of dependence between these events. The expected entropy reflects the complexity or uncertainty of the event sequence between storage nodes. If the expected entropy of a branch is high, it means that there are more inconsistencies in the event sequence, that is, the data between the storage nodes is inconsistent or there are more disagreements; on the contrary, if the expected entropy is low, it means that the event sequence between the nodes is more consistent or stable, and the data between the nodes is consistent.

[0099] Step S33: identifying the minimum expected entropy path of the divergence fluctuation graph according to the expected entropy data of several branches, and obtaining the order consistent path milestone branch, wherein the order consistent path milestone branch corresponds to the ordered event sequence that most nodes in the storage node cluster agree on;

[0100] Specifically, for example, based on several branch expected entropy data and their corresponding path branches, the branch with the lowest expected entropy is selected as the order consistent path milestone branch. The event sequence corresponding to this branch is widely recognized in the storage node cluster because most nodes participate in the formation of this sequence, making this branch represent the highest consistency level. For example, suppose there are three branches with expected entropies of 0.1, 0.3, and 0.2, respectively. In this case, the branch with an expected entropy of 0.1 is selected as the order consistent path milestone branch because it has the lowest expected entropy and represents the highest consistency level in the storage node cluster.

[0101] Step S34: performing consensus convergence modeling on the divergence fluctuation graph to obtain an order consensus pruner; using the order consensus pruner to perform abnormal branch scanning and identification on the divergence fluctuation graph to obtain an abnormal branch graph of storage nodes;

[0102] Specifically, for example, the Byzantine fault-tolerant algorithm is selected as the consensus algorithm, and a pruning strategy is designed to require that the event sequence of any storage node must be consistent with the event sequence of most nodes, and a consensus pruner is implemented that can scan the divergence fluctuation graph and identify all abnormal branches that do not meet the consistency standard. For example, if an event sequence is found in the divergence fluctuation graph, in which the behavior of nodes A and B is inconsistent with other nodes, then this event sequence will be identified as an abnormal branch.

[0103] Step S35: using the order consensus pruner to perform abnormal path pruning and implantation operations on the divergent fluctuation graph according to the storage node abnormal branch graph and the order consistent path milestone branch, and obtaining the storage node normalized graph;

[0104] Specifically, for example, suppose there is an abnormal branch in the storage node abnormal branch graph that is inconsistent with the order consistent path milestone branch. Use the order consensus pruner to prune this abnormal branch. Then, the order consistent path milestone branch is implanted into the position of the pruned abnormal branch to replace the original inconsistent branch. Finally, the corrected storage node graph can reflect the consistency of the data in the storage node cluster.

[0105] Step S36: construct a consensus topology based on the normalized graph of storage nodes to obtain an order consistency event chain;

[0106] Specifically, for example, a consensus topology is constructed based on the event sequence in the normalized graph of storage nodes. The event sequence in the graph is traversed, and an event sequence that all nodes can agree on is determined based on the protocol and consensus mechanism between the nodes. When building the consensus topology, ensure that all nodes can recognize the final event sequence to ensure the consistency of the order data. After the consensus topology is built, an order consistency event chain is obtained, which is an ordered event sequence recognized by all nodes in the storage node cluster. For example, assume that there are three event sequences in the normalized graph of storage nodes: [A, B, C], [A, B, D], and [A, E, F]. Through the consensus topology construction, the event sequence [A, B, D] is determined because all nodes recognize this sequence. Therefore, [A, B, D] becomes part of the order consistency event chain. The reasoning process is as follows: In the normalized graph of storage nodes, there are three event sequences: [A, B, C], [A, B, D], and [A, E, F], corresponding to nodes 1, 2, and 3, respectively. First, identify the common prefixes of all event sequences as [A] and [A, B]. Analyze the recognition of each node, node 1 recognizes [A, B, C], node 2 recognizes [A, B, D], and node 3 recognizes [A, E, F]. In order to reach a consensus, it is necessary to find an event sequence that is recognized by the majority of nodes (more than half). After [A, B], the event sequence branches into [C], [D], and [E, F]. If [A, B, C] is to be recognized by the majority of nodes, more than half of the nodes must contain this sequence. However, only node 1 contains [A, B, C], while nodes 2 and 3 do not contain [C], so [A, B, C] is not a sequence recognized by the majority of nodes. On the contrary, [A, B, D] appears in node 2, and node 1 and node 2 agree on the prefix [A, B]. Although the complete sequence of node 1 is [A, B, C], the sequence of [A, B, D] contains [D] recognized by more nodes. Assume that a sequence must be selected to reach global consensus. Although [A, B, C] exists in node 1, it does not exist in node 2 and node 3. [A, B, D] exists in node 2, and node 1 and node 3 must choose a branch that is closer to its sequence to reach consensus. Node 3's [A, E, F] is completely different from the sequences of the other two nodes, so it must choose a sequence that has a common part with other nodes to reach consensus. Therefore, through the consensus mechanism and the analysis of node recognition, [A, B, D] is confirmed by most nodes and becomes part of the order consistency event chain. This explains why [A, B, D] is the path confirmed by most nodes, while [A, B, C] cannot become part of the consistency event chain because it does not appear in more than half of the nodes.

[0107] Step S37: Decode and restore the order consistency event chain according to the storage node mapping table to obtain a consistent order data block sequence, and write the consistent order data block sequence back to the original storage node cluster to complete the synchronization and obtain the synchronized total order data.

[0108] Specifically, for example, according to the storage node mapping table, the order consistency event chain is decoded and restored. This means the process of converting the event sequence back to the order data block sequence. After decoding and restoration, a consistent order data block sequence is obtained. Next, the consistent order data block sequence is written back to the corresponding node in the original storage node cluster to complete the data synchronization. By writing the consistent order data block sequence back to the storage node cluster, the entire storage node cluster obtains the synchronized total order data. For example, suppose the storage node mapping table indicates that the storage node corresponding to a certain event sequence [A, B, D] is node X. The event sequence is restored to an order data block sequence, and these data blocks are written back to storage node X to complete the synchronization.

[0109] The present invention constructs a divergence fluctuation map based on the data inconsistency detection results. This fluctuation map clearly shows the causal dependency between storage nodes and the branching of inconsistent storage nodes. This can accurately locate and describe the source and scope of data inconsistency. By calculating the expected entropy for each branch in the divergence fluctuation map, several branch expected entropy data are obtained. This helps to evaluate the importance and impact of inconsistent branches and provides an important reference for subsequent repairs. The order consistency path milestone branch is identified by using the expected entropy data. The order consistency path milestone branch represents an ordered sequence of events that most nodes in the storage node cluster agree on, providing the direction and path for subsequent consistent data repair. By performing consensus convergence modeling, an order consensus pruner is obtained. This pruner can identify and mark the abnormal branch graph of the storage node, which helps to further analyze and handle abnormal situations. The order consensus pruner prunes the divergence fluctuation map by abnormal paths and implants operations to obtain a normalized map of the storage node. This ensures the effectiveness and accuracy of the data repair process and provides a guarantee for data consistency. By constructing a consensus topology based on the normalized map of the storage node, an order consistency event chain is obtained. This consistency event chain represents the entire process of order data repair, ensuring the smooth progress of the repair process and the ultimate realization of data consistency. By decoding and restoring the order consistency event chain according to the storage node mapping table, a consistent order data block sequence is obtained and written back to the original storage node cluster to complete the synchronization. This ensures the consistency and integrity of the order data and improves the data management efficiency and reliability of the system.

[0110] Preferably, step S32 includes the following steps:

[0111] Step S321: Perform storage node encoding on each branch in the divergence fluctuation map to obtain a storage node encoding sequence, wherein each storage node is assigned a unique number according to the topological order in the branch;

[0112] Specifically, for example, each branch in the divergence wave map is identified, and each storage node is encoded in topological order so that each node has a unique number. For example, suppose there are the following branches in the divergence wave map: branch 1 contains nodes A, B, and C, branch 2 contains nodes A, D, and E, and branch 3 contains nodes F, G, and H. Encode these nodes in topological order, and the storage node encoding sequence of branch 1 is [1, 2, 3], the storage node encoding sequence of branch 2 is [1, 4, 5], and the storage node encoding sequence of branch 3 is [6, 7, 8].

[0113] Step S322: extracting node trust values ​​from the storage node code sequence of each branch to obtain a storage node trust value set;

[0114] Specifically, for example, the trust value of each storage node is extracted based on its historical behavior and performance indicators, and these trust values ​​are obtained based on the node's response time, data accuracy and consistency factors. For example, assume that the trust value of node A is 0.9, the trust value of node B is 0.85, the trust value of node C is 0.8, the trust value of node D is 0.75, the trust value of node E is 0.7, the trust value of node F is 0.95, the trust value of node G is 0.9, and the trust value of node H is 0.85. Thus, the following storage node trust value sets can be obtained: the trust value set of branch 1 is [0.9, 0.85, 0.8], the trust value set of branch 2 is [0.9, 0.75, 0.7], and the trust value set of branch 3 is [0.95, 0.9, 0.85]. Among them, the specific calculation process of the trust value is as follows: The calculation of the trust value is based on the historical behavior and performance indicators of the storage node, including response time (RT), data accuracy (DA) and consistency (C) factors. The calculation formula of the trust value is: Trust value = w RT ×Response time score + w DA ×Data accuracy score + w C × consistency score, where w RT 、w DA and w C are the weights of response time, data accuracy and consistency, and their sum is 1. Set w RT =0.3, w DA =0.4, w C =0.3.

[0115] Step S323: performing Markov chain trust transfer modeling on each branch based on the storage node trust value set to obtain a branch trust transfer matrix;

[0116] Specifically, for example, for branch 1, its initial trust value set is [0.9, 0.85, 0.8], the trust transfer from node A to node B is calculated to be 0.9×0.85=0.765, and the trust transfer from node B to node C is 0.765×0.8=0.612. Therefore, the trust transfer matrix of branch 1 is: For branch 2, its initial trust value set is [0.9, 0.75, 0.7], and the trust transfer matrix is: For branch 3, its initial trust value set is [0.95, 0.9, 0.85], and the trust transfer matrix is: Through these steps, the trust transfer matrix of each branch can be obtained, thereby evaluating the trust level of each branch and providing basic data for subsequent branch analysis and selection.

[0117] Step S324: Calculate the overall trust weight value of each branch according to the branch trust transfer matrix to obtain the branch trust weight value;

[0118] Specifically, the trust values ​​in the branch trust transfer matrix are summarized and normalized to calculate the overall trust weight value of each branch. For example, for branch 1, its trust transfer matrix is: The overall trust weight value is calculated as the cumulative value of the trust values ​​of each node. Using the cumulative trust value method, the overall trust weight value of branch 1 is 0.9+0.765+0.612=2.277. Similarly, for branches 2 and 3, their overall trust weight values ​​are calculated respectively.

[0119] Step S325: performing topological length measurement on each branch to obtain branch length data;

[0120] Specifically, for example, for branch 1, its storage node encoding sequence is [1, 2, 3], and the distance between nodes is 1 unit distance, then the topological length of branch 1 is 2 (that is, the number of connections between nodes). Similarly, the topological length of branch 2 is 2, and the topological length of branch 3 is 2. In this way, the topological length data of each branch can be obtained.

[0121] Step S326: Calculate the expected entropy of the corresponding branch based on the branch trust weight value and the branch length data to obtain branch expected entropy data. After completely traversing each branch in the divergence fluctuation map, several branches of expected entropy data are obtained.

[0122] Specifically, for example, for branch 1, its trust weight value is 2.277 and the topological length is 2. The expected entropy is calculated using the following formula: expected entropy = trust weight value / topological length. For branch 1, the expected entropy is 2.277 / 2 = 1.1385. Similarly, the expected entropy of branches 2 and 3 is calculated to obtain the expected entropy data respectively. By completely traversing each branch in the divergence fluctuation map, several branch expected entropy data can be obtained.

[0123] The present invention performs storage node encoding on each branch in the divergence fluctuation map. This encoding sequence assigns a unique number to each storage node and sorts them according to their topological order in the branch. This helps to accurately describe and identify the position and relationship of the storage nodes in different branches. By extracting the node trust value of the storage node encoding sequence of each branch, a storage node trust value set is obtained. This can evaluate the credibility of the node based on the historical behavior of the node and the data consistency factor. By performing Markov chain trust transfer modeling on each branch based on the storage node trust value set, a branch trust transfer matrix is ​​obtained. This can effectively describe the trust transfer relationship between storage nodes and provide methods and means for evaluating the overall credibility of the branch. By calculating the overall trust weight value of each branch according to the branch trust transfer matrix, the branch trust weight value is obtained. This value represents the importance and credibility of each branch in the overall data repair process. By measuring the topological length of each branch, the branch length data is obtained. This data reflects the topological structure and complexity of each branch. By calculating the expected entropy of each branch based on the branch trust weight value and the branch length data, the branch expected entropy data is obtained. This data takes into account trust weight and branch length factors, providing comprehensive information and guidance for assessing the importance and difficulty of repair of each branch.

[0124] Preferably, the present invention further provides an order data synchronization system for executing the order data synchronization method as described above, the order data synchronization system comprising:

[0125] The order data collection module is used to collect orders from the e-commerce system in real time and obtain the original order data; the original order data is divided into blocks and hash mapped to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node;

[0126] A distributed database construction module is used to construct a distributed database based on an ordered order data block sequence and a storage node cluster mapping table to obtain a distributed database; maintain a logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result;

[0127] The consistency data processing module is used to reach a multi-data voting consensus on the storage node cluster mapping table based on the data inconsistency detection results to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection results include the storage nodes with inconsistent order data and their corresponding event divergence point data.

[0128] In the present invention, the order data acquisition module is used to realize the real-time collection of order data of the e-commerce system, ensuring the timely acquisition and processing of order data. By dividing the original order data into blocks and hash mapping, the data is accurately allocated to specific storage nodes, and the order data block sequence and the storage node cluster mapping table are managed in an orderly manner, thereby improving the systematicness and efficiency of data management. An efficient distributed database is constructed based on the ordered order data block sequence and the storage node cluster mapping table through the distributed database construction module, which can effectively cope with the high concurrency and large data volume processing requirements of the e-commerce system. By maintaining the logical vector clock record and the global orderly maintenance of the timestamp sequence, the consistency detection and maintenance of the data are realized, ensuring that the system still maintains high performance and high efficiency under a high load environment. By maintaining the timestamp sequence in a global orderly manner, the problem of data inconsistency can be discovered and located in time, ensuring the data consistency in the system. By adopting a distributed architecture, the order data is dispersedly stored in multiple nodes, and even if an individual node fails, it will not affect the overall operation of the system, thereby improving the fault tolerance and reliability of the system. By the consistency data processing module, the consistency order data is written back to the original storage node, ensuring the consistency and integrity of the data, and can effectively operate even in a complex network environment.

[0129] Preferably, the present invention also provides an order data synchronization device, comprising:

[0130] Memory for storing computer programs;

[0131] A processor is used to implement any of the above-mentioned order data synchronization methods when executing the computer program.

[0132] Therefore, the embodiments should be regarded as illustrative and non-restrictive from all points, and the scope of the present invention is limited by the appended claims rather than the above description, and it is therefore intended that all changes falling within the meaning and range of equivalent elements of the application documents are included in the present invention.

[0133] The above description is only a specific embodiment of the present invention, so that those skilled in the art can understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but should conform to the widest scope consistent with the principles and novel features invented herein.

Claims

1. A method for synchronizing order data, characterized in that: The following steps are involved: Step S1: collect orders from the e-commerce system in real time to obtain original order data; The original order data is divided into blocks and hash mapped to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node; wherein step S1 includes the following steps: Step S11: deploying order event probes on the e-commerce system to obtain an order event monitoring agent; Step S12: using an order event monitoring agent to capture order transaction data streams of the e-commerce system in real time to obtain original order data; Step S13: normalizing the format of the original order data to obtain standardized order data; Step S14: Divide the standard order data into blocks to obtain an ordered sequence of order data blocks; Step S15: Based on the ordered order data block sequence, each order data block is mapped to a storage node through a ring virtual node mapping to obtain a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node; wherein step S15 includes the following steps: Step S151: constructing an N-dimensional ring virtual node space for the e-commerce system to obtain an N-dimensional ring virtual node space, wherein the physical storage nodes in the N-dimensional ring virtual node space are mapped to the N-dimensional ring according to the consistent hashing algorithm, and each physical storage node has M virtual nodes evenly distributed on the ring; Step S152: using a fractal hash coding algorithm to encode each order data block in the ordered order data block sequence to obtain a number of order N-dimensional hash vectors; Step S153: Based on the N-dimensional ring virtual node space, the distance measurement is performed on each order N-dimensional hash vector to obtain the nearest virtual node distance data; wherein step S153 includes the following steps: Step S1531: Perform non-Euclidean geometric space mapping on the N-dimensional ring-shaped virtual node space to obtain an N-dimensional non-Euclidean space; Step S1532: Perform self-similarity measurement analysis on the order N-dimensional hash vector to obtain the fractal dimension of the order hash vector; Step S1533: performing fractal similarity measurement on each virtual node in the N-dimensional non-Euclidean space and the fractal dimension of the order hash vector to obtain a number of node-order fractal distance values; Step S1534: performing nearest matching on the fractal dimension of the order hash vector according to a number of node-order fractal distance values ​​to obtain the nearest virtual node distance data; Step S154: Map the virtual node corresponding to the nearest virtual node distance data back to the physical storage node, and record the order data block and the corresponding physical storage node in the storage node cluster mapping table; Step S155: repeat steps S153 and S154 until all order data blocks have completed node mapping, that is, a storage node cluster mapping table is obtained, in which the ordered order data block sequence corresponds one-to-one with the storage node cluster mapping table, and each order data block corresponds to a unique storage node; Step S2: construct a distributed database based on the ordered order data block sequence and the storage node cluster mapping table to obtain a distributed database; maintain the logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result; wherein step S2 includes the following steps: Step S21: Based on the ordered order data block sequence and the storage node cluster mapping table, a Gossip database instance is deployed on each storage node in the storage node cluster to obtain a distributed database; Step S22: Initialize the vector clock of each Gossip database instance in the distributed database to obtain the node initial vector clock state data; Step S23: Based on the node initial vector clock state data, the vector clock coordinator in each Gossip database instance in the distributed database is used to exchange anti-entropy states with the storage nodes in the storage node cluster in a preset period to obtain node update vector clock state data, wherein, in the anti-entropy state exchange process, after receiving the incremental state of other storage nodes, the incremental state is merged according to the causal order and the node number to update the local vector clock; Step S24: periodically archiving the node update vector clock state data of each storage node according to a preset archiving period, generating an ordered vector clock tuple, and broadcasting it to the entire storage node cluster to obtain ordered vector clock tuple data; Step S25: Summarize the ordered vector clock tuple data of each storage node in the storage node cluster to obtain a global ordered maintenance timestamp sequence, wherein the global ordered maintenance timestamp sequence includes a plurality of ordered vector clock tuple data; Step S26: Perform data consistency detection on the order data blocks stored in each storage node according to the global orderly maintenance timestamp sequence to obtain a data inconsistency detection result; wherein step S26 includes the following steps: Step S261: construct a topological model of the global order data event based on the global ordered maintenance timestamp sequence to obtain a global ordered event graph, in which each ordered vector clock tuple data corresponds to an event node, and the dependency between nodes is determined by the causal order and the node number; Step S262: extracting a local order data block sequence from each storage node based on the storage node cluster to obtain a local order data block sequence; Step S263: Perform local event topology modeling on the corresponding storage node according to the local order data block sequence to obtain a local ordered event graph; Step S264: performing topological isomorphism detection on the local ordered event graph and the global ordered event graph to obtain consistency detection result data; Step S265: If the consistency detection result data shows that there is an inequality relationship between the local ordered event graph and the global ordered event graph, then the vector clock state of the local ordered event graph is backtracked based on the global ordered maintenance timestamp sequence to obtain event divergence point data; Step S266: constructing a storage node mapping for the event divergence point data to obtain a data inconsistency detection result, wherein the data inconsistency detection result includes a storage node with inconsistent order data and its corresponding event divergence point data; Step S3: Based on the data inconsistency detection result, a multi-data voting consensus is reached on the storage node cluster mapping table to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection result includes the storage node with inconsistent order data and its corresponding event divergence point data; wherein step S3 includes the following steps: Step S31: Based on the data inconsistency detection result, a fluctuation map is constructed for the storage nodes with inconsistent order data to obtain a divergent fluctuation map, wherein the nodes in the divergent fluctuation map represent storage nodes, the edges represent causal dependencies, and the inconsistent storage nodes construct branches according to the corresponding event divergence point data; Step S32: Calculate the expected entropy of each branch in the divergence fluctuation map to obtain a number of branch expected entropy data; Step S33: identifying the minimum expected entropy path of the divergence fluctuation graph according to the expected entropy data of several branches, and obtaining the order consistent path milestone branch, wherein the order consistent path milestone branch corresponds to the ordered event sequence that most nodes in the storage node cluster agree on; Step S34: performing consensus convergence modeling on the divergence fluctuation graph to obtain an order consensus pruner; using the order consensus pruner to perform abnormal branch scanning and identification on the divergence fluctuation graph to obtain an abnormal branch graph of storage nodes; Step S35: using the order consensus pruner to perform abnormal path pruning and implantation operations on the divergent fluctuation graph according to the storage node abnormal branch graph and the order consistent path milestone branch, and obtaining the storage node normalized graph; Step S36: construct a consensus topology based on the normalized graph of storage nodes to obtain an order consistency event chain; Step S37: Decode and restore the order consistency event chain according to the storage node mapping table to obtain a consistent order data block sequence, and write the consistent order data block sequence back to the original storage node cluster to complete the synchronization and obtain the synchronized total order data.

2. The method for synchronizing order data according to claim 1, characterized in that: Step S32 includes the following steps: Step S321: Perform storage node encoding on each branch in the divergence fluctuation map to obtain a storage node encoding sequence, wherein each storage node is assigned a unique number according to the topological order in the branch; Step S322: extracting node trust values ​​from the storage node code sequence of each branch to obtain a storage node trust value set; Step S323: performing Markov chain trust transfer modeling on each branch based on the storage node trust value set to obtain a branch trust transfer matrix; Step S324: Calculate the overall trust weight value of each branch according to the branch trust transfer matrix to obtain the branch trust weight value; Step S325: performing topological length measurement on each branch to obtain branch length data; Step S326: Calculate the expected entropy of the corresponding branch based on the branch trust weight value and the branch length data to obtain branch expected entropy data. After completely traversing each branch in the divergence fluctuation map, several branches of expected entropy data are obtained.

3. A synchronization system for order data, characterized in that: For executing the method for synchronizing order data as claimed in claim 1, the order data synchronization system comprises: The order data collection module is used to collect orders from the e-commerce system in real time and obtain the original order data; the original order data is divided into blocks and hash mapped to obtain an ordered order data block sequence and a storage node cluster mapping table, wherein the ordered order data block sequence corresponds to the storage node cluster mapping table one by one, and each order data block corresponds to a unique storage node; A distributed database construction module is used to construct a distributed database based on an ordered order data block sequence and a storage node cluster mapping table to obtain a distributed database; maintain a logical vector clock record for each storage node in the distributed database to obtain a global ordered maintenance timestamp sequence; perform data consistency detection on the order data block of the corresponding storage node according to the global ordered maintenance timestamp sequence to obtain a data inconsistency detection result; The consistency data processing module is used to reach a multi-data voting consensus on the storage node cluster mapping table based on the data inconsistency detection results to obtain consistent order data; based on the storage node cluster mapping table, the consistent order data is written back to the original storage node to obtain the synchronized total order data, wherein the data inconsistency detection results include the storage nodes with inconsistent order data and their corresponding event divergence point data.

4. A synchronization device for order data, characterized in that: include: Memory for storing computer programs; A processor, configured to implement the method for synchronizing order data as described in any one of claims 1 to 2 when executing the computer program.

Citation Information

Patent Citations

  • Method and system for managing special car order data in different libraries and different tables

    CN114493799A

  • Generation and application method of consistency check points of distributed database

    CN114510539A

  • Flexible data partition routing method for complex service scene

    CN116431615A

  • Block chain system with artificial intelligence nodes

    CN117251887A

  • Request processing method and device based on hash ring, electronic equipment and storage medium

    CN117459597A