Storage data consistency management methods, devices, terminals and storage media

Abstract

This invention provides a method, apparatus, terminal, and storage medium for managing data consistency, relating to the field of data storage technology. It captures data change actions in the database; converts these actions into real-time online messages; generates an ecosystem link based on the real-time online messages; and performs anomaly detection on the ecosystem link to obtain erroneous data. This addresses the problems of traditional data consistency verification methods lacking specificity, exhibiting delays in anomaly repair, and being ill-suited for application scenarios involving massive amounts of business data. It enables real-time detection, avoids delays in anomaly repair, and, by capturing data change actions in the database, provides greater targeting, enabling rapid data auditing and comparison. Furthermore, it is applicable to application scenarios involving massive amounts of business data.

Description

Technical Field

[0001] This invention relates to the field of data storage technology, and in particular to a method, apparatus, terminal, and storage medium for managing data consistency. Background Technology

[0002] In the telecommunications industry, business data is fundamental to user applications. Data anomalies can lead to business failures, making data accuracy governance crucial. Microservice architectures involve business chains with multiple heterogeneous databases, requiring consistency verification when data changes occur. For single-database scenarios involving long-flow business processes across multiple tables, the accuracy of user-received business data also needs verification. Current technologies involve data auditing based on business processes, utilizing the database's built-in SQL script auditing and comparison capabilities, or collecting data from different tables into a unified database and then establishing corresponding auditing and comparison algorithms to determine accuracy. However, this offline data governance model requires operations personnel to extract various business data into a common database beforehand, using collection, extraction, and comparison to determine anomalies. This approach lacks specificity and results in significant delays in anomaly repair. Furthermore, the massive amounts of business data are constantly changing, and operational efficiency cannot keep pace with the rate of data generation resulting from evolving business scenarios, making it difficult to guarantee data accuracy. Summary of the Invention

[0003] This invention provides a storage data consistency management method, device, terminal, and storage medium to address the shortcomings of traditional storage data consistency verification methods, such as lack of specificity, delayed abnormal data repair, and difficulty in handling application scenarios with massive amounts of business data.

[0004] This invention provides a method for managing data consistency, comprising:

[0005] Capture data change actions in the database;

[0006] The data change action is converted into an online real-time message;

[0007] An ecosystem link is generated based on the aforementioned online real-time messages;

[0008] Anomaly detection is performed on the aforementioned ecological links to obtain error data.

[0009] According to a storage data consistency management method provided by the present invention, the step of capturing data change actions in the database includes:

[0010] A first connector is generated, and a corresponding capture task is generated through the first connector. The capture task listens for database change actions.

[0011] When a database change action is detected, capture the database change log.

[0012] According to a storage data consistency management method provided by the present invention, the step of converting the data change action into an online real-time message includes:

[0013] The database change log is read through the second connector and converted into online real-time messages.

[0014] According to a storage data consistency management method provided by the present invention, the step of generating an ecosystem link based on the online real-time message includes:

[0015] Obtain business data;

[0016] The business data is divided into corresponding business types;

[0017] Remove invalid business data under each business type;

[0018] The starting node of the ecosystem link is determined based on the retained business data under each business type;

[0019] The starting node is combined with free business data with the same business identifier to generate an ecosystem link.

[0020] According to a storage data consistency management method provided by the present invention, the step of detecting anomalies in the ecosystem link and obtaining erroneous data includes:

[0021] Match the ecological link with the preset master link;

[0022] If the match fails and the ecological similarity is greater than a preset threshold, output error data.

[0023] According to a storage data consistency management method provided by the present invention, the method for obtaining ecological similarity includes:

[0024] Obtain the total time of generating the ecosystem link, divide each business node in the ecosystem link into multiple stages, and obtain the matching information entropy value of each stage;

[0025] Calculate the overall matching information entropy value based on the matching information entropy value of each stage;

[0026] Calculate the link time compliance based on the preset maximum matching time period;

[0027] Ecological similarity is calculated based on the overall matching information entropy value and the link time conformity.

[0028] According to a storage data consistency control method provided by the present invention, after a failure to match the ecological link with the preset master link, the method further includes:

[0029] Self-healing data is implanted into the ecological link where there is erroneous data, and the self-healing data is generated according to the corresponding master link.

[0030] The present invention also provides a storage data consistency management device, comprising:

[0031] The capture module is used to capture data change actions in the database;

[0032] The conversion module is used to convert the data change action into an online real-time message;

[0033] The generation module is used to generate an ecosystem link based on the online real-time messages;

[0034] The detection module is used to detect anomalies in the ecological link and obtain error data.

[0035] The present invention also provides a terminal, including a memory, a transceiver, and a processor;

[0036] A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations:

[0037] Capture data change actions in the database;

[0038] The data change action is converted into an online real-time message;

[0039] An ecosystem link is generated based on the aforementioned online real-time messages;

[0040] Anomaly detection is performed on the aforementioned ecological links to obtain error data.

[0041] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, wherein the computer program, when executed by a processor, implements the storage data consistency management method described in any of the preceding claims.

[0042] The storage data consistency management method, device, terminal, and storage medium provided by this invention capture data change actions in the database; convert the data change actions into online real-time messages; generate an ecosystem link based on the online real-time messages; and perform anomaly detection on the ecosystem link to obtain erroneous data. This enables real-time detection, avoids delays in abnormal data repair, and is more targeted by capturing data change actions in the database, enabling rapid data auditing and comparison. Furthermore, it is applicable to application scenarios with massive amounts of business data. Attached Figure Description

[0043] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0044] Figure 1 This is one of the flowcharts illustrating the storage data consistency control method provided by the present invention;

[0045] Figure 2 This is a schematic diagram of the log message conversion and filtering process provided by the present invention;

[0046] Figure 3 This is the third flowchart of the storage data consistency control method provided by the present invention;

[0047] Figure 4 This is a schematic diagram of sample distribution based on partitioning efficiency provided by the present invention;

[0048] Figure 5 This is the fourth flowchart of the storage data consistency control method provided by the present invention;

[0049] Figure 6 This is a flowchart of the Ecological Similarity Algorithm provided by this invention;

[0050] Figure 7 This is a flowchart of the data ecosystem reorganization detection and self-healing repair process provided by the present invention;

[0051] Figure 8 This is a schematic diagram of the storage data consistency management device provided by the present invention;

[0052] Figure 9 This is a schematic diagram of the terminal provided by the present invention. Detailed Implementation

[0053] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0054] Figure 1 A flowchart of the storage data consistency management method provided in the embodiments of the present invention is shown below. Figure 1As shown, the storage data consistency management method provided in this embodiment of the invention includes:

[0055] Step 101: Capture data change actions in the database;

[0056] In this embodiment of the invention, capturing data change actions in the database specifically includes:

[0057] Step 1011: Generate the first connector and generate a corresponding capture task through the first connector. The capture task listens for database change actions.

[0058] Step 1012: When a database change action is detected, capture the database change log.

[0059] Step 102: Convert the data change action into an online real-time message;

[0060] In this embodiment of the invention, converting data change actions into online real-time messages includes: reading the database change log through a second connector and converting the database change log into online real-time messages.

[0061] Step 103: Generate an ecosystem link based on real-time online messages;

[0062] Step 104: Perform anomaly detection on the ecological link and obtain error data.

[0063] This invention can intelligently determine the accuracy of data changes by reconstructing and capturing the changed data links and comparing them with the standard processes of the various services handled in the telecommunications industry during their interaction with the database, based on actions such as additions, modifications, and deletions. This invention applies data ecosystem reconstructing computing technology to the traditional telecommunications industry, ensuring data accuracy.

[0064] In the current telecommunications industry, major operators have begun to move away from Oracle databases, introducing multiple types of databases into their system architectures. These heterogeneous databases are complex and diverse in key business processes. For example, a single business flow based on orders may span multiple databases such as AntDB, GoldenDB, and OceanBase. Order interaction may involve rollback operations due to interface service timeouts. Routine maintenance often reveals that uncontrolled transaction consistency leads to inconsistencies across multiple databases in the business chain, causing service unavailability and resulting in user complaints. Traditional data auditing based on business processes relies on the database's own SQL script auditing and comparison capabilities, or collects data from different tables into a unified database and then establishes corresponding auditing and comparison algorithms to determine data accuracy. This data governance model is offline; maintenance personnel need to extract various business data into a public database in advance and determine data anomalies through collection, extraction, and comparison. This approach lacks specificity, and anomaly repair is too delayed. Massive amounts of business data are constantly changing, and maintenance efficiency cannot keep up with the rate at which different business scenarios generate varying amounts of data, making it difficult to guarantee data accuracy.

[0065] The data consistency management method provided in this invention captures data change actions in the database; converts these actions into real-time online messages; generates an ecosystem link based on the real-time online messages; and performs anomaly detection on the ecosystem link to obtain erroneous data. This enables real-time detection, avoiding delays in anomaly data repair. By capturing data change actions in the database, the method is more targeted, enabling rapid data auditing and comparison. Furthermore, it is applicable to application scenarios involving massive amounts of business data. This invention can be applied in the telecommunications infrastructure and IT support fields, and is suitable for data consistency and accuracy governance when business chains in microservice architectures involve data changes across multiple types of heterogeneous databases. It can also ensure the accuracy of user-received business data when a single-type database is used in long-process business operations spanning multiple tables.

[0066] Based on any of the above embodiments, such as Figure 2As shown, the first connector is, for example, the Kafka Connector application, and the second connector is, for example, the Debezium connector deployed for databases such as MySQL and PostgreSQL. Through log capture and transformation, it can capture change events from various database types, convert change actions and data into messages, and then write the messages to the Kafka message middleware, completing the message transformation and storage of user business data. By converting business data into messages, centrally storing and processing them, the results are more accurate and less complex than existing log traffic analysis methods. Depending on the specific business scenario, it can quickly achieve accurate monitoring, auditing, and remediation of user data, promoting healthy management of user business data. It can be extended to various industries for accuracy governance and assurance of any user business data.

[0067] Based on any of the above embodiments, such as Figure 3 As shown, the ecosystem chain is generated based on online real-time messages, including:

[0068] Step 301: Obtain business data;

[0069] Step 302: Assign business data to the corresponding business type;

[0070] In this embodiment of the invention, a data application frequency decreasing algorithm is selected for data business affiliation partitioning. The algorithm-related parameters include: business group Q, business partitioning probability Pc, accurate initial probability F, and the partitioning efficiency calculation formula is as follows:

[0071]

[0072] Among them, the accurate initial probability F is classified hierarchically according to the hot business. When no division is made, F0 = 0. As the number of business divisions increases, the probability of F value approaches 1.

[0073] In business scenarios, business probabilities are distributed based on hotspot frequency. For example, for business A, Pc = A / Q, where A represents the size of business A. Verification is performed based on partitioning efficiency by calculating Pc for each of the H assigned business samples. Pc describes the actual partitioning success rate. In the final successful partitioning state, the success rate of the corresponding H businesses under multiple partitions can be observed, such as... Figure 4 As shown.

[0074] Step 303: Remove invalid business data under each business type;

[0075] In this embodiment of the invention, the filtering and processing of business data messages are shown in Table 1:

[0076] Table 1. Relationship Matrix Between Business Scenarios and Filtering Processes

[0077]

[0078]

[0079] In Table 1, each filtering step filters business data messages. If the business data in a business data message does not match the core business data, the business data is removed.

[0080] If a piece of business data is frequently used in multiple business scenarios, removing that data can filter out invalid starting node data and simplify the length of the ecosystem chain.

[0081] Step 304: Determine the starting node of the ecosystem link based on the retained business data under each business type;

[0082] Step 305: Combine the starting node with the free business data with the same business identifier to generate an ecosystem link.

[0083] In this embodiment of the invention, by classifying, eliminating, and filtering business data by category and starting node, business data with business type differentiation can be used as the starting node, which can effectively reduce the frequency of business link generation and improve the accuracy of link comparison.

[0084] Based on any of the above embodiments, such as Figure 5 As shown, anomaly detection is performed on the ecological link to obtain error data, including:

[0085] Step 501: Match the ecosystem link with the preset master link;

[0086] Step 502: If the matching fails and the ecological similarity is greater than the preset threshold, output the error data.

[0087] In this embodiment of the invention, business data is captured in real time and exists in the Kafka Connect ecosystem. The initial link start node is obtained through the data attribution filter. The data in the filter is matched with the parent link and combined with the free data to form the corresponding ecosystem link. Combined with Flink streaming batch technology, the consistency and accuracy of the business link data flow are compared and consumed. Abnormal change data is focused and converged, improving the retrieval efficiency of abnormal business links.

[0088] In this embodiment of the invention, the method for obtaining ecological similarity includes:

[0089] Step 5021: Obtain the total time of generating the ecosystem link, divide each business node in the ecosystem link into multiple stages, and obtain the matching information entropy value of each stage;

[0090] Step 5022: Calculate the overall matching information entropy value based on the matching information entropy value of each stage;

[0091] Step 5023: Calculate the link time compliance based on the preset maximum matching time period;

[0092] Step 5024: Calculate the ecological similarity based on the overall matching information entropy value and the link time conformity.

[0093] In this embodiment of the invention, the ecological similarity algorithm simulates business links, generating a total link time of TIMn. Free atomic data waits for a synthesis opportunity within a specified time range, and fails after the maximum time period MAXT. Link generation can divide the similarity of each business node into n stages, with each stage having a predictable entropy value. The initial similarity is T1, increasing to Tn as the number of link nodes increases. The ecological similarity (ESA) is calculated using the following formula:

[0094] Calculate the overall matching information entropy value X1' = (T1X1 + T2X2 + ... + TnXn)

[0095] Link time compliance (TIM) = (1 - MAXT / TIMn)

[0096]

[0097] A higher ESA indicates a higher degree of business link matching. This algorithm can match abnormal data in change messages to find a better self-healing solution. In the case of a large amount of inconsistent data in the system, it can self-heal more efficiently and avoid the risk of self-healing stagnation caused by locking mechanisms, thereby greatly improving business operation and maintenance efficiency.

[0098] In this embodiment of the invention, the Ecological Similarity Algorithm (ESA) is used to reorganize business data links. The algorithm flow is as follows: Figure 6 As shown, the core of this algorithm lies in the data within the ecosystem. Based on the integrity of the business mapping, it maps the most similar data combination links and calculates business combinations in real-time in batches. Each combination generates a traceId identifier. This link replicates the child version business data in the master version business configuration, performing core field detection and data migration self-healing on the data under the traceId identifier. Combined with the ecosystem similarity algorithm, it performs real-time data comparison, resulting in faster response and higher error detection and convergence efficiency compared to traditional offline analysis modes.

[0099] In this embodiment of the invention, the ESA algorithm can be used to track the occurrence of erroneous data, and the accuracy of erroneous data matching can be improved from about 75% before the application of the algorithm to 97.8%, thereby achieving end-to-end business chain accuracy management of the entire business data.

[0100] In some embodiments of the present invention, after the matching between the ecological link and the preset master link fails, the method further includes:

[0101] Self-healing data is implanted into the ecological link where there is erroneous data, and the self-healing data is generated according to the corresponding master link.

[0102] In this embodiment of the invention, upon detecting a data anomaly, an automated operation and maintenance script is triggered to quickly complete the repair by matching pre-configured self-healing rules and conditions, ensuring timely self-healing correction of abnormal data before customer complaints. This effectively improves and compensates for the lack of a systematic governance and protection solution for user error data in existing technical solutions. The data ecosystem reorganization detection and self-healing repair process is as follows: Figure 7 As shown:

[0103] Step 1: Enable data change capture to convert all business data changes, including the content before and after the change, the change action (add, delete, modify), the change time, the changed employee ID, the changed database IP and data table, into messages; Step 2: Batch send the change messages to Kafka for business chain initial data classification;

[0104] Step 3: Consume messages in Kafka using Flink streaming and batch processing to combine data into the business logic;

[0105] Step 4: Based on the Ecological similarity algorithm, obtain similar business data links, complete the comparison and output of abnormal data, and determine whether it belongs to an abnormal message;

[0106] Step 5: Batch send the erroneous data to ClinkHouse to prepare for real-time comparison of the erroneous data;

[0107] Step Six: Based on the business_id comparison data flow model, ClinkHouse error data is implanted by deploying a self-healing plan to achieve rapid and automatic repair of batch errors.

[0108] Traditional data inconsistency governance and remediation operate offline. Operations personnel need to extract various business data into a common database beforehand, using collection, extraction, and comparison capabilities to determine if anomalies have occurred. This approach lacks specificity and is prone to reactive, reactive responses to user complaints, often only revealing data errors after they have occurred, resulting in significant delays in anomaly remediation. Current technologies for detecting business data errors rely on audit comparisons and user complaints, and are limited to specific business scenarios, failing to detect anomalies across the entire business chain. This leads to poor end-to-end data governance and remediation capabilities. In microservice architectures, databases exhibit diverse heterogeneity, and massive amounts of business data are constantly changing. When using database tables as the object of operations and maintenance analysis, the ability to detect anomalies in user business data is weak. Analysis of logs and traffic data is complex, lacks versatility, and is less accurate than database data comparisons. Furthermore, traditional database languages ​​are inefficient as operations and maintenance analysis tools, resulting in operations efficiency lagging behind the rate of data generation due to changes in business scenarios. This places immense pressure on operations personnel to ensure data accuracy.

[0109] The storage data consistency management method provided in this invention uses data change capture technology to transform database add, delete, and modify actions into real-time messages. It then reassembles the data chain, identifies the integrity of the business chain, implants data into defective chains, and verifies and repairs the integrity. Furthermore, it employs an Ecological similarity algorithm for massive messages to detect and output abnormal data. Based on the highest business chain matching degree, it reassembles data chains that conform to the business chain, achieving message-based data accuracy operation and maintenance and intelligent self-healing solutions. This meets the end-to-end data chain assurance requirements of the business process and improves customer experience.

[0110] The storage data consistency management device provided by the present invention is described below. The storage data consistency management device described below can be referred to in correspondence with the storage data consistency management method described above.

[0111] Figure 8 This is a schematic diagram of a storage data consistency management device provided in an embodiment of the present invention, as shown below. Figure 8 As shown, the storage data consistency management device provided in this embodiment of the invention includes:

[0112] Capture module 801 is used to capture data change actions in the database;

[0113] The conversion module 802 is used to convert data change actions into online real-time messages;

[0114] The generation module 803 is used to generate ecological links based on online real-time messages;

[0115] The detection module 804 is used to detect anomalies in the ecological link and obtain error data.

[0116] The storage data consistency management device provided in this invention captures data change actions in the database; converts these actions into online real-time messages; generates an ecosystem link based on the online real-time messages; and performs anomaly detection on the ecosystem link to obtain erroneous data. This enables real-time detection, avoiding delays in abnormal data repair. By capturing data change actions in the database, it is more targeted, enabling rapid data auditing and comparison. Furthermore, it is applicable to application scenarios involving massive amounts of business data. This invention can be applied in the telecommunications industry infrastructure and IT support fields. It is suitable for data consistency and accuracy governance when business chains in microservice architectures involve data changes across multiple types of heterogeneous databases. It can also ensure the accuracy of user-received business data when a single type of database is used in long-process business operations spanning multiple tables.

[0117] In this embodiment of the invention, the capture module 801 is configured as follows:

[0118] A first connector is generated, and a corresponding capture task is generated through the first connector. The capture task listens for database change actions.

[0119] When a database change action is detected, capture the database change log.

[0120] In this embodiment of the invention, the conversion module 802 is configured as follows:

[0121] The database change log is read through the second connector and converted into online real-time messages.

[0122] In this embodiment of the invention, the generation module 803 is configured as follows:

[0123] Obtain business data;

[0124] The business data is divided into corresponding business types;

[0125] Remove invalid business data under each business type;

[0126] The starting node of the ecosystem link is determined based on the retained business data under each business type;

[0127] The starting node is combined with free business data with the same business identifier to generate an ecosystem link.

[0128] In this embodiment of the invention, the detection module 804 is configured as follows:

[0129] Match the ecosystem link with the preset master link;

[0130] If the match fails and the ecological similarity is greater than a preset threshold, output error data.

[0131] In this embodiment of the invention, the method for obtaining ecological similarity includes:

[0132] Obtain the total time of generating the ecosystem link, divide each business node in the ecosystem link into multiple stages, and obtain the matching information entropy value of each stage;

[0133] Calculate the overall matching information entropy value based on the matching information entropy value of each stage;

[0134] Calculate the link time compliance based on the preset maximum matching time period;

[0135] Ecological similarity is calculated based on the overall matching information entropy value and the link time conformity.

[0136] In this embodiment of the invention, a self-healing module is also included, which is used to implant self-healing data into the ecological link with erroneous data after the ecological link fails to match the preset master link. The self-healing data is generated according to the corresponding master link.

[0137] The storage data consistency management device provided in this invention uses data change capture technology to transform database add, delete, and modify actions into real-time messages. It then reassembles the data chain, identifies the integrity of the business chain, implants data into defective chains, and verifies and repairs the integrity. Furthermore, it employs an Ecological similarity algorithm to detect and output abnormal data for massive amounts of messages. Based on the highest business chain matching degree, it reassembles data chains that conform to the business chain, achieving message-based data accuracy operation and maintenance and intelligent self-healing solutions. This meets the end-to-end data chain assurance requirements of the business process and improves customer experience.

[0138] Figure 9 An example is a schematic diagram of the physical structure of a terminal, such as... Figure 9 As shown, the terminal may include a processor 910, a transceiver 920, a memory 930, and a communication bus 940. The processor 910, transceiver 920, and memory 930 communicate with each other via the communication bus 940. The processor 910 can invoke logical instructions in the memory 930 to execute a data consistency management method. This method includes: capturing data change actions in the database; converting the data change actions into online real-time messages; generating an ecosystem link based on the online real-time messages; and performing anomaly detection on the ecosystem link to obtain error data.

[0139] On the other hand, the present invention also provides a non-transitory computer-readable storage medium storing a computer program thereon. When executed by a processor, the computer program implements the storage data consistency management method provided by the above methods. The method includes: capturing data change actions in the database; converting the data change actions into online real-time messages; generating an ecosystem link based on the online real-time messages; and performing anomaly detection on the ecosystem link to obtain error data.

[0140] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0141] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0142] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for managing data consistency, characterized in that, include: Capture data change actions in the database and convert the data change actions into online real-time messages; An ecosystem link is generated based on the aforementioned online real-time messages; Anomaly detection is performed on the aforementioned ecological links to obtain error data; The process of anomaly detection in the ecological link and obtaining error data includes: Match the ecological link with the preset master link; If the match fails and the ecological similarity is greater than the preset threshold, output error data. The method for obtaining ecological similarity includes: obtaining the total time of generating ecological links; dividing each business node in the ecological links into multiple stages; obtaining the matching information entropy value of each stage; calculating the overall matching information entropy value based on the matching information entropy value of each stage; calculating the link time compliance based on a preset maximum matching time period; and calculating the ecological similarity based on the overall matching information entropy value and the link time compliance.

2. The storage data consistency control method according to claim 1, characterized in that, The data change actions captured in the database include: A first connector is generated, and a corresponding capture task is generated through the first connector. The capture task listens for database change actions. When a database change action is detected, capture the database change log.

3. The storage data consistency control method according to claim 2, characterized in that, The step of converting the data change action into an online real-time message includes: The database change log is read through the second connector and converted into online real-time messages.

4. The storage data consistency control method according to claim 1 or 3, characterized in that, The process of generating an ecosystem link based on the online real-time messages includes: Obtain business data; The business data is divided into corresponding business types; Remove invalid business data under each business type; The starting node of the ecosystem link is determined based on the retained business data under each business type; The starting node is combined with free business data with the same business identifier to generate an ecosystem link.

5. The storage data consistency control method according to claim 1, characterized in that, If the matching between the ecological link and the preset master link fails, the following steps are also included: Self-healing data is implanted into the ecological link where there is erroneous data, and the self-healing data is generated according to the corresponding master link.

6. A storage data consistency control device, characterized in that, include: The capture module is used to capture data change actions in the database; The conversion module is used to convert the data change action into an online real-time message; The generation module is used to generate an ecosystem link based on the online real-time messages; The detection module is used to perform anomaly detection on the ecological link and obtain error data, including: Match the ecological link with the preset master link; If the match fails and the ecological similarity is greater than the preset threshold, output error data. The method for obtaining ecological similarity includes: obtaining the total time of generating ecological links; dividing each business node in the ecological links into multiple stages; obtaining the matching information entropy value of each stage; calculating the overall matching information entropy value based on the matching information entropy value of each stage; calculating the link time compliance based on a preset maximum matching time period; and calculating the ecological similarity based on the overall matching information entropy value and the link time compliance.

7. A terminal, characterized in that, Includes memory, transceiver, and processor; A memory for storing computer programs; a transceiver for sending and receiving data under the control of the processor; and a processor for reading the computer programs from the memory and performing the following operations: Capture data change actions in the database; The data change action is converted into an online real-time message; An ecosystem link is generated based on the aforementioned online real-time messages; Anomaly detection is performed on the aforementioned ecological link to obtain error data, including: Match the ecological link with the preset master link; If the match fails and the ecological similarity is greater than the preset threshold, output error data. The method for obtaining ecological similarity includes: obtaining the total time of generating ecological links; dividing each business node in the ecological links into multiple stages; obtaining the matching information entropy value of each stage; calculating the overall matching information entropy value based on the matching information entropy value of each stage; calculating the link time compliance based on a preset maximum matching time period; and calculating the ecological similarity based on the overall matching information entropy value and the link time compliance.

8. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the storage data consistency management method as described in any one of claims 1 to 5.

Images