A data synchronization transmission method, device and medium for a heterogeneous database

By constructing cross-database field semantic mapping and transaction feature vector analysis, the problems of manual intervention and complex conflict identification in heterogeneous database data synchronization are solved, achieving efficient and automated data synchronization and transmission, and ensuring data consistency and the stability of the synchronization process.

CN121365109BActive Publication Date: 2026-02-27CAPITAL CLOUD TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511949018.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-12-23
Publication Date
2026-02-27
Estimated Expiration
2045-12-23

AI Technical Summary

Technical Problem

Existing heterogeneous database data synchronization solutions rely on manual intervention, which is difficult to meet rapidly changing business needs. Furthermore, traditional methods struggle to identify and handle complex transaction conflicts, resulting in low data synchronization efficiency and difficulty in ensuring consistency.

Method used

By constructing a semantic mapping mechanism for cross-database fields, utilizing field attribute clustering algorithms and transaction feature vector mining, a set of synchronization constraints is generated, and a minimum cut graph partitioning algorithm is used to construct a transaction synchronization partition structure, thereby achieving automated data synchronization and transmission.

Benefits of technology

It improves the accuracy and automation of heterogeneous database data synchronization, ensures the correctness of transaction-level synchronization and data consistency, and enhances the performance and stability of synchronization transmission.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN121365109B_ABST
    Figure CN121365109B_ABST
Patent Text Reader

Abstract

The present application relates to the technical field of data transmission, and discloses a data synchronization transmission method and device for heterogeneous databases and a medium, comprising: obtaining data table structure definitions from a source end database and a target end database, using a field attribute clustering algorithm to construct a semantic mapping table reflecting cross-database field implicit association; extracting a transaction feature vector set based on a transaction log, mining implicit conflict relationships between transactions through the transaction feature vector set, and generating a synchronization constraint set reflecting implicit constraints between transactions; based on the synchronization constraint set, using a minimum cut graph segmentation algorithm to construct a transaction synchronization partition structure with dense conflict relationship aggregation; and according to the transaction synchronization partition structure, performing transaction granularity data synchronization transmission of the heterogeneous databases using the semantic mapping table in units of transaction synchronization partitions; the present application realizes implicit semantic mapping and transaction conflict identification between heterogeneous databases, effectively improving the consistency and accuracy of data synchronization transmission.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This invention relates to the field of data transmission technology, and more specifically, to a method, apparatus, and medium for data synchronization transmission of heterogeneous databases. Background Technology

[0002] With the rapid development of information technology and the Internet, database systems have become a crucial infrastructure for enterprise information management and business operations. To meet the diverse business scenarios, system scales, and performance requirements, enterprises or organizations typically deploy multiple types of database systems simultaneously. These databases differ in system architecture, storage methods, data types, and transaction processing mechanisms, resulting in a heterogeneous database environment.

[0003] In heterogeneous database environments, data collaboration between different databases has become a significant challenge in the field of information technology. Typically, efficient data synchronization between different database systems is required to ensure data consistency and integrity. However, due to differences in data models, field definitions, semantic expressions, and transaction mechanisms among heterogeneous database systems, data synchronization often faces technical challenges such as difficulties in data mapping, low synchronization efficiency, complex handling of transaction conflicts, and difficulty in guaranteeing data accuracy.

[0004] Existing heterogeneous database data synchronization solutions typically require significant human intervention, relying on manually predefined data mapping relationships and synchronization strategies. This results in insufficient system scalability, making it difficult to meet rapidly changing business needs. Furthermore, traditional synchronization methods have limited ability to identify and handle complex transaction conflicts, making it difficult to achieve fine-grained synchronization at the transaction level. This can lead to inconsistencies in transaction data or data synchronization errors, seriously impacting the reliability of business operations and the security of data management.

[0005] Therefore, overcoming the aforementioned technical problems and designing a precise, efficient, and highly automated method for synchronizing heterogeneous databases has become an urgent issue to be addressed in the field of database technology. Summary of the Invention

[0006] To overcome the above-mentioned deficiencies of the prior art, embodiments of the present invention provide a method, device and medium for data synchronization transmission of heterogeneous databases.

[0007] To achieve the above objectives, the present invention provides the following technical solution:

[0008] A method for data synchronization and transmission in heterogeneous databases, the method comprising:

[0009] The data table structure definitions are obtained from the source database and the target database, and a semantic mapping table that reflects the implicit relationship between fields across databases is constructed using a field attribute clustering algorithm.

[0010] extract a transaction feature vector set based on the transaction log, mine the implicit conflict relationship between transactions through the transaction feature vector set, and generate a synchronization constraint set reflecting the implicit constraints between transactions;

[0011] Based on the synchronization constraint set, a transaction synchronization partition structure with dense conflict relationship aggregation is constructed by using a minimum cut graph segmentation algorithm;

[0012] According to the transaction synchronization partition structure, transaction granularity data synchronization transmission of heterogeneous databases is performed using the semantic mapping table in units of transaction synchronization partitions.

[0013] An electronic device includes a memory, a processor, and a computer program stored on the memory and running on the processor, and the processor implements the data synchronization transmission method for heterogeneous databases described in any of the above.

[0014] A computer readable storage medium has a computer program stored thereon, and the computer program is executed to implement the data synchronization transmission method for heterogeneous databases described in any of the above.

[0015] Compared with the prior art, the beneficial effects of the present application are:

[0016] The present application constructs a semantic mapping mechanism across the library field, accurately mines the implicit association relationship between different database systems using a field attribute clustering algorithm, realizes automatic semantic mapping of cross-database fields, avoids data mapping errors and low efficiency caused by manual definition, and improves the accuracy and automation of the heterogeneous database data synchronization process.

[0017] The present application mines transaction feature vectors and analyzes implicit conflict relationships to deeply identify complex implicit operation conflicts between transactions, generates a synchronization constraint rule set, solves the implicit transaction conflict problem that is difficult to identify by traditional methods, and ensures the correctness and data consistency of transaction granularity synchronization transmission of heterogeneous databases.

[0018] The present application constructs a transaction synchronization partition structure with dense aggregation of transaction conflict relationships, optimizes the execution order and synchronization unit division of transaction synchronization, realizes effective management and efficient transmission in the transaction synchronization process, avoids synchronization blocking and data inconsistency risks caused by conflicts between transactions, and significantly improves the overall performance and stability of data synchronization transmission in a heterogeneous database environment. BRIEF DESCRIPTION OF DRAWINGS

[0019] Figure 1 A flowchart of a data synchronization transmission method for heterogeneous databases is provided in the present application.

[0020] Figure 2 A schematic diagram of the structure of an electronic device provided by the present invention;

[0021] Figure 3 This is a schematic diagram of the structure of a computer-readable storage medium provided by the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] Example 1

[0024] Please see Figure 1 As shown in the figure, this embodiment discloses a data synchronization and transmission method for heterogeneous databases, the method comprising:

[0025] S101: Obtain the data table structure definition from the source database and the target database, and use the field attribute clustering algorithm to construct a semantic mapping table that reflects the implicit relationship between fields across databases;

[0026] In specific implementation, the construction process of the field attribute clustering algorithm includes: starting from the table structure level of heterogeneous databases, extracting and vectorizing the attributes, types, constraints, etc. of each field, and then identifying the implicit semantic relationship between fields through a clustering model that combines local density and global distance, so as to achieve automatic mapping of cross-database field semantic consistency.

[0027] It should be noted that, in this embodiment, "heterogeneous database" refers to database systems with differences in structure definition, data type, indexing rules, or constraint logic. For example, the source may be a MySQL database, and the target may be an Oracle or PostgreSQL database. The two may differ in field naming rules, data type compatibility, and the way integrity constraints are expressed.

[0028] In a specific implementation, the step of constructing a semantic mapping table that reflects implicit cross-database field relationships using a field attribute clustering algorithm includes:

[0029] Extract the data type, field length, and integrity constraint information of the fields from the source database and the target database respectively to form the initial semantic features of the source field and the initial semantic features of the target field;

[0030] It should be noted that the field information extraction process is realized through the metadata interface or system table of the database system. For example, in the MySQL system, the name, data type, length, nullability, and constraint rules of the field can be obtained through the information_schema.columns table; in the Oracle system, the same type of data extraction can be realized through the ALL_TAB_COLUMNS view; after standardization through a unified interface, the field attribute table is generated.

[0031] Further, in order to ensure the integrity of the semantic feature description, the initial semantic feature of each field is composed of the following three dimensions:

[0032] Field data type feature: used to reflect the data expression method of the field in the storage structure and logical structure, including integer type, floating point type, character type, date and time type, Boolean type, etc.;

[0033] Field length feature: indicating the data length or value range allowed to be stored by the field, used to measure the compatibility and information density between different fields;

[0034] Integrity constraint feature: reflecting the logical role of the field in the database, such as primary key, foreign key, unique constraint, non-null constraint, or default value constraint, etc.

[0035] Exemplarily, when the source database table user_info contains the field user_id (int type, primary key, non-null), and the target database table tb_user contains the field uid (number type, primary key, non-null), the initial semantic feature table structure is shown in Table 1 as follows:

[0036] Table 1: Example of initial semantic feature table structure

[0037]

[0038] In specific implementation, in order to facilitate subsequent semantic modeling, the system integrates the structured features (type, length, constraint) extracted in Table 1 into a standardized "initial semantic feature description" text.

[0039] Further, if there are complex fields (such as JSON fields or composite type fields) in the database, the system extracts the data types and hierarchical relationships of the internal key values by recursively parsing the field structure, and includes them in the "initial semantic feature description" text, ensuring that the semantic feature table covers all hierarchical fields.

[0040] On this basis, the initial semantic feature description text generated by the system (as shown in Table 1) provides a standardized input basis for the semantic embedding model in the subsequent steps.

[0041] Based on the source field initial semantic feature and the target field initial semantic feature, a multi-granularity semantic embedding model is constructed to generate the field semantic feature vectors of the source and the target respectively;

[0042] It should be noted that the "multi-granularity semantic embedding model" in the embodiment is a semantic feature representation model constructed by combining deep learning and multi-scale feature fusion technology, which specifically includes but is not limited to pre-trained language models (such as BERT, RoBERTa, GPT) and multi-scale feature fusion networks.

[0043] In a specific implementation, first, the generated field initial semantic feature text is subjected to preliminary semantic embedding processing through a pre-trained language model to obtain a high-dimensional basic semantic feature vector. For example, for the source field user_id, taking "integer field, as primary key to identify the uniqueness of the user" as the input text, a 768-dimensional basic semantic vector representation is obtained after embedding by the BERT model.

[0044] Further, in view of the possible fine-grained differences and different granularity semantic information between field semantics, a multi-granularity feature fusion network is constructed on the basis of the basic feature embedding vector to capture the correlation between local and global semantic features. Exemplarily, the multi-granularity feature fusion network includes:

[0045] Parallel convolution branch: a plurality of convolution kernels of different sizes (such as 1x1, 3x3, 5x5) are set to perform convolution operations on the basic semantic feature vector respectively to obtain local semantic features of different granularities;

[0046] Fully connected layer: the local semantic feature vectors output by different convolution branches are spliced and fused to further abstract global semantic features through a fully connected network;

[0047] Nonlinear activation and dimension reduction: efficient and low-redundancy semantic feature vector representation is realized through a ReLU activation function and a dimension reduction layer to obtain the final multi-granularity field semantic feature vector, and the vector dimension is exemplarily 256.

[0048] It should be further noted that the specific calculation formula of the multi-granularity semantic feature vector is as follows:

[0049]

[0050] In the formula: is the final multi-granularity semantic feature vector; is the basic semantic feature vector; respectively represent the feature output by the convolution branch of different scales; is a feature splicing operation; is a fully connected layer.

[0051] By using a field clustering algorithm based on joint judgment of local semantic density and global semantic distance, the implicit correlation between the semantic feature vectors of source and target fields is analyzed to obtain cross-database field clusters;

[0052] The analysis of the implicit correlation between the semantic feature vectors of the source and target fields includes:

[0053] Calculate the semantic density of the semantic feature vectors of the source and target fields in the local region, and determine the initial local semantic clustering center;

[0054] In specific implementation, the local semantic density is calculated using a Gaussian kernel function, and the calculation formula is as follows: In the formula: Let be the local density of the feature vector of the i-th field; Let be the Euclidean distance between the i-th and j-th eigenvectors; The cutoff distance parameter in density calculation is determined by the average nearest neighbor distance.

[0055] Furthermore, by sorting the local density, several feature vectors with density peaks greater than a threshold are selected as initial local semantic cluster centers for subsequent cross-database semantic association clustering analysis.

[0056] Based on the complementary features of the initial local semantic clustering center and the global semantic distance, a composite semantic association evaluation index for cross-database fields is constructed.

[0057] In practice, the composite semantic association evaluation index considers the local compactness (density) and global discriminativeness (distance) between field semantic feature vectors, and is defined as follows:

[0058]

[0059] In the formula: The composite correlation index is the feature vector of the i-th field. For local density; It is the distance between the nearest neighbor vector with a higher feature vector density in the i-th field.

[0060] By using the composite semantic association evaluation index of cross-database fields, joint clustering of semantic feature vectors of source and target fields is performed to obtain a cross-database field cluster structure that can express the deep implicit semantic association between cross-database fields;

[0061] Specifically, the feature vectors with the highest composite semantic association evaluation index are used as the final cluster centers, and the semantic feature vectors of the source and target fields are assigned to the corresponding cluster centers according to the nearest neighbor principle, thereby forming a cross-database field cluster structure.

[0062] For example, a cross-database field cluster structure is shown in Table 2 below:

[0063] Table 2: Cross-database field cluster structure table

[0064]

[0065] Based on the implicit semantic relationships between fields in cross-database field clusters, construct a cross-database field semantic mapping table;

[0066] It should be noted that the cross-database field semantic mapping table in this embodiment is a structured data mapping table, specifically including source database fields, target database fields, and corresponding field semantic descriptions, used to guide subsequent data synchronization operations. An example of the semantic mapping table structure is shown in Table 3 below:

[0067] Table 3: Structure of the Semantic Mapping Table

[0068]

[0069] It should be further noted that this semantic mapping table not only provides explicit mapping relationships between fields, but also lays the semantic foundation for subsequent transaction-level data synchronization and transmission (such as insert, update, and delete operations), ensuring the accuracy and consistency of the heterogeneous database data synchronization process.

[0070] S102: Extract a set of transaction feature vectors based on the transaction log, mine the implicit conflict relationships between transactions through the set of transaction feature vectors, and generate a set of synchronization constraints that reflect the implicit constraints between transactions;

[0071] It should be noted that the core purpose of this implementation method is to uncover implicit transaction conflict relationships in transaction logs, which is especially suitable for complex or implicit dependencies that are difficult to identify through conventional log analysis, thereby providing clear and accurate constraint rules for subsequent data synchronization.

[0072] In a specific implementation, generating a set of synchronization constraints that embodies implicit constraints between transactions includes:

[0073] Extract the operation behavior, access field range and operation type during transaction execution from the transaction log to form an initial transaction operation feature set;

[0074] In a specific implementation, the "transaction log" refers to the data change operation log recorded by the database system for recovery and replication, such as the binlog in MySQL, the redo log in Oracle, or the WAL log in PostgreSQL.

[0075] It should be noted that the specific methods for extracting transaction log information in this embodiment include, but are not limited to:

[0076] Operation behavior: Insert, Delete, Update and Select operations performed by the transaction;

[0077] Access field range: Data tables and specific fields involved in the transaction, such as id and name fields in the user table;

[0078] Operation type: Including read, write, lock, unlock and other operation features, used to identify the root cause of possible conflicts between transactions.

[0079] Exemplarily, taking the MySQL database transaction log as an example, an example of initial operation feature record of a transaction log segment is shown in Table 4:

[0080] Table 4: Initial operation feature record table of transaction log

[0081]

[0082] A transaction feature vector set reflecting the complex operation relationship between transactions is generated by using a transaction operation feature enhancement method based on a multi-scale graph attention mechanism;

[0083] In specific implementation, the "multi-scale graph attention mechanism" is a graph neural network (GNN) method that can effectively extract and enhance the complex relationship features between transactions from the initial operation features of the transactions.

[0084] Among them, the transaction feature vector set reflecting the complex operation relationship between transactions includes:

[0085] The initial semantic graph structure of transaction operation is constructed based on the initial transaction operation feature set, and multi-scale semantic connection edges between transaction nodes are established in the graph;

[0086] In specific implementation, each transaction is represented as a node in the graph, and the initial features of each node are composed of operation behavior, access field range and operation type. For example:

[0087] The node feature example is represented as a vector [operation type (read = 0, write = 1), table ID, field set code].

[0088] Further, multi-scale semantic connection edges are constructed between nodes, which specifically include:

[0089] Table-level semantic edge: representing the association relationship between transactions involving the same data table;

[0090] Field-level semantic edge: representing the association relationship between transactions involving specific fields in the same data table;

[0091] Operation level semantic edge: represents the association between transactions involving the same type of operation (e.g., read-read, read-write).

[0092] Exemplarily, the transaction initial semantic graph structure is defined as follows:

[0093] Node: transaction T101, T102, T103;

[0094] Edge example: a "table level" and "field level" connection edge is constructed between transaction T101 and T103 (both involving the user table); there is no direct association between transaction T101 and T102 (involving the user table and the order table, respectively).

[0095] The node features of the transaction operation initial semantic graph are fused and enhanced by a multi-scale graph attention mechanism to generate enhanced transaction operation feature representations;

[0096] It should be noted that the "multi-scale graph attention mechanism" is realized by a Graph Attention Network (GAT) structure in specific implementations, and its specific process is disclosed as follows:

[0097] First, the local attention weight calculation formula is defined as: ; in which: is the local attention weight from node i to node j; , is the node feature vector; is the shared weight matrix; is the learning weight of the attention mechanism; denotes the feature concatenation operation.

[0098] Further, the global attention weight calculation formula is defined as: ; in which: is the global attention weight; d is the dimension of the node feature vector.

[0099] Then, the final enhanced node feature representation is calculated by fusing the local and global attention:

[0100]

[0101] in which: is the enhanced transaction operation node feature vector; is the fusion coefficient (e.g., 0.6) of the local and global attention weights; is the set of adjacent nodes of node i.

[0102] The above method ensures that the local and global transaction semantic features are enhanced at the same time, and finally obtains an enhanced feature representation that can reflect the complex implicit relationship between transactions.

[0103] According to the feature projection of the enhanced transaction operation feature representation, a transaction feature vector set capable of accurately reflecting the complex implicit operation conflict relationship between transactions is outputted;

[0104] It should be noted that in the present embodiment, the transaction operation node feature vector after enhancement may be high in dimension or have redundancy, resulting in reduced accuracy and efficiency of transaction conflict relationship mining. Therefore, the present sub-step further adopts a feature projection method for dimension reduction processing to highlight the most representative transaction features.

[0105] In specific implementation, the present step realizes transaction feature projection and dimension reduction processing through an autoencoder (Autoencoder). Specifically:

[0106] Firstly, a transaction feature projection autoencoder model is constructed, including the following two parts:

[0107] Encoder (Encoder): maps high-dimensional transaction operation feature vectors to a low-dimensional latent space, retaining key operation relationship information between transactions;

[0108] Decoder (Decoder): reconstructs low-dimensional latent features back to high-dimensional transaction operation feature space to train and optimize encoder model parameters.

[0109] The objective function in the training process of the autoencoder model is defined as: ; is the enhanced original transaction operation feature vector set; is the reconstructed transaction operation feature vector set; represents the squared Euclidean distance between vectors.

[0110] It should be further noted that the present embodiment obtains a low-dimensional transaction feature vector set after repeated training until the loss function converges, for example, each transaction feature vector dimension is reduced from 256 to 32, significantly improving the efficiency and accuracy of subsequent transaction conflict relationship mining.

[0111] According to the transaction feature vector set, operation semantic conflict analysis between transactions is performed to mine transaction implicit operation conflict relationships that cannot be obtained through conventional log analysis methods;

[0112] In specific implementation, operation semantic conflict analysis between transactions is realized by calculating the similarity between transaction feature vectors, as follows:

[0113] Firstly, the distance metric in transaction conflict analysis is defined as cosine similarity : ; in the formula: A and B are feature vectors of two transactions.

[0114] Further need to explain that the similarity Greater than a given threshold (for example 0.8) transaction feature vector between the existence of potential semantic conflict relationship, and record, the specific conflict relationship structure is formed as shown in Table 5 below example:

[0115] Table 5: conflict relationship structure table

[0116]

[0117] It should be noted that the conflict relationship mined by the above method is an implicit relationship that is not explicitly recorded in the transaction log but will cause data inconsistency or conflict in the process of synchronous transmission, which can provide effective decision support for subsequent transaction granularity data synchronization transmission.

[0118] According to the transaction implicit operation conflict relationship mined, a synchronization constraint set reflecting the implicit constraints between transactions is generated;

[0119] In specific implementation, the construction of transaction implicit constraint set is based on the obtained conflict relationship between transactions, and each implicit conflict relationship will be converted into a synchronization constraint rule, which is specifically disclosed as follows:

[0120] First, the general data structure of transaction implicit constraint set is defined as shown in Table 6 below:

[0121] Table 6: general data structure table of transaction implicit constraint set

[0122]

[0123] Further need to explain that the transaction implicit constraint type mainly includes:

[0124] Mutual exclusion constraint: when a transaction involves the same data write, it must be executed in series and cannot be executed concurrently;

[0125] Dependency constraint: the execution result of the subsequent transaction depends on the completion of the previous transaction, and there is an explicit execution sequence requirement;

[0126] Consistency constraint: concurrent transaction execution cannot cause cross-database data consistency damage, and should ensure the integrity and validity of the execution result data.

[0127] Exemplarily, for the above mutual exclusion constraint (R001), the system automatically generates a specific synchronization execution strategy suggestion:

[0128] If transactions T101 and T105 involve the same data update operation, the execution order of the two transactions is explicitly required during actual synchronization, for example, transaction T101 is executed first, and then transaction T105 is executed, to avoid concurrent conflict.

[0129] Similarly, for the dependency constraint (R002), the system-generated synchronization execution strategy suggestion is:

[0130] In the transaction granularity synchronization process, it is ensured that the execution of transaction T102 is completed before the synchronization transmission of transaction T109 is performed, so as to guarantee the correctness and consistency of data.

[0131] The synchronization constraint set constructed in the above manner provides explicit and operable implicit constraint rules for subsequent transaction granularity heterogeneous database synchronization, guarantees the consistency and accuracy of data synchronization transmission, and avoids data anomalies and conflict risks that may occur in the synchronization process.

[0132] S103: Based on the synchronization constraint set, a transaction synchronization partition structure with dense aggregation of conflict relationships is constructed using a minimum cut graph segmentation algorithm;

[0133] It should be noted that the purpose of the present embodiment step is to obtain a transaction synchronization constraint set, and to accurately divide the transaction implicit conflict relationship network through a minimum cut algorithm in graph theory, to form a transaction synchronization partition structure that is convenient for efficient management and execution of synchronization operations.

[0134] In specific embodiments, the construction of the transaction synchronization partition structure with dense aggregation of conflict relationships includes:

[0135] Based on the synchronization constraint set, an initial transaction conflict network is constructed with transactions as nodes and implicit conflict relationships as edges;

[0136] In specific implementations, the initial transaction conflict network is represented by a graph structure, wherein:

[0137] The node is defined as the transaction itself, and the node characteristics are represented by the transaction unique identifier (transaction ID) for example;

[0138] The edge is defined as the implicit conflict relationship between transactions, and the characteristics of the specific edge are represented as conflict strength or association type (such as dependency relationship, mutual exclusion relationship, etc.);

[0139] It should be noted that the transaction conflict network of the present embodiment is a weighted undirected graph, wherein the weight of the edge depends on the strength of the conflict relationship between transactions. For example, when a transaction involves the same data field and the operation type is both write operation, the edge weight is set to be high (such as weight 10), to represent a high degree of conflict; and when a transaction involves the same data but is read-write operation, the weight is set to be low (such as weight 3), to represent a relatively low degree of conflict.

[0140] For example, the edge weight structure of the transaction conflict network is shown in Table 7 below:

[0141] Table 7: Edge weight structure table of transaction conflict network

[0142]

[0143] A hybrid metric algorithm based on edge correlation strength and node local density is used to assign comprehensive capacity weight to each edge of the initial transaction conflict network;

[0144] In specific implementation, this step aims to further refine the capacity weight setting of the edges of the constructed transaction conflict network, so as to achieve more accurate and effective transaction synchronization partition division, which specifically includes:

[0145] First, the local density value of the transaction node is calculated to reflect the conflict intensity of the network area where the node is located;

[0146] In specific implementation, the local density of the transaction node is calculated by summing the edge weights of the connected local network nodes, and the formula is: ; in the formula: denotes the local density of the i-th transaction node; N(i) denotes the neighbor node set directly connected to node i; denotes the weight of the edge between node i and node j.

[0147] Further, according to the local density of the transaction node associated with the edge and the conflict strength of the edge itself, the comprehensive capacity weight of each edge in the transaction conflict network is calculated;

[0148] In specific implementation, the comprehensive capacity weight of the edge is calculated as follows: ; in the formula: is the initial edge weight between node i and node j (i.e. transaction conflict strength); are the local density values of node i and node j, respectively.

[0149] It should be further noted that the above comprehensive capacity weight not only reflects the direct conflict strength between transaction nodes, but also considers the overall conflict intensity of the local network environment where the transaction node is located, effectively improving the accuracy of network division.

[0150] For example, taking the node pair (T201, T204) as an example, if the edge weight is 10, the local densities of nodes T201 and T204 are 15 and 12 respectively, then the comprehensive capacity weight of the edge is calculated as: .

[0151] On the transaction conflict network with comprehensive capacity weight, an improved minimum cut algorithm with random multi-start and parallel search is used for segmentation to determine the boundary of the nonlinearly distributed transaction synchronization partition;

[0152] ​​It should be noted that the "improved minimum cut algorithm of random multi-start parallel search" in the embodiment is aimed at the problem that the traditional minimum cut algorithm is easy to fall into local optimization. By introducing multiple randomly selected search starting points and performing minimum cut search in parallel, the global search performance and partition accuracy are effectively improved.

[0153] The determination of the non-linearly distributed transaction synchronization partition boundary comprises:

[0154] In the transaction conflict network, a plurality of dispersed initial search starting points are randomly selected to construct a starting point parallel search space.

[0155] In a specific implementation, the initial search starting points are randomly selected and dispersed in the transaction conflict network, and the number of starting points is determined according to the size of the network nodes. For example, when the number of nodes exceeds 100, the number of starting points is set to about 10, to ensure effective coverage of the search space.

[0156] Local cut search is simultaneously performed from each search starting point, and the search path is dynamically adjusted according to the change trend of the edge capacity weight.

[0157] In a specific implementation, this step performs local cut search in parallel at each search starting point, and the search path is determined by the decreasing trend of the edge capacity weight. Specifically, the search path at each starting point preferentially advances in the direction with the fastest decreasing capacity weight, until the network boundary is reached or a local minimum point of the capacity weight is reached, and the current starting point search process is stopped.

[0158] The cut point results of the multiple starting point searches are cross-regionally fused, and the fused cut point distribution is secondarily corrected to obtain the non-linearly distributed transaction synchronization partition boundary.

[0159] It should be noted that the specific implementation of this step is to gather the local cut point results obtained by multiple parallel starting points, and to form a stable and accurate non-linear transaction synchronization partition boundary through cross-regional fusion and distribution correction.

[0160] In a specific implementation, the cross-regional fusion process is as follows:

[0161] First, the local search cut points of multiple starting points are summarized to form a cut point set, which is defined as: In the formula: represents the cut point set obtained by the kth search starting point; K is the number of search starting points; and CutPoints is the summary set of all cut points.

[0162] Further, to avoid redundancy caused by overlapping cut points generated by local search, the above cut point set is preliminarily de-duplicated according to the repeatability of transaction nodes, that is, only the cut point record with the minimum capacity weight is retained for multiple occurrences of the same transaction node, thereby obtaining a preliminarily fused cut point set.

[0163] It should be further explained that the cut point distribution secondary correction method of the embodiment is as follows:

[0164] Based on the capacity weight of the cut point, the kernel density estimation method is used to analyze the density of the spatial distribution of the cut point set;

[0165] According to the density distribution of the cut point, a density threshold parameter (such as the 75% quantile of the density distribution curve) is set for secondary distribution correction. The cut points in the high-density area are preferentially used as stable partition boundary points, and the cut points with low density or isolation are regarded as abnormal points and are removed or re-evaluated.

[0166] Exemplarily, the KDE density analysis calculation formula is: ; In the formula: is the kernel density estimation value of the position x; is the position of the cut point (node number or index representation); h is the bandwidth parameter for controlling the smoothing degree; is the kernel function, and the Gaussian kernel function is selected;

[0167] It should be further explained that the cut point distribution after the above secondary correction can more objectively reflect the node positions that have partition boundary significance in the transaction conflict network. The obtained nonlinear transaction synchronization partition boundary has higher stability and accuracy. Exemplarily, as shown in the following table 8:

[0168] Table 8: Partition boundary cut point data table

[0169]

[0170] According to the determined transaction synchronization partition boundary, the transaction synchronization partition structure with intensive conflict relationship aggregation is obtained;

[0171] In the specific implementation, after determining the transaction synchronization partition boundary (cut point), the transaction synchronization partition structure is constructed through the transaction node attribution principle, and the specific disclosure is as follows:

[0172] First, according to the above determined nonlinear transaction synchronization partition boundary, the transaction conflict network is segmented into node sets:

[0173] If the transaction node is located within the partition boundary cut point (i.e., not directly adjacent to or connected with the cut point), it belongs to an independent transaction synchronization partition;

[0174] If the transaction node is directly adjacent to the partition boundary cut point and is not explicitly located in a specific partition, the final attribution of the node is determined according to the edge capacity weight between the node and the cut point. Specifically, the side with larger capacity weight is the transaction synchronization partition to which the node belongs.

[0175] It should be further noted that the transaction synchronization partition structure is defined as a series of node sets. Within each set, the conflict density between transaction nodes is high, and they should be treated as a single unit for transaction synchronization transmission. However, the conflict level between sets is low, and transaction synchronization operations can be performed in parallel and independently. An example of the data representation of the transaction synchronization partition structure is shown in Table 9 below:

[0176] Table 9: Data Tables for Transaction Synchronization Partition Structure

[0177]

[0178] It is worth noting that the transaction synchronization partition structure obtained through the above method ensures that nodes with dense transaction conflict relationships are aggregated into the same synchronization unit, which significantly improves the execution efficiency and stability of subsequent transaction-level synchronization operations and avoids transaction anomalies or data inconsistencies caused by conflicts during data synchronization.

[0179] S104: Based on the transaction synchronization partition structure, the semantic mapping table is used to perform transaction-granular data synchronization transmission of heterogeneous databases in units of transaction synchronization partitions;

[0180] It should be noted that the purpose of this implementation step is to utilize the constructed cross-database field semantic mapping table and the obtained transaction synchronization partition structure to perform partition-level data synchronization transmission for transactions between heterogeneous databases, so as to ensure data consistency, correctness, and efficiency of the synchronization transmission process.

[0181] In a specific implementation, the step of performing transaction-level data synchronization transmission of heterogeneous databases includes:

[0182] Based on the implicit conflict relationships between transactions that have been discovered, the transactions within the transaction synchronization partition are topologically sorted to determine the reasonable execution order of transactions during synchronization transmission.

[0183] In practice, the topological sorting process of transactions within a transaction synchronization partition is executed based on implicit constraints between transactions to ensure the correct execution order of dependencies during data synchronization and transmission. The specific details are as follows:

[0184] First, for each transaction synchronization partition, construct a directed dependency graph of transaction nodes within the partition, where:

[0185] A node represents the transaction itself;

[0186] Directed edges represent the execution order of dependencies between transactions, and the direction of the edge indicates the order of execution.

[0187] Furthermore, the directed dependency graph is constructed as follows:

[0188] If there is a dependency constraint in the partition (for example, transaction T301 depends on the data write completion of transaction T302), a directed edge is established between the nodes;

[0189] If there is a mutual exclusion constraint, a reasonable execution order is determined according to the timestamp of the transaction or the execution order of the transaction log, and a corresponding directed edge is established.

[0190] Exemplarily, the transaction dependency graph of the transaction synchronization partition is represented as follows:

[0191] The transaction nodes {T301, T302, T303} in the synchronization partition Cluster-1:

[0192] If T302 depends on T301, a directed edge T301 → T302 is established;

[0193] If T303 and T302 are mutually exclusive, and the log record shows that T302 precedes T303, a directed edge T302 → T303 is established;

[0194] Secondly, on the basis of the constructed directed dependency graph, a classic topological sorting algorithm (such as Kahn algorithm) is used to calculate the transaction execution order, and the specific process is as follows:

[0195] Select a transaction node with an in-degree of 0 from the directed graph as the starting node of transaction sorting;

[0196] After deleting the transaction node and its associated directed edge, update the in-degree of each node;

[0197] Repeat the above process until all nodes are sorted.

[0198] After the above topological sorting, the exemplary transaction execution order is determined as shown in Table 10:

[0199] Table 10: Transaction execution order table

[0200]

[0201] It should be noted that the above topological sorting determines the synchronous execution order of the transaction, which ensures the data consistency and reasonable execution order in the transaction synchronization process between heterogeneous databases, and avoids data anomalies caused by dependency or mutual exclusion conflicts in the synchronization operation.

[0202] Based on the differences in database operation characteristics between the source and the target, transaction operation semantic conversion rules for heterogeneous databases are constructed to realize semantic conversion of source transaction operations to target compatible transaction operations;

[0203] In specific implementation, the "transaction operation semantic conversion rule" refers to a rule set for converting the transaction operation type and specific operation behavior (such as insertion, update, deletion) of the source end database into the transaction operation type and syntax structure compatible with the target end database. The specific disclosure mode is as follows:

[0204] First, the rule is constructed according to the transaction semantic differences between the specific types of database systems (such as MySQL and Oracle), which exemplarily include:

[0205] Transaction isolation level difference;

[0206] Field data type and constraint expression difference;

[0207] SQL syntax and function expression difference (such as date function, sequence operation function);

[0208] Index and constraint mechanism difference (such as foreign key constraint, unique constraint).

[0209] Further, the construction of the transaction operation semantic conversion rule specifically includes the following steps:

[0210] Syntax analysis and abstract expression of the source end database transaction operation;

[0211] In specific implementation, the specific transaction operation statement (such as INSERT, UPDATE statement) in the source end transaction log is parsed to obtain the syntax structure and abstract semantic expression form of the transaction operation.

[0212] Exemplarily, for the transaction operation of the source end database (such as MySQL): UPDATE user SET name='Alice' WHERE user_id=1001;

[0213] After syntax analysis, the abstract operation expression is obtained: operation type: update (UPDATE); table name involved: user; field and value: name='Alice'; condition field: user_id=1001.

[0214] Structure mapping and conversion expression for target end database operation semantics;

[0215] In specific implementation, the abstract operation expression of the source end database transaction is converted into the operation semantics and SQL syntax structure compatible with the target end database, combined with the field semantic mapping table constructed in the foregoing S101 step.

[0216] Exemplarily, taking Oracle as the target end database, the aforementioned UPDATE operation is mapped and converted as follows: UPDATE tb_user SET uname='Alice' WHERE uid=1001;

[0217] wherein the table name is converted from user of the source end to tb_user of the target end; the field name is converted from name of the source end to uname of the target end; and the condition field user_id is converted to uid.

[0218] It is further needed to be explained that the transaction operation semantic conversion rule set example data structure is as shown in Table 11:

[0219] Table 11: Transaction operation semantic conversion rule set example data structure table

[0220]

[0221] The cross-database field semantic mapping table is used to perform cross-database mapping processing on the data fields involved in the transaction operation after semantic conversion, and to construct a transaction data transmission structure adapted to the target end database structure and operation logic;

[0222] It is needed to be explained that the purpose of the present step is to further use the constructed cross-database field semantic mapping table to perform cross-database mapping processing on the transaction data field level for the converted transaction operation statement, and finally to obtain a transaction data transmission structure which can be directly applied to the target end database.

[0223] In specific implementation, the detailed process of the present step is as follows:

[0224] For the converted transaction operation statement, the specific data field set involved in the transaction is obtained by parsing;

[0225] In specific implementation, the converted transaction operation statement (such as the UPDATE and INSERT statements of Oracle database) is parsed to obtain the field set of the target end database involved in the operation.

[0226] Exemplarily, taking the target end database transaction statement as an example: INSERT INTO tb_order(order_id, order_date, amount) VALUES(2001, SYSDATE, 100.50); the target end field set obtained after parsing is: {order_id, order_date, amount}.

[0227] According to the cross-database field semantic mapping table, the accurate mapping relationship between the source end and the target end is determined;

[0228] In implementation, according to the cross-database field semantic mapping table, the corresponding source field is determined for the target field set to ensure the consistency and correctness of the data transmission structure.

[0229] Exemplarily, the cross-database field mapping relationship is determined as shown in Table 12:

[0230] Table 12: Cross-database field mapping relationship table

[0231]

[0232] It should be noted that the above field type conversion rule is obtained by presetting the database field data type mapping relationship to ensure the adaptability of the target database operation logic and data structure.

[0233] The determined field mapping relationship is used to construct the transaction data transmission structure adapted to the target database;

[0234] In implementation, based on the data values involved in the transaction operation statement, the field name and data type conversion are accurately performed through the field mapping relationship, and finally the transaction data transmission structure that can be recognized and received by the target database is obtained.

[0235] Exemplarily, taking the source MySQL transaction operation as an example:

[0236] INSERT INTO orders(order_no, created_at, total_price) VALUES(2001, '2023-04-01 10:20:30', 100.50);

[0237] The transaction data transmission structure after field mapping processing is an example of an Oracle database statement:

[0238] INSERT INTO tb_order(order_id, order_date, amount) VALUES(2001, TO_DATE('2023-04-01 10:20:30','YYYY-MM-DD HH24:MI:SS'), 100.50);

[0239] It should be noted that the transaction data transmission structure constructed by the above-mentioned manner can realize the double conversion of data fields and data types, and guarantee the availability, accuracy and effectiveness of the operation in the heterogeneous database data synchronization process.

[0240] According to the transaction data transmission structure, the transaction granularity data synchronization instruction sequence of the heterogeneous database is generated in units of transaction synchronization partitions, and the transaction granularity data synchronization transmission of the heterogeneous database is realized.

[0241] In practice, the purpose of this step is to generate a sequence of executable transaction synchronization instructions for the target database, based on the constructed transaction data transmission structure and the determined transaction execution order, using transaction synchronization partitions as units. This ultimately achieves transaction-level data synchronization between heterogeneous databases. Details are as follows:

[0242] The transaction data transmission structure is organized according to the transaction execution order determined by the topology sort, with the transaction synchronization partition as the basic synchronization unit.

[0243] In practice, the transaction data transmission structure is organized strictly according to the execution order of the transactions, based on the transaction topology sorting result determined in S104.1, taking the transaction synchronization partition (such as the aforementioned Cluster-1) as the unit, to form a transaction execution sequence.

[0244] For example, taking the synchronous partition Cluster-1 as an example, its transaction topology sorting order is: T301 → T302 → T303.

[0245] The following table 13 shows an example of the transaction data transmission structure organization:

[0246] Table 13: Transaction Data Transmission Structure Organization Table

[0247]

[0248] Generate a data synchronization instruction sequence at the transaction granularity of the target database based on the transaction execution sequence;

[0249] In practice, the above transaction data transmission structure is used to generate a standard transaction synchronization instruction sequence based on the target database specification (such as Oracle, SQL Server, PostgreSQL).

[0250] It should be noted that the data synchronization instruction sequence at the transaction granularity is encapsulated using the target database transaction control structure, which specifically includes transaction start markers (such as BEGIN TRANSACTION), transaction commit markers (such as COMMIT), and exception handling mechanisms (such as ROLLBACK).

[0251] For example, a sequence of data synchronization instructions at the transaction granularity is shown below:

[0252] BEGIN TRANSACTION;

[0253] INSERT INTO tb_user(uid, uname) VALUES(1001, 'Alice');

[0254] UPDATE tb_user SET uname='Bob' WHERE uid=1001;

[0255] DELETE FROM tb_user WHERE uid=1002;

[0256] COMMIT;

[0257] Further need to be explained is that, in the transaction granularity data synchronization instruction sequence execution process, if the synchronization exception (for example, violation of constraint) appears, then the transaction rollback is triggered automatically through the preset exception handling mechanism, and the data consistency of the target end database is guaranteed.

[0258] Embodiment 2

[0259] Please refer to Figure 2 The embodiment discloses an electronic device, including a memory, a processor and a computer program stored in the memory and running on the processor, and the processor implements any one of the methods provided by the above methods for data synchronization transmission of heterogeneous databases when executing the computer program.

[0260] Since the electronic device introduced in the embodiment is the electronic device used to implement the data synchronization transmission method for heterogeneous databases in the embodiment, the specific implementation of the electronic device and its various forms can be understood by those skilled in the art based on the data synchronization transmission method for heterogeneous databases introduced in the embodiment, so the implementation of the method in the embodiment will not be introduced in detail. As long as the electronic device used to implement the data synchronization transmission method for heterogeneous databases in the embodiment is implemented by those skilled in the art, it belongs to the scope of the application.

[0261] Embodiment 3

[0262] Please refer to Figure 3 The embodiment discloses a computer readable storage medium, including a memory, a processor and a computer program stored in the memory and running on the processor, and the processor implements any one of the methods provided by the above methods for data synchronization transmission of heterogeneous databases when executing the computer program.

[0263] The above formulas are all dimensionless numerical calculations, and the formulas are obtained by collecting a large amount of data to simulate the most recent real situation. The preset parameters, weights and threshold values in the formula are set by those skilled in the art according to the actual situation.

[0264] The above-described embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented by software, the above-described embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are wholly or partially generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transferred from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions can be transferred from one website, computer, server, or data center to another website, computer, server, or data center through a wired network or a wireless network. The computer-readable storage medium can be any available medium accessible by a computer or a data storage device such as a server, data center, etc. containing one or more available medium collections. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium. The semiconductor medium can be a solid-state disk.

[0265] Those skilled in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed in the present application can be realized by electronic hardware or a combination of computer software and electronic hardware. Whether the functions are realized in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0266] Those skilled in the art can clearly understand that, for the convenience and brevity of the description, the specific working processes of the above-described system, device, and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described here.

[0267] In several embodiments provided by the present application, it should be understood that the disclosed system, device, and method can be implemented in other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only one, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the shown or discussed units can be indirect coupling or communication connection through some interfaces, devices, or units, which can be electrical, mechanical, or other forms.

[0268] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed to multiple network units. Part or all of the units can be selected to achieve the purpose of the embodiment of the present application according to actual needs.

[0269] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0270] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of changes or replacements within the technical range disclosed by the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

[0271] Finally: the above is only a preferred embodiment of the present application and is not used to limit the present application, any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application should be included in the protection scope of the present application.

Claims

1. A method for data synchronization and transmission in heterogeneous databases, characterized in that, The method includes: The data table structure definitions are obtained from the source and target databases, and a semantic mapping table reflecting implicit cross-database field relationships is constructed using a field attribute clustering algorithm; including: Extract the data type, field length, and integrity constraint information of the fields from the source database and the target database respectively to form the initial semantic features of the source field and the initial semantic features of the target field; Based on the initial semantic features of the source field and the initial semantic features of the target field, a multi-granularity semantic embedding model is constructed to generate semantic feature vectors of the source and target fields, respectively. By using a field clustering algorithm based on the joint judgment of local semantic density and global semantic distance, the implicit correlation between the semantic feature vectors of source and target fields is analyzed to obtain cross-database field clusters; Based on the implicit semantic relationships between fields in cross-database field clusters, construct a cross-database field semantic mapping table; Based on the transaction log, a set of transaction feature vectors is extracted. The implicit conflict relationships between transactions are then explored through the set of transaction feature vectors to generate a set of synchronization constraints that reflect the implicit constraints between transactions. Based on the aforementioned set of synchronization constraints, a transaction synchronization partition structure with densely aggregated conflict relationships is constructed using the minimum cut graph partitioning algorithm; including: Based on the set of synchronization constraints, an initial transaction conflict network is constructed with transactions as nodes and implicit conflict relationships as edges. A hybrid metric algorithm based on edge association strength and node local density is used to assign a comprehensive capacity weight to each edge of the initial transaction conflict network. On a transaction conflict network with comprehensive capacity weights, an improved minimum cut algorithm with random multi-start parallel search is used for partitioning to determine the non-linear distribution of transaction synchronization partition boundaries. Based on the determined transaction synchronization partition boundaries, a transaction synchronization partition structure with densely aggregated conflict relationships is obtained; Based on the transaction synchronization partition structure, the semantic mapping table is used to perform transaction-granular data synchronization transmission of heterogeneous databases, with each transaction synchronization partition as a unit.

2. The data synchronization and transmission method for heterogeneous databases according to claim 1, characterized in that, The analysis of the implicit correlation between the semantic feature vectors of the source and target fields includes: Calculate the semantic density of the semantic feature vectors of the source and target fields in the local region, and determine the initial local semantic clustering center; Based on the complementary features of the initial local semantic clustering center and the global semantic distance, a composite semantic association evaluation index for cross-database fields is constructed. By utilizing the composite semantic association evaluation index of cross-database fields, joint clustering of semantic feature vectors of source and target fields is performed to obtain a cross-database field cluster structure that can express the deep implicit semantic association between cross-database fields.

3. The data synchronization and transmission method for heterogeneous databases according to claim 2, characterized in that, The generation of the synchronization constraint set, which embodies implicit constraints between transactions, includes: Extract the operation behavior, access field range and operation type during transaction execution from the transaction log to form an initial transaction operation feature set; A transaction operation feature enhancement method based on a multi-scale graph attention mechanism is adopted to generate a set of transaction feature vectors that reflect the complex operation relationships between transactions; Based on the set of transaction feature vectors, semantic conflict analysis of inter-transaction operations is performed to uncover implicit operation conflict relationships that cannot be obtained through conventional log analysis methods. Based on the discovered implicit operational conflict relationships between transactions, a set of synchronization constraints that reflects the implicit constraints between transactions is generated.

4. The data synchronization and transmission method for heterogeneous databases according to claim 3, characterized in that, The generation of the transaction feature vector set reflecting the complex operational relationships between transactions includes: The initial semantic graph structure of transaction operations is constructed using the initial transaction operation feature set, and multi-scale semantic connection edges are established between transaction nodes in the graph. For the node features of the initial semantic graph of transaction operations, a multi-scale graph attention mechanism is used to achieve the fusion and enhancement of local and global operation semantics, generating enhanced feature representations of transaction operations; Based on the enhanced transaction operation feature representation, feature projection is performed to output a set of transaction feature vectors that can accurately reflect the complex implicit operation conflict relationships between transactions.

5. The data synchronization and transmission method for heterogeneous databases according to claim 4, characterized in that, The determination of the transaction synchronization partition boundary for nonlinear distribution includes: In the transaction conflict network, several dispersed initial search starting points are randomly selected to construct a parallel search space for the starting points; Local cut searches are performed simultaneously from each search starting point, and the search path is dynamically adjusted based on the changing trend of edge capacity weights. The cut point results from multiple starting points are fused across regions, and the distribution of the fused cut points is then corrected to obtain the non-linear distribution of the transaction synchronization partition boundary.

6. The data synchronization and transmission method for heterogeneous databases according to claim 5, characterized in that, The execution of transaction-level data synchronization transmission across heterogeneous databases includes: Based on the implicit conflict relationships between transactions that have been discovered, the transactions within the transaction synchronization partition are topologically sorted to determine the reasonable execution order of transactions during synchronization transmission. Based on the differences in database operation characteristics between the source and target ends, semantic transformation rules for transaction operations of heterogeneous databases are constructed to achieve semantic transformation of source-end transaction operations into target-end compatible transaction operations. By using a cross-database field semantic mapping table, cross-database mapping is performed on the data fields involved in the transaction operation after semantic transformation, and a transaction data transmission structure adapted to the target database structure and operation logic is constructed. Based on the transaction data transmission structure, a sequence of transaction-granular data synchronization instructions for heterogeneous databases is generated, with transaction synchronization partitions as the unit, to achieve transaction-granular data synchronization transmission for heterogeneous databases.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the data synchronization transmission method for heterogeneous databases as described in any one of claims 1 to 6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the data synchronization transmission method for heterogeneous databases as described in any one of claims 1 to 6.

Citation Information

Patent Citations

  • Data synchronous transmission system and method for heterogeneous database

    CN119003653A

  • Database multi-source heterogeneous data synchronization method and device

    CN120596576A