A Real-time Optimization Method and System for Preventing Conflicts in Heterogeneous Database DDL Synchronization

Abstract

The present invention relates to the technical field of data processing, and particularly relates to a method and system for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts. The present invention detects DDL operation conflicts through timestamps and resource overlap degrees, can eliminate the dependencies of the intermediate layer, processes DDL operations in sequence by adopting a global time sequence, improves the performance of the overall system, can more accurately judge the order and conflicts of each operation by sorting the DDL operation timestamps, ensures that operations are executed in the order of priorities by dynamically adjusting the execution order and priorities, can respond quickly and reduce latency, can flexibly adjust the execution order according to the current environment by obtaining the preset weight coefficient of each operation and calculating the priority value, and can quickly adapt to the occurrence of errors and re-evaluate and adjust the execution strategy of each operation by obtaining the execution result feedback in real time and adjusting the execution order, so as to avoid the global impact caused by single point failure.

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and particularly relates to a method and system for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts. Background Art

[0002] The Database Schema Definition Language (DDL) is a language used to describe the real-world entities to be stored in a database. Currently, the methods for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts mainly coordinate the DDL synchronization between different databases by introducing an intermediate layer and reduce the probability of conflicts generated when multiple databases modify the same table structure or data simultaneously through incremental synchronization. To a certain extent, it can ensure the efficient synchronization of data. However, due to the increased complexity of the system architecture caused by the intermediate layer, it may lead to performance bottlenecks and poor real-time performance. Especially in the case of asynchronous synchronization, there may be synchronization delays and it is impossible to achieve true real-time synchronization, with relatively high complexity. Summary of the Invention

[0003] The main object of the present invention is to provide a method for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts, aiming to solve the technical problems in the prior art.

[0004] The present invention proposes a method for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts, including:

[0005] Real-time obtaining multiple DDL operation records of each heterogeneous database, and extracting the operation timestamps of each of the DDL operation records;

[0006] Sorting all the DDL operation records according to each of the operation timestamps to obtain a global time sequence;

[0007] Successively obtaining the target table structures of every two adjacent DDL operation records according to the global time sequence, and obtaining the resource overlap degree of the corresponding two DDL operation records according to the two target table structures;

[0008] Judging whether the resource overlap degree is greater than a preset overlap degree;

[0009] If the resource overlap degree is greater than the preset overlap degree, it is determined that the corresponding two adjacent DDL operation records conflict, and the preset weight coefficients of the corresponding two DDL operation records are obtained;

[0010] Obtaining the priority value of the corresponding DDL operation record according to each of the preset weight coefficients, and dynamically adjusting the execution order identification table according to the priority value to obtain an execution DDL operation sequence;

[0011] Transmit each DDL operation record to the corresponding node of the heterogeneous database for execution in sequence according to the executed DDL operation sequence, and obtain the execution result feedback in real time;

[0012] If the execution result feedback fails, return to the step of obtaining the preset weight coefficient of the corresponding two DDL operation records until the execution result feedback is successful.

[0013] Preferably, the step of obtaining the resource overlap degree of the corresponding two DDL operation records according to every two of the target table structures includes:

[0014] Obtain the key elements of each target table structure, where the key elements include column names, data types, constraints, and indexes;

[0015] Obtain the first quantity of the same column names in two target table structures, and obtain the quantity of column names in the two target table structures;

[0016] Obtain the total quantity of column names according to the quantity of column names in the two target table structures, and obtain the column name overlap degree according to the first quantity and the total quantity of column names;

[0017] Obtain the second quantity of the same data types in two target table structures, and obtain the quantity of data types in the two target table structures;

[0018] Obtain the total quantity of data types according to the quantity of data types in the two target table structures, and obtain the data type overlap degree according to the second quantity and the total quantity of data types;

[0019] Obtain the third quantity of the same constraints in two target table structures, and obtain the quantity of constraints in the two target table structures;

[0020] Obtain the total quantity of constraints according to the quantity of constraints in the two target table structures, and obtain the constraint overlap degree according to the third quantity and the total quantity of constraints;

[0021] Obtain the fourth quantity of the same indexes in two target table structures, and obtain the quantity of indexes in the two target table structures;

[0022] Obtain the total quantity of indexes according to the quantity of indexes in the two target table structures, and obtain the index overlap degree according to the fourth quantity and the total quantity of indexes;

[0023] Obtain the resource overlap degree of the corresponding two DDL operation records according to the index overlap degree, constraint overlap degree, data type overlap degree, and column name overlap degree.

[0024] Preferably, the step of obtaining the priority value of the corresponding DDL operation record according to each preset weight coefficient includes:

[0025] Obtain the resource utilization rate of the corresponding heterogeneous database according to the DDL operation record;

[0026] Obtain the first minimum adjustment factor of the corresponding heterogeneous database under high load and the first maximum adjustment factor under low load according to the resource utilization rate;

[0027] Calculate the system load adjustment factor according to the first maximum adjustment factor, the first minimum adjustment factor and the resource utilization rate, where the calculation formula is:

[0028] F(TZ) = Z(XT) + [Z(DT) - Z(XT)] * [1 - Z(LY)];

[0029] Wherein, F(TZ) represents the system load adjustment factor, Z(XT) represents the first minimum adjustment factor, Z(DT) represents the first maximum adjustment factor, and Z(LY) represents the resource utilization rate;

[0030] Obtain the resource overlap degree of two DDL operation records, and respectively obtain the low conflict frequency and the high conflict frequency according to the resource overlap degree;

[0031] Obtain the second minimum adjustment factor according to the low conflict frequency, and obtain the second maximum adjustment factor according to the high conflict frequency;

[0032] Obtain the conflict frequency adjustment factor according to the second minimum adjustment factor, the second maximum adjustment factor and the resource overlap degree;

[0033] Obtain the corresponding operation timestamp and the current time according to the DDL operation record, and obtain the time difference according to the current time and the operation timestamp;

[0034] Obtain the preset weight coefficient, and calculate the priority value of the corresponding DDL operation record according to the time difference, the conflict frequency adjustment factor, the system load adjustment factor, the preset weight coefficient and the resource overlap degree, where the calculation formula is:

[0035] Y(XJ) = F(TZ) * S(JC) + C(TZ) * Y(QZ) * Z(CD);

[0036] Wherein, Y(XJ) represents the priority value, F(TZ) represents the system load adjustment factor, S(JC) represents the time difference, C(TZ) represents the conflict frequency adjustment factor, Y(QZ) represents the preset weight coefficient, and Z(CD) represents the resource overlap degree.

[0037] Preferably, the step of dynamically adjusting the execution order identification table according to the priority value to obtain the DDL operation execution sequence includes:

[0038] Obtain the first priority value and the second priority value of two adjacent DDL operation records with conflicts in the execution order identification table according to the priority value;

[0039] Determine whether the first priority value is less than the second priority value;

[0040] If the first priority value is less than the second priority value, adjust the DDL operation record corresponding to the first priority value to after the DDL operation record corresponding to the second priority value;

[0041] If the first priority value is not less than the second priority value, obtain the third priority value of the DDL operation record one position before the DDL operation record corresponding to the first priority value from the execution order identification table, and determine whether the third priority value is less than the first priority value;

[0042] If the third priority value is less than the first priority value, adjust the DDL operation record corresponding to the third priority value to after the DDL operation record corresponding to the first priority value;

[0043] If the third priority value is not less than the first priority value, obtain the fourth priority value of the DDL operation record one position after the DDL operation record corresponding to the second priority value from the execution order identification table, and determine whether the fourth priority value is greater than the second priority value;

[0044] If the fourth priority value is greater than the second priority value, adjust the DDL operation record corresponding to the fourth priority value to before the DDL operation record corresponding to the second priority value until all conflicting DDL operation records in the execution order table are traversed to obtain the DDL operation execution sequence.

[0045] Preferably, the step of sequentially transmitting each DDL operation record to the corresponding node of the heterogeneous database according to the DDL operation execution sequence includes:

[0046] Obtain the real-time network load and the initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the DDL operation execution sequence;

[0047] Set a maximum load threshold, and calculate and adjust the transmission rate according to the maximum load threshold, the real-time network load and the initial transmission rate, where the calculation formula is:

[0048]

[0049] Among them, T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold;

[0050] Determine the master node and sub - nodes of each DDL operation record according to the executed DDL operation sequence, and respectively obtain the receive window sizes of each master node and sub - node;

[0051] Divide each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size;

[0052] Transmit each data packet to the nodes of the corresponding heterogeneous database for execution through an adaptive transmission protocol.

[0053] Preferably, the step of determining the master node and sub - nodes of each DDL operation record according to the executed DDL operation sequence includes:

[0054] Obtain the operation type feature of each DDL operation record according to the executed DDL operation sequence, where the operation type feature includes at least one of create operation, modify operation, delete operation, truncate operation, rename operation, and index operation;

[0055] Judge whether the operation type feature belongs to the modify operation;

[0056] If the operation type feature belongs to the modify operation, transmit the DDL operation record corresponding to the operation type feature to the master node for execution;

[0057] If the operation type feature does not belong to the modify operation, transmit the DDL operation record corresponding to the operation type feature to the sub - node for execution.

[0058] This application also provides a real - time optimization system for preventing conflicts in DDL synchronization of heterogeneous databases, including:

[0059] An extraction module, configured to obtain multiple DDL operation records of each heterogeneous database in real - time, and extract the operation timestamps of each DDL operation record;

[0060] A sorting module, configured to sort all DDL operation records according to each operation timestamp to obtain a global time sequence;

[0061] An acquisition module, configured to sequentially obtain the target table structures of every two adjacent DDL operation records according to the global time sequence, and obtain the resource overlap degree of the corresponding two DDL operation records according to every two target table structures;

[0062] A judgment module, configured to judge whether the resource overlap degree is greater than a preset overlap degree;

[0063] If the resource overlap degree is greater than the preset overlap degree, it is determined that a conflict occurs between the corresponding two adjacent DDL operation records, and the preset weight coefficients of the corresponding two DDL operation records are obtained;

[0064] An adjustment module, configured to obtain the priority value of the corresponding DDL operation record according to each preset weight coefficient, and dynamically adjust the execution order identification table according to the priority value to obtain an execution DDL operation sequence;

[0065] A transmission module, configured to sequentially transmit each DDL operation record to the nodes of the corresponding heterogeneous database for execution according to the execution DDL operation sequence, and obtain real-time feedback on the execution result;

[0066] If the execution result feedback fails, return to the step of obtaining the preset weight coefficients of the corresponding two DDL operation records until the execution result feedback is successful.

[0067] Preferably, the transmission module includes:

[0068] An acquisition unit, configured to obtain the real-time network load and the initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the execution DDL operation sequence;

[0069] A calculation unit, configured to set a maximum load threshold, and calculate and adjust the transmission rate according to the maximum load threshold, the real-time network load, and the initial transmission rate, where the calculation formula is:

[0070]

[0071] where, T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold;

[0072] A determination unit, configured to determine the primary node and the secondary node of each DDL operation record according to the execution DDL operation sequence, and respectively obtain the receive window sizes of each primary node and secondary node;

[0073] A splitting unit, configured to split each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size;

[0074] A transmission unit, configured to transmit each data packet to the nodes of the corresponding heterogeneous database for execution through an adaptive transmission protocol.

[0075] The present invention also provides a computer device, including a memory and a processor, where the memory stores a computer program, and when the processor executes the computer program, the steps of the above heterogeneous database DDL synchronization conflict prevention and real-time optimization method are implemented.

[0076] The present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned heterogeneous database DDL synchronization conflict prevention real-time optimization method are realized.

[0077] The beneficial effects of the present invention are as follows: The present invention detects DDL operation conflicts through timestamps and resource overlap degrees, can eliminate the dependence on the middle layer, thereby simplifying the system architecture. By using a global time sequence to process DDL operations in sequence, the dependence on the middle layer is reduced, and the performance of the overall system is improved. By sorting the timestamps of DDL operations, the order and conflicts of each operation can be judged more accurately, the waiting time in the traditional synchronization process is reduced, and the real-time performance is improved. Whether there is a conflict is judged according to the resource overlap degree between two DDL operations, avoiding the coarse-grained judgment through a simple time window or trigger mechanism. This method can more accurately identify possible conflicting operations, thereby improving the accuracy of synchronization operations and avoiding potential errors or inconsistencies. By directly obtaining the preset weight coefficient of the DDL operation and performing priority sorting, the system can avoid the intervention of the middle layer, thereby improving the overall performance of the system. By dynamically adjusting the execution order and priority, it is ensured that the operations are executed in the order of priority, which can quickly respond and reduce latency. By dynamically adjusting the execution order of DDL operations according to the preset weight coefficient and priority value, conflicts can be processed in real time and it is ensured that higher-priority operations are executed first. Through the priority dynamic adjustment mechanism based on the preset weight coefficient, different priorities can be set according to different business requirements to ensure that critical DDL operations can be executed first, avoiding unnecessary delays or errors caused by conflicts. By obtaining the preset weight coefficient of each operation and calculating the priority value, the system can flexibly adjust the execution order according to the current environment. By obtaining the execution result feedback in real time and adjusting the execution order, the system can quickly adapt to the occurrence of errors, re-evaluate and adjust the execution strategy of each operation, and avoid the global impact caused by a single point of failure. In a heterogeneous database environment, directly transmitting the DDL operation record to the corresponding database node for execution can ensure that each database independently processes its own DDL operations without relying on a central coordination layer. This not only simplifies the cross-database coordination process but also reduces the performance bottleneck caused by the out-of-sync coordination mechanism. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 It is a schematic flowchart of the method according to an embodiment of the present invention.

[0079] Figure 2 It is a schematic structural diagram of the device according to an embodiment of the present invention.

[0080] Figure 3 It is a schematic internal structure diagram of a computer device according to an embodiment of the present application.

[0081] The implementation, functional features and advantages of the present invention will be further described in conjunction with embodiments with reference to the accompanying drawings. Specific embodiments

[0082] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0083] As Figures 1 - 3 shown, the present application provides a method for real-time optimization of heterogeneous database DDL synchronization to prevent conflicts, including:

[0084] S1. Obtain multiple DDL operation records of each heterogeneous database in real time, and extract the operation timestamps of each of the DDL operation records;

[0085] S2. Sort all the DDL operation records according to each of the operation timestamps to obtain a global time sequence;

[0086] S3. Obtain the target table structures of every two adjacent DDL operation records in sequence according to the global time sequence, and obtain the resource overlap degree of the corresponding two DDL operation records according to every two of the target table structures;

[0087] S4. Determine whether the resource overlap degree is greater than a preset overlap degree;

[0088] If the resource overlap degree is greater than the preset overlap degree, it is determined that a conflict occurs in the corresponding two adjacent DDL operation records, and the preset weight coefficients of the corresponding two DDL operation records are obtained;

[0089] S5. Obtain the priority value of the corresponding DDL operation record according to each of the preset weight coefficients, and dynamically adjust the execution order identification table according to the priority value to obtain an execution DDL operation sequence;

[0090] S6. Transmit each DDL operation record to the nodes of the corresponding heterogeneous database for execution in sequence according to the execution DDL operation sequence, and obtain the execution result feedback in real time;

[0091] If the execution result feedback fails, return to the step of obtaining the preset weight coefficients of the corresponding two DDL operation records until the execution result feedback is successful.

[0092] As described in the above steps S1 - S6, the current real - time optimization method for preventing conflicts in heterogeneous database DDL synchronization mainly coordinates DDL synchronization between different databases by introducing an intermediate layer and reduces the probability of conflicts generated when multiple databases modify the same table structure or data simultaneously through incremental synchronization. To a certain extent, it can ensure the efficient synchronization of data. However, due to the addition of the intermediate layer increasing the complexity of the system architecture, it may lead to performance bottlenecks and poor real - time performance. Especially in the case of asynchronous synchronization, there may be synchronization delays and it cannot achieve true real - time synchronization, with a relatively high complexity. In the present invention, by extracting the operation timestamps of each DDL operation record, sorting all DDL operation records according to each operation timestamp to obtain a global time sequence, sequentially obtaining the target table structures of every two adjacent DDL operation records through the global time sequence, and obtaining the resource overlap degree of the corresponding two DDL operation records according to every two target table structures, determining whether the resource overlap degree is greater than a preset overlap degree. If the resource overlap degree is greater than the preset overlap degree, it is determined that the corresponding two adjacent DDL operation records conflict. The present invention directly relies on timestamps and resource overlap degrees to detect DDL operation conflicts, can eliminate the dependence on the intermediate layer, thereby simplifying the system architecture. By using the global time sequence to sequentially process DDL operations, it reduces the dependence on the intermediate layer and improves the overall performance of the system. By sorting the DDL operation timestamps, it can more accurately judge the order and conflicts of each operation, reduces the waiting time in the traditional synchronization process, and improves real - time performance. By judging whether there is a conflict according to the resource overlap degree between two DDL operations, it avoids the coarse - grained judgment through a simple time window or trigger mechanism. This method can more accurately identify possible conflicting operations, thereby improving the accuracy of synchronization operations and avoiding potential errors or inconsistencies. By adjusting the preset resource overlap degree threshold, different degrees of DDL operation conflicts can be flexibly managed. For example, if the cost of operation conflicts is small, a certain degree of overlap can be tolerated; if the cost is large, the system logic can be adjusted to adopt more strict conflict resolution measures, so as to better meet the requirements of different business scenarios. Through the conflict detection mechanism based on operation timestamps and resource overlap degrees, the system can more intelligently process DDL operations between different databases, reduce the synchronization mechanism that depends on the intermediate layer, and enhance the adaptability to the asynchronous environment. The system independently performs conflict detection and synchronization judgment through each database node, avoiding the bottleneck problems that may occur in the intermediate layer during large - scale expansion. In this way, as the system scale grows, good performance and scalability can be maintained, and the preset weight coefficients corresponding to the two DDL operation records are obtained. According to each preset weight coefficient, the priority value of the corresponding DDL operation record is obtained, and the execution order identification table is dynamically adjusted according to the priority value to obtain an execution DDL operation sequence. By executing the execution DDL operation sequence, each DDL operation record is sequentially transmitted to the nodes of the corresponding heterogeneous database for execution and the execution result feedback is obtained in real - time.If the execution result feedback fails, return to the step of obtaining the preset weight coefficients corresponding to the two DDL operation records until the execution result feedback is successful. By directly obtaining the preset weight coefficients of the DDL operations and performing priority sorting, the system can avoid the intervention of the middle layer, thereby improving the overall performance of the system. By dynamically adjusting the execution order and priority, it is ensured that the operations are executed in the order of priority, which can quickly respond and reduce latency. Especially in an asynchronous synchronization environment, it can avoid unnecessary waiting to the greatest extent. By dynamically adjusting the execution order of the DDL operations according to the preset weight coefficients and priority values, conflicts can be processed in real time and it is ensured that higher-priority operations are executed first. This real-time priority scheduling allows the DDL operations to be executed on demand, avoiding the latency caused by the middle layer processing in the traditional synchronization mechanism, thereby improving the real-time performance of the system. If a DDL operation fails, the system will return to re-obtain the weight coefficients and adjust the execution order until it is successful. This feedback mechanism ensures that the system can quickly react and adjust when encountering execution problems, thereby achieving more efficient fault recovery and real-time execution. When it is found that two adjacent DDL operations conflict, through the priority dynamic adjustment mechanism based on the preset weight coefficients, different priorities can be set according to different business requirements to ensure that critical DDL operations can be executed first, avoiding unnecessary latency or errors caused by conflicts. By obtaining the preset weight coefficients of each operation and calculating the priority values, the system can flexibly adjust the execution order according to the current environment. For example, when the load is light, low-priority operations can be executed quickly, while when the system load is high, important DDL operations can be executed first. By obtaining the execution result feedback in real time and adjusting the execution order, the system can quickly adapt to the occurrence of errors, re-evaluate and adjust the execution strategy of each operation, and avoid the global impact caused by a single point of failure. In a heterogeneous database environment, directly transmitting the DDL operation records to the corresponding database nodes for execution can ensure that each database independently processes its own DDL operations without relying on a central coordination layer. This not only simplifies the cross-database coordination process but also reduces the performance bottleneck caused by the out-of-sync coordination mechanism.

[0093] In one embodiment, the step S3 of obtaining the resource overlap degree of the corresponding two DDL operation records according to every two of the target table structures includes:

[0094] S31. Obtain the key elements of each of the target table structures, where the key elements include column names, data types, constraints, and indexes;

[0095] S32. Obtain the first quantity of the same column names in the two target table structures, and obtain the quantity of column names in the two target table structures;

[0096] S33. Obtain the total number of column names based on the number of column names in the two target table structures, and obtain the column name overlap degree according to the first quantity and the total number of column names;

[0097] S34. Obtain the second quantity of the same data types in the two target table structures, and obtain the number of data types in the two target table structures;

[0098] S35. Obtain the total number of data types based on the number of data types in the two target table structures, and obtain the data type overlap degree according to the second quantity and the total number of data types;

[0099] S36. Obtain the third quantity of the same constraints in the two target table structures, and obtain the number of constraints in the two target table structures;

[0100] S37. Obtain the total number of constraints based on the number of constraints in the two target table structures, and obtain the constraint overlap degree according to the third quantity and the total number of constraints;

[0101] S38. Obtain the fourth quantity of the same indexes in the two target table structures, and obtain the number of indexes in the two target table structures;

[0102] S39. Obtain the total number of indexes based on the number of indexes in the two target table structures, and obtain the index overlap degree according to the fourth quantity and the total number of indexes;

[0103] S310. Obtain the resource overlap degree of the corresponding two DDL operation records according to the index overlap degree, constraint overlap degree, data type overlap degree, and column name overlap degree.

[0104] As described in the above steps S31 - S310, the present invention obtains the column names, data types, constraints, and indexes of the key elements of each target table structure, obtains the total number of column names through the number of column names of the two target table structures, and obtains the column name overlap degree according to the first number of identical column names and the total number of column names in the two target table structures. It obtains the total number of data types according to the number of data types of the two target table structures, and obtains the data type overlap degree according to the second number of identical data types and the total number of data types in the two target table structures. It obtains the total number of constraints according to the number of constraints of the two target table structures, and obtains the constraint overlap degree according to the third number of identical constraints and the total number of constraints in the two target table structures. It obtains the total number of indexes according to the number of indexes of the two target table structures, and obtains the index overlap degree according to the fourth number of identical indexes and the total number of indexes in the two target table structures. It obtains the resource overlap degree of the corresponding two DDL operation records according to the index overlap degree, constraint overlap degree, data type overlap degree, and column name overlap degree. By obtaining the overlap degrees of column names, data types, constraints, and indexes, it can help identify the similarities and differences between target tables. These overlap degree indicators (such as column name overlap degree, data type overlap degree, etc.) can help developers or system designers formulate more effective synchronization strategies. For example, when the overlap degrees of column names and data types between two tables are relatively high, it may not be necessary to fully synchronize all their data or DDL operations. Instead, incremental synchronization or only synchronizing the non - overlapping parts can be selected, thereby reducing the burden on the system. By quantifying the overlap degrees between different target table structures, the system can, to a certain extent, automatically identify which tables have tight synchronization requirements and which have loose ones. In this way, the synchronization mechanism can be optimized, and the complexity of intermediate layer coordination can be reduced. Using these overlap degree indicators can reduce the differences in table structures processed by the system during synchronization and avoid introducing excessive complexity in the architecture design. For example, if there is a high overlap degree of column names or data types in the DDL operations between two databases, more efficient synchronization of these table structures can be selected, while avoiding unnecessary intermediate layer processing. By calculating and adjusting the overlap degree, it can help optimize the synchronization operation.For example, in the case of asynchronous synchronization, the system can intelligently select the synchronization frequency, synchronization strategy, and the table data to be synchronized, which can reduce the burden on the intermediate layer and avoid the need to process DDL operations for all tables during each synchronization, thereby alleviating the system's performance bottleneck. By calculating the overlap degree, the amount of computation and data transmission during each synchronization can be reduced, resulting in a significant improvement in the overall performance and response speed of the system. As the database structure evolves, the overlap degree can be calculated and monitored in real time, and the synchronization strategy can be dynamically adjusted. For changes in the overlap degree of different tables, the synchronization frequency, method, and content can be flexibly adjusted to ensure that the system is always in an optimal state. Based on the overlap degree of column names, data types, constraints, and indexes, better resource coordination can be achieved, which helps manage cross-database DDL operations, avoid resource contention or duplicate operations, and improve the overall resource utilization rate of the system.

[0105] In one embodiment, step S5 of obtaining the priority value of the corresponding DDL operation record according to each preset weight coefficient includes:

[0106] S51. Obtain the resource utilization rate of the corresponding heterogeneous database according to the DDL operation record;

[0107] S52. Obtain the first minimum adjustment factor under high load and the first maximum adjustment factor under low load of the corresponding heterogeneous database according to the resource utilization rate;

[0108] S53. Calculate the system load adjustment factor according to the first maximum adjustment factor, the first minimum adjustment factor, and the resource utilization rate, where the calculation formula is:

[0109] F(TZ) = Z(XT) + [Z(DT) - Z(XT)] * [1 - Z(LY)];

[0110] Among them, F(TZ) represents the system load adjustment factor, Z(XT) represents the first minimum adjustment factor, Z(DT) represents the first maximum adjustment factor, and Z(LY) represents the resource utilization rate;

[0111] S54. Obtain the resource overlap degree of two DDL operation records, and respectively obtain the low conflict frequency and the high conflict frequency according to the resource overlap degree;

[0112] S55. Obtain the second minimum adjustment factor according to the low conflict frequency, and obtain the second maximum adjustment factor according to the high conflict frequency;

[0113] S56. Obtain the conflict frequency adjustment factor according to the second minimum adjustment factor, the second maximum adjustment factor, and the resource overlap degree;

[0114] S57. Obtain the corresponding operation timestamp and the current time according to the DDL operation record, and obtain the time difference according to the current time and the operation timestamp;

[0115] S58. Obtain the preset weight coefficient, and calculate the priority value of the corresponding DDL operation record according to the time difference, conflict frequency adjustment factor, system load adjustment factor, preset weight coefficient, and resource overlap degree, where the calculation formula is:

[0116] Y(XJ) = F(TZ) * S(JC) + C(TZ) * Y(QZ) * Z(CD);

[0117] Among them, Y(XJ) represents the priority value, F(TZ) represents the system load adjustment factor, S(JC) represents the time difference, C(TZ) represents the conflict frequency adjustment factor, Y(QZ) represents the preset weight coefficient, and Z(CD) represents the resource overlap degree.

[0118] As described in the above steps S51 - S58, the present invention obtains the resource utilization rate of the corresponding heterogeneous database through DDL operation records, obtains the first minimum adjustment factor under high load and the first maximum adjustment factor under low load of the corresponding heterogeneous database according to the resource utilization rate, calculates the system load adjustment factor based on the first maximum adjustment factor, the first minimum adjustment factor and the resource utilization rate, obtains the low conflict frequency and the high conflict frequency respectively through the resource overlap degree of two DDL operation records, obtains the second minimum adjustment factor according to the low conflict frequency, obtains the second maximum adjustment factor according to the high conflict frequency, and obtains the conflict frequency adjustment factor through the second minimum adjustment factor, the second maximum adjustment factor and the resource overlap degree. Among them, the method of calculating the conflict frequency adjustment factor through the second minimum adjustment factor, the second maximum adjustment factor and the resource overlap degree is the same as the calculation method of the system load adjustment factor. By directly calculating based on the resource utilization rate, the load adjustment factor and the conflict frequency adjustment factor to coordinate the heterogeneous database, rather than relying on the middle layer to handle the synchronization logic, the complexity of the architecture can be effectively reduced. Although the introduction of the middle layer can play a coordinating role, it will increase the additional maintenance and management costs and may lead to performance bottlenecks. By improving the calculation mechanism, the dependence on the middle layer is reduced, and the complexity of the system architecture is lowered. By obtaining the resource utilization rate of the database in real time and adjusting the corresponding load and conflict frequency adjustment factors, it can ensure that the resources of different databases are optimally utilized under high load and low load. The calculation of the load adjustment factor is based on the resource utilization rate, which helps to appropriately adjust the database in both high load and low load situations, ensuring the overall stability and performance of the system. The middle layer often causes delays in data synchronization, especially in the asynchronous synchronization mode, and such delays may lead to data consistency problems.Through the dynamic adjustment of the resource utilization rate and conflict frequency adjustment factors, the latency during the synchronization process can be reduced, improving real-time performance. The dynamic calculation of the load and conflict adjustment factors enables the system to quickly respond to changing workloads and resource conditions, avoiding the inevitable latency issues in traditional intermediate layer synchronization methods. Through the optimized calculation of the system load and conflict frequency adjustment factors, the bottleneck problems that may occur in the database when the load is high can be effectively reduced. Especially in the case of a high degree of resource overlap and a large conflict frequency, the calculation method of the adjustment factors can help alleviate the conflicts and latency of database operations, thereby improving the system's throughput and response speed. By calculating the conflict frequency adjustment factor, the data synchronization frequency can be precisely adjusted, especially optimized for low-conflict and high-conflict situations. Through the calculation of the resource overlap degree and conflict frequency, the synchronization timing can be predicted and controlled more accurately, thus avoiding latency and unnecessary synchronization conflicts. The system dynamically adjusts the synchronization operation timing according to the load and conflict situations, contributing to faster data synchronization and lower latency. By adopting a dynamic calculation method based on the resource utilization rate, load adjustment factor, and conflict frequency, the system can adaptively adjust according to the real-time resource status and load conditions. This enables the system to better handle different types of load changes, enhancing the system's stability and flexibility. Especially in resource-constrained or high-concurrency scenarios, the system can automatically adjust to meet the challenges. There are often differences in resources, architectures, and loads between heterogeneous databases. This method takes these differences into account by dynamically calculating the adjustment factors, thereby improving the coordination and consistency between different databases, avoiding over-reliance on the intermediate layer for coordination work, and helping to reduce problems caused by synchronization inconsistencies between heterogeneous systems. By obtaining the corresponding operation timestamp and the current time from the DDL operation record, and calculating the time difference based on the current time and the operation timestamp. By obtaining the preset weight coefficient, and calculating the priority value of the corresponding DDL operation record according to the time difference, conflict frequency adjustment factor, system load adjustment factor, preset weight coefficient, and resource overlap degree. By calculating the priority value based on dynamic parameters such as the operation timestamp, time difference, conflict frequency, and system load, the system can adjust the priority of the synchronization operation according to the real-time situation. For example, some DDL operations may require more urgent processing (such as operations involving data consistency), while others may be able to be delayed.This fine-grained priority calculation helps optimize the operation execution order, avoiding unnecessary blocking and delays. By combining the conflict frequency adjustment factor and resource overlap degree to adjust the operation priority, it is possible to reduce resource conflicts and competition between databases, avoid data consistency problems caused by synchronization operations between multiple databases, and an effective resource scheduling mechanism can ensure the coordinated execution of DDL operations for each database, reducing performance degradation caused by conflicts. The introduction of the system load adjustment factor enables flexible adjustment of the DDL synchronization priority according to the current system state (such as CPU, memory, etc.) under different load conditions. This means that the system can automatically adjust the synchronization operation according to the current operating conditions, avoiding bottlenecks caused by excessive synchronization operations during high load or quickly completing operations during low load, improving the overall resource utilization rate. By calculating the specific priority, the system can achieve finer-grained control of the synchronization process. For example, when facing multiple database synchronizations, the system can accurately divide priorities between operations, avoiding the inefficiency of blind synchronization, improving the real-time synchronization ability and overall system coordination. Through priority adjustment based on the above methods, the system can dynamically manage the operation order and delay according to the real-time situation, minimizing the negative impact of asynchronous synchronization. For example, if it is detected that the delay of certain operations is relatively high, the system may automatically adjust their priorities, first process other operations or trigger other emergency measures.

[0119] In one embodiment, the step S5 of dynamically adjusting the execution order identification table according to the priority value to obtain the DDL operation execution sequence includes:

[0120] S59. Obtain the first priority value and the second priority value of two adjacent DDL operation records with conflicts in the execution order identification table according to the priority value;

[0121] S510. Determine whether the first priority value is less than the second priority value;

[0122] If the first priority value is less than the second priority value, adjust the DDL operation record corresponding to the first priority value to after the DDL operation record corresponding to the second priority value;

[0123] If the first priority value is not less than the second priority value, obtain the third priority value of the DDL operation record before the DDL operation record corresponding to the first priority value from the execution order identification table, and determine whether the third priority value is less than the first priority value;

[0124] If the third priority value is less than the first priority value, adjust the DDL operation record corresponding to the third priority value to after the DDL operation record corresponding to the first priority value;

[0125] If the third priority value is not less than the first priority value, obtain the fourth priority value of the DDL operation record that is one position after the DDL operation record corresponding to the second priority value from the execution order identification table, and determine whether the fourth priority value is greater than the second priority value;

[0126] If the fourth priority value is greater than the second priority value, adjust the DDL operation record corresponding to the fourth priority value to before the DDL operation record corresponding to the second priority value until all conflicting DDL operation records in the execution order table are traversed to obtain the DDL operation execution sequence.

[0127] As described in the above steps S59 - S510, the present invention obtains the first priority value and the second priority value of two adjacent DDL operation records with conflicts in the execution order identification table through the priority value, and determines whether the first priority value is less than the second priority value. If the first priority value is less than the second priority value, the DDL operation record corresponding to the first priority value is adjusted to be after the DDL operation record corresponding to the second priority value. If the first priority value is not less than the second priority value, the third priority value of the DDL operation record before the DDL operation record corresponding to the first priority value is obtained from the execution order identification table, and it is determined whether the third priority value is less than the first priority value. If the third priority value is less than the first priority value, the DDL operation record corresponding to the third priority value is adjusted to be after the DDL operation record corresponding to the first priority value. If the third priority value is not less than the first priority value, the fourth priority value of the DDL operation record after the DDL operation record corresponding to the second priority value is obtained from the execution order identification table, and it is determined whether the fourth priority value is greater than the second priority value. If the fourth priority value is greater than the second priority value, the DDL operation record corresponding to the fourth priority value is adjusted to be before the DDL operation record corresponding to the second priority value, until all the conflicting DDL operation records in the execution order table are traversed to obtain the DDL operation execution sequence. By using the priority value to precisely control the order of DDL operations, it is ensured that adjacent DDL operations can be re - sorted according to the priority when conflicts occur. By adjusting the priority and judging the priority relationship between adjacent operations, the occurrence of conflicts can be effectively avoided, thus ensuring the consistency and integrity of data. By directly adjusting the operation order in the execution order identification table without the need to introduce a complex intermediate layer coordination mechanism, traditional solutions may require multiple levels of coordination and synchronization, which will increase the complexity of the system and bring potential performance bottlenecks. By adjusting the priority value in the execution order identification table, conflicts can be directly handled at the database level, thereby reducing the complexity of the system architecture. By avoiding the introduction of an additional intermediate layer, the system can schedule and execute DDL operations more efficiently, reducing the synchronization delay caused by the intermediate layer. Through the priority mechanism, the execution order can be flexibly adjusted to ensure that the operation order is reasonable under specific conditions. This mechanism is not limited to the current conflict resolution, but also provides an extended space for new operations or requirements that may appear in the future. By dynamically adjusting the operation order, problems caused by synchronization delay can be reduced, especially in the case of asynchronous synchronization. Since there is no dependence on an external coordination layer, adjustments can be directly made in the execution sequence, thereby reducing the synchronization time and delay and improving real - time performance.This enables the system to synchronize more efficiently and accurately among multiple databases. By introducing a method of adjusting the execution order of DDL operations through priority values, the present invention solves the complexity and performance bottleneck problems brought by traditional DDL synchronization between different databases through an intermediate layer, thereby effectively improving the real-time performance, performance, and flexibility of the system, reducing synchronization latency, and optimizing the architecture of the entire system.

[0128] In one embodiment, step S6 of sequentially transmitting each DDL operation record to the corresponding node of the heterogeneous database according to the DDL operation execution sequence includes:

[0129] S61. Obtain the real-time network load and initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the DDL operation execution sequence;

[0130] S62. Set a maximum load threshold, and calculate the adjusted transmission rate according to the maximum load threshold, real-time network load, and initial transmission rate. The calculation formula is:

[0131]

[0132] where T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold;

[0133] S63. Determine the master node and slave nodes of each DDL operation record according to the DDL operation execution sequence, and respectively obtain the receive window sizes of each master node and slave node;

[0134] S64. Divide each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size;

[0135] S65. Transmit each data packet to the corresponding node of the heterogeneous database for execution through an adaptive transmission protocol.

[0136] As described in the above steps S61 - S65, the present invention obtains the real - time network load and the initial transmission rate of the heterogeneous database corresponding to each DDL operation record by executing the DDL operation sequence. By setting a maximum load threshold and calculating and adjusting the transmission rate based on the maximum load threshold, real - time network load, and initial transmission rate. By executing the DDL operation sequence to determine the master node and slave nodes of each DDL operation record, and respectively obtaining the receive window sizes of each master node and slave node. By adjusting the transmission rate and corresponding to the receive window size, each DDL operation record is divided into multiple data packets. By the adaptive transmission protocol, each data packet is transmitted to the nodes of the corresponding heterogeneous database for execution. The present invention avoids the complexity of using an intermediate layer in the traditional method. By optimizing the network load and transmission rate, it directly synchronizes between heterogeneous databases without the need for an additional synchronization management layer or middleware, directly operating on database nodes, reducing the architecture level and management cost. By setting a maximum load threshold and dynamically adjusting according to the real - time network load and initial transmission rate, it avoids excessive load and network congestion, thus improving the transmission efficiency and performance. By dividing each DDL operation record into multiple data packets and adjusting according to the receive window size and transmission rate, it helps to balance the data flow and avoid network congestion caused by an overly large single data packet, thus enhancing the efficiency and stability of the transmission. The present invention can dynamically adjust the network load and transmission rate and directly control the transmission path (master node and slave nodes), which can avoid the problems of delay accumulation and synchronization delay in the traditional method. Through the adaptive transmission protocol and the strategy of dividing data packets, it ensures that DDL operations can be synchronized to the target database nodes more quickly and accurately, solving the problem of poor real - time performance that may exist in asynchronous synchronization. By setting a load threshold and dynamically adjusting the transmission rate, it effectively avoids the synchronization delay that may be caused by excessive network load or resource limitations in the traditional method. Each database node (master node and slave node) has an independent receive window. By precisely controlling the window size, it ensures that data packets can be efficiently and quickly delivered to the target node, avoiding delays caused by untimely transmission. By setting up an independent adjustment mechanism (such as adjusting the transmission rate, receive window size, etc.) for different database nodes, it can flexibly adapt to the performance characteristics and requirements of different heterogeneous databases, further improving the overall efficiency and stability of the system. Through precise packet segmentation, transmission rate adjustment, and receive window control, this method has good scalability and can adapt to database clusters of different scales and the increasing demand for DDL operations.

[0137] In one embodiment, step S63 of determining the master node and slave nodes of each DDL operation record according to the execution of the DDL operation sequence includes:

[0138] S631. Obtain the operation type feature of each of the DDL operation records according to the DDL operation sequence to be executed, where the operation type feature includes at least one of a creation operation, a modification operation, a deletion operation, a truncation operation, a rename operation, and an index operation;

[0139] S632. Determine whether the operation type feature belongs to a modification operation;

[0140] If the operation type feature belongs to a modification operation, transmit the DDL operation record corresponding to the operation type feature to the master node for execution;

[0141] If the operation type feature does not belong to a modification operation, transmit the DDL operation record corresponding to the operation type feature to the slave node for execution.

[0142] As described in the above steps S631 - S632, the present invention obtains the operation type characteristics of each of the DDL operation records by executing a DDL operation sequence. Among them, the operation type characteristics include at least one of create operation, modify operation, delete operation, truncate operation, rename operation, and index operation. It is judged whether the operation type characteristics belong to the modify operation. If the operation type characteristics belong to the modify operation, the DDL operation record corresponding to the operation type characteristics is transmitted to the master node for execution. If the operation type characteristics do not belong to the modify operation, the DDL operation record corresponding to the operation type characteristics is transmitted to the slave node for execution. The present invention directly transmits the DDL operation to the appropriate node (master node or slave node) for execution according to the operation type characteristics, avoiding the introduction of a complex intermediate layer for coordination. Through classification operations (such as modify operation, delete operation, etc.), it is possible to clearly determine the execution location of each DDL operation, avoiding the cumbersome scheduling process and additional coordination mechanism that may occur in the traditional method. The modify operation is handed over to the master node for execution, and other operations (such as create, delete, etc.) are handed over to the slave node for execution, which can optimize the node load according to the characteristics of different operations. The master node is usually used to process operations that require global consistency, and the slave node processes other operations to achieve load balancing, avoiding the performance bottleneck caused by centralized processing. By intelligently judging the operation type and distributing it to different nodes, it is avoided to execute all DDL operations on a single node, thereby reducing the operation latency. Transmitting the DDL operation to the master node or slave node quickly according to the operation type can effectively avoid the latency problem in the asynchronous synchronization process. The timely execution of the modify operation ensures data consistency, and the scattered execution of other operations ensures that no additional synchronization latency is generated. By classifying operations and distributing them to different nodes (master node or slave node), it has high scalability. Compared with the traditional complex method coordinated by the intermediate layer, this intelligent scheduling - based scheme can synchronize data more efficiently, reduce latency, and improve execution efficiency when processing large - scale DDL operations, thereby achieving better performance and real - time synchronization in a heterogeneous database environment.

[0143] The present application also provides a heterogeneous database DDL synchronization conflict - prevention real - time optimization system, including:

[0144] An extraction module, configured to obtain multiple DDL operation records of each heterogeneous database in real - time and extract the operation timestamp of each of the DDL operation records;

[0145] A sorting module, configured to sort all DDL operation records according to each of the operation timestamps to obtain a global time sequence;

[0146] An acquisition module, configured to sequentially acquire the target table structures of every two adjacent DDL operation records according to the global timing sequence, and acquire the resource overlap degree of the corresponding two DDL operation records according to every two target table structures;

[0147] A judgment module, configured to judge whether the resource overlap degree is greater than a preset overlap degree;

[0148] If the resource overlap degree is greater than the preset overlap degree, it is determined that a conflict occurs between the corresponding two adjacent DDL operation records, and the preset weight coefficients of the corresponding two DDL operation records are acquired;

[0149] An adjustment module, configured to acquire the priority value of the corresponding DDL operation record according to each preset weight coefficient, and dynamically adjust the execution order identification table according to the priority value to obtain an execution DDL operation sequence;

[0150] A transmission module, configured to sequentially transmit each DDL operation record to the nodes of the corresponding heterogeneous database for execution according to the execution DDL operation sequence, and obtain real-time execution result feedback;

[0151] If the execution result feedback fails, return to the step of acquiring the preset weight coefficients of the corresponding two DDL operation records until the execution result feedback is successful.

[0152] In one embodiment, the transmission module includes:

[0153] An acquisition unit, configured to acquire the real-time network load and initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the execution DDL operation sequence;

[0154] A calculation unit, configured to set a maximum load threshold, and calculate and adjust the transmission rate according to the maximum load threshold, real-time network load and initial transmission rate, where the calculation formula is:

[0155]

[0156] where, T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold;

[0157] A determination unit, configured to determine the primary node and sub-nodes of each DDL operation record according to the execution DDL operation sequence, and respectively acquire the receive window sizes of each primary node and sub-nodes;

[0158] A segmentation unit, configured to segment each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size;

[0159] A transmission unit, configured to transmit each data packet to a node of a corresponding heterogeneous database for execution through an adaptive transmission protocol.

[0160] It should be noted that each module and unit in the heterogeneous database DDL synchronization anti-collision real-time optimization system corresponds one by one to the steps in the heterogeneous database DDL synchronization anti-collision real-time optimization method.

[0161] As Figure 3 shown, the present application also provides a computer device, which may be a server, and its internal structure may be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store all the data required for the process of the heterogeneous database DDL synchronization anti-collision real-time optimization method. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes the heterogeneous database DDL synchronization anti-collision real-time optimization method.

[0162] Those skilled in the art can understand that Figure 3 the structure shown in

[0163] is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied.

[0164] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be obtained in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0165] It should be noted that in this article, the terms "including", "comprising", or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, device, article, or method including a series of elements not only includes those elements but also includes other elements not explicitly listed, or further includes elements inherent to such a process, device, article, or method. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of another identical element in the process, device, article, or method including that element.

[0166] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A real-time optimization method for preventing conflicts in heterogeneous database DDL synchronization, characterized in that, Including: Obtain multiple DDL operation records of each heterogeneous database in real time, and extract the operation timestamps of each of the DDL operation records; Sort all the DDL operation records according to each of the operation timestamps to obtain a global time sequence; Obtain the target table structures of every two adjacent DDL operation records in sequence according to the global time sequence, and obtain the resource overlap degree of the corresponding two DDL operation records according to every two of the target table structures; Determine whether the resource overlap degree is greater than a preset overlap degree; If the resource overlap degree is greater than the preset overlap degree, then determine that the corresponding two adjacent DDL operation records conflict, and obtain the preset weight coefficients of the corresponding two DDL operation records; Obtain the priority value of the corresponding DDL operation record according to each of the preset weight coefficients, and dynamically adjust the execution order identification table according to the priority value to obtain an execution DDL operation sequence; Transmit each DDL operation record to the node of the corresponding heterogeneous database for execution in sequence according to the execution DDL operation sequence, and obtain the execution result feedback in real time; If the execution result feedback fails, then return to the step of obtaining the preset weight coefficients of the corresponding two DDL operation records until the execution result feedback is successful.

2. The real-time optimization method for preventing conflicts in heterogeneous database DDL synchronization according to claim 1, characterized in that The step of obtaining the resource overlap degree of the corresponding two DDL operation records according to every two of the target table structures includes: Obtain the key elements of each of the target table structures, where the key elements include column names, data types, constraints, and indexes; Obtain the first quantity of the same column names in two target table structures, and obtain the quantity of column names of the two target table structures; Obtain the total quantity of column names according to the quantity of column names of the two target table structures, and obtain the column name overlap degree according to the first quantity and the total quantity of column names; Obtain the second quantity of the same data types in two target table structures, and obtain the quantity of data types of the two target table structures; Obtain the total quantity of data types according to the quantity of data types of the two target table structures, and obtain the data type overlap degree according to the second quantity and the total quantity of data types; Obtain the third quantity of the same constraints in two target table structures, and obtain the quantity of constraints of the two target table structures; Obtain the total quantity of constraints according to the quantity of constraints of the two target table structures, and obtain the constraint overlap degree according to the third quantity and the total quantity of constraints; Obtain the fourth quantity of the same indexes in two target table structures, and obtain the quantity of indexes of the two target table structures; Obtain the total quantity of indexes according to the quantity of indexes of the two target table structures, and obtain the index overlap degree according to the fourth quantity and the total quantity of indexes; Obtain the resource overlap degree of the corresponding two DDL operation records according to the index overlap degree, constraint overlap degree, data type overlap degree, and column name overlap degree.

3. The real-time optimization method for preventing conflicts in heterogeneous database DDL synchronization according to claim 1, characterized in that, The step of obtaining the priority value of the corresponding DDL operation record according to each of the preset weight coefficients includes: Obtain the resource utilization rate of the corresponding heterogeneous database according to the DDL operation record; Obtain the first minimum adjustment factor under high load and the first maximum adjustment factor under low load of the corresponding heterogeneous database according to the resource utilization rate; Calculate the system load adjustment factor according to the first maximum adjustment factor, the first minimum adjustment factor, and the resource utilization rate. The calculation formula is as follows: F(TZ) = Z(XT) + [Z(DT) - Z(XT)] * [1 - Z(LY)]; Wherein, F(TZ) represents the system load adjustment factor, Z(XT) represents the first minimum adjustment factor, Z(DT) represents the first maximum adjustment factor, and Z(LY) represents the resource utilization rate; Obtain the resource overlap degree of two DDL operation records, and respectively obtain the low conflict frequency and the high conflict frequency according to the resource overlap degree; Obtain the second minimum adjustment factor according to the low conflict frequency, and obtain the second maximum adjustment factor according to the high conflict frequency; Obtain the conflict frequency adjustment factor according to the second minimum adjustment factor, the second maximum adjustment factor, and the resource overlap degree; Obtain the corresponding operation timestamp and the current time according to the DDL operation record, and obtain the time difference according to the current time and the operation timestamp; Obtain a preset weight coefficient, and calculate the priority value of the corresponding DDL operation record according to the time difference, the conflict frequency adjustment factor, the system load adjustment factor, the preset weight coefficient, and the resource overlap degree. The calculation formula is as follows: Y(XJ) = F(TZ) * S(JC) + C(TZ) * Y(QZ) * Z(CD); Wherein, Y(XJ) represents the priority value, F(TZ) represents the system load adjustment factor, S(JC) represents the time difference, C(TZ) represents the conflict frequency adjustment factor, Y(QZ) represents the preset weight coefficient, and Z(CD) represents the resource overlap degree.

4. The real-time optimization method for preventing conflict in heterogeneous database DDL synchronization according to claim 1, characterized in that The step of dynamically adjusting the execution order identification table according to the priority value to obtain the DDL operation sequence to be executed includes: Obtain the first priority value and the second priority value of two adjacent DDL operation records with conflicts in the execution order identification table according to the priority value; Judge whether the first priority value is less than the second priority value; If the first priority value is less than the second priority value, adjust the DDL operation record corresponding to the first priority value to after the DDL operation record corresponding to the second priority value; If the first priority value is not less than the second priority value, obtain the third priority value of the DDL operation record before the DDL operation record corresponding to the first priority value from the execution order identification table, and judge whether the third priority value is less than the first priority value; If the third priority value is less than the first priority value, adjust the DDL operation record corresponding to the third priority value to after the DDL operation record corresponding to the first priority value; If the third priority value is not less than the first priority value, obtain the fourth priority value of the DDL operation record after the DDL operation record corresponding to the second priority value from the execution order identification table, and judge whether the fourth priority value is greater than the second priority value; If the fourth priority value is greater than the second priority value, adjust the DDL operation record corresponding to the fourth priority value to before the DDL operation record corresponding to the second priority value until all conflicting DDL operation records in the execution order list are traversed to obtain an execution DDL operation sequence.

5. The real-time optimization method for preventing conflict in heterogeneous database DDL synchronization according to claim 1, characterized in that The step of sequentially transmitting each DDL operation record to a node of a corresponding heterogeneous database according to the execution DDL operation sequence includes: Obtain the real-time network load and initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the execution DDL operation sequence; Set a maximum load threshold, and calculate and adjust the transmission rate according to the maximum load threshold, real-time network load, and initial transmission rate, where the calculation formula is: Where, T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold; Determine the primary node and sub-node of each DDL operation record according to the execution DDL operation sequence, and respectively obtain the receive window size of each primary node and sub-node; Divide each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size; Transmit each data packet to a node of a corresponding heterogeneous database for execution through an adaptive transmission protocol.

6. The real-time optimization method for preventing conflict in heterogeneous database DDL synchronization according to claim 5, characterized in that, The step of determining the primary node and sub-node of each DDL operation record according to the execution DDL operation sequence includes: Obtain the operation type feature of each DDL operation record according to the execution DDL operation sequence, where the operation type feature includes at least one of a create operation, a modify operation, a delete operation, a truncate operation, a rename operation, and an index operation; Determine whether the operation type feature belongs to a modify operation; If the operation type feature belongs to a modify operation, transmit the DDL operation record corresponding to the operation type feature to the primary node for execution; If the operation type feature does not belong to a modify operation, transmit the DDL operation record corresponding to the operation type feature to the sub-node for execution.

7. A real-time optimization system for preventing conflicts in heterogeneous database DDL synchronization, characterized in that, Includes: An extraction module for obtaining multiple DDL operation records of each heterogeneous database in real time and extracting the operation timestamp of each DDL operation record; A sorting module for sorting all DDL operation records according to each operation timestamp to obtain a global time sequence; An acquisition module for sequentially obtaining the target table structure of every two adjacent DDL operation records according to the global time sequence, and obtaining the resource overlap degree of the corresponding two DDL operation records according to the two target table structures; A judgment module for judging whether the resource overlap degree is greater than a preset overlap degree; If the resource overlap degree is greater than the preset overlap degree, determine that the corresponding two adjacent DDL operation records conflict, and obtain the preset weight coefficient of the corresponding two DDL operation records; An adjustment module, configured to obtain the priority value of the corresponding DDL operation record according to each of the preset weight coefficients, and dynamically adjust the execution order identification table according to the priority value to obtain an execution DDL operation sequence; A transmission module, configured to sequentially transmit each DDL operation record to the nodes of the corresponding heterogeneous database for execution according to the execution DDL operation sequence, and obtain real-time feedback of the execution result; If the execution result feedback fails, return to the step of obtaining the preset weight coefficients of the corresponding two DDL operation records until the execution result feedback is successful.

8. The heterogeneous database DDL synchronization conflict prevention real-time optimization system according to claim 7, characterized in that The transmission module includes: An obtaining unit, configured to obtain the real-time network load and the initial transmission rate of the heterogeneous database corresponding to each DDL operation record according to the execution DDL operation sequence; A calculating unit, configured to set a maximum load threshold, and calculate an adjusted transmission rate according to the maximum load threshold, the real-time network load, and the initial transmission rate, where the calculation formula is: where T(CS) represents the adjusted transmission rate, C(CS) represents the initial transmission rate, S(FZ) represents the real-time network load, and Z(FZ) represents the maximum load threshold; A determining unit, configured to determine the master node and the slave node of each DDL operation record according to the execution DDL operation sequence, and respectively obtain the receive window sizes of each master node and slave node; A splitting unit, configured to split each DDL operation record into multiple data packets according to the adjusted transmission rate and the corresponding receive window size; A transmission unit, configured to transmit each data packet to the nodes of the corresponding heterogeneous database for execution through an adaptive transmission protocol.

9. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images