Universal data and data table automatic synchronization method

Abstract

The invention discloses an automatic synchronization method for general data and a data table. The automatic synchronization method comprises the following steps: acquiring Category and query parameters; querying the redis cache on the basis of the Category and query parameters, and determining a first query result; when it is determined that the target general data does not exist in the first query result, whether a target data table exists in a local database or not is queried based on query parameters, and a second query result is obtained; when it is determined that the target data table does not exist in the second query result, the local database initiates a synchronization request to the remote server; after the local database receives the incremental data sent by the remote server, synchronously updating the local database based on the incremental data, and storing the incremental data into a redis cache; and high-performance reading of the general data is realized based on a three-level caching strategy, and automatic synchronization of the general data and a data table is realized.

Description

Technical Field

[0001] The present invention relates to the field of data synchronization, and in particular to a method for automatic synchronization of general data and data tables. Background Art

[0002] With the development of computer technology, big data technology has emerged. Big data technology can process a large amount of data in a centralized manner, improving data processing efficiency. In big data processing, the synchronization of common data in different tables and data tables is a very important part.

[0003] In the prior art, (1) synchronization tools are usually used to achieve data synchronization. However, when using synchronization tools for data synchronization, excessive system resources will be occupied, affecting other operations, resulting in slow synchronization or synchronization interruption; (2) There is a problem with the trigger mechanism of data synchronization. In the prior art, synchronization is often performed according to a certain time period, which may result in a certain delay and fail to reflect the latest data changes in real time; if the cycle setting is unreasonable, data may not be updated in a timely manner; (3) When obtaining target data, the remote server is often directly accessed, resulting in increased load pressure on the remote server.

[0004] Therefore, there is an urgent need for a general data and data table automatic synchronization method to solve the above technical problems. Summary of the invention

[0005] The present invention aims to solve at least one of the technical problems in the above-mentioned technology to a certain extent. To this end, the purpose of the present invention is to propose a method for automatic synchronization of general data and data tables, realize high-performance reading of general data based on a three-level cache strategy, and realize automatic synchronization of general data and data tables.

[0006] To achieve the above object, an embodiment of the present invention provides a method for automatically synchronizing general data and data tables, including:

[0007] Get Category and query parameters;

[0008] Query the redis cache based on Category and query parameters to determine the first query result;

[0009] When it is determined that the target general data does not exist in the first query result, query whether the target data table exists in the local database based on the query parameter to obtain a second query result;

[0010] If it is determined that the target data table does not exist in the second query result, the local database initiates a synchronization request to the remote server;

[0011] After the local database receives the incremental data sent by the remote server, it synchronously updates the local database based on the incremental data and stores the incremental data in the redis cache.

[0012] Preferably, it also includes:

[0013] If the target data table exists in the second query result, query whether the target general data exists in the target data table based on the query parameter;

[0014] If the target common data exists in the target data table, the target common data is stored in the redis cache.

[0015] Preferably, it also includes:

[0016] If the target data does not exist in the target data table, the remote server is queried based on the query parameters;

[0017] If the target data corresponding to the query parameter exists in the remote server, the local database initiates a synchronization request to the remote server;

[0018] The remote server determines the incremental data based on the synchronization request;

[0019] The local database is synchronized based on the incremental data, and the incremental data is stored in the redis cache.

[0020] Preferably, querying whether the target data table exists in the local database based on the query parameter to obtain the second query result includes:

[0021] Convert the query parameters into SQL statements;

[0022] Based on the SQL statement, it is queried whether the target data table exists in the local database, and a second query result is determined.

[0023] Preferably, after the local database initiates a synchronization request to the remote server, the method further includes:

[0024] Obtaining current load information and configuration information of the remote server;

[0025] Determine whether to perform data synchronization immediately based on the current load information and configuration information of the remote server;

[0026] When it is determined to perform data synchronization immediately, the remote server sends incremental data to the local database.

[0027] Preferably, determining whether to immediately perform data synchronization based on the current load information and configuration information of the remote server includes:

[0028] Determine the load evaluation index of the remote server based on the preset evaluation index algorithm according to the current load information and configuration information of the remote server;

[0029] Comparing the load evaluation index with a preset evaluation index threshold;

[0030] Whether to immediately perform data synchronization is determined based on the comparison result.

[0031] Preferably, before the remote server sends the incremental data to the local database, the method further includes:

[0032] Acquire the data transmission channel between the remote server and the local database, and obtain several data transmission channels;

[0033] Classifying a plurality of data transmission channels to obtain a first data transmission channel and a second data transmission channel;

[0034] The remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel.

[0035] Preferably, a plurality of data transmission channels are classified to obtain a first data transmission channel and a second data transmission channel, including:

[0036] Obtain a statistical table of historical transmission data information corresponding to each data transmission channel within a preset time period;

[0037] Determine the historical maximum transmission rate corresponding to each data transmission channel based on the historical transmission data information statistical table;

[0038] Comparing the historical maximum transmission rate with a preset transmission rate threshold;

[0039] The data transmission channel whose historical maximum transmission rate is greater than or equal to a preset transmission rate threshold is used as a first data transmission channel;

[0040] The data transmission channel whose historical maximum transmission rate is less than a preset transmission rate threshold is used as the second data transmission channel.

[0041] Preferably, the remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel, including:

[0042] Acquire historical transmission data corresponding to the historical maximum transmission rate of the first data transmission channel;

[0043] Clustering the historical transmission data to obtain a plurality of first classification data corresponding to the historical transmission data;

[0044] Acquire the data volume corresponding to each first classification data, and use the first classification data corresponding to the maximum data volume among the plurality of first classification data as the first classification sub-data of the first data transmission channel;

[0045] Traversing all first data transmission channels to obtain first classification sub-data corresponding to each first data transmission channel;

[0046] Get the preset data classification model;

[0047] Inputting the database change information into a preset data classification model for classification to obtain a plurality of second classification data;

[0048] Randomly select a second classification data as the target classification data;

[0049] Calculate the similarity between the target classification data and each first classification sub-data respectively, and if the maximum similarity is greater than or equal to a preset similarity threshold, use the first data transmission channel corresponding to the first classification sub-data as the target data transmission channel of the target classification data;

[0050] If the maximum similarity value is less than a preset similarity threshold, the target classification data is used as the data to be allocated;

[0051] Traversing all the second classification data, determining a number of target data transmission channels corresponding to the second classification data and a number of data to be allocated;

[0052] Transmitting the plurality of second classified data based on the target data transmission channels corresponding to the plurality of second classified data;

[0053] Transmitting a plurality of data to be allocated based on a second data transmission channel;

[0054] Incremental data is sent to a local database based on a target data transmission channel and a second data transmission channel corresponding to a plurality of second classified data.

[0055] Preferably, the method for determining the incremental data includes:

[0056] After receiving the synchronization request from the local database, the remote server obtains the version difference data between the local database and the remote server; determines the incremental data based on the version difference data; the incremental data includes target common data and target data table.

[0057] The present invention discloses a method for automatic synchronization of general data and data tables. By querying the redis cache first, the frequency of direct access to the local database and remote server is reduced, and the operation pressure of the remote server is relieved. High-performance reading of general data and automatic synchronization of data tables are realized based on a three-level cache strategy, and missing data in the local database can be discovered in time and a synchronization request can be automatically initiated to ensure real-time updating of data. The automated synchronization process reduces manual intervention, reduces management cost and complexity, and reasonably allocates data query and synchronization resources to improve overall resource utilization efficiency.

[0058] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the written description and the accompanying drawings.

[0059] The technical solution of the present invention is further described in detail below through the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0061] Figure 1 is a flow chart of a method for automatic synchronization of general data and data tables according to an embodiment of the present invention;

[0062] Figure 2 is a flow chart of obtaining a second query result according to an embodiment of the present invention;

[0063] Figure 3 is a flowchart of determining whether to perform data synchronization immediately according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0065] Example 1

[0066] like Figure 1 As shown, a method for automatically synchronizing common data and data tables includes steps S1-S5:

[0067] S1: Get Category and query parameters;

[0068] S2: query the redis cache based on Category and query parameters to determine the first query result;

[0069] S3: When it is determined that the target general data does not exist in the first query result, query whether the target data table exists in the local database based on the query parameter to obtain a second query result;

[0070] S4: When it is determined that the target data table does not exist in the second query result, the local database initiates a synchronization request to the remote server;

[0071] S5: After receiving the incremental data sent by the remote server, the local database is synchronously updated based on the incremental data, and the incremental data is stored in the redis cache.

[0072] In this embodiment, Category is the classification of data, which is essentially an identifier.

[0073] In this embodiment, query is a corresponding database query parameter.

[0074] In this embodiment, redis is a memory cache technology with extremely fast data reading and writing speeds.

[0075] In this embodiment, the synchronization request includes the version number of the current local database.

[0076] In this embodiment, the incremental data includes target general data and a target data table.

[0077] The working principle of the above technical solution is: establish a three-layer cache mechanism, in which the redis cache is the first-level cache; the local database is the second-level cache; and the remote server is the third-level cache; by preferentially querying the redis cache, when the target common data does not exist in the redis cache, query whether the target data table exists in the local database based on the query parameter to obtain a second query result; when it is determined that the target data table does not exist in the second query result, the local database initiates a synchronization request to the remote server; after the local database receives the incremental data sent by the remote server, it synchronously updates the local database based on the incremental data and stores the incremental data in the redis cache.

[0078] The beneficial effects of the above technical solution are: by giving priority to querying the redis cache, the frequency of direct access to the local database and remote server is reduced, alleviating the operating pressure of the remote server; based on the three-level cache strategy, high-performance reading of common data and automatic synchronization of data tables are achieved, which can timely detect missing data in the local database and automatically initiate synchronization requests to ensure real-time data updates; the automated synchronization process reduces manual intervention and reduces management costs and complexity; and data query and synchronization resources are reasonably allocated to improve overall resource utilization efficiency.

[0079] Example 2

[0080] Also includes:

[0081] If the target data table exists in the second query result, query whether the target general data exists in the target data table based on the query parameter;

[0082] If the target common data exists in the target data table, the target common data is stored in the redis cache.

[0083] The beneficial effects of the above technical solution are: by first determining whether the target data table exists and then querying the specific data, unnecessary query operations are avoided; using Redis cache can speed up subsequent access to target general data and improve the overall performance of the system; optimizing the data management process, making data acquisition and storage more orderly and efficient; enhancing the stability and reliability of the system, and reducing the possibility of errors through reasonable process design.

[0084] Example 3

[0085] Also includes:

[0086] If the target data does not exist in the target data table, the remote server is queried based on the query parameters;

[0087] If the target data corresponding to the query parameter exists in the remote server, the local database initiates a synchronization request to the remote server;

[0088] The remote server determines the incremental data based on the synchronization request;

[0089] The local database is synchronized based on the incremental data, and the incremental data is stored in the redis cache.

[0090] In this embodiment, the synchronization request includes the version number of the current local server.

[0091] The beneficial effects of the above technical solution are: it further expands the way of data acquisition. When the target data does not exist in the local data table, it can be searched from the remote server, thereby improving the completeness of the data; it realizes the data synchronization mechanism between the local database and the remote server to ensure the consistency and real-time nature of the data; through incremental data synchronization, the data transmission volume and synchronization time are reduced, thereby improving efficiency; storing incremental data in the Redis cache can provide services for subsequent requests faster and improve response speed; it enhances the flexibility and adaptability of the system and can better cope with various data situations; it ensures the continuous updating and accuracy of the data, thereby helping the system to better operate and serve users.

[0092] Example 4

[0093] like Figure 2 As shown, based on the query parameter, it is queried whether the target data table exists in the local database to obtain the second query result, including steps S31-S32:

[0094] S31: converting the query parameters into SQL statements;

[0095] S32: Query whether the target data table exists in the local database based on the SQL statement, and determine a second query result.

[0096] In this embodiment, the query parameter is converted into a SQL statement, and the conditions, fields and other information contained in the query parameter are analyzed; corresponding SQL keywords and expressions are constructed based on this information, such as SELECT followed by a list of fields to be queried, FROM specifies a data table, and the WHERE clause adds conditions, etc.; for the values ​​in the parameters, appropriate format conversion and splicing are performed to ensure compatibility with SQL syntax; and finally a complete SQL statement is combined.

[0097] The beneficial effects of the above technical solution are: by converting parameters into SQL statements, mature database query mechanisms can be used to ensure accuracy and reliability; query results can be obtained quickly, which helps to make corresponding decisions in a timely manner, such as whether it is necessary to further obtain data from other data sources; it is beneficial to optimize system performance and avoid unnecessary data operations and waste of resources; it enhances the maintainability and scalability of the system, and facilitates flexible adjustment according to different needs.

[0098] Example 5

[0099] like Figure 3 As shown, after the local database initiates a synchronization request to the remote server, steps S41-S43 are also included:

[0100] S41: Obtaining current load information and configuration information of the remote server;

[0101] S42: Determine whether to perform data synchronization immediately based on the current load information and configuration information of the remote server;

[0102] S43: When it is determined to perform data synchronization immediately, the remote server sends incremental data to the local database.

[0103] In this embodiment, the load information and configuration information include but are not limited to the server's data transmission rate, the server's data throughput rate, the server's stable parameter value, CPU usage, memory usage, network bandwidth usage, disk I / O status, CPU model and number of cores, network interface type and rate, etc.

[0104] The beneficial effects of the above technical solution are: by obtaining load information, it is possible to avoid synchronization when the remote server is highly loaded, which may cause performance degradation and achieve more reasonable resource scheduling; determine whether to synchronize immediately based on configuration information to ensure that it is performed at the right time, reduce unnecessary waiting and errors, and improve the efficiency and accuracy of data synchronization; avoid excessive pressure on the remote server due to inappropriate synchronization timing, thereby helping to maintain the stable operation of the entire system; only transmit incremental data at the right time, saving network bandwidth and time, and improving the quality and speed of data synchronization; and be able to flexibly adjust the synchronization strategy according to different server states to make the system more adaptable and robust.

[0105] Example 6

[0106] Determining whether to immediately perform data synchronization based on the current load information and configuration information of the remote server includes:

[0107] Determine the load evaluation index of the remote server based on the preset evaluation index algorithm according to the current load information and configuration information of the remote server;

[0108] Comparing the load evaluation index with a preset evaluation index threshold;

[0109] Whether to immediately perform data synchronization is determined based on the comparison result.

[0110] In this embodiment, the preset evaluation index algorithm includes:

[0111]

[0112] Among them, T represents the load evaluation index of the remote server; C represents the idle rate of the CPU in the server; μ represents the data throughput of the server; η represents the data transmission rate of the server; ξ represents the stable parameter value of the server, and the value range is [0.85, 0.95]; P represents the operating efficiency of the service.

[0113] In this embodiment, the preset evaluation index threshold is set in advance based on industry experience.

[0114] The beneficial effects of the above technical solution are: the load evaluation index can more accurately evaluate the current working status of the server so as to make accurate synchronization decisions; it can be dynamically adjusted according to different loads and configurations, thereby enhancing the system's adaptability to various environments; ensuring synchronization when the server load is low and conditions are suitable helps to optimize the overall system performance and avoid performance degradation due to excessive pressure on the server caused by synchronization; a reasonable decision-making mechanism can reduce errors or failures caused by inappropriate synchronization and improve the reliability of the system; judgment based on the evaluation index threshold can better utilize server resources and avoid waste or excessive use of resources; accurate decisions can ensure the quality and integrity of data synchronization and reduce risks such as data inconsistency.

[0115] Example 7

[0116] Before the remote server sends incremental data to the local database, it also includes:

[0117] Acquire the data transmission channel between the remote server and the local database, and obtain several data transmission channels;

[0118] Classifying a plurality of data transmission channels to obtain a first data transmission channel and a second data transmission channel;

[0119] The remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel.

[0120] In this embodiment, the first data transmission channel is a data transmission channel whose historical maximum transmission rate is greater than or equal to a preset transmission rate threshold.

[0121] In this embodiment, the second data transmission channel is a data transmission channel whose historical maximum transmission rate is less than a preset transmission rate threshold.

[0122] The beneficial effects of the above technical scheme are: by classifying the data transmission channels, it is possible to more specifically select appropriate channels for data transmission, optimize the transmission path, and thus improve the transmission efficiency; by distinguishing different types of channels, it is possible to better ensure the stability of data transmission according to their characteristics and reduce the possibility of transmission interruption or errors; it is possible to flexibly adjust the use of different channels according to actual conditions to adapt to various network environments and conditions; it is possible to rationally utilize different channel resources to avoid excessive load on a single channel and achieve optimal allocation of resources; and it is possible to classify and manage channels according to their characteristics, which helps to better implement data security strategies and ensure the security of incremental data transmission.

[0123] Example 8

[0124] Classifying a plurality of data transmission channels to obtain a first data transmission channel and a second data transmission channel includes:

[0125] Obtain a statistical table of historical transmission data information corresponding to each data transmission channel within a preset time period;

[0126] Determine the historical maximum transmission rate corresponding to each data transmission channel based on the historical transmission data information statistical table;

[0127] Comparing the historical maximum transmission rate with a preset transmission rate threshold;

[0128] The data transmission channel whose historical maximum transmission rate is greater than or equal to a preset transmission rate threshold is used as a first data transmission channel;

[0129] The data transmission channel whose historical maximum transmission rate is less than a preset transmission rate threshold is used as the second data transmission channel.

[0130] In this embodiment, the preset transmission rate threshold is set in advance based on industry experience.

[0131] The beneficial effects of the above technical scheme are: through classification, better transmission strategies for different channels can be formulated to improve the overall transmission efficiency and effect; the first data transmission channel is clearly defined as a high-speed channel to ensure that important data or data with high rate requirements can use these channels first to improve the transmission quality; classification is carried out according to rate differences to achieve reasonable allocation of resources and avoid inefficient utilization; understanding the historical maximum transmission rate of each channel helps to select a more stable and reliable channel during the transmission process; adapting to different network environments and data transmission requirements, and flexibly adjusting the transmission plan; after classification, it is convenient to monitor and manage the transmission conditions of different types of channels in a targeted manner; classification based on historical data makes data transmission planning more scientific and reasonable.

[0132] Example 9

[0133] The remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel, including:

[0134] Acquire historical transmission data corresponding to the historical maximum transmission rate of the first data transmission channel;

[0135] Clustering the historical transmission data to obtain a plurality of first classification data corresponding to the historical transmission data;

[0136] Acquire the data volume corresponding to each first classification data, and use the first classification data corresponding to the maximum data volume among the plurality of first classification data as the first classification sub-data of the first data transmission channel;

[0137] Traversing all first data transmission channels to obtain first classification sub-data corresponding to each first data transmission channel;

[0138] Get the preset data classification model;

[0139] Inputting the database change information into a preset data classification model for classification to obtain a plurality of second classification data;

[0140] Randomly select a second classification data as the target classification data;

[0141] Calculate the similarity between the target classification data and each first classification sub-data respectively, and if the maximum similarity is greater than or equal to a preset similarity threshold, use the first data transmission channel corresponding to the first classification sub-data as the target data transmission channel of the target classification data;

[0142] If the maximum similarity value is less than a preset similarity threshold, the target classification data is used as the data to be allocated;

[0143] Traversing all the second classification data, determining a number of target data transmission channels corresponding to the second classification data and a number of data to be allocated;

[0144] Transmitting the plurality of second classified data based on the target data transmission channels corresponding to the plurality of second classified data;

[0145] Transmitting a plurality of data to be allocated based on a second data transmission channel;

[0146] Incremental data is sent to a local database based on a target data transmission channel and a second data transmission channel corresponding to a plurality of second classified data.

[0147] In this embodiment, the clustering method for clustering the historical transmission data includes but is not limited to K-Means clustering and fuzzy clustering.

[0148] In this embodiment, the data classification model includes but is not limited to decision tree, naive Bayes, support vector machine, neural network and other models.

[0149] In this embodiment, first, the historical transmission data corresponding to the historical maximum transmission rate of the first data transmission channel is obtained, and clustered to obtain multiple first classification data, the first classification data with the largest data volume is found as the first classification sub-data, and all first data transmission channels are traversed to obtain respective first classification sub-data. Next, the database change information is classified using a preset data classification model to obtain second classification data. Then, for each second classification data, its similarity with each first classification sub-data is calculated. If the maximum similarity reaches or exceeds the preset similarity threshold, the corresponding first data transmission channel is determined as the target data transmission channel of the target classification data; if it is less than the preset similarity threshold, the target classification data is marked as data to be allocated. By traversing all second classification data, the target data transmission channel and the data to be allocated of each second classification data are determined. Finally, the second classification data is transmitted according to the corresponding target data transmission channel, and the data to be allocated is transmitted through the second data transmission channel, so as to realize the sending of incremental data to the local database based on different channels. In this way, reasonable allocation can be made according to the characteristics of the data and the characteristics of the transmission channel, thereby improving the efficiency and accuracy of data transmission.

[0150] The beneficial effects of the above technical solution are: by analyzing and classifying historical transmission data and reasonably allocating transmission channels according to similarities, the advantages of different channels can be used more efficiently to improve the transmission speed of incremental data; targeted allocation can be made according to data characteristics and channel characteristics, so that transmission resources can be used more reasonably to avoid resource waste or imbalance; it can adapt to different database changes and transmission environments, and flexibly adjust the transmission strategy according to actual conditions to ensure the reliability and stability of data transmission; accurate classification and channel allocation help ensure that data is accurately transmitted to the local database and reduce the occurrence of transmission errors.

[0151] Example 10

[0152] The method for determining the incremental data includes:

[0153] After receiving the synchronization request from the local database, the remote server obtains the version difference data between the local database and the remote server; determines the incremental data based on the version difference data; the incremental data includes target common data and target data table.

[0154] The beneficial effects of the above technical solution are: by clarifying the version difference to determine the incremental data, more accurate database synchronization is achieved, and unnecessary data transmission is avoided; only the incremental part is transmitted, which effectively reduces the data transmission volume, saves network resources and transmission time;: the required target common data and target data tables are quickly determined, and the efficiency and speed of data synchronization are improved; due to the reduction in the amount of transmitted data, the demand for resources such as network bandwidth is reduced, thereby reducing costs; the local database can be updated in time according to the version difference, enhancing the real-time and availability of data.

[0155] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A method for automatic synchronization of general data and data tables, characterized in that: include: Get Category and query parameters; Query the redis cache based on Category and query parameters to determine the first query result; When it is determined that the target general data does not exist in the first query result, query whether the target data table exists in the local database based on the query parameter to obtain a second query result; If it is determined that the target data table does not exist in the second query result, the local database initiates a synchronization request to the remote server; After the local database receives the incremental data sent by the remote server, it synchronously updates the local database based on the incremental data and stores the incremental data in the redis cache.

2. The method for automatically synchronizing common data and data tables as claimed in claim 1, characterized in that: Also includes: If the target data table exists in the second query result, query whether the target general data exists in the target data table based on the query parameter; If the target common data exists in the target data table, the target common data is stored in the redis cache.

3. The method for automatically synchronizing common data and data tables as claimed in claim 2, characterized in that: Also includes: If the target data does not exist in the target data table, the remote server is queried based on the query parameters; If the target data corresponding to the query parameter exists in the remote server, the local database initiates a synchronization request to the remote server; The remote server determines the incremental data based on the synchronization request; The local database is synchronized based on the incremental data, and the incremental data is stored in the redis cache.

4. The method for automatically synchronizing common data and data tables as claimed in claim 2, characterized in that: Based on the query parameter, query whether the target data table exists in the local database, and obtain the second query result, including: Convert the query parameters into SQL statements; Based on the SQL statement, it is queried whether the target data table exists in the local database, and a second query result is determined.

5. The method for automatically synchronizing common data and data tables as claimed in claim 1, characterized in that: After the local database initiates a synchronization request to the remote server, it also includes: Obtaining current load information and configuration information of the remote server; Determine whether to perform data synchronization immediately based on the current load information and configuration information of the remote server; When it is determined to perform data synchronization immediately, the remote server sends incremental data to the local database.

6. The method for automatically synchronizing common data and data tables as claimed in claim 5, characterized in that: Determining whether to immediately perform data synchronization based on the current load information and configuration information of the remote server includes: Determine the load evaluation index of the remote server based on the preset evaluation index algorithm according to the current load information and configuration information of the remote server; Comparing the load evaluation index with a preset evaluation index threshold; Whether to immediately perform data synchronization is determined based on the comparison result.

7. The method for automatically synchronizing common data and data tables as claimed in claim 6, characterized in that: Before the remote server sends incremental data to the local database, it also includes: Acquire the data transmission channel between the remote server and the local database, and obtain several data transmission channels; Classifying a plurality of data transmission channels to obtain a first data transmission channel and a second data transmission channel; The remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel.

8. The method for automatically synchronizing common data and data tables as claimed in claim 7, characterized in that: Classifying a plurality of data transmission channels to obtain a first data transmission channel and a second data transmission channel includes: Obtain a statistical table of historical transmission data information corresponding to each data transmission channel within a preset time period; Determine the historical maximum transmission rate corresponding to each data transmission channel based on the historical transmission data information statistical table; Comparing the historical maximum transmission rate with a preset transmission rate threshold; The data transmission channel whose historical maximum transmission rate is greater than or equal to a preset transmission rate threshold is used as a first data transmission channel; The data transmission channel whose historical maximum transmission rate is less than a preset transmission rate threshold is used as the second data transmission channel.

9. The method for automatically synchronizing common data and data tables as claimed in claim 7, characterized in that: The remote server sends incremental data to the local database based on the first data transmission channel and the second data transmission channel, including: Acquire historical transmission data corresponding to the historical maximum transmission rate of the first data transmission channel; Clustering the historical transmission data to obtain a plurality of first classification data corresponding to the historical transmission data; Acquire the data volume corresponding to each first classification data, and use the first classification data corresponding to the maximum data volume among the plurality of first classification data as the first classification sub-data of the first data transmission channel; Traversing all first data transmission channels to obtain first classification sub-data corresponding to each first data transmission channel; Get the preset data classification model; Inputting the database change information into a preset data classification model for classification to obtain a plurality of second classification data; Randomly select a second classification data as the target classification data; Calculate the similarity between the target classification data and each first classification sub-data respectively, and if the maximum similarity is greater than or equal to a preset similarity threshold, use the first data transmission channel corresponding to the first classification sub-data as the target data transmission channel of the target classification data; If the maximum similarity value is less than a preset similarity threshold, the target classification data is used as the data to be allocated; Traversing all the second classification data, determining a number of target data transmission channels corresponding to the second classification data and a number of data to be allocated; Transmitting the plurality of second classified data based on the target data transmission channels corresponding to the plurality of second classified data; Transmitting a plurality of data to be allocated based on a second data transmission channel; Incremental data is sent to a local database based on a target data transmission channel and a second data transmission channel corresponding to a plurality of second classified data.

10. The method for automatically synchronizing common data and data tables as claimed in claim 1, characterized in that: The method for determining the incremental data includes: After receiving the synchronization request from the local database, the remote server obtains the version difference data between the local database and the remote server; determines the incremental data based on the version difference data; the incremental data includes target common data and target data table.

Images