Exogenous data synchronization method and device

Abstract

The invention relates to the technical field of computers, and provides an exogenous data synchronization method and device. The method comprises the following steps: determining to-be-synchronized data and a corresponding external data source to establish a data synchronization task; determining a real-time requirement of the data synchronization task according to the business requirement; obtaining a first weight of the external data source based on the real-time requirement; acquiring historical synchronization task execution information corresponding to the external data source to obtain the busy degree and the second weight of the external data source; determining a corresponding priority scoring mode according to the busy degree to obtain a priority score of the external data source; adjusting the bandwidth of the external data source according to the priority score, and determining a network bandwidth configuration strategy; and obtaining a corresponding data synchronization mode based on the real-time requirement, and synchronizing the to-be-synchronized data to the target database by using the corresponding data synchronization mode according to the network bandwidth configuration strategy. Through the embodiment of the invention, the exogenous data can be efficiently and conveniently synchronized to the local target database.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and more particularly to a method and device for synchronizing external data. Background Art

[0002] In complex projects that interact with multiple external data sources, it is often necessary to synchronize a steady stream of different types of data from external data sources to local data through relevant interfaces. External data sources are complex and changeable, and the corresponding data sources are often in a variety of forms. In the prior art, inter-library synchronization can usually only be achieved between databases of the same type. Moreover, since inter-library synchronization is achieved through file export, data synchronization through file export requires a large amount of disk space, and disk space needs to be cleaned up regularly to ensure the normal operation of the system. For data synchronization scenarios with multiple external data sources of different types, due to the complexity of the external data sources themselves, there is still a problem of low synchronization efficiency. Therefore, there is an urgent need for an external data synchronization method to achieve efficient and convenient synchronization of external data to local data. Summary of the invention

[0003] In view of the present situation, the present solution is proposed to overcome the above problems or at least partially solve the above problems.

[0004] On the one hand, some embodiments of this specification aim to provide a method for synchronizing external data, the method comprising:

[0005] Receive data synchronization request;

[0006] Determine the data to be synchronized and the corresponding external data source according to the data synchronization request, and establish a data synchronization task based on the external data source and the data to be synchronized;

[0007] Determine the real-time requirements of data synchronization tasks based on preset business needs;

[0008] Based on the real-time requirement of the data synchronization task, obtaining a first weight of the external data source;

[0009] Obtaining historical synchronization task execution information of the external data source within a preset time interval;

[0010] Calculating the busyness of the external data source according to the historical synchronization task execution information, and obtaining a second weight of the external data source according to the busyness;

[0011] Determine a corresponding priority scoring method according to the busyness, and calculate the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight;

[0012] Adjusting the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source;

[0013] The data synchronization mode corresponding to the data synchronization task is obtained based on the real-time requirement matching, and according to the network bandwidth configuration strategy, the data to be synchronized is synchronized to the target database using the data synchronization mode corresponding to the data synchronization task.

[0014] Furthermore, the real-time requirements of the data synchronization task are determined according to the preset business requirements, including:

[0015] According to the preset business requirements, a collection of external data sources with real-time requirements and a collection of data to be synchronized with real-time requirements are obtained;

[0016] Using the external data source collection and the to-be-synchronized data collection, respectively judging whether the external data source and the to-be-synchronized data corresponding to the data synchronization task have real-time requirements, and obtaining a judgment result;

[0017] The data synchronization tasks are divided according to the judgment results, and the real-time requirements of the data synchronization tasks are determined according to the division results.

[0018] Furthermore, the data synchronization tasks are divided according to the judgment result, and the real-time requirements of the data synchronization tasks are determined according to the division result, including:

[0019] If both the external data source and the data to be synchronized corresponding to the data synchronization task have real-time requirements, the data synchronization task is classified as the first level;

[0020] If the external data source and the data to be synchronized corresponding to the data synchronization task do not have real-time requirements, the data synchronization task is classified as the second level;

[0021] If the external data source corresponding to the data synchronization task has real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task does not have real-time requirements, the data synchronization task is classified into the third level;

[0022] If the external data source corresponding to the data synchronization task does not have real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task has real-time requirements, the data synchronization task is classified into the fourth level;

[0023] The real-time requirement of the data synchronization task is obtained by matching from a preset real-time requirement matching table using the level corresponding to the data synchronization task and the size of the data to be synchronized.

[0024] Further, based on the real-time requirement of the data synchronization task, a first weight of the external data source is obtained, including:

[0025] Count all data synchronization tasks corresponding to each external data source;

[0026] Determine the real-time requirements of each data synchronization task among all data synchronization tasks corresponding to each external data source, as well as the real-time requirements of each external data source itself;

[0027] The real-time requirement of each data synchronization task among all data synchronization tasks corresponding to each external data source and the real-time requirement of each external data source itself are used as linear weighting factors, and a linear weighting operation is performed to obtain a first weight of the external data source.

[0028] Furthermore, the historical synchronization task execution information includes one or more of the synchronization task amount, synchronization data amount, update times, update data amount and access times of the external data source.

[0029] Further, calculating the busyness of the external data source according to the historical synchronization task execution information, and obtaining a second weight of the external data source according to the busyness, includes:

[0030] Performing nonlinear attenuation processing on the historical synchronization task execution information using a logarithmic function;

[0031] Using the historical synchronization task execution information after nonlinear attenuation processing as a linear weighting factor to perform a linear weighting operation to obtain the busyness of the external data source;

[0032] The busyness of the external data source is used as the second weight of the external data source.

[0033] Further, determining a corresponding priority scoring method according to the busyness, and calculating the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight, includes:

[0034] Determining whether the busyness exceeds a preset threshold;

[0035] If the busyness does not exceed the preset threshold, the second weight is used as a gain parameter, and a linear addition operation is performed on the first weight and the second weight to obtain an operation result;

[0036] If the busyness exceeds a preset threshold, the second weight is used as a decay parameter, and a linear subtraction operation is performed on the first weight and the second weight to obtain an operation result;

[0037] The calculation result is optimized according to the business importance of the external data source to obtain a priority score of the external data source.

[0038] Furthermore, the calculation result is optimized according to the business importance of the external data source to obtain the priority score of the external data source, further comprising:

[0039] Determining the business importance of the external data source according to the business requirement and the first weight;

[0040] The business importance is added to the calculation result to obtain a priority score of the external data source.

[0041] Further, adjusting the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source, including:

[0042] Acquire communication status information of an external data source; the communication status information includes one or more of bandwidth, latency, jitter, packet loss rate, and storage utilization rate;

[0043] Adjusting the bandwidth of all external data sources according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, and optimizing the specified type of data in the communication status information according to the adjusted bandwidth;

[0044] The data transmission cost of each external data source is calculated using the optimized communication state information, and the data transmission cost of each external data source is added together to obtain the total data transmission cost;

[0045] Repeat the above step of adjusting the bandwidth of all external data sources according to the priority scores to obtain a total data transmission cost, thereby obtaining a plurality of total data transmission costs;

[0046] A minimum total data transmission cost is selected from a plurality of total data transmission costs, and a network bandwidth configuration strategy is determined according to a bandwidth of an external data source corresponding to the minimum total data transmission cost.

[0047] Further, optimizing the specified type of data in the communication state information according to the adjusted bandwidth includes:

[0048] Determine whether the adjusted bandwidth is less than the preset minimum bandwidth;

[0049] If the adjusted bandwidth is not less than the preset minimum bandwidth, there is no need to adjust the specified type of data in the communication status information; the specified type of data includes one or more of delay, jitter and packet loss rate;

[0050] If the adjusted bandwidth is less than the preset minimum bandwidth, the preset optimization gradient is used to optimize the value of the specified type of data.

[0051] Furthermore, the data synchronization mode includes incremental synchronization, full synchronization and full-incremental hybrid synchronization.

[0052] On the other hand, some embodiments of this specification further provide an external data synchronization device, the device comprising:

[0053] A receiving module, used for receiving a data synchronization request;

[0054] A determination module, configured to determine the data to be synchronized and the corresponding external data source according to the data synchronization request, so as to establish a data synchronization task based on the external data source and the data to be synchronized;

[0055] The classification module is used to determine the real-time requirements of data synchronization tasks according to preset business needs;

[0056] A first sorting module, configured to obtain a first weight of the external data source based on the real-time requirement of the data synchronization task;

[0057] An acquisition module, used to acquire historical synchronization task execution information of the external data source within a preset time interval;

[0058] A second sorting module is used to calculate the busyness of the external data source according to the historical synchronization task execution information to obtain a second weight of the external data source;

[0059] An evaluation module, configured to determine a corresponding priority scoring method according to the busyness, and calculate a priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight;

[0060] A matching module, configured to adjust the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source;

[0061] The synchronization module is used to obtain the data synchronization mode corresponding to the data synchronization task based on the real-time requirement matching, and synchronize the data to be synchronized to the target database using the data synchronization mode corresponding to the data synchronization task according to the network bandwidth configuration strategy.

[0062] On the other hand, some embodiments of the present specification further provide a computer device, including a memory, a processor, and a computer program stored in the memory, wherein when the computer program is executed by the processor, the instructions of the above method are executed.

[0063] On the other hand, some embodiments of the present specification further provide a computer storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0064] On the other hand, some embodiments of the present specification further provide a computer program product, wherein the computer program product comprises a computer program, and when the computer program is executed by a processor of a computer device, the instructions of the above method are executed.

[0065] One or more technical solutions provided by some embodiments of this specification have at least the following technical effects:

[0066] The embodiments of the present specification first automatically receive a data synchronization request, determine the data to be synchronized and the corresponding external data source according to the data synchronization request, establish a data synchronization task, and determine the real-time requirement of the data synchronization task according to the preset business needs, thereby obtaining a first weight of the external data source based on the real-time requirement of the data synchronization task, and using the first weight to numerically measure the real-time of each data synchronization task. At the same time, historical synchronization task execution information of the external data source within a preset time interval is obtained, and the busyness of each external data source is calculated according to the historical synchronization task execution information to obtain the corresponding second weight, thereby using the second weight to numerically measure the busyness of the external data source, and then the external data source is calculated according to the first weight and the second weight. The priority score of the data source is determined by the priority score. The priority scoring method corresponding to the priority score is determined according to the busyness. This is because when external data sources with different busyness perform data synchronization, the efficiency of data synchronization will be affected. After the priority score is calculated, the bandwidth of the external data source is allocated and adjusted according to the priority score, so that the bandwidth size of the external data source is positively correlated with the priority score, and the network bandwidth configuration strategy corresponding to the external data source is determined. According to the real-time requirements, the data synchronization method corresponding to the data synchronization task is matched, so that when the corresponding data synchronization method and network bandwidth configuration strategy are used to synchronize the data to be synchronized in multiple external data sources to the target database, the process of automatic data synchronization can be realized efficiently and quickly.

[0067] The above description is only an overview of the technical solutions of some embodiments of this specification. In order to more clearly understand the technical means of some embodiments of this specification, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of some embodiments of this specification more obvious and easy to understand, the specific implementation methods of some embodiments of this specification are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] In order to more clearly illustrate some embodiments of this specification or technical solutions in the prior art, the following briefly introduces the drawings required for the embodiments or prior art descriptions. Obviously, the drawings described below are only some embodiments recorded in this specification. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. In the drawings:

[0069] Figure 1 A schematic diagram of an implementation system of an external data synchronization method in some embodiments of this specification is shown;

[0070] Figure 2 A flowchart of an external data synchronization method in some embodiments of this specification is shown;

[0071] Figure 3 A schematic diagram of steps for determining the real-time requirements of a data synchronization task in some embodiments of this specification;

[0072] Figure 4 A schematic diagram of the steps of obtaining a first weight of an external data source in some embodiments of this specification;

[0073] Figure 5 A schematic diagram of the steps of obtaining a second weight of an external data source in some embodiments of this specification;

[0074] Figure 6 A schematic diagram of steps for obtaining a priority score of an external data source in some embodiments of this specification;

[0075] Figure 7 A schematic diagram of the steps for determining a network bandwidth configuration strategy in some embodiments of this specification;

[0076] Figure 8 This is a schematic diagram of the structure of an external data synchronization device in some embodiments of this specification;

[0077] Fig. 9 This is a schematic diagram of the computer device structure provided in some embodiments of this specification.

[0078] [Description of Reference Numerals]

[0079] 101. Terminal;

[0080] 102. Server;

[0081] 801, receiving module;

[0082] 802. Determine a module;

[0083] 803, classification module;

[0084] 804, a first sorting module;

[0085] 805, acquisition module;

[0086] 806, second sorting module;

[0087] 807. Evaluation module;

[0088] 808, matching module;

[0089] 809, synchronization module;

[0090] 902. Computer equipment;

[0091] 904, processor;

[0092] 906. Memory;

[0093] 908, driving mechanism;

[0094] 910, input / output interface;

[0095] 912. Input device;

[0096] 914. Output device;

[0097] 916. Presentation equipment;

[0098] 918. Graphical user interface;

[0099] 920, network interface;

[0100] 922, communication link;

[0101] 924. Communication bus. DETAILED DESCRIPTION

[0102] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in some embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on some embodiments in this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this specification.

[0103] It should be noted that the terms "first", "second", etc. in the specification and claims of this article and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of this article described here can be implemented in an order other than those illustrated or described here. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, device, product or equipment that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0104] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of relevant laws and regulations.

[0105] It should be noted that in the embodiments of this specification, certain software, components, models and other existing solutions in the industry may be mentioned, which should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of this application, but it does not mean that the applicant has or will necessarily use the solution.

[0106] like Figure 1 The diagram is a schematic diagram of an implementation system of an external data synchronization method according to an embodiment of the present invention, which may include: a terminal 101 and a server 102, wherein the terminal 101 and the server 102 communicate with each other through a network, and the network may include a local area network (LAN), a wide area network (WAN), the Internet or a combination thereof, and is connected to a website, a user device (such as a computing device) and a back-end system. A staff member may send a data synchronization request to the server 102 through the terminal 101, and after receiving the data synchronization request, the server 102 determines the data to be synchronized and the corresponding external data source according to the data synchronization request to establish a data synchronization task, thereby synchronizing the data to be synchronized in the external data source to the target database based on the data synchronization task.

[0107] In addition, it should be noted that Figure 1 What is shown is only an application environment provided by the present disclosure. In actual application, multiple terminals 101 may be included, and this specification does not limit this.

[0108] Figure 2It is a flowchart of an external data synchronization method provided by an embodiment of the present invention. This specification provides method operation steps as described in the embodiment or flowchart, but may include more or fewer operation steps based on conventional or non-creative labor. The order of steps listed in the embodiment is only one way of executing the steps among many orders, and does not represent the only order of execution. When the actual system or device product is executed, it can be executed in the order or in parallel according to the method shown in the embodiment or the accompanying drawings. Specifically, Figure 2 As shown, applied to the above-mentioned server side, the method may include:

[0109] S201: receiving a data synchronization request;

[0110] S202: Determine the data to be synchronized and the corresponding external data source according to the data synchronization request, and establish a data synchronization task based on the external data source and the data to be synchronized;

[0111] S203: Determine the real-time requirement of the data synchronization task according to the preset business requirements;

[0112] S204: Obtaining a first weight of the external data source based on the real-time requirement of the data synchronization task;

[0113] S205: Acquire historical synchronization task execution information of the external data source within a preset time interval;

[0114] S206: Calculate the busyness of the external data source according to the historical synchronization task execution information, and obtain a second weight of the external data source according to the busyness;

[0115] S207: determining a corresponding priority scoring method according to the busyness, and calculating the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight;

[0116] S208: adjusting the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source;

[0117] S209: obtaining a data synchronization method corresponding to the data synchronization task based on the real-time requirement matching, and synchronizing the to-be-synchronized data to a target database using the data synchronization method corresponding to the data synchronization task according to the network bandwidth configuration strategy.

[0118] The embodiments of the present specification first automatically receive a data synchronization request, determine the data to be synchronized and the corresponding external data source according to the data synchronization request, establish a data synchronization task, and determine the real-time requirement of the data synchronization task according to the preset business needs, thereby obtaining a first weight of the external data source based on the real-time requirement of the data synchronization task, and using the first weight to numerically measure the real-time of each data synchronization task. At the same time, historical synchronization task execution information of the external data source within a preset time interval is obtained, and the busyness of each external data source is calculated according to the historical synchronization task execution information to obtain the corresponding second weight, thereby using the second weight to numerically measure the busyness of the external data source, and then the external data source is calculated according to the first weight and the second weight. The priority score of the data source is determined by the priority score. The priority scoring method corresponding to the priority score is determined according to the busyness. This is because when external data sources with different busyness perform data synchronization, the efficiency of data synchronization will be affected. After the priority score is calculated, the bandwidth of the external data source is allocated and adjusted according to the priority score, so that the bandwidth size of the external data source is positively correlated with the priority score, and the network bandwidth configuration strategy corresponding to the external data source is determined. According to the real-time requirements, the data synchronization method corresponding to the data synchronization task is matched, so that when the corresponding data synchronization method and network bandwidth configuration strategy are used to synchronize the data to be synchronized in multiple external data sources to the target database, the process of automatic data synchronization can be realized efficiently and quickly.

[0119] It can be understood that, in some embodiments, the synchronization between the external data source and the target database can be database synchronization, file synchronization, and outbound API synchronization, etc. The external data source stores the data to be synchronized required by the target database. In addition to data synchronization, the external data source itself also has other tasks that need to be performed, such as writing new data, modifying old data, etc. When multiple external data sources need to synchronize data to the target database within the same time period, and under the premise that the total bandwidth is limited, it is necessary to formulate a suitable network bandwidth configuration strategy and use a suitable data synchronization method to accurately, efficiently, and orderly synchronize all data to be synchronized to the target database.

[0120] See attached Figure 3 In some embodiments, determining the real-time requirements of the data synchronization task according to preset business requirements may include:

[0121] S301: According to preset business requirements, a collection of external data sources with real-time requirements and a collection of data to be synchronized with real-time requirements are obtained;

[0122] S302: using the external data source collection and the to-be-synchronized data collection, respectively judging whether the external data source and the to-be-synchronized data corresponding to the data synchronization task have real-time requirements, and obtaining a judgment result;

[0123] S303: Divide the data synchronization tasks according to the judgment result, and determine the real-time requirements of the data synchronization tasks according to the division results.

[0124] It can be understood that in some embodiments, the preset business needs pre-set the real-time requirements of the external data source and the real-time requirements of the data to be synchronized. Through the preset business needs, a collection of external data sources with real-time requirements and a collection of data to be synchronized with real-time requirements can be quickly obtained. Thereafter, the collection of external data sources and the collection of data to be synchronized can be used to respectively determine whether the external data source and the data to be synchronized corresponding to the current data synchronization task to be executed have the real-time requirements, and obtain a judgment result. Based on the judgment result, the data synchronization tasks can be quickly and accurately divided and classified according to the real-time requirements.

[0125] Furthermore, in some embodiments, the data synchronization tasks are divided according to the judgment result, and the real-time requirements of the data synchronization tasks are determined according to the division result, including:

[0126] If both the external data source and the data to be synchronized corresponding to the data synchronization task have real-time requirements, the data synchronization task is classified as the first level;

[0127] If the external data source and the data to be synchronized corresponding to the data synchronization task do not have real-time requirements, the data synchronization task is classified as the second level;

[0128] If the external data source corresponding to the data synchronization task has real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task does not have real-time requirements, the data synchronization task is classified into the third level;

[0129] If the external data source corresponding to the data synchronization task does not have real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task has real-time requirements, the data synchronization task is classified into the fourth level;

[0130] The real-time requirement of the data synchronization task is obtained by matching from a preset real-time requirement matching table using the level corresponding to the data synchronization task and the size of the data to be synchronized.

[0131] It can be understood that, in some embodiments, if both the external data source and the data to be synchronized corresponding to the current data synchronization task have real-time requirements, it means that the real-time requirement of the current data synchronization task is relatively high, and the current data synchronization task is divided into the first level; if both the external data source and the data to be synchronized corresponding to the current data synchronization task do not have real-time requirements, it means that the real-time requirement of the current data synchronization task is relatively low, and the current data synchronization task is divided into the fourth level; accordingly, when the external data source corresponding to the data synchronization task does not have real-time requirements, and the data to be synchronized corresponding to the data synchronization task has real-time requirements, the data synchronization task is divided into the fourth level; when the external data source corresponding to the data synchronization task has real-time requirements, and the data to be synchronized corresponding to the data synchronization task does not have real-time requirements, the data synchronization task is divided into the third level, wherein the data synchronization task of the first level has the highest real-time requirement, and the data synchronization task of the fourth level has the lowest real-time requirement; the difference between the data synchronization tasks of the second level and the third level can be adjusted according to business needs; it can generally be considered that the real-time requirement of the data synchronization task of the second level is higher than the real-time requirement of the data synchronization task of the third level, thereby realizing rapid and refined distinction of the real-time of the data synchronization tasks.

[0132] See attached Figure 4 In some embodiments, obtaining the first weight of the external data source based on the real-time requirement of the data synchronization task may include:

[0133] S401: Count all data synchronization tasks corresponding to each external data source;

[0134] S402: Determine the real-time requirement of each data synchronization task among all data synchronization tasks corresponding to each external data source, and the real-time requirement of each external data source itself;

[0135] S403: taking the real-time requirement of each data synchronization task among all data synchronization tasks corresponding to each external data source and the real-time requirement of each external data source itself as linear weighting factors, performing linear weighting operation to obtain a first weight of the external data source.

[0136] It can be understood that, in some embodiments, after obtaining the real-time requirement of the data synchronization task, it is also necessary to quantify the real-time requirement, and use the external data source as a measurement benchmark to measure the real-time requirement of each external data source through the first weight. Specifically, it is necessary to first count all data synchronization tasks corresponding to each external data source. If for an external data source X, among all the data synchronization tasks corresponding to the external data source X, the more data synchronization tasks with high real-time requirements, the higher the real-time requirement of the external data source X. In addition, each external data source itself also has requirements for real-time. Afterwards, the real-time requirement of each data synchronization task among all the data synchronization tasks corresponding to each external data source and the real-time requirement of each external data source itself are used as linear weighting factors to perform linear weighted operations to obtain the first weight of the external data source. Specifically, in some embodiments, when performing linear weighted operations, the real-time requirement of a data synchronization task can be matched from a preset real-time requirement matching table through the level corresponding to the data synchronization task and the size of the data to be synchronized, thereby realizing the rapid and accurate calculation of the first weight of the external data source.

[0137] Furthermore, in some embodiments, the historical synchronization task execution information includes one or more of the synchronization task volume, synchronization data volume, update times, update data volume, and access times of the external data source.

[0138] It can be understood that in some embodiments, the historical synchronization task execution information is often data within the most recent time interval, so as to ensure the accuracy of the subsequent calculation of the busyness of the external data source, where the synchronization task volume is the total amount of data synchronization tasks, the synchronization data volume is the total amount of data synchronized to the target database, the update number is the number of times the external data source updates data, the update data volume is the total amount of data updated by the external data source, and the access number is the total number of times the external data source is accessed. In some embodiments, the historical synchronization task execution information can also be other types of data, which is not limited in this article.

[0139] See attached Figure 5 In some embodiments, calculating the busyness of the external data source according to the historical synchronization task execution information, and obtaining the second weight of the external data source according to the busyness may include:

[0140] S501: Performing nonlinear attenuation processing on the historical synchronization task execution information using a logarithmic function;

[0141] S502: Using the historical synchronization task execution information after nonlinear attenuation processing as a linear weighting factor to perform a linear weighting operation to obtain the busyness of the external data source;

[0142] S503: Taking the busyness of the external data source as the second weight of the external data source.

[0143] It can be understood that in some embodiments, since the magnitude of the number in the historical synchronization task execution information is large, the historical synchronization task execution information is first subjected to nonlinear attenuation processing through a logarithmic function. Typically, a natural logarithm function is used to take the natural logarithm of the data in the historical synchronization task execution information, thereby achieving nonlinear attenuation processing. Afterwards, the historical synchronization task execution information after nonlinear attenuation processing is used as a linear weighting factor to perform a linear weighted operation to obtain the busyness of the external data source. The busyness is used to measure the busyness of the external data source for data synchronization, access, and update, thereby using the busyness as the second weight of the external data source to achieve fast and accurate calculation of the second weight.

[0144] Further, in some embodiments, determining a corresponding priority scoring method according to the busyness, and calculating the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight, includes:

[0145] Determining whether the busyness exceeds a preset threshold;

[0146] If the busyness does not exceed the preset threshold, the second weight is used as a gain parameter, and a linear addition operation is performed on the first weight and the second weight to obtain an operation result;

[0147] If the busyness exceeds a preset threshold, the second weight is used as a decay parameter, and a linear subtraction operation is performed on the first weight and the second weight to obtain an operation result;

[0148] The calculation result is optimized according to the business importance of the external data source to obtain a priority score of the external data source.

[0149] It can be understood that in some embodiments, after obtaining the first weight and the second weight, it is necessary to determine the appropriate priority score first, first whether the busyness exceeds the preset threshold. If the busyness does not exceed the preset threshold, it means that the external data source at this time is idle to perform data synchronization tasks. At this time, the second weight corresponding to the data source is used as a gain parameter, and the first weight and the second weight are linearly added to increase the priority score of the external data source. If the busyness does not exceed the preset threshold, it means that the external data source at this time is not idle to perform data synchronization tasks. Therefore, the first weight and the second weight are linearly subtracted to reduce the priority score of the external data source, thereby realizing the rapid and automatic calculation of the priority score of the external data source.

[0150] Further, in some embodiments, after obtaining the operation result, refer to the attached Figure 6In some embodiments, optimizing the calculation result according to the business importance of the external data source to obtain the priority score of the external data source may further include:

[0151] S601: Determine the business importance of the external data source according to the business requirement and the first weight;

[0152] S602: Add the business importance to the calculation result to obtain a priority score of the external data source.

[0153] It can be understood that in some embodiments, a collection of external data sources with real-time requirements and a collection of data to be synchronized can be obtained through business needs. The first weight performs a quantitative assessment of the real-time performance of the external data source. On this basis, the business importance of the external data source can be determined according to business needs and the level of the first weight. The business importance is regarded as an adjustment parameter and added to the calculation result, so as to quickly and accurately obtain the priority score of the external data source to ensure the effectiveness of the subsequent determination of the network bandwidth configuration.

[0154] See attached Figure 7 In some embodiments, adjusting the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source, may include:

[0155] S701: Acquire communication status information of an external data source;

[0156] The communication status information includes one or more of bandwidth, delay, jitter, packet loss rate and storage utilization rate;

[0157] S702: adjusting the bandwidth of all external data sources according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, and optimizing the specified type of data in the communication status information according to the adjusted bandwidth;

[0158] S703: Calculate the data transmission cost of each external data source using the optimized communication state information, and add up the data transmission costs of each external data source to obtain a total data transmission cost;

[0159] S704: Repeat the above step of adjusting the bandwidth of all external data sources according to the priority scores to obtain a total data transmission cost, thereby obtaining a plurality of total data transmission costs;

[0160] S705: Select a minimum total data transmission cost from multiple total data transmission costs, and determine a network bandwidth configuration strategy according to a bandwidth of an external data source corresponding to the minimum total data transmission cost.

[0161] It can be understood that, in some embodiments, when adjusting the bandwidth of the external data source according to the priority score, firstly obtain the communication status information of the external data source, the communication status information can be used to evaluate the communication smoothness of the current external data source, and then adjust the bandwidth of all external data sources according to the priority score, so that the bandwidth size of the external data source is positively correlated with the priority score, and optimize the specified type of data in the communication status information according to the adjusted bandwidth. Further, in some embodiments, optimizing the specified type of data in the communication status information according to the adjusted bandwidth may include:

[0162] Determine whether the adjusted bandwidth is less than the preset minimum bandwidth;

[0163] If the adjusted bandwidth is not less than the preset minimum bandwidth, there is no need to adjust the specified type of data in the communication status information; the specified type of data includes one or more of delay, jitter and packet loss rate;

[0164] If the adjusted bandwidth is less than the preset minimum bandwidth, the preset optimization gradient is used to optimize the value of the specified type of data.

[0165] It can be understood that, in some embodiments, if the adjusted bandwidth is not less than the preset minimum bandwidth, it means that the adjustment of the bandwidth at this time does not have much impact on the specified type of data. At this time, the adjusted bandwidth and information such as delay, jitter and packet loss rate are still adapted to each other, and there is no need to adjust the specified type of data in the communication status information. If the adjusted bandwidth is less than the preset minimum bandwidth, the specified type of data will be affected by changes in the bandwidth, such as longer delay, increased jitter rate and increased packet loss rate. Therefore, it is necessary to optimize the value of the specified type of data so that as the bandwidth is adjusted, the specified type of data in the communication status information can also change in accordance with the actual situation, thereby ensuring the accuracy of the subsequent calculation of the data transmission cost.

[0166] Furthermore, in some embodiments, the data transmission cost of each external data source may be calculated using the optimized communication status information according to the following formula:

[0167]

[0168] Wherein, time represents delay, bandwidth represents bandwidth, jitter represents jitter rate, drop represents packet loss rate, a, b, c, d, and f represent parameters for adjusting delay, bandwidth, jitter rate, packet loss rate, and inverse effect of bandwidth, respectively, k represents an exponential parameter, and e represents a natural constant.

[0169] Afterwards, the data transmission cost of each external data source is added together to obtain the total data transmission cost. Since it is impossible to determine whether the current network bandwidth allocation strategy is optimal, it is necessary to repeat the above steps of adjusting the bandwidth of all external data sources according to the priority score to obtain the total data transmission cost, thereby obtaining multiple total data transmission costs. It can be understood that the total data transmission cost is regarded as an objective function value, and the optimization goal is to take the minimum objective function value, that is, to take the minimum total data transmission cost, so as to determine the network bandwidth configuration strategy according to the bandwidth of the external data source corresponding to the minimum total data transmission cost, so as to ensure the effectiveness of the network bandwidth configuration strategy, so as to facilitate the subsequent accurate and efficient synchronization of the data to be synchronized to the target database.

[0170] Furthermore, in some embodiments, the data synchronization methods include incremental synchronization, full synchronization, and full-incremental mixed synchronization.

[0171] It can be understood that in some embodiments, the preset data synchronization mode matching table is used to match the data synchronization mode corresponding to the data synchronization task based on the real-time requirements. In the full synchronization, the local corresponding data is first cleared and the external data is fully inserted. In the incremental synchronization mode, the data of the external data source is incremented daily, and the local corresponding data is not cleared. The data of the external data source is synchronized in by updating if there is data and inserting if there is none. The full-incremental hybrid synchronization is that the local data is not cleared, and the data of the external data source is synchronized in by updating if there is data and inserting if there is none. After determining the data synchronization mode corresponding to the data synchronization task, the data synchronization task is executed, and according to the network bandwidth configuration strategy, the corresponding data synchronization mode is used to efficiently and conveniently synchronize the data to be synchronized to the target database.

[0172] It should be noted that, although the operations of the method of the present invention are described in a specific order in the above embodiments and the accompanying drawings, this does not require or imply that the operations must be performed in the specific order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0173] Corresponding to the above-mentioned external data synchronization method, some embodiments of this specification also provide an external data synchronization device, referring to Figure 8 As shown, in some embodiments, the apparatus may include:

[0174] Receiving module 801, used for receiving a data synchronization request;

[0175] A determination module 802 is used to determine the data to be synchronized and the corresponding external data source according to the data synchronization request, so as to establish a data synchronization task based on the external data source and the data to be synchronized;

[0176] The classification module 803 is used to determine the real-time requirements of the data synchronization task according to the preset business requirements;

[0177] A first sorting module 804, configured to obtain a first weight of the external data source based on the real-time requirement of the data synchronization task;

[0178] An acquisition module 805 is used to acquire historical synchronization task execution information of the external data source within a preset time interval;

[0179] A second sorting module 806 is used to calculate the busyness of the external data source according to the historical synchronization task execution information to obtain a second weight of the external data source;

[0180] An evaluation module 807 is used to determine a corresponding priority scoring method according to the busyness, and calculate the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight;

[0181] A matching module 808, configured to adjust the bandwidth of the external data source according to the priority score, so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source;

[0182] The synchronization module 809 is used to obtain the data synchronization mode corresponding to the data synchronization task based on the real-time requirement matching, and synchronize the data to be synchronized to the target database using the data synchronization mode corresponding to the data synchronization task according to the network bandwidth configuration strategy.

[0183] For the convenience of description, the above device is described in various units according to their functions. Of course, when implementing this specification, the functions of each unit can be implemented in the same or multiple software and / or hardware.

[0184] It should be noted that in the embodiments of this specification, the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user and fully authorized by all parties.

[0185] It should be noted that the computer program product described in this specification is a software product that mainly implements the method described in this specification through a computer program.

[0186] The embodiments of this specification also provide a computer device. Fig. 9As shown, in some embodiments of the present specification, the computer device 902 may include one or more processors 904, such as one or more central processing units (CPUs) or graphics processing units (GPUs), and each processing unit may implement one or more hardware threads. The computer device 902 may also include any memory 906, which is used to store any kind of information such as code, settings, data, etc. In a specific embodiment, the computer program on the memory 906 and can be run on the processor 904, when the computer program is run by the processor 904, the instructions of the method described in any of the above embodiments may be executed. Non-limiting, for example, the memory 906 may include any one or more combinations of the following: any type of RAM, any type of ROM, flash memory device, hard disk, optical disk, etc. More generally, any memory may use any technology to store information. Further, any memory may provide volatile or non-volatile retention of information. Further, any memory may represent a fixed or removable component of the computer device 902. In one case, when the processor 904 executes an associated instruction stored in any memory or a combination of memories, the computer device 902 may perform any operation of the associated instruction. The computer device 902 also includes one or more drive mechanisms 908 for interacting with any storage, such as a hard disk drive mechanism, an optical disk drive mechanism, etc.

[0187] The computer device 902 may also include an input / output interface 910 (I / O) for receiving various inputs (via input devices 912) and for providing various outputs (via output devices 914). A specific output mechanism may include a presentation device 916 and an associated graphical user interface 918 (GUI). In other embodiments, the input / output interface 910 (I / O), input device 912, and output device 914 may not be included, and the computer device 902 may be used as a computer device in a network. The computer device 902 may also include one or more network interfaces 920 for exchanging data with other devices via one or more communication links 922. One or more communication buses 924 couple the components described above together.

[0188] The communication link 922 may be implemented in any manner, for example, through a local area network, a wide area network (e.g., the Internet), a point-to-point connection, etc., or any combination thereof. The communication link 922 may include any combination of hardwired links, wireless links, routers, gateway functions, name servers, etc. governed by any protocol or combination of protocols.

[0189] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), computer-readable storage media, and computer program products of some embodiments of the present specification. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processor to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processor generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0190] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processor to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0191] These computer program instructions can also be loaded onto a computer or other programmable data processor so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0192] In a typical configuration, a computer device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0193] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0194] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computer device. As defined in this specification, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0195] Those skilled in the art will appreciate that the embodiments of this specification may be provided as methods, systems or computer program products. Therefore, the embodiments of this specification may take the form of complete hardware embodiments, complete software embodiments or embodiments combining software and hardware. Moreover, the embodiments of this specification may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0196] The present specification embodiments may be described in the general context of computer-executable instructions executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform specific tasks or implement specific abstract data types. The present specification embodiments may also be practiced in distributed computing environments where tasks are performed by remote processors connected through a communication network. In a distributed computing environment, program modules may be located in local and remote computer storage media, including storage devices.

[0197] It should also be understood that in the embodiments of this specification, the term "and / or" is only a description of the association relationship of the associated objects, indicating that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship.

[0198] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0199] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the embodiments of this specification. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0200] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for synchronizing external data, characterized in that: The method comprises: Receive data synchronization request; Determine the data to be synchronized and the corresponding external data source according to the data synchronization request, and establish a data synchronization task based on the external data source and the data to be synchronized; Determine the real-time requirements of data synchronization tasks based on preset business needs; Based on the real-time requirement of the data synchronization task, obtaining a first weight of the external data source; Obtaining historical synchronization task execution information of the external data source within a preset time interval; Calculating the busyness of the external data source according to the historical synchronization task execution information, and obtaining a second weight of the external data source according to the busyness; Determine a corresponding priority scoring method according to the busyness, and calculate the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight; Adjusting the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source; The data synchronization mode corresponding to the data synchronization task is obtained based on the real-time requirement matching, and according to the network bandwidth configuration strategy, the data to be synchronized is synchronized to the target database using the data synchronization mode corresponding to the data synchronization task.

2. The method according to claim 1, characterized in that Determine the real-time requirements of data synchronization tasks based on preset business requirements, including: According to the preset business requirements, a collection of external data sources with real-time requirements and a collection of data to be synchronized with real-time requirements are obtained; Using the external data source collection and the to-be-synchronized data collection, respectively judging whether the external data source and the to-be-synchronized data corresponding to the data synchronization task have real-time requirements, and obtaining a judgment result; The data synchronization tasks are divided according to the judgment results, and the real-time requirements of the data synchronization tasks are determined according to the division results.

3. The method according to claim 2, characterized in that The data synchronization tasks are divided according to the judgment results, and the real-time requirements of the data synchronization tasks are determined according to the division results, including: If both the external data source and the data to be synchronized corresponding to the data synchronization task have real-time requirements, the data synchronization task is classified as the first level; If the external data source and the data to be synchronized corresponding to the data synchronization task do not have real-time requirements, the data synchronization task is classified as the second level; If the external data source corresponding to the data synchronization task has real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task does not have real-time requirements, the data synchronization task is classified into the third level; If the external data source corresponding to the data synchronization task does not have real-time requirements, and the to-be-synchronized data corresponding to the data synchronization task has real-time requirements, the data synchronization task is classified into the fourth level; The real-time requirement of the data synchronization task is obtained by matching from a preset real-time requirement matching table using the level corresponding to the data synchronization task and the size of the data to be synchronized.

4. The method according to claim 3, characterized in that Based on the real-time requirement of the data synchronization task, obtaining a first weight of the external data source includes: Count all data synchronization tasks corresponding to each external data source; Determine the real-time requirements of each data synchronization task among all data synchronization tasks corresponding to each external data source, as well as the real-time requirements of each external data source itself; The real-time requirement of each data synchronization task among all data synchronization tasks corresponding to each external data source and the real-time requirement of each external data source itself are used as linear weighting factors, and a linear weighting operation is performed to obtain a first weight of the external data source.

5. The method according to claim 1, characterized in that The historical synchronization task execution information includes one or more of the synchronization task amount, synchronization data amount, update times, update data amount, and access times of the external data source.

6. The method according to claim 5, characterized in that Calculating the busyness of the external data source according to the historical synchronization task execution information, and obtaining a second weight of the external data source according to the busyness, including: Performing nonlinear attenuation processing on the historical synchronization task execution information using a logarithmic function; Using the historical synchronization task execution information after nonlinear attenuation processing as a linear weighting factor to perform a linear weighting operation to obtain the busyness of the external data source; The busyness of the external data source is used as the second weight of the external data source.

7. The method according to claim 1, characterized in that Determining a corresponding priority scoring method according to the busyness, and calculating the priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight, including: Determining whether the busyness exceeds a preset threshold; If the busyness does not exceed the preset threshold, the second weight is used as a gain parameter, and a linear addition operation is performed on the first weight and the second weight to obtain an operation result; If the busyness exceeds a preset threshold, the second weight is used as a decay parameter, and a linear subtraction operation is performed on the first weight and the second weight to obtain an operation result; The calculation result is optimized according to the business importance of the external data source to obtain a priority score of the external data source.

8. The method according to claim 7, characterized in that Optimizing the calculation result according to the business importance of the external data source to obtain a priority score of the external data source further includes: Determining the business importance of the external data source according to the business requirement and the first weight; The business importance is added to the calculation result to obtain a priority score of the external data source.

9. The method according to claim 1, characterized in that: The bandwidth of the external data source is adjusted according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source, including: Acquire communication status information of an external data source; the communication status information includes one or more of bandwidth, latency, jitter, packet loss rate, and storage utilization rate; Adjusting the bandwidth of all external data sources according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, and optimizing the specified type of data in the communication status information according to the adjusted bandwidth; The data transmission cost of each external data source is calculated using the optimized communication state information, and the data transmission cost of each external data source is added together to obtain the total data transmission cost; Repeat the above step of adjusting the bandwidth of all external data sources according to the priority scores to obtain a total data transmission cost, thereby obtaining a plurality of total data transmission costs; A minimum total data transmission cost is selected from a plurality of total data transmission costs, and a network bandwidth configuration strategy is determined according to a bandwidth of an external data source corresponding to the minimum total data transmission cost.

10. The method according to claim 9, characterized in that Optimize the specified type of data in the communication status information according to the adjusted bandwidth, including: Determine whether the adjusted bandwidth is less than the preset minimum bandwidth; If the adjusted bandwidth is not less than the preset minimum bandwidth, there is no need to adjust the specified type of data in the communication status information; the specified type of data includes one or more of delay, jitter and packet loss rate; If the adjusted bandwidth is less than the preset minimum bandwidth, the preset optimization gradient is used to optimize the value of the specified type of data.

11. The method according to claim 1, characterized in that: The data synchronization methods include incremental synchronization, full synchronization, and full-incremental hybrid synchronization.

12. An external data synchronization device, characterized in that: The device comprises: A receiving module, used for receiving a data synchronization request; A determination module, configured to determine the data to be synchronized and the corresponding external data source according to the data synchronization request, so as to establish a data synchronization task based on the external data source and the data to be synchronized; The classification module is used to determine the real-time requirements of data synchronization tasks according to preset business needs; A first sorting module, configured to obtain a first weight of the external data source based on the real-time requirement of the data synchronization task; An acquisition module, used to acquire historical synchronization task execution information of the external data source within a preset time interval; A second sorting module is used to calculate the busyness of the external data source according to the historical synchronization task execution information to obtain a second weight of the external data source; An evaluation module, configured to determine a corresponding priority scoring method according to the busyness, and calculate a priority score of the external data source using the corresponding priority scoring method according to the first weight and the second weight; A matching module, configured to adjust the bandwidth of the external data source according to the priority score so that the bandwidth of the external data source is positively correlated with the priority score, thereby determining a network bandwidth configuration strategy corresponding to the external data source; The synchronization module is used to obtain the data synchronization mode corresponding to the data synchronization task based on the real-time requirement matching, and synchronize the data to be synchronized to the target database using the data synchronization mode corresponding to the data synchronization task according to the network bandwidth configuration strategy.

13. A computer device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: When the computer program is executed by the processor, the computer program executes the instructions of the method according to any one of claims 1 to 11.

14. A computer storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor of a computer device, the computer program executes the instructions of the method according to any one of claims 1 to 11.

15. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the computer program executes instructions of the method according to any one of claims 1-11.

Images