Data synchronization method and device

Abstract

The invention discloses a data synchronization method and device, and relates to the technical field of computers. A specific embodiment of the method comprises the steps of obtaining residual storage space data, data increase speed and data migration speed data of a database, calculating storage parameters of incremental data, determining the number of target slave libraries based on the parameters, and storing migration data in the target slave libraries for data synchronization. According to the embodiment mode, the incremental data can be stored in the slave library, the situation that data migration cannot be carried out when the disk space of the main library of the database is insufficient is avoided, normal use of services is guaranteed, and the purposes of non-perception and smooth transition migration of the services are achieved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a method and device for data synchronization. Background Art

[0002] At present, when performing data migration of MYSQL (relational database management system) database, data migration tools are usually used to migrate both stock data and incremental data. During data migration, the moment of data migration is used as the cut point between stock data and incremental data, and then the full data is migrated first. After the full data migration is completed, the corresponding incremental data point is found through the time point recorded at the beginning of the migration to perform incremental data synchronization migration.

[0003] However, when the above method is used for data migration, if the disk space is small, the writing of business data will be stopped, and subsequent operations can only be performed after the data migration, which affects the business. Summary of the invention

[0004] In view of this, an embodiment of the present invention provides a method for data synchronization, which can avoid business interruption caused by insufficient space in the database master database by storing incremental data in the slave database, thereby ensuring the normal operation of the business while realizing data migration.

[0005] To achieve the above object, according to a first aspect of an embodiment of the present invention, a method for data synchronization is provided, comprising:

[0006] Obtain database remaining storage space data, data growth rate, and data migration speed data;

[0007] Calculate incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data;

[0008] Determine the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database;

[0009] According to the number of the target slave repositories and the preset storage data volume, the migration data is sequentially stored in the target slave repositories, wherein the migration data includes stock data and incremental data;

[0010] A data synchronization operation is performed based on the migration data.

[0011] Optionally, the data migration speed data includes an existing data migration speed and an incremental data migration speed; and the step of calculating the incremental data storage parameter based on the remaining storage space data of the database, the data growth rate and the data migration speed data includes:

[0012] Calculating the stock data migration duration based on the stock data migration speed;

[0013] Calculating the incremental data migration duration based on the incremental data migration speed;

[0014] Calculate the time required for database space to be exhausted based on the remaining storage space data of the database and the data growth rate;

[0015] Incremental data storage parameters are calculated based on the stock data migration duration, the incremental data migration duration, the data growth rate, and the duration required for the database space to be exhausted.

[0016] Optionally, determining the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database includes:

[0017] When the incremental data storage parameter is less than 0, determining the number of target slave libraries to be 1;

[0018] When the incremental data storage parameter is greater than or equal to 0 and less than or equal to the remaining storage space of the database, determining the number of target slave libraries to be 2;

[0019] When the incremental data storage parameter is greater than the remaining storage space of the database, the formula for calculating the number of target slave libraries is:

[0020] Number of target slaves = (incremental data storage parameter / remaining database storage space) + 2

[0021] The value of the incremental data storage parameter / the remaining storage space of the database is an integer.

[0022] Optionally, according to the number of the target slave repositories and the preset storage data volume, before sequentially storing the migration data in the target slave repositories, wherein the migration data includes stock data and incremental data, the method includes:

[0023] The preset storage data volume is calculated based on the remaining storage space of the database and the number of the target slave libraries.

[0024] Optionally, according to the number of the target slave repositories and the preset storage data volume, the migration data is sequentially stored in the target slave repositories, and the migration data includes stock data and incremental data, including:

[0025] A first slave library storing the stock data in the target slave library;

[0026] When the number of the target slave database is 1, storing the incremental data in the target slave database;

[0027] When the number of the target slave libraries is greater than or equal to 2, the incremental data is stored in the target slave libraries in sequence according to the preset storage data volume.

[0028] Optionally, when the number of the target slave libraries is greater than or equal to 2, after storing the incremental data in the target slave libraries in sequence according to the preset storage data amount, the method further includes:

[0029] Delete the target data stored in the target slave library, where the target data is the incremental data stored in the previous slave library of the target slave library.

[0030] Optionally, deleting the target data stored in the target slave library includes:

[0031] Obtain the incremental data storage timestamp of the previous slave library of the target slave library;

[0032] Delete the incremental data before the timestamp stored in the target repository.

[0033] According to a second aspect of an embodiment of the present invention, there is provided a data synchronization device, including:

[0034] The acquisition module is used to obtain the remaining storage space data, data growth rate and data migration speed data of the database;

[0035] A calculation module, used to calculate incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data;

[0036] A determination module, configured to determine the number of target slave libraries based on the incremental data storage parameters and the remaining storage space data of the database;

[0037] A storage module, used to store migration data in the target slave library in sequence according to the number of the target slave libraries and a preset storage data volume, wherein the migration data includes stock data and incremental data;

[0038] A synchronization module is used to perform a data synchronization operation based on the migration data.

[0039] Optionally, the data migration speed data includes an existing data migration speed and an incremental data migration speed; the calculation module is used to calculate the incremental data storage parameter based on the remaining storage space data of the database, the data growth rate and the data migration speed data, including:

[0040] The calculation module is used to calculate the migration duration of existing data based on the migration speed of existing data; calculate the migration duration of incremental data based on the migration speed of incremental data; calculate the time required for database space exhaustion based on the remaining storage space data of the database and the data growth rate; calculate incremental data storage parameters based on the migration duration of existing data, the migration duration of incremental data, the data growth rate and the time required for database space exhaustion.

[0041] Optionally, the determination module is used to determine the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database, including:

[0042] The determination module is used to determine that the number of target slave libraries is 1 when the incremental data storage parameter is less than 0; and to determine that the number of target slave libraries is 2 when the incremental data storage parameter is greater than or equal to 0 and less than or equal to the remaining storage space of the database; the calculation module is used to calculate the number of target slave libraries when the incremental data storage parameter is greater than the remaining storage space of the database, and the formula is: the number of target slave libraries = (incremental data storage parameter / remaining storage space of the database) + 2; wherein the value of the incremental data storage parameter / remaining storage space of the database takes the integer part.

[0043] Optionally, the device further comprises: the calculation module, configured to calculate the preset storage data volume based on the remaining storage space of the database and the number of the target slave libraries.

[0044] Optionally, the storage module is used to store the migration data in the target slave library in sequence according to the number of the target slave libraries and a preset storage data volume, wherein the migration data includes stock data and incremental data, including:

[0045] The storage module is used to store the existing data in the first slave library in the target slave library; when the number of the target slave libraries is 1, the incremental data is stored in the target slave library; when the number of the target slave libraries is greater than or equal to 2, the incremental data is stored in the target slave libraries in sequence according to the preset storage data volume.

[0046] Optionally, the device further includes: a deleting module, configured to delete target data stored in the target slave library, wherein the target data is incremental data stored in a previous slave library of the target slave library.

[0047] Optionally, the deletion module is used to delete the target data stored in the target slave library, including: an acquisition module, used to obtain the incremental data storage timestamp of the previous slave library of the target slave library; the deletion module is used to delete the incremental data stored in the target slave library before the timestamp.

[0048] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0049] One or more processors; a storage device for storing one or more programs,

[0050] When the one or more programs are executed by the one or more processors, the one or more processors implement the method provided in the first aspect of the embodiment of the present invention.

[0051] According to a fourth aspect of an embodiment of the present invention, a computer-readable medium is provided, on which a computer program is stored, and when the program is executed by a processor, the method provided by the first aspect of the embodiment of the present invention is implemented.

[0052] An embodiment of the above invention has the following advantages or beneficial effects: the number of target slave libraries is determined by calculating the incremental data storage parameters, and the incremental data can be stored in the target slave libraries for subsequent data migration, thereby avoiding the situation where the database master library has insufficient storage space and cannot perform data migration, and the business has to be stopped, thereby ensuring the normal progress of the business and the normal progress of the data migration process. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 is a schematic diagram of the main process of the data synchronization method according to an embodiment of the present invention;

[0054] Figure 2 It is a schematic diagram of a data migration method;

[0055] Figure 3 is a schematic diagram of main modules of a data synchronization device according to an embodiment of the present invention;

[0056] Figure 4 is an exemplary system architecture diagram to which embodiments of the present invention may be applied;

[0057] Figure 5 It is a schematic diagram of the structure of a computer system of a terminal device or a server suitable for implementing an embodiment of the present invention. DETAILED DESCRIPTION

[0058] The following is a description of exemplary embodiments of the present invention in conjunction with the accompanying drawings, including various details of the embodiments of the present invention to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present invention. Similarly, for clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0059] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information, network security and national security.

[0060] Figure 1 A data synchronization method according to an embodiment of the present invention is as follows. Figure 1 As shown, including:

[0061] Step S101, obtaining database remaining storage space data, data growth rate and data migration rate data.

[0062] Calculate the remaining disk space P of the MYSQL database main library through the operating system; obtain historical disk exhaustion time data, for example, obtain historical data: after N days of use, the incremental data usage space of the disk is U, then the calculated data growth rate is K=U / N, that is, the daily data growth amount is K; evaluate the data migration speed according to real-time hardware resource conditions.

[0063] Step S102, calculating incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data.

[0064] The time required for the database disk storage space to be exhausted is calculated based on the remaining data storage space and the growth rate of data in the database. The required incremental data storage parameters of the slave database are calculated based on the calculated time and data migration speed, which are used for the subsequent number of slave databases based on the total storage parameters of the slave database.

[0065] Step S103, determining the target number of slave libraries based on the incremental data storage parameters and the remaining storage space data of the database.

[0066] After calculating the incremental data storage parameter of the slave database, if the storage parameter is less than 0, it means that the database master database can meet the data migration requirements, and no additional slave database is needed to store incremental data. Only one slave database is needed to store the database data; if the storage parameter is greater than or equal to 0, it means that the remaining storage space of the database master database cannot meet the data migration requirements, and an additional slave database is needed to store incremental data. At this time, the value of the storage parameter is the total storage space required by the slave database. Calculate the ratio of the total storage space required by the slave database to the remaining storage space of the database master database to get the required target number of slave databases.

[0067] Step S104, according to the number of target slave repositories and the preset storage data volume, the migration data is stored in the target slave repositories in sequence, and the migration data includes stock data and incremental data.

[0068] Data is stored according to the calculated number of target slave libraries and the preset storage data volume. Specifically, the slave libraries are sorted, and the existing data in the database and the incremental data that meets the preset storage data volume are stored in the first slave library. Incremental data that does not exceed the preset storage data volume is stored in the subsequent slave libraries in sequence until all the incremental data is stored. Among them, the preset storage data volume can be a set fixed value or a set fixed time value. The amount of data stored in each slave library is determined based on the fixed time value, and the data storage operation of each slave library is performed based on the data volume. Among them, the server resource configuration of the master and slave databases is consistent, specifically mainly referring to the consistency of the hardware physical parameters and setting parameters (such as the number of CPU cores is 4 cores, the CPU main frequency is 1GHz) of the physical server's CPU (central processing unit), disk, memory, network card, etc.

[0069] Step S105: performing data synchronization operation based on the migration data.

[0070] Through the data migration tool, the existing data and incremental data stored in the slave database are migrated in sequence until all data are migrated, thus achieving delay-free synchronization with the master database.

[0071] Alternatively, if Figure 2 As shown, the data migration time is t1-t5. Multiple slave databases replicate data information in the master database at the same time, and all slave databases synchronize the master database data in real time. After calculating the specific number of slave databases, the master-slave synchronization of one of the slave databases is stopped every t2-t1 time or when a certain disk storage space is reached. At this time, the data migration tool starts to migrate the incremental data in slave1. When the incremental data migration of slave1 is completed, the incremental data in the slave2 database is migrated immediately, and so on until the data migration is completed, that is, the synchronization with the master database is completed in real time without delay.

[0072] According to the data synchronization method provided in an embodiment of the present invention, it is possible to calculate incremental data storage parameters through database remaining storage space data, data growth rate and data migration rate data, and further determine the number of target slave libraries for storing incremental data through the target slave libraries, as well as subsequent data synchronization processes, thereby ensuring the normal progress of business processes and data migration when the database master library disk space is insufficient.

[0073] Optionally, the data migration speed data includes the migration speed of stock data and the migration speed of incremental data; the incremental data storage parameters are calculated based on the remaining storage space data of the database, the data growth rate and the data migration speed data, including: calculating the migration time of stock data based on the migration speed of stock data; calculating the migration time of incremental data based on the migration speed of incremental data; calculating the time required for database space exhaustion based on the remaining storage space data of the database and the data growth rate; calculating the incremental data storage parameters based on the migration time of stock data, the migration time of incremental data, the data growth rate and the time required for database space exhaustion. The migration speed of stock data is estimated based on the real-time conditions of the data migration tool to obtain the incremental data migration speed of the data migration tool. The migration time H of stock data is calculated based on the migration speed of stock data and the stock data stored in the slave database; the incremental data migration time L is calculated according to the remaining storage space P of the database and the migration speed S of incremental data by the formula: P / S=L; the disk exhaustion time M is calculated according to the remaining storage space P of the database and the data growth rate K, the formula is: P / K=M; the incremental data storage parameter I is calculated based on the above-mentioned calculated data, the formula is: (H+LM)*K=I. The calculated incremental data parameter I is the storage space required for the slave database after the second slave database. When the calculated I is less than 0, it means that the remaining space of the database master database can meet the data migration requirements without the need for additional slave databases for incremental data storage.

[0074] Optionally, the number of target slave libraries is determined based on the incremental data storage parameter and the remaining storage space data of the database, including: when the incremental data storage parameter is less than 0, the number of target slave libraries is determined to be 1; when the incremental data storage parameter is greater than or equal to 0 and less than or equal to the remaining storage space of the database, the number of target slave libraries is determined to be 2; when the incremental data storage parameter is greater than the remaining storage space of the database, the formula for calculating the number of target slave libraries is: number of target slave libraries = (incremental data storage parameter / remaining storage space of the database) + 2; wherein the value of the incremental data storage parameter / remaining storage space of the database takes the integer part.

[0075] The specific steps for calculating the number of slave libraries V according to the incremental data storage parameters are as follows: determine the relationship between the incremental data storage parameter I and the remaining storage space P of the database. If I<0, it means that the remaining storage space of the database can meet the migration requirements, and the required slave library data volume V=1+0; if 0≤I≤P, the required number of slave libraries V=1+1, that is, two slave libraries; if I>P, the required number of slave libraries V=(I / P)+1+1, where the value of I / P takes the integer part.

[0076] Optionally, according to the number of target slave libraries and the preset storage data volume, the migration data is stored in the target slave library in sequence, and before the migration data includes the stock data and the incremental data, the method includes: calculating the preset storage data volume based on the remaining storage space of the database and the number of target slave libraries. Before synchronizing the master-slave database data, it is also necessary to calculate the space for storing incremental data in each slave library, and the formula is: E=P / V, where P is the remaining storage space of the database and V is the number of target slave libraries.

[0077] Optionally, according to the number of target slave libraries and the preset storage data volume, the migration data is stored in the target slave library in sequence, and the migration data includes stock data and incremental data, including: storing the stock data in the first slave library in the target slave library; when the number of target slave libraries is 1, storing the incremental data in the target slave library; when the number of target slave libraries is greater than or equal to 2, storing the incremental data in the target slave library in sequence according to the preset storage data volume. When storing migration data in the target slave library, the stock data in the database is first stored in the first slave library in the target slave library, and when the number of target slave libraries is 1, the incremental data is also stored in the above-mentioned first slave library; when the target slave library data volume is greater than or equal to 2, the preset storage data volume stores the incremental data in the target slave library in sequence. Specifically, when storing data, all target slave libraries will synchronize the stock data and incremental data. When the incremental data stored in the first slave library reaches the preset storage data volume, the first slave library stops data synchronization, and the stop time t2 is marked at this time. Other databases continue to synchronize, and other databases can delete the data synchronized before time t2, and so on. Repeat the above steps until the incremental data is synchronized.

[0078] Optionally, when performing data migration based on the above storage method, start from the first slave library, migrate its stored data, and after all the data are migrated, read the data of the second slave library, and so on, until the migration of data in all slave libraries is completed and data synchronization is completed. Among them, since the data storage time of the first slave library is stopped at t2, and the start time of the data stored in the second slave library is t2, the data in the two slave libraries are continuous data. By migrating all slave library data, accurate and continuous stock data and incremental data can be obtained.

[0079] Optionally, when the number of target slave libraries is greater than or equal to 2, after storing incremental data in the target slave libraries in sequence according to the preset storage data volume, the method further includes: deleting the target data stored in the target slave library, where the target data is the incremental data stored in the previous slave library of the target slave library. Specifically, since all slave libraries start synchronizing data with the database master library at the same time, each slave library needs to delete the data stored in the previous database to avoid duplicate storage of data and waste of storage resources.

[0080] Optionally, deleting the target data stored in the target slave library includes: obtaining the incremental data storage timestamp of the previous slave library of the target slave library; and deleting the incremental data stored in the target slave library before the timestamp. Since all target slave libraries start synchronizing the database master library data at the same time, when deleting the data stored in the previous slave library, the timestamp when the previous slave library data synchronization stopped, that is, the incremental data storage timestamp, can be obtained, and the data before the timestamp can be deleted.

[0081] Optionally, in order to ensure the storage space of the database master, the data furthest from the current time will be deleted periodically according to the disk space, and some of the latest incremental data will be retained to maintain the master-slave database relationship.

[0082] According to the data synchronization method provided by the embodiment of the present invention, it is possible to calculate the incremental data storage parameters through the remaining storage space data of the database, the data growth rate and the data migration speed data, and further determine the number of target slave libraries for storing incremental data through the target slave libraries, as well as the subsequent data synchronization process, thereby avoiding business termination and data migration process stoppage due to insufficient disk space of the database master library, and achieving the purpose of business-unaware and smooth transition migration by exchanging space for time through the target slave library.

[0083] Figure 3 3 is a schematic diagram of a data synchronization device 300 provided in an embodiment of the present invention, including:

[0084] The acquisition module 301 is used to acquire the remaining storage space data, data growth rate and data migration rate data of the database;

[0085] A calculation module 302 is used to calculate incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data;

[0086] Determination module 303, used to determine the number of target slave libraries based on the incremental data storage parameters and the remaining storage space data of the database;

[0087] The storage module 304 is used to store the migration data in the target slave database in sequence according to the number of the target slave databases and the preset storage data volume, where the migration data includes the stock data and the incremental data;

[0088] The synchronization module 305 is used to perform data synchronization operations based on the migration data.

[0089] The data synchronization device 300 provided according to an embodiment of the present invention can calculate incremental data storage parameters through database remaining storage space data, data growth rate and data migration rate data, and further determine the number of target slave libraries for storing incremental data through the target slave libraries, as well as subsequent data synchronization processes, thereby ensuring the normal progress of business processes and data migration when the database master library disk space is insufficient.

[0090] Optionally, the data migration speed data includes the stock data migration speed and the incremental data migration speed; the calculation module 302 is used to calculate the incremental data storage parameters based on the database remaining storage space data, the data growth rate and the data migration speed data, including:

[0091] Calculation module 302 is used to calculate the migration duration of existing data based on the migration speed of existing data; calculate the migration duration of incremental data based on the migration speed of incremental data; calculate the time required for database space to be exhausted based on the remaining storage space data of the database and the data growth rate; calculate the incremental data storage parameters based on the migration duration of existing data, the migration duration of incremental data, the data growth rate and the time required for database space to be exhausted.

[0092] Optionally, the determination module 303 is used to determine the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database, including:

[0093] The determination module 303 is used to determine that the number of target slave libraries is 1 when the incremental data storage parameter is less than 0; when the incremental data storage parameter is greater than or equal to 0 and less than or equal to the remaining storage space of the database, the number of target slave libraries is determined to be 2; the calculation module 302 is used to calculate the number of target slave libraries when the incremental data storage parameter is greater than the remaining storage space of the database. The formula is: the number of target slave libraries = (incremental data storage parameter / remaining storage space of the database) + 2; wherein the value of the incremental data storage parameter / remaining storage space of the database takes the integer part.

[0094] Optionally, the device further includes: a calculation module 302, configured to calculate a preset storage data volume based on the remaining storage space of the database and the number of target slave libraries.

[0095] Optionally, the storage module 304 is used to store the migration data in the target slave database in sequence according to the number of the target slave databases and the preset storage data volume, and the migration data includes the stock data and the incremental data, including:

[0096] The storage module 304 is used to store the existing data in the first slave library in the target slave library; when the number of target slave libraries is 1, the incremental data is stored in the target slave library; when the number of target slave libraries is greater than or equal to 2, the incremental data is stored in the target slave library in sequence according to the preset storage data volume.

[0097] Optionally, the device further includes: a deleting module 305, which is used to delete the target data stored in the target slave library, where the target data is the incremental data stored in the previous slave library of the target slave library.

[0098] Optionally, the deletion module 305 is used to delete the target data stored in the target slave library, including: an acquisition module 301, used to obtain the incremental data storage timestamp of the previous slave library of the target slave library; the deletion module 305 is used to delete the incremental data before the timestamp stored in the target slave library.

[0099] The data synchronization device provided in an embodiment of the present invention can calculate the incremental data storage parameters through the remaining storage space data of the database, the data growth rate and the data migration speed data, and further determine the number of target slave libraries for storing incremental data through the target slave libraries, as well as the subsequent data synchronization process, so as to avoid business termination and data migration process stoppage caused by insufficient disk space of the database master library, and achieve the purpose of business-unaware and smooth transition migration by exchanging space for time through the target slave library.

[0100] Figure 4 An exemplary system architecture 400 is shown to which the method or apparatus for data synchronization according to an embodiment of the present invention may be applied.

[0101] like Figure 4 As shown, system architecture 400 may include terminal devices 401, 402, 403, network 404 and server 405. Network 404 is used to provide a medium for communication links between terminal devices 401, 402, 403 and server 405. Network 404 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0102] Users can use terminal devices 401, 402, 403 to interact with server 405 through network 404 to receive or send messages, etc. Various communication client applications can be installed on terminal devices 401, 402, 403, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (only for example).

[0103] The terminal devices 401 , 402 , and 403 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0104] The server 405 may be a server that provides various services, such as a background management server (only an example) that provides support for shopping websites browsed by users using the terminal devices 401, 402, and 403. The background management server may analyze and process the received data such as the data synchronization request, and feed back the processing result (such as synchronization data - only an example) to the terminal device.

[0105] It should be noted that the data synchronization method provided in the embodiment of the present invention is generally executed by the server 405 , and accordingly, the data synchronization device is generally set in the server 405 .

[0106] It should be understood that Figure 4 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to the implementation requirements.

[0107] Reference below Figure 5 , which shows a schematic diagram of the structure of a computer system 500 of a terminal device suitable for implementing an embodiment of the present invention. Figure 5 The terminal device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0108] like Figure 5 As shown, the computer system 500 includes a central processing unit (CPU) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage part 508 into a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the system 500 are also stored. The CPU 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0109] The following components are connected to the I / O interface 505: an input section 506 including a keyboard, a mouse, etc.; an output section 507 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 508 including a hard disk, etc.; and a communication section 509 including a network interface card such as a LAN card, a modem, etc. The communication section 509 performs communication processing via a network such as the Internet. A drive 510 is also connected to the I / O interface 505 as needed. A removable medium 511, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 510 as needed, so that a computer program read therefrom is installed into the storage section 508 as needed.

[0110] In particular, according to the embodiments disclosed in the present invention, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present invention include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 509, and / or installed from the removable medium 511. When the computer program is executed by the central processing unit (CPU) 501, the above-mentioned functions defined in the system of the present invention are executed.

[0111] It should be noted that the computer-readable medium shown in the present invention may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In the present invention, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0112] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a module, a program segment, or a part of a code, and the above-mentioned module, program segment, or a part of a code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flow chart, and the combination of the boxes in the block diagram or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0113] The modules involved in the embodiments of the present invention may be implemented by software or hardware. The modules described may also be set in a processor, for example, it may be described as: a processor includes an acquisition module, a calculation module, a determination module and a storage module. The names of these modules do not constitute a limitation on the modules themselves in some cases, for example, the acquisition module may also be described as "a module for obtaining database remaining storage space data, data growth rate and data migration speed data".

[0114] As another aspect, the present invention further provides a computer-readable medium, which may be included in the device described in the above embodiment; or may exist independently without being assembled into the device. The above computer-readable medium carries one or more programs, and when the above one or more programs are executed by a device, the device includes:

[0115] Obtain database remaining storage space data, data growth rate, and data migration speed data;

[0116] Calculate incremental data storage parameters based on database remaining storage space data, data growth rate, and data migration rate data;

[0117] Determine the number of target slave databases based on incremental data storage parameters and database remaining storage space data;

[0118] According to the number of target slaves and the preset storage data volume, the migration data is stored in the target slaves in sequence. The migration data includes the existing data and the incremental data.

[0119] Perform data synchronization based on the migrated data.

[0120] According to the technical solution of the embodiment of the present invention, by adopting a slave database to store incremental data, it is possible to avoid the situation where data migration cannot be performed when the disk space of the database master database is insufficient, thereby ensuring the normal use of the business and achieving the purpose of business-unaware and smooth transition migration.

[0121] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for data synchronization, characterized in that: The method comprises: Obtain database remaining storage space data, data growth rate, and data migration speed data; Calculate incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data; Determine the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database; According to the number of the target slave repositories and the preset storage data volume, the migration data is sequentially stored in the target slave repositories, wherein the migration data includes stock data and incremental data; A data synchronization operation is performed based on the migration data.

2. The method according to claim 1, characterized in that The data migration speed data includes the stock data migration speed and the incremental data migration speed; the incremental data storage parameter is calculated based on the remaining storage space data of the database, the data growth rate and the data migration speed data, including: Calculating the stock data migration duration based on the stock data migration speed; Calculating the incremental data migration duration based on the incremental data migration speed; Calculate the time required for database space to be exhausted based on the remaining storage space data of the database and the data growth rate; Incremental data storage parameters are calculated based on the stock data migration duration, the incremental data migration duration, the data growth rate, and the duration required for the database space to be exhausted.

3. The method according to claim 1, characterized in that The determining the number of target slave libraries based on the incremental data storage parameter and the remaining storage space data of the database includes: When the incremental data storage parameter is less than 0, determining the number of target slave libraries to be 1; When the incremental data storage parameter is greater than or equal to 0 and less than or equal to the remaining storage space of the database, determining the number of target slave libraries to be 2; When the incremental data storage parameter is greater than the remaining storage space of the database, the formula for calculating the number of target slave libraries is: The number of target slaves = (incremental data storage parameter / remaining database storage space) + 2; The value of the incremental data storage parameter / the remaining storage space of the database is an integer.

4. The method according to claim 1, characterized in that: According to the number of the target slave libraries and the preset storage data volume, before sequentially storing the migration data in the target slave library, wherein the migration data includes stock data and incremental data, the method includes: The preset storage data volume is calculated based on the remaining storage space of the database and the number of the target slave libraries.

5. The method according to claim 3, characterized in that: According to the number of the target slave libraries and the preset storage data volume, the migration data is sequentially stored in the target slave library, wherein the migration data includes stock data and incremental data, including: A first slave library storing the stock data in the target slave library; When the number of the target slave database is 1, storing the incremental data in the target slave database; When the number of the target slave libraries is greater than or equal to 2, the incremental data is stored in the target slave libraries in sequence according to the preset storage data volume.

6. The method according to claim 5, characterized in that When the number of the target slave libraries is greater than or equal to 2, after the incremental data is stored in the target slave libraries in sequence according to the preset storage data amount, the method further includes: Delete the target data stored in the target slave library, where the target data is the incremental data stored in the previous slave library of the target slave library.

7. The method according to claim 6, characterized in that The deleting the target data stored in the target slave library includes: Obtain the incremental data storage timestamp of the previous slave library of the target slave library; Delete the incremental data before the timestamp stored in the target repository.

8. A data synchronization device, characterized in that: include: The acquisition module is used to obtain the remaining storage space data, data growth rate and data migration speed data of the database; A calculation module, used to calculate incremental data storage parameters based on the remaining storage space data of the database, the data growth rate and the data migration rate data; A determination module, configured to determine the number of target slave libraries based on the incremental data storage parameters and the remaining storage space data of the database; A storage module, used to store migration data in the target slave library in sequence according to the number of the target slave libraries and a preset storage data volume, wherein the migration data includes stock data and incremental data; A synchronization module is used to perform a data synchronization operation based on the migration data.

9. An electronic device, characterized in that: include: one or more processors; a storage device for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 7.

10. A computer readable medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images