A method, device and system for incremental data storage in an online database

Abstract

An embodiment of the present invention provides a method, device and system for warehousing incremental data in an online database, wherein the method includes: obtaining incremental files to be stored; loading the data in the incremental files to be stored into a pre-created intermediate database; generating a script file for the data of the incremental files to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental files to be stored in the intermediate database to the online target database; executing the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database, thereby realizing automatic online incremental data storage of the online target database.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a method, device and system for storing incremental data in an online database. Background Art

[0002] As business development places more and more complex demands on data, the amount of business data stored in the database is also growing rapidly. Business data is changing, which makes the need to update business data in the database more frequent. When business data in the database needs to be updated, the large amount of data for full update can easily lead to slow data update speed and low efficiency, which in turn introduces the method of incremental data storage.

[0003] In the related art, when updating business data in an online database, it is necessary to temporarily close the online database. After the online database is closed, the business data that needs to be updated in the database is incrementally stored in the database through manual replacement, and the online use of the database is restored after the data update is completed.

[0004] However, the above method requires a lot of manpower to maintain the database to achieve incremental data storage, resulting in high cost of incremental data storage, and it is impossible to achieve incremental data storage in the online database. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a method, device and system for incremental data storage in an online database to solve at least one of the above problems. The specific technical solution is as follows:

[0006] In a first aspect, an embodiment of the present invention provides a method for incrementally storing data in an online database, the method comprising:

[0007] Get the incremental files to be stored;

[0008] Loading the data in the incremental file to be stored into a pre-created intermediate database;

[0009] Generate a script file of the incremental file data to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0010] Execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0011] Optionally, the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified;

[0012] The target script file contains identification information of data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of data to be modified and data corresponding to the identification information of the data to be modified.

[0013] Optionally, when the target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted,

[0014] The executing the target script file to update the data of the incremental file to be stored in the intermediate database to the online target database includes:

[0015] Matching the identification information of the data to be inserted with the identification information of the data stored in the online target database;

[0016] If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be inserted, then updating the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be inserted;

[0017] If identification information of data stored in the online target database does not contain identification information identical to the identification information of the data to be inserted, the identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted are written into the online target database.

[0018] Optionally, when the target script file contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified,

[0019] The executing the target script file to update the data of the incremental file to be stored in the intermediate database to the online target database includes:

[0020] Matching the identification information of the data to be modified with the identification information of the data stored in the online target database;

[0021] If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be modified, then updating the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be modified;

[0022] If identification information of data stored in the online target database does not contain identification information identical to the identification information of the data to be modified, the identification information of the data to be modified and data corresponding to the identification information of the data to be modified are written into the online target database.

[0023] Optionally, after updating the data of the incremental file to be stored in the intermediate database to the online target database, the method further includes:

[0024] Delete all data in the intermediate database.

[0025] In a second aspect, an embodiment of the present invention provides a device for storing incremental data in an online database, the device comprising:

[0026] The file acquisition module is used to obtain the incremental files to be stored;

[0027] A data loading module is used to load the data in the incremental file to be stored into a pre-created intermediate database;

[0028] A file generation module, used to generate a script file of the incremental file data to be stored in the intermediate database, and obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0029] The data storage module is used to execute the target script file and update the data of the incremental files to be stored in the intermediate database to the online target database.

[0030] Optionally, the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified;

[0031] The target script file contains identification information of data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of data to be modified and data corresponding to the identification information of the data to be modified.

[0032] In a third aspect, an embodiment of the present invention provides a data incremental storage system for an online database, the system comprising: a client device and a server device, wherein:

[0033] The client device is used to obtain the incremental file to be stored, load the data in the incremental file to be stored into a locally pre-created intermediate database, and generate a script file of the incremental file data to be stored in the intermediate database to obtain a target script file, and send the target script file to the server device; wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0034] The server device is used to receive the target script file, execute the target script file, and update the data of the incremental file to be stored in the intermediate database to the online target database.

[0035] In a fourth aspect, an embodiment of the present invention provides an electronic device, including a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0036] Memory, used to store computer programs;

[0037] The processor is used to implement any of the above-mentioned method steps when executing the program stored in the memory.

[0038] In a fifth aspect, an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the method steps described above is implemented.

[0039] In a sixth aspect, an embodiment of the present invention further provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the method steps described above.

[0040] Beneficial effects of the embodiments of the present invention:

[0041] The embodiments of the present invention provide a method, device and system for incremental data entry into an online database, which can load the data in the incremental files to be entered into a pre-created intermediate database, generate a script file for the data of the incremental files to be entered in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be entered in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online incremental data entry into the online target database, reducing the high cost caused by manual data update for incremental data entry, and improving the efficiency of incremental data entry into the online target database.

[0042] Of course, it is not necessary to achieve all of the advantages described above at the same time to implement any product or method of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0044] Figure 1 A schematic diagram of a flow chart of a method for storing incremental data in an online database provided by an embodiment of the present invention;

[0045] Figure 2 A schematic diagram of an implementation method of a data incremental storage method provided by an embodiment of the present invention;

[0046] Figure 3 A schematic diagram of another implementation method of an incremental data storage method provided by an embodiment of the present invention;

[0047] Figure 4 A schematic diagram of a flow chart of another method for storing incremental data in an online database provided by an embodiment of the present invention;

[0048] Figure 5 A schematic diagram of the structure of a data increment storage device for an online database provided by an embodiment of the present invention;

[0049] Figure 6 A schematic diagram of the structure of an online database data increment storage system provided by an embodiment of the present invention;

[0050] Figure 7 A schematic diagram of the structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field based on this application belong to the scope of protection of the present invention.

[0052] In order to solve the problem that a large amount of manpower cost is required to maintain the database to realize the incremental storage of data in the related technology, resulting in high cost of incremental storage of data, and the inability to realize incremental storage of online databases, an embodiment of the present invention provides a method for incremental storage of data in an online database, the method comprising:

[0053] Obtain the incremental files to be stored; load the data in the incremental files to be stored into a pre-created intermediate database; generate a script file for the data of the incremental files to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental files to be stored in the intermediate database to the online target database; execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0054] A method for incremental data entry into an online database provided by an embodiment of the present invention can load data in incremental files to be entered into a pre-created intermediate database, generate a script file for the data of the incremental files to be entered in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be entered in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online incremental data entry into the online target database, reducing the high cost caused by manual data update for incremental data entry, and improving the efficiency of incremental data entry into the online target database.

[0055] The following is a detailed description of a method for incrementally storing data in an online database provided by an embodiment of the present invention:

[0056] The method for incremental data entry into an online database provided by an embodiment of the present invention can be applied to a server device or a server cluster where the online database is located, or can be applied to a client device that is connected to the online database and can operate the online database, or can be applied to cloud storage, cloud servers and other devices.

[0057] The method for storing incremental data in an online database provided by an embodiment of the present invention can be implemented by a separate device, for example, by a server device or a server cluster where the online database is located. It can also be implemented by device interaction, for example, by obtaining the incremental file to be stored through a client device, uploading the incremental file to be stored to the server device or server cluster where the online database is located, so that the server device or server cluster where the online database is located executes to update the data of the incremental file to be stored to the online target database, or by obtaining the incremental file to be stored through a client device, loading the data in the incremental file to be stored into a pre-created intermediate database, generating a script file for the data of the incremental file to be stored in the intermediate database, obtaining the target script file, and sending the target script file to the server device or server cluster where the online database is located, so that the server device or server cluster where the online database is located executes to update the data of the incremental file to be stored to the online target database, and so on.

[0058] like Figure 1 As shown, Figure 1 A schematic diagram of a flow chart of a method for incrementally storing data in an online database provided by an embodiment of the present invention, the method may include:

[0059] S101, obtaining the incremental files to be stored.

[0060] The incremental file to be stored may be an incremental file within a predetermined time period relative to the data stored in the online database, such as a day, a week, or a month, etc. The incremental file to be stored may also be an incremental file relative to the most recent data stored in the online database, etc. The incremental file to be stored may include incremental data to be updated in the online database, such incremental data may be data to be modified in the online database, and / or data to be inserted into the online database, etc.

[0061] In the embodiment of the present invention, the online database can be a MySQL database, an Oracle database, or a SQL Server database, etc. In the embodiment of the present invention, the online database is a MySQL database as an example for description, but it does not constitute a specific limitation of the present invention.

[0062] S102, loading the data in the incremental file to be stored into a pre-created intermediate database.

[0063] In the embodiment of the present invention, an intermediate database may be created in advance. The intermediate database may be created locally or deployed separately in other devices other than the local device, and is used to temporarily cache the incremental file data to be stored.

[0064] After the incremental file to be stored is obtained, the data in the incremental file to be stored may be loaded into a pre-created intermediate database, so that the intermediate database is used to temporarily cache the data in the incremental file to be stored.

[0065] S103, generating a script file of the incremental file data to be stored in the intermediate database, and obtaining a target script file.

[0066] Among them, the target script file is used to instruct the update of the data of the incremental files to be stored in the intermediate database to the online target database.

[0067] In an example of an embodiment of the present invention, the online database and the intermediate database are MySQL databases as an example for explanation, and mysqldump can be used to generate a corresponding script file for the data of the incremental file to be stored in the intermediate database to obtain a target script file. Among them, mysqldump is a utility program used by MySQL to transfer the storage database, which can generate a SQL (Structured Query Language) script file.

[0068] In an embodiment of the present invention, mysqldump can be used to generate a SQL script file for a Replace operation for the data of the incremental files to be stored in the intermediate database, and the SQL script file can instruct to update the data of the incremental files to be stored in the intermediate database to the online target database. Exemplarily, the generated SQL script file can include a Replace operation command, as well as relevant information such as the data of the incremental files to be stored.

[0069] S104, executing the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0070] After generating a target script file that can instruct the update of the data of the incremental files to be stored in the intermediate database to the online target database, the target script file can be triggered to execute to update the data of the incremental files to be stored in the intermediate database to the online target database, thereby realizing automatic storage of incremental data in the online target database.

[0071] An embodiment of the present invention provides a method for incremental data entry into an online database. The method can load data in incremental files to be entered into a pre-created intermediate database, generate a script file for the data of the incremental files to be entered in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be entered in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online incremental data entry into the online target database, and being able to perform online incremental data entry into the incremental files to be entered without stopping the online database from providing external services, thereby better meeting the business party's use requirements for the online database, improving the availability of the online database, and at the same time reducing the high cost caused by manual data update for incremental data entry, thereby improving the efficiency of incremental data entry into the online target database.

[0072] In a possible implementation, the incremental file to be stored may include identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified.

[0073] In one example, the incremental file to be stored may contain identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, which is used to update the data newly added compared to the online target database into the online target database. Alternatively, the incremental file to be stored may contain identification information of the data to be modified and data corresponding to the identification information of the data to be modified, which is used to update the newly modified (or updated) data compared to the online target database into the online target database. Alternatively, the incremental file to be stored may contain identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and identification information of the data to be modified and data corresponding to the identification information of the data to be modified, which is used to update the data newly added compared to the online target database, and the newly modified data into the online target database.

[0074] Correspondingly, the target script file obtained above may include identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified.

[0075] In an example, when the above-mentioned incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, the corresponding target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted; when the above-mentioned incremental file to be stored contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified, the corresponding target script file contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified; when the above-mentioned incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and identification information of the data to be modified and data corresponding to the identification information of the data to be modified, the corresponding target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and identification information of the data to be modified and data corresponding to the identification information of the data to be modified.

[0076] In a possible implementation, when the target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, correspondingly, the above step S104 executes the target script file to update the data of the incremental file to be stored in the intermediate database to the implementation of the online target database, such as Figure 2 As shown, it may include:

[0077] S201, matching identification information of the data to be inserted with identification information of the data stored in the online target database.

[0078] In one example, the target script file obtained may include a Replace operation command, identification information of the data to be inserted, and data corresponding to the identification information of the data to be inserted, etc., and triggering the execution of the target script file triggers the execution of the Replace operation command. Specifically, the identification information of the data to be inserted may be matched with the identification information of the data stored in the online target database, and it is checked whether the identification information of the data stored in the online target database contains the same identification information as the identification information of the data to be inserted. If so, step S202 is executed, and if not, step S203 is executed.

[0079] S202: If the identification information of the data stored in the online target database contains identification information that is identical to the identification information of the data to be inserted, update the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be inserted.

[0080] S203: If the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be inserted, write the identification information of the data to be inserted and the data corresponding to the identification information of the data to be inserted into the online target database.

[0081] In the case where the identification information of the data stored in the online target database contains the same identification information as the identification information of the data to be inserted, it is necessary to perform a data replacement operation to update the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be inserted, so as to complete the online data incremental storage for the online target database. In the case where the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be inserted, it is necessary to perform a data insertion operation to write the identification information of the data to be inserted and the data corresponding to the identification information of the data to be inserted into the online target database, so as to complete the online data incremental storage for the online target database.

[0082] In a possible implementation, when the target script file contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified, correspondingly, the above step S104 executes the target script file to update the data of the incremental file to be stored in the intermediate database to the implementation of the online target database, such as Figure 3 As shown, it may include:

[0083] S301, matching identification information of the data to be modified with identification information of the data stored in the online target database.

[0084] In one example, the target script file obtained may include a Replace operation command, identification information of the data to be modified, and data corresponding to the identification information of the data to be modified, etc. Triggering the execution of the target script file triggers the execution of the Replace operation command. Specifically, the identification information of the data to be modified may be matched with the identification information of the data stored in the online target database, and it is checked whether the identification information of the data stored in the online target database contains the same identification information as the identification information of the data to be modified. If so, step S302 is executed, and if not, step S303 is executed.

[0085] S302: If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be modified, update the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be modified.

[0086] S303: If the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be modified, write the identification information of the data to be modified and the data corresponding to the identification information of the data to be modified into the online target database.

[0087] In the case where the identification information of the data stored in the online target database contains the same identification information as the identification information of the data to be modified, it is necessary to perform a data replacement operation to update the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be modified, so as to complete the online data incremental storage for the online target database. In the case where the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be modified, it is necessary to perform a data insertion operation to write the identification information of the data to be modified and the data corresponding to the identification information of the data to be modified into the online target database, so as to complete the online data incremental storage for the online target database.

[0088] In a possible implementation, Figure 4 As shown, another method for storing incremental data in an online database provided by an embodiment of the present invention may include:

[0089] S401, obtaining the incremental files to be stored.

[0090] S402, loading the data in the incremental file to be stored into a pre-created intermediate database.

[0091] S403, generating a script file of the incremental file data to be stored in the intermediate database, and obtaining a target script file.

[0092] Among them, the target script file is used to instruct the update of the data of the incremental files to be stored in the intermediate database to the online target database.

[0093] S404, execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0094] The implementation process of step S401 to step S404 may refer to the implementation process of step S101 to step S104 described above, and the embodiments of the present invention will not be repeated here.

[0095] S405, delete all data in the intermediate database.

[0096] In an embodiment of the present invention, after the data of the incremental files to be stored in the intermediate database are updated to the online target database, all data cached in the intermediate database can be deleted so that the intermediate database can cache the incremental files to be stored when the data is incrementally stored next time or in the next cycle.

[0097] An embodiment of the present invention provides a method for incremental data entry into an online database. The method can load data in incremental files to be entered into a pre-created intermediate database, generate a script file for the data of the incremental files to be entered in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be entered in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online incremental data entry into the online target database, and being able to perform online incremental data entry into the incremental files to be entered without stopping the online database from providing external services, thereby better meeting the business party's use requirements for the online database, improving the availability of the online database, and at the same time reducing the high cost caused by manual data update for incremental data entry, thereby improving the efficiency of incremental data entry into the online target database.

[0098] In one example, an embodiment of the present invention can deploy an intermediate database in a server device, a server cluster, or a cloud server device where an online database is located, and then the server device, the server cluster, or the cloud server device can obtain the incremental files to be stored, load the data in the incremental files to be stored into a pre-created intermediate database, generate a script file for the data of the incremental files to be stored in the intermediate database, obtain a target script file, and further execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database. Among them, the server device, the server cluster, or the cloud server device obtains the incremental files to be stored, which can be the server device, the server cluster, or the cloud server device itself obtaining the incremental files to be stored, or the server device, the server cluster, or the cloud server device obtaining the incremental files to be stored from other devices (such as client devices, etc.).

[0099] In one example, an embodiment of the present invention can also deploy an intermediate database separately. For example, the online database is deployed in a server device, and the intermediate database is deployed in a client device. When implementing online data incremental warehousing for the online database, the client device can obtain the incremental files to be stored, load the data in the incremental files to be stored into a pre-created local intermediate database, generate a script file for the incremental file data to be stored in the intermediate database, obtain a target script file, send the target script file to the server device, and the server device executes the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0100] In a possible implementation, the embodiment of the present invention can also create a historical backtracking database, which can record the reception time, update time and valid time of each data in the online target database, so as to facilitate subsequent analysis and tracking of the data in the online target database.

[0101] Corresponding to the method for storing incremental data in an online database provided in the above embodiment, the embodiment of the present invention further provides a method device for storing incremental data in an online database, such as Figure 5 As shown, the device may include:

[0102] The file acquisition module 501 is used to acquire the incremental files to be stored;

[0103] The data loading module 502 is used to load the data in the incremental file to be stored into the pre-created intermediate database;

[0104] The file generation module 503 is used to generate a script file of the incremental file data to be stored in the intermediate database, and obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0105] The data storage module 504 is used to execute the target script file and update the data of the incremental files to be stored in the intermediate database to the online target database.

[0106] An embodiment of the present invention provides a device for incremental data entry into an online database, which can load data in incremental files to be entered into a pre-created intermediate database, generate a script file for the data of the incremental files to be entered in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be entered in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online incremental data entry into the online target database, reducing the high cost caused by manual data update for incremental data entry, and improving the efficiency of incremental data entry into the online target database.

[0107] In a possible implementation manner, the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified;

[0108] The target script file contains identification information of data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of data to be modified and data corresponding to the identification information of the data to be modified.

[0109] In a possible implementation manner, when the target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, the data storage module 504 is specifically used to:

[0110] Match the identification information of the data to be inserted with the identification information of the data stored in the online target database;

[0111] If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be inserted, then the data corresponding to the identification information of the data stored in the online target database is updated to the data corresponding to the identification information of the data to be inserted;

[0112] If the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be inserted, the identification information of the data to be inserted and the data corresponding to the identification information of the data to be inserted are written into the online target database.

[0113] In a possible implementation manner, when the target script file contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified, the data storage module 504 is specifically used to:

[0114] Matching the identification information of the data to be modified with the identification information of the data stored in the online target database;

[0115] If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be modified, then the data corresponding to the identification information of the data stored in the online target database is updated to the data corresponding to the identification information of the data to be modified;

[0116] If the identification information of the data stored in the online target database does not contain the same identification information as the identification information of the data to be modified, the identification information of the data to be modified and the data corresponding to the identification information of the data to be modified are written into the online target database.

[0117] In a possible implementation manner, the above device further includes:

[0118] The data deletion module is used to delete all data in the intermediate database.

[0119] Corresponding to the method for incrementally storing data in an online database provided in the above embodiment, the embodiment of the present invention further provides a system for incrementally storing data in an online database, such as Figure 6 As shown, the system 600 may include: a client device 601 and a server device 602, wherein:

[0120] The client device 601 is used to obtain the incremental file to be stored, load the data in the incremental file to be stored into the locally pre-created intermediate database, generate a script file of the incremental file data to be stored in the intermediate database, obtain a target script file, and send the target script file to the server device 602; wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0121] The server device 602 is used to receive the target script file, execute the target script file, and update the data of the incremental file to be stored in the intermediate database to the online target database.

[0122] An embodiment of the present invention provides an online database data incremental warehousing system, which can load the data in the incremental files to be stored into a pre-created intermediate database, generate a script file for the incremental file data to be stored in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be stored in the intermediate database to the online target database without closing the online target database, thereby realizing automatic online data incremental warehousing of the online target database, reducing the high cost caused by manual data update for data incremental warehousing, and improving the efficiency of data incremental warehousing of the online target database.

[0123] The embodiment of the present invention further provides an electronic device, such as Figure 7 As shown, it includes a processor 701, a communication interface 702, a memory 703 and a communication bus 704, wherein the processor 701, the communication interface 702, and the memory 703 communicate with each other through the communication bus 704.

[0124] Memory 703, used for storing computer programs;

[0125] The processor 701 is used to execute the program stored in the memory 703 to implement the following steps:

[0126] Get the incremental files to be stored;

[0127] Load the data in the incremental file to be stored into the pre-created intermediate database;

[0128] Generate a script file for the incremental file data to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database;

[0129] Execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database.

[0130] An electronic device provided by an embodiment of the present invention can load data in incremental files to be stored into a pre-created intermediate database, generate a script file for the incremental file data to be stored in the intermediate database, obtain a target script file, execute the target script file, and update the data of the incremental files to be stored in the intermediate database to an online target database without closing the online target database, thereby realizing automatic online incremental data storage in the online target database, reducing the high cost caused by manual data update for incremental data storage, and improving the efficiency of incremental data storage in the online target database.

[0131] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0132] The communication interface is used for communication between the above electronic device and other devices.

[0133] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0134] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0135] In another embodiment provided by the present invention, a computer-readable storage medium is also provided, which stores a computer program. When the computer program is executed by a processor, it implements any of the above-mentioned methods for incremental data entry into an online database to achieve the same technical effect.

[0136] In another embodiment provided by the present invention, a computer program product containing instructions is also provided. When the computer is run on a computer, the computer executes any one of the methods for incrementally storing data in an online database in the above embodiments to achieve the same technical effect.

[0137] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk Solid State Disk (SSD)), etc.

[0138] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0139] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device / system / electronic device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0140] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A method for incrementally storing data in an online database, characterized in that: The method comprises: Get the incremental files to be stored; Loading the data in the incremental file to be stored into a pre-created intermediate database; Generate a script file for the data of the incremental file to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct updating the data of the incremental file to be stored in the intermediate database to the online target database; the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; the target script file contains operation commands, identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; Execute the target script file to update the data of the incremental files to be stored in the intermediate database to the online target database; Delete all data in the intermediate database.

2. The method according to claim 1, characterized in that In the case where the target script file contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, The executing the target script file to update the data of the incremental file to be stored in the intermediate database to the online target database includes: Matching the identification information of the data to be inserted with the identification information of the data stored in the online target database; If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be inserted, then updating the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be inserted; If identification information of data stored in the online target database does not contain identification information identical to the identification information of the data to be inserted, the identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted are written into the online target database.

3. The method according to claim 1, characterized in that In the case where the target script file contains identification information of the data to be modified and data corresponding to the identification information of the data to be modified, The executing the target script file to update the data of the incremental file to be stored in the intermediate database to the online target database includes: Matching the identification information of the data to be modified with the identification information of the data stored in the online target database; If the identification information of the data stored in the online target database contains identification information that is the same as the identification information of the data to be modified, then updating the data corresponding to the identification information of the data stored in the online target database to the data corresponding to the identification information of the data to be modified; If identification information of data stored in the online target database does not contain identification information identical to the identification information of the data to be modified, the identification information of the data to be modified and data corresponding to the identification information of the data to be modified are written into the online target database.

4. A data increment storage device for an online database, characterized in that: The device comprises: The file acquisition module is used to obtain the incremental files to be stored; A data loading module is used to load the data in the incremental file to be stored into a pre-created intermediate database; A file generation module is used to generate a script file of the incremental file data to be stored in the intermediate database to obtain a target script file, wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database; the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; the target script file contains operation commands, identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; A data storage module is used to execute the target script file and update the data of the incremental file to be stored in the intermediate database to the online target database; The data deletion module is used to delete all data in the intermediate database.

5. A data incremental storage system for an online database, characterized in that: The system comprises: a client device and a server device, wherein: The client device is used to obtain the incremental file to be stored, load the data in the incremental file to be stored into a locally pre-created intermediate database, and generate a script file of the incremental file data to be stored in the intermediate database to obtain a target script file, and send the target script file to the server device; wherein the target script file is used to instruct to update the data of the incremental file to be stored in the intermediate database to the online target database; the incremental file to be stored contains identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; the target script file contains operation commands, as well as identification information of the data to be inserted and data corresponding to the identification information of the data to be inserted, and / or identification information of the data to be modified and data corresponding to the identification information of the data to be modified; The server device is used to receive the target script file, execute the target script file, and update the data of the incremental file to be stored in the intermediate database to the online target database; The client device is also used to delete all data in the intermediate database after the server device updates the data of the incremental files to be stored in the intermediate database to the online target database.

6. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 3 when executing a program stored in a memory.

7. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method steps described in any one of claims 1 to 3 are implemented.

Images