Data processing method, device, electronic device, storage medium and program product

Abstract

The present application provides a data processing method, device, electronic device, storage medium and program product. The method provided by the present application includes: obtaining data migration parameters, the data migration parameters include: the identifier of the target structured query language SQL script corresponding to the target subsystem, the target SQL script is used to indicate the method of extracting data to be migrated to the target subsystem from the source database; the target subsystem is any one of at least one subsystem; according to the identifier of the target SQL script, obtaining the target SQL script; using the target SQL script, exporting the target data file to be migrated to the target subsystem from the source database, the target data file is in a preset format, and the identifier of the target data file is the same as the identifier of the target SQL script; sending the target data file to the second device; the second device is a device of the target subsystem. The present application improves the efficiency of data migration.

Description

Technical Field

[0001] The present application relates to database technology, and in particular to a data processing method, device, electronic device, storage medium and program product. Background Art

[0002] The e-commerce platform includes a source system and multiple subsystems. Among them, the source database in the source system stores the business data of the above-mentioned multiple subsystems. Take the source database storing business data 1 of subsystem 1 and business data 2 of subsystem 2 as an example, wherein there may be some overlapping data between business data 1 and business data 2. If subsystem 1 modifies the partially overlapping data, when subsystem 2 uses the partially overlapping data, it will use the data modified by subsystem 1, which may cause errors in the business based on the partially overlapping data in subsystem 2. At present, the business data corresponding to each subsystem in the source database can be migrated to the subdatabase of each subsystem. Then the subsystem can perform data operations on the business data in the subdatabase of the subsystem to avoid errors in the operation of other subsystems caused by the data operations of one subsystem on its business data.

[0003] However, when using the existing data migration method to export data from the source database, only table-level operations can be performed on the data in the source database. For example, if it is necessary to migrate part of the data in table a in the source database, the existing data migration method is to export the data of the entire table a and then write the data of the entire table a into the sub-database of the subsystem. Therefore, the existing data migration method has the problem of data redundancy, which may lead to low efficiency of data migration. Summary of the invention

[0004] The present application provides a data processing method, device, electronic device, storage medium and program product to improve the efficiency of data migration.

[0005] In a first aspect, the present application provides a data processing method, where an e-commerce platform includes a source system and at least one subsystem, the source system has a source database, and the subsystem has a sub-database. The method is applied to a first device in the source system, and the method includes:

[0006] Acquire data migration parameters, the data migration parameters including: an identifier of a target structured query language SQL script corresponding to a target subsystem, the target SQL script being used to indicate a method of extracting data to be migrated to the target subsystem from the source database; the target subsystem being any one of the at least one subsystem;

[0007] According to the identifier of the target SQL script, obtaining the target SQL script;

[0008] Using the target SQL script, exporting a target data file to be migrated to the target subsystem from the source database, wherein the target data file is in a preset format and an identifier of the target data file is the same as an identifier of the target SQL script;

[0009] The target data file is sent to a second device; the second device is a device of the target subsystem. The second device is used to determine the target data table of the data to be written into the target data file in the target subdatabase according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target subdatabase of the target subsystem.

[0010] Optionally, the using the target SQL script to export the target data file to be migrated to the target subsystem from the source database includes:

[0011] Obtaining a configuration file of the source database, wherein the configuration file of the source database includes: an access path of the source database, a user name and a password required to access the source database;

[0012] According to the access path of the source database, log in to the source database using the user name and the password;

[0013] The target SQL script is used to export the target data file to be migrated to the target subsystem from the source database.

[0014] Optionally, after using the target SQL script to export the target data file to be migrated to the target subsystem from the source database, the method further includes:

[0015] Compressing the target data file to obtain a compressed target data file;

[0016] The sending the target data file to the second device includes:

[0017] Send the compressed target data file to the second device.

[0018] Optionally, the method further includes:

[0019] If the target data file to be migrated to the target subsystem is not successfully exported from the source database using the target SQL script, information about data export failure is recorded.

[0020] In a second aspect, the present application provides a data processing method, where an e-commerce platform includes a source system and at least one subsystem, the source system has a source database, the subsystem has a sub-database, the method is applied to a second device of a target subsystem in the at least one subsystem, and the method includes:

[0021] receiving a target data file from a first device of the source system; the target data file is a data file to be migrated to the target subsystem, which is exported from the source database by the first device using a target SQL script, the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database, the target data file is in a preset format, and an identifier of the target data file is the same as an identifier of the target SQL script;

[0022] Determine the target data table of the data to be written into the target data file in the target subdatabase according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target subdatabase of the target subsystem;

[0023] A write operation is performed on the target data table, wherein the write operation is used to write the data in the target data file.

[0024] Optionally, performing a write operation on the target data table includes:

[0025] Acquire a configuration file of the target sub-database, wherein the configuration file of the target sub-database includes: an access path of the target sub-database, a user name and a password required to access the target sub-database;

[0026] According to the access path of the target sub-database, log in to the target sub-database using the user name and the password;

[0027] A write operation is performed on the target data table.

[0028] Optionally, performing a write operation on the target data table includes:

[0029] Acquire data writing parameters, the data writing parameters including: an identifier of the target data file, an access path of the target data file, an identifier of whether the data in the target data file belongs to incremental data of the target data table, and a writing method;

[0030] A write operation is performed on the target data table according to the data write parameter.

[0031] Optionally, performing a write operation on the target data table includes:

[0032] If the write operation is interrupted and the data in the target data file belongs to incremental data of the target data table, the data written into the target data table by executing the write operation is deleted from the target data table, and the write operation is re-executed on the target data table.

[0033] Optionally, the receiving the target data file from the first device of the source system includes:

[0034] receiving a compressed target data file from the first device;

[0035] The compressed target data file is decompressed to obtain the target data file.

[0036] Optionally, after performing the write operation to the target data table, the method further includes:

[0037] After the data in the target data file is successfully written into the target data table, the target data file is deleted.

[0038] In a third aspect, the present application provides a data processing apparatus, wherein an e-commerce platform includes a source system and at least one subsystem, wherein the source system has a source database, and the subsystem has a sub-database, and the apparatus is applied to a first device in the source system, and the apparatus includes:

[0039] A first acquisition module is used to acquire data migration parameters, wherein the data migration parameters include: an identifier of a target structured query language SQL script corresponding to a target subsystem, wherein the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database; the target subsystem is any one of the at least one subsystem;

[0040] A second acquisition module, configured to acquire the target SQL script according to the identifier of the target SQL script;

[0041] An export module, used to export a target data file to be migrated to the target subsystem from the source database using the target SQL script, wherein the target data file is in a preset format and an identifier of the target data file is the same as an identifier of the target SQL script;

[0042] The sending module is used to send the target data file to a second device; the second device is a device of the target subsystem.

[0043] In a fourth aspect, the present application provides a data processing device, wherein an e-commerce platform includes a source system and at least one subsystem, wherein the source system has a source database, and the subsystem has a sub-database, and the device is applied to a second device of a target subsystem in the at least one subsystem, and the device includes:

[0044] a receiving module, configured to receive a target data file from a first device of the source system; the target data file is a data file to be migrated to the target subsystem, which is exported from the source database by the first device using a target SQL script, the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database, the target data file is in a preset format, and an identifier of the target data file is the same as an identifier of the target SQL script;

[0045] A determination module, configured to determine a target data table of the data to be written into the target data file in the target subdatabase according to an identifier of the target data file and a mapping relationship between an identifier of a preset SQL script and a data table of a target subdatabase of the target subsystem;

[0046] An execution module is used to perform a write operation on the target data table, wherein the write operation is used to write the data in the target data file.

[0047] In a fifth aspect, the present application provides an electronic device, including: at least one processor and a memory;

[0048] The memory stores computer-executable instructions;

[0049] The at least one processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method described in any one of the first aspects.

[0050] In a sixth aspect, the present application provides a computer-readable storage medium, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the method described in any one of the first aspects is implemented.

[0051] In a seventh aspect, the present application provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method described in any one of the first aspects is implemented.

[0052] The data processing method, device, electronic device, storage medium and program product provided by the present application can obtain the target SQL script according to the identifier of the target SQL script. Using the target SQL script, the target data file of the target subsystem to be migrated can be exported from the source database. By exporting data from the source database by SQL script, data export below the table level in the source database can be realized. In other words, the data in the target data file only includes data that needs to be migrated. Compared with the existing data migration method, the above method reduces the data redundancy in the migration data, thereby improving the efficiency of the first device to export the target data file. Reducing the data redundancy in the migration data also reduces the data volume of the target data file, thereby reducing the transmission bandwidth required for the first device to send the target data file to the second device, reducing the use of network resources, and reducing the transmission delay. For the second device, reducing the data redundancy in the migration data also improves the efficiency of the second device to write the target data file to the target data table of the target sub-database. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0054] Figure 1 A schematic diagram of an application scenario of the data processing method;

[0055] Figure 2 A flowchart of a data processing method provided in this application;

[0056] Figure 3 A flowchart of a method for performing a write operation to a target data table provided by the present application;

[0057] Figure 4 A flowchart of another data processing method provided for this application;

[0058] Figure 5 A flowchart of another data processing method provided by the present application;

[0059] Figure 6 A schematic diagram of the structure of a data processing device 500 provided in this application;

[0060] Figure 7 A schematic diagram of the structure of another data processing device 600 provided in the present application;

[0061] Figure 8A schematic diagram of the structure of an electronic device provided in this application.

[0062] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0063] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in this application. Obviously, the described embodiments are part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0064] The following is an explanation of some of the concepts involved in this application:

[0065] E-commerce platform: An e-commerce platform is a platform for users to conduct online transactions. The e-commerce platform needs to coordinate and integrate transaction information related data (such as transaction amount information, goods information, etc.). The e-commerce platform includes a source system and at least one subsystem. Each subsystem performs different business.

[0066] Structured Query Language (SQL): A database query and programming language used to access data and query, update, and manage relational database systems. By accessing the database through SQL scripts, you can access any row or column of data in a table in the database.

[0067] Shell script: You can use commands under the Linux operating system or Unix operating system. Through the Shell script, you can put multiple different types of commands into a file in advance, so that the electronic device can execute the above multiple commands at one time, which can improve the efficiency of completing the above multiple commands.

[0068] awk and sed: Both are text processing tools. For example, the commands in awk or sed can be used to split a string.

[0069] sqlldr: A database data loading command.

[0070] sqlplus: A client tool for interacting with the database. In sqlplus, you can run sqlplus commands and sql scripts.

[0071] spool: A data export instruction.

[0072] Chinese Internal Code Specification (GBK): A standard data format that can be recognized by various types of databases.

[0073] Direct write mode: A mode that can be used when writing data to a data table. If the direct write mode is used to write data to a data table, the data can be written directly to the data table without going through the System Global Area (SGA). After the data is written, the index of the data needs to be rebuilt.

[0074] Index writing mode: a writing mode that can be used when writing data into a data table. If the index writing mode is used to write data into a data table, the data needs to be written into the data table according to the data index list.

[0075] General Data Transfer Platform (GTP): is a data transfer platform that provides general transmission functions to meet the needs of enterprise-level applications according to demand.

[0076] Database operation level: Database operation levels include: global level operations, database level operations, table level operations, and operations at a row (or column) level in a table.

[0077] Figure 1 Figure 1 is a schematic diagram of an application scenario of the data processing method. Figure 1 As shown, the source database of the source system of the e-commerce platform may store business data of multiple subsystems. For example, taking the source database storing business data of N subsystems (N is an integer greater than or equal to 1) as an example, the business data of each subsystem may be as shown in the following Table 1:

[0078] Table 1

[0079] Serial number Subsystem Subsystem business data 1 Subsystem 1 Business data 1 2 Subsystem 2 Business data 2 … … … N Subsystem N Business dataN

[0080] Taking the business data 1 and business data 2 shown in Table 1 as an example, the user corresponding to subsystem 1 can perform data operations on business data 1 through the electronic device of the source system. The user corresponding to subsystem 2 can perform data operations on business data 2 through the electronic device of the source system. Exemplarily, the above data operations may include operations such as deletion, modification, or reading.

[0081] like Figure 1As shown, in the source database, there may be some overlapping data between business data 1 and business data 2. If subsystem 1 modifies or deletes the partially overlapping data, when subsystem 2 uses the partially overlapping data, it will use the data modified or deleted by subsystem 1, which may cause errors in the business based on the partially overlapping data in subsystem 2. In other words, after each subsystem performs data operations on the data in the source database, it may cause errors in the business execution of other subsystems.

[0082] At present, in order to avoid errors in the execution of other subsystems' business after each subsystem performs data operations on the data in the source database, such as Figure 1 As shown, a sub-database of the sub-system can be set in each sub-system, and the business data corresponding to each sub-system can be migrated to the sub-database corresponding to each sub-system. Taking sub-system 1 as an example, business data 1 can be stored in sub-database 1 in sub-system 1, and then the user corresponding to sub-system 1 can perform data operations on business data 1 in sub-database 1 through electronic device 1. Through the above method, even if sub-system 1 performs data operations on the overlapping data between business data 1 and business data 2, the above-mentioned overlapping data in sub-database 2 in sub-system 2 will not be changed, so it will not affect the business processing based on the part of overlapping data in sub-system 2.

[0083] When migrating business data corresponding to each subsystem in the source database to the subdatabase corresponding to each subsystem, it is first necessary to export the business data corresponding to each subsystem in the source database, and then write the exported data into the subdatabase of the corresponding subsystem.

[0084] However, when using existing data processing methods to export data from a source database, only table-level operations can be performed on the data in the source database. For example, if data migration is required for part of the data in table a in the source database, the existing data processing method is to export the data of the entire table a, and then write the data of the entire table a into the sub-database of the corresponding subsystem. In other words, the existing data processing method will not only migrate the data that needs to be migrated, but also migrate the data that does not need to be migrated. Therefore, the existing data migration method has the problem of data redundancy, which may lead to low efficiency of data migration.

[0085] Considering that the reason why the existing data processing methods have low efficiency in data migration is that there is data redundancy when migrating data, this application proposes a method for accurately exporting the data to be migrated from the source database through SQL scripts. Through this method, only the data that needs to be migrated is exported, which improves the efficiency of data export from the source data, and the efficiency of writing the data to be migrated into the sub-database of the subsystem, thereby improving the efficiency of data migration.

[0086] In a specific implementation, the electronic device that executes the above data processing method may be one electronic device or at least two electronic devices.

[0087] Taking the electronic device executing the above data processing method as an electronic device as an example, the electronic device may be, for example, an electronic device in the source system that can export the data to be migrated from the source database and write the data to be migrated into the sub-database of each sub-system. Alternatively, the electronic device may also be, for example, an electronic device in each sub-system that can export the data to be migrated from the source database and write the data to be migrated into the sub-database of the sub-system.

[0088] Take two electronic devices as an example to execute the above data processing method. Figure 1 As shown, the two electronic devices may be, for example, an electronic device in a source system and an electronic device in a subsystem. In this scenario, the electronic device in the source system may export the data to be migrated from the source database in the source system. Then, the electronic device in the source system sends the exported data to the electronic device in the subsystem. Accordingly, the electronic device in the subsystem may receive the exported data and write the data into the sub-database of the subsystem.

[0089] The following is an example in which the electronic device in the source system exports the data to be migrated from the source database and then sends it to the electronic device of the subsystem; the electronic device of the subsystem receives the data to be migrated and writes the data to be migrated into the database of the subsystem, and the technical solution of the present application is described in detail in combination with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0090] Figure 2 A flowchart of a data processing method provided in this application. Figure 2 As shown, the method comprises the following steps:

[0091] S101. A first device obtains data migration parameters.

[0092] The first device is a device in the source system. The data migration parameters include: an identifier of a target SQL script corresponding to a target subsystem. The target subsystem may be any subsystem of the subsystems of the e-commerce platform. Exemplarily, the identifier of the target SQL script may be, for example, the file name of the file where the target SQL script is located, or other identification information (such as numbers or letters).

[0093] The target SQL script is used to indicate a method for extracting data to be migrated to the target subsystem from the source database. Exemplarily, the method for extracting data to be migrated to the target subsystem from the source database may be, for example, extracting data from any at least one column or any at least one row in a data table in the source database.

[0094] It should be understood that this application does not limit whether the above data migration parameters include other parameters. Exemplarily, the above data migration parameters may also include, for example, the current date when the first device obtains the data migration parameters, the identifier of the target subsystem, the identifier corresponding to the data migration action, the identifier of the data to be migrated to the target subsystem, etc. In specific implementation, what other parameters the above data migration parameters may include may be determined, for example, based on the need to access the source database of the source system.

[0095] Optionally, the first device may receive, for example, data migration parameters input by a user. In this implementation, the first device may obtain the data migration parameters input by the user, for example, through an application programming interface (API) or a graphical user interface (GUI). Alternatively, the first device may also receive, for example, data migration parameters sent by an electronic device of the target subsystem.

[0096] S102: The first device obtains a target SQL script according to an identifier of the target SQL script.

[0097] After obtaining the identifier of the target SQL script, taking the identifier of the target SQL script as the file name of the file where the target SQL script is located as an example, the first device may obtain the target SQL script from the SQL scripts stored in the first device according to the file name.

[0098] Taking the target SQL script identifier as identification information such as numbers or letters as an example, the first device can, for example, obtain the target SQL script according to the identifier of the target SQL script and the mapping relationship between the identifier of the SQL script and the SQL script.

[0099] S103: The first device uses the target SQL script to export the target data file to be migrated to the target subsystem from the source database.

[0100] The data in the target data file to be migrated to the target subsystem is the data to be migrated to the target subsystem. The format of the target data file is a preset format. The identifier of the target data file is the same as the identifier of the target SQL script. Exemplarily, the preset format can be any standard data format (such as GBK format) so that the sub-database of the target subsystem can recognize the data format.

[0101] As a possible implementation, before using the target SQL script to export the target data file to be migrated to the target subsystem from the source database, the first device may first obtain the configuration file of the source database. The configuration file of the source database includes: the access path of the source database, and the user name and password required to access the source database. Exemplarily, the first device may obtain the above configuration file through an API or a GUI.

[0102] Then, the first device can log in to the source database using a user name and password according to the access path of the source database. After logging in to the source database, the first device can use the target SQL script to export the target data file to be migrated to the target subsystem from the source database.

[0103] As another possible implementation, if a user name is not required to access the source database, but only a password to access the source database is required, the first device may also log in to the source database according to the access path and password after obtaining the access path to the source database and the password required to access the source database. Alternatively, if only the access path to the source database is required to access the source database, the first device may also log in to the source database according to the access path after obtaining the access path to the source database. Then, the target SQL script is used to export the target data file to be migrated to the target subsystem from the source database.

[0104] S104: The first device sends the target data file to the second device.

[0105] The second device is a device of the target subsystem.

[0106] After exporting the target data file to be migrated to the target subsystem from the source database, the first device may send the target data file to the second device, and correspondingly, the second device may receive the target data file.

[0107] S105. The second device determines a target data table of the target sub-database for the data to be written into the target data file according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target sub-database of the target sub-system.

[0108] The target data tables corresponding to the identifiers of different target data files may be the same or different.

[0109] Exemplarily, the mapping relationship between the identifier of the above SQL script and the data table of the target sub-database of the target sub-system may be as shown in the following Table 2 (where M is an integer greater than or equal to 1):

[0110] Table 2

[0111] Serial number SQL script identifier Data Sheet 1 Logo 1 Data Table 1 2 Logo 2 Data Table 2 … … … M Logo M Data Sheet M

[0112] After receiving the target data file, the second device can obtain the identifier of the target data file. Taking the mapping relationship shown in Table 2 as an example, assuming that the identifier of the target data file is identifier 2, the second device can determine that the target data table of the data to be written into the target data file is data table 2.

[0113] Optionally, the mapping relationship between the identifier of the above-mentioned SQL script and the data table of the target sub-database of the target sub-system may be pre-stored in the second device by the user, for example.

[0114] S106: The second device performs a write operation on the target data table.

[0115] The above-mentioned writing operation is used to write data in the target data file.

[0116] As a possible implementation, before performing a write operation to the target data table, the second device may first obtain a configuration file of the target sub-database. The configuration file of the target sub-database includes: an access path to the target sub-database, and a user name and password required to access the target sub-database. Exemplarily, the first device may obtain the above configuration file through an API or a GUI.

[0117] Then, the second device can log in to the target sub-database using the user name and password according to the access path of the target sub-database. After logging in to the target sub-database, the second device can perform a write operation on the target data table to write the data in the target data file into the target data table.

[0118] As another possible implementation, if a user name is not required to access the target sub-database, but only a password to access the target sub-database is required, the second device may also log in to the target sub-database according to the access path and password after acquiring the access path of the target sub-database and the password required to access the target sub-database. Alternatively, if only the access path of the target sub-database is required to access the target sub-database, the second device may also log in to the target sub-database according to the access path after acquiring the access path of the target sub-database. Then, a write operation is performed on the target data table to write the data in the target data file into the target data table.

[0119] In this embodiment, the target SQL script can be obtained according to the identifier of the target SQL script. Using the target SQL script, the target data file of the target subsystem to be migrated can be exported from the source database. By exporting data from the source database through the SQL script, data export below the table level in the source database can be achieved. In other words, the data in the target data file only includes data that needs to be migrated. Compared with the existing data migration method, the above method reduces the data redundancy in the migration data, thereby improving the efficiency of the first device in exporting the target data file. Reducing the data redundancy in the migration data also reduces the data volume of the target data file, thereby reducing the transmission bandwidth required for the first device to send the target data file to the second device, reducing the use of network resources, and reducing the transmission delay. For the second device, reducing the data redundancy in the migration data also improves the efficiency of the second device in writing the target data file into the target data table of the target sub-database.

[0120] As a possible implementation manner, the target data file sent by the first device to the second device can also be a compressed target data file to reduce the data volume of the target data file, thereby improving the efficiency of the first device sending the target data file to the second device, thereby further improving the efficiency of data migration.

[0121] Specifically, after using the target SQL script to export the target data file to be migrated to the target subsystem from the source database, the first device may also compress the target data file to obtain a compressed target data file. The first device may then send the compressed target data file to the second device. Correspondingly, the second device may receive the compressed target data file from the first device. The second device may then decompress the compressed target data file to obtain the target data file.

[0122] As a possible implementation, the first device may also determine whether the target data file to be migrated to the target subsystem is successfully exported from the source database, and record the data export information so that the user can know whether the data export is successful. Specifically, when the target data file to be migrated to the target subsystem is not successfully exported from the source database using the target SQL script, the first device may record the data export failure information. When the target data file to be migrated to the target subsystem is successfully exported from the source database using the target SQL script, the first device may also record the data export success information.

[0123] In this implementation, exemplarily, if the first device obtains an identifier "used to indicate that the target data file was not successfully exported from the source database" after executing all SQL scripts in the target SQL script, indicating that the target data file was not successfully exported from the source database, the first device can record information about the data export failure. If the first device does not obtain an identifier "used to indicate that the target data file was not successfully exported from the source database" after executing all SQL scripts in the target SQL script, indicating that the target data file was successfully exported from the source database, the first device can record information about the data export success. Exemplarily, the above-mentioned identifier "used to indicate that the target data file was not successfully exported from the source database" can be represented by "error", for example.

[0124] The following describes in detail how the second device performs a write operation on the target data table. Figure 3 The present invention provides a flowchart of a method for performing a write operation on a target data table. Figure 3 As shown, as a possible implementation, the aforementioned step S106 may include the following steps:

[0125] S201, obtaining data writing parameters.

[0126] The data writing parameters include: an identifier of a target data file, an access path to the target data file, an identifier of whether the data in the target data file belongs to incremental data of a target data table, and a writing method.

[0127] Specifically, according to the identifier of the target data file, the second device may determine the target data table of the data to be written into the target data file in the target sub-database. For details, reference may be made to the method described in the above embodiment, which will not be described in detail here.

[0128] With respect to the access path of the target data file, the second device may store the target data file after receiving the target data file sent by the first device. The storage path of the target data file is the access path of the target data file. Through the access path of the target data file, the second device may obtain the target data file to write the data in the target data file into the target data table.

[0129] The identifier of whether the data in the target data file belongs to the incremental data of the target data table is used to indicate whether the data in the target data file belongs to the incremental data of the target data table. Exemplarily, the identifier may be a number or a letter.

[0130] The writing mode of the data in the target data file may be, for example, a writing mode according to an index, or a direct writing mode, etc. Exemplarily, taking the writing mode as a direct writing mode as an example, the direct writing mode may be represented by, for example, "direct=true". That is, if the value of the direct parameter in the data writing parameter is true (correct), the second device may determine that the writing mode of the data in the target data file is a direct writing mode.

[0131] S202: Perform a write operation on the target data table according to the data write parameters.

[0132] During the process of performing a write operation to the target data table according to the above data write parameters, the second device may also determine whether the write operation is interrupted. For example, if during the process of performing a write operation to the target data table, the second device obtains an identifier indicating that the write operation is interrupted, it can be determined that the write operation is interrupted.

[0133] If the write operation is interrupted, and the data write parameter includes an indication that the data in the target data file belongs to the incremental data of the target data table, the second device may delete the data that has been written into the target data table by the write operation from the target data table, and re-execute the write operation to the target data table to avoid re-writing the data that has been written into the target data table. If the write operation is interrupted, and the data write parameter includes an indication that the data in the target data file does not belong to the incremental data of the target data table, the second device may delete all the data in the target data table, and re-execute the write operation to the target data table.

[0134] If the write operation is not interrupted, it means that the second device successfully writes the data in all the target data files into the target data table. Optionally, the second device may also generate an identifier indicating that the data is written successfully.

[0135] Further, as a possible implementation, after the second device performs a write operation to the target data table, the second device may also delete the target data file in the second device to save storage space of the second device. Specifically, the second device may delete the target data file after successfully writing the data in the target data file to the target data table.

[0136] Optionally, after performing the write operation to the target data table, the second device may determine whether an identifier indicating that the target data file is successfully written into the target data table is obtained, so as to determine whether the data in the target data file is successfully written into the target data table. If the second device obtains the identifier indicating that the target data file is successfully written into the target data table, it can be determined that the data in the target data file has been successfully written into the target data table, and the second device may delete the target data file.

[0137] Based on the above embodiments, for the process of the first device exporting the target data file from the source database, Figure 4 A flowchart of another data processing method provided by this application. Figure 4 As shown, the method comprises the following steps:

[0138] S301: A first device obtains data migration parameters.

[0139] The data migration parameters include: the current date when the first device obtains the data migration parameters, the identifier of the target subsystem, the identifier corresponding to the data migration action, the identifier of the data to be migrated to the target subsystem, and the file name of the file where the target SQL script corresponding to the target subsystem is located.

[0140] Among them, the identifier of the above-mentioned target subsystem can also be called an application identifier. Exemplarily, the application identifier can be an application identity identification number (Identity document, ID). The identifier corresponding to the above-mentioned data migration action can also be called the identifier of the job chain. Exemplarily, the identifier of the job chain can be a job chain ID. The identifier of the above-mentioned data to be migrated to the target subsystem can also be called the identifier of the job. Exemplarily, the identifier of the job can be a job ID.

[0141] Optionally, the above data migration parameters may also include a target path for storing the target data file and an identifier of a source database. The first device may store the exported target data file in the first device according to the target path for storing the target data file. According to the identifier of the source database, the user may record the operation history of the source database. If the above data migration parameters do not include a target path for storing the target data file and an identifier of the source database, the first device may store the target data file according to a preset storage path and access a default source database.

[0142] The first device may receive data migration parameters input by a user. When the user inputs the data migration parameters, the first device obtains four parameters, namely, the current date of the data migration parameters, the identifier of the target subsystem, the identifier corresponding to the data migration action, and the identifier of the data to be migrated to the target subsystem. The four parameters may be separated by spaces, and the field lengths of the four parameters may be fixed.

[0143] The file name of the target SQL script file corresponding to the target subsystem, the target path for storing the target data file, and the source database identifier may be separated by special characters (eg, ","). The first device may parse the three parameters by using awk.

[0144] Optionally, the first device may also determine the number of received parameters to ensure that all parameters required to access the source database are received. If the number of received parameters is not equal to the preset number, the first device may no longer execute subsequent steps. If the received parameters are equal to the preset number, the first device may continue to execute subsequent steps.

[0145] S302: The first device obtains a target SQL script according to an identifier of the target SQL script.

[0146] S303: The first device obtains a configuration file of a source database.

[0147] The configuration file includes: the access path of the source database, and the user name and encrypted password required to access the source database. The encrypted password may also be referred to as a password stored in ciphertext.

[0148] S304: The first device decrypts the encrypted password to obtain a password required to access the source database.

[0149] Exemplarily, the first device may use a symmetric database password encryption and decryption program written using awk and sed to decrypt the encrypted password.

[0150] S305: The first device logs into the source database using the user name and password according to the access path of the source database.

[0151] S306: The first device uses the target SQL script to export the target data file to be migrated to the target subsystem from the source database.

[0152] Exemplarily, the first device can run the target SQL script in sqlplus under the Shell master control script, connect to the source database, and then execute the target SQL script through spool to export the target data file to be migrated to the target subsystem from the source database.

[0153] S307: The first device determines whether the target data file to be migrated to the target subsystem is successfully exported from the source database.

[0154] Specifically, the first device may determine whether the target data file to be migrated to the target subsystem is successfully exported from the source database by judging whether an error flag is generated during the process of exporting the target data file to be migrated to the target subsystem from the source database.

[0155] If not, execute step S308. If yes, execute step S309.

[0156] S308. The first device may record information on data export failure.

[0157] The first device may record the information of the data export failure into a preset log file according to a preset log file path (including the log file name).

[0158] S309: The first device records the time when step S306 is started and the time when the target data file is acquired.

[0159] S310. The first device transcodes the data in the target data file into a standard data format GBK.

[0160] S311. The first device compresses the target data file to obtain a compressed target data file.

[0161] S312. The first device moves the compressed target data file to the GTP source path, and sends the compressed target data file to the second device of the target subsystem through GTP.

[0162] Optionally, the first device may also perform the above steps S301-S312 at a preset time and according to a preset frequency. Optionally, the above preset time and preset frequency may be pre-stored in the first device by a user, for example.

[0163] As an example, the following is the Figure 4 The program code corresponding to the method embodiment shown is:

[0164] SQLplus –s / nolog< <eof>$sqllogfile

[0165] conn${DB_USER} / ${DB PWD}@${DB_URL}

[0166] set colsep|!;

[0167] set echo off;

[0168] set verify off;

[0169] set linesize 10000;

[0170] set feedback off;

[0171] set termout off;

[0172] set trimout on;

[0173] set heading off;

[0174] set newpage none;

[0175] set pagesize 0;

[0176] set trimspool on;

[0177] spool$TEMP_FILE_PATH / $FINAL_FILE_NAME;

[0178] @$QUERY_FILE_NAME $OPER_DATE $PARMETER_ONE $PARMETER_TWO

[0179] spool off;

[0180] quit

[0181] EOF

[0182] Based on the above embodiments, for the process in which the second device writes the data in the target data file into the sub-database of the target sub-system, Figure 5 A flowchart of another data processing method provided by this application is shown below. Figure 5 As shown, the method comprises the following steps:

[0183] S401. The second device receives the compressed target data file sent by the first device.

[0184] After receiving the compressed target data file, the second device may decompress the compressed target data file to obtain the target data file.

[0185] S402: The second device obtains a configuration file of the target sub-database.

[0186] The configuration file of the target sub-database includes the access path of the target sub-database, as well as the user name and encrypted password required to access the target sub-database. The encrypted password may also be referred to as a password stored in ciphertext.

[0187] S403: The second device decrypts the encrypted password to obtain a password required to access the target sub-database.

[0188] Exemplarily, the second device may use a symmetric database password encryption and decryption program written using awk and sed to decrypt the encrypted password.

[0189] S404: The second device logs into the target sub-database using the user name and password according to the access path of the target sub-database.

[0190] S405: The second device determines the target data table of the target sub-database for the data to be written into the target data file according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target sub-database of the target sub-system.

[0191] S406: The second device obtains data writing parameters.

[0192] Among them, the data writing parameters include: the current date when the second device obtains the data writing parameters, the identifier of the target subsystem, the identifier corresponding to the data migration action, the identifier of the data to be migrated to the target subsystem, the identifier of the target data file, the access path of the target data file, the identifier of whether the data in the target data file belongs to the incremental data of the target data table, and the writing method.

[0193] The second device may receive data writing parameters input by a user. When the user inputs the above data writing parameters, the above second device obtains the current date of the data writing parameters, the identifier of the target subsystem, the identifier corresponding to the data migration action, and the identifier of the data to be migrated to the target subsystem. The four parameters may be separated by spaces, and the field lengths of the above four parameters may be fixed.

[0194] The identification of the target data file, the access path of the target data file, the identification of whether the data in the target data file belongs to the incremental data of the target data table, and the writing methods can be separated by special characters (such as ","). The first device can parse the above four parameters through awk.

[0195] S407: The second device performs a write operation on the target data table according to the data write parameters.

[0196] For details, please refer to the method described in the above embodiment, which will not be described again here.

[0197] S408: The second device determines whether the data writing is successful.

[0198] If successful, execute step S409. If unsuccessful, execute step S411.

[0199] S409: After successfully writing the data in the target data file into the target data table, the second device deletes the target data file.

[0200] Optionally, after the data in the target data file is successfully written into the target data table, the second device may retain the compressed target data file and delete the decompressed target data file.

[0201] S410. The second device records the time when step S407 is started and the time when the data in the target data file is successfully written into the target data table.

[0202] Optionally, the second device may record the time when the step S407 is started and the time when the data in the target data file is successfully written into the target data table in a log file, so that the user can estimate the writing progress of the data in the target data file.

[0203] S411. The second device records information on data writing failure.

[0204] Optionally, the second device may record information about the failure to write the above data into a log file.

[0205] As an example, the following is the Figure 5 The program code corresponding to the method embodiment shown is:

[0206] OPTIONS(ROWS=2000,BINDSIZE=100000,READSIZE=100000)

[0207] LOAD DATA

[0208] CHARACTERSET ZHS16GBK

[0209] infile '%data_file_name'

[0210] TRUNCATE INTO TABLE T_TABLE_NAME

[0211] FIELDS TERMINATED BY '|!'

[0212] TRAILING NULLCOLS

[0213] (ID,

[0214] P_ID,

[0215] CODE,

[0216] NAME,

[0217] SHORT_NAME,

[0218] PROVINCE,

[0219] LEVEL,

[0220] IS_DEL,

[0221] UPDATE_DATE)

[0222] Figure 6 Schematic diagram of a data processing device 500 provided in the present application. The device 500 is applied to a first device in a source system in an e-commerce platform, such as Figure 6 As shown, the device 500 includes: a first acquisition module 501, a second acquisition module 502, an export module 503, and a sending module 504.

[0223] The first acquisition module 501 is used to acquire data migration parameters, wherein the data migration parameters include: an identifier of a target structured query language SQL script corresponding to a target subsystem, the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database; the target subsystem is any one of the at least one subsystem.

[0224] The second acquisition module 502 is used to acquire the target SQL script according to the identifier of the target SQL script.

[0225] The export module 503 is used to export the target data file to be migrated to the target subsystem from the source database using the target SQL script, wherein the target data file is in a preset format and the identifier of the target data file is the same as the identifier of the target SQL script.

[0226] The sending module 504 is used to send the target data file to a second device. The second device is a device of the target subsystem. The second device is used to determine the target data table of the data to be written into the target data file in the target subdatabase according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target subdatabase of the target subsystem.

[0227] Optionally, the export module 503 is specifically used to obtain the configuration file of the source database; log in to the source database using the user name and the password according to the access path of the source database; and use the target SQL script to export the target data file to be migrated to the target subsystem from the source database. The configuration file of the source database includes: the access path of the source database, the user name and the password required to access the source database.

[0228] Optionally, the apparatus 500 may further include a compression module 505, which is used to compress the target data file to be migrated to the target subsystem after exporting the target data file from the source database using the target SQL script to obtain a compressed target data file. In this implementation, the sending module 504 is specifically used to send the compressed target data file to the second device.

[0229] Optionally, the device 500 may further include a recording module 506 for recording information of data export failure when the target data file to be migrated to the target subsystem is not successfully exported from the source database using the target SQL script.

[0230] The data processing device 500 provided in this embodiment is used to execute the data processing method embodiment executed by the aforementioned first device. Its implementation principle and technical effect are similar and will not be described in detail.

[0231] Figure 7 Schematic diagram of another data processing device 600 provided in the present application. The device 600 is applied to a second device of a target subsystem in at least one subsystem of an e-commerce platform. Figure 7 As shown, the device 600 includes: a receiving module 601, a determining module 602, and an executing module 603.

[0232] The receiving module 601 is used to receive a target data file from a first device of the source system, wherein the target data file is a data file to be migrated to the target subsystem, which is exported from the source database by the first device using a target SQL script, and the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database, and the target data file is in a preset format, and the identifier of the target data file is the same as the identifier of the target SQL script.

[0233] The determination module 602 is used to determine the target data table of the target sub-database for the data to be written into the target data file according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target sub-database of the target sub-system.

[0234] The execution module 603 is used to perform a write operation on the target data table, wherein the write operation is used to write the data in the target data file.

[0235] Optionally, the execution module 603 is specifically used to obtain a configuration file of the target sub-database; log in to the target sub-database using the user name and the password according to the access path of the target sub-database; and perform a write operation on the target data table. The configuration file of the target sub-database includes: the access path of the target sub-database, the user name and the password required to access the target sub-database.

[0236] Optionally, the execution module 603 is specifically used to obtain data writing parameters, and perform a write operation to the target data table according to the data writing parameters. The data writing parameters include: the identifier of the target data file, the access path of the target data file, the identifier of whether the data in the target data file belongs to the incremental data of the target data table, and the writing method.

[0237] Optionally, the execution module 603 is specifically used to delete the data that has been written into the target data table by executing the write operation from the target data table when the write operation is interrupted and the data in the target data file belongs to incremental data of the target data table, and re-execute the write operation to the target data table.

[0238] Optionally, the receiving module 601 is specifically configured to receive a compressed target data file from the first device; and decompress the compressed target data file to obtain the target data file.

[0239] Optionally, the device 600 may further include a deleting module 604, which is used to delete the target data file after the write operation is performed on the target data table and the data in the target data file is successfully written into the target data table.

[0240] The data processing device 600 provided in this embodiment is used to execute the data processing method embodiment executed by the aforementioned second device. Its implementation principle and technical effect are similar and will not be described in detail.

[0241] Figure 8 This is a schematic diagram of the structure of an electronic device provided in this application. The electronic device may be the aforementioned first device or the second device. Figure 8 As shown, the electronic device 700 may include: at least one processor 701 and a memory 702 .

[0242] The memory 702 is used to store programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions.

[0243] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0244] The processor 701 is used to execute the computer-executable instructions stored in the memory 702 to implement the data processing method described in the aforementioned method embodiment. The processor 701 may be a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0245] Optionally, the electronic device 700 may further include a communication interface 703. In a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are implemented independently, the communication interface 703, the memory 702 and the processor 701 may be interconnected through a bus and communicate with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0246] Optionally, in a specific implementation, if the communication interface 703, the memory 702 and the processor 701 are integrated on a chip, the communication interface 703, the memory 702 and the processor 701 can communicate through an internal interface.

[0247] The present application also provides a computer-readable storage medium, which may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, and other media that can store program codes. Specifically, the computer-readable storage medium stores program instructions, and the program instructions are used for the methods in the above embodiments.

[0248] The present application also provides a program product, which includes an execution instruction, which is stored in a readable storage medium. At least one processor of the electronic device can read the execution instruction from the readable storage medium, and at least one processor executes the execution instruction so that the electronic device implements the data processing method provided by the various embodiments described above.

[0249] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application.< / eof>

Claims

1. A data processing method, characterized in that: The e-commerce platform includes a source system and at least one subsystem, the source system has a source database, the subsystem has a sub-database, the method is applied to a first device in the source system, and the method includes: Acquire data migration parameters, the data migration parameters including: an identifier of a target SQL script corresponding to a target subsystem, the target SQL script being used to indicate a method of extracting data to be migrated to the target subsystem from the source database, the target SQL script including extracting data of at least one column or at least one row in a data table in the source database; the target subsystem being any one of the at least one subsystem; According to the identifier of the target SQL script, obtaining the target SQL script; Using the target SQL script, exporting a target data file to be migrated to the target subsystem from the source database, wherein the target data file is in a preset format and an identifier of the target data file is the same as an identifier of the target SQL script; Sending the target data file to a second device; the second device is a device of the target subsystem; the second device is used to determine the target data table of the target subdatabase to be written into the target data file according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target subdatabase of the target subsystem, and perform a write operation on the target data table to write the data in the target data file into the target data table; The using the target SQL script to export the target data file to be migrated to the target subsystem from the source database includes: Obtaining a configuration file of the source database, wherein the configuration file of the source database includes: an access path of the source database, a user name and a password required to access the source database; According to the access path of the source database, log in to the source database using the user name and the password; Using the target SQL script, exporting the target data file to be migrated to the target subsystem from the source database; After the target data file to be migrated to the target subsystem is exported from the source database using the target SQL script, the method further includes: Compressing the target data file to obtain a compressed target data file; The sending the target data file to the second device includes: Sending the compressed target data file to the second device; If the target data file to be migrated to the target subsystem is not successfully exported from the source database using the target SQL script, information about data export failure is recorded.

2. A data processing method, characterized in that: The e-commerce platform includes a source system and at least one subsystem, the source system has a source database, the subsystem has a sub-database, the method is applied to a second device of a target subsystem in the at least one subsystem, and the method includes: receiving a target data file from a first device of the source system; the target data file is a data file to be migrated to the target subsystem, which is exported from the source database by the first device using a target SQL script, the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database, the target SQL script includes extracting data of at least one column or at least one row in a data table in the source database, the target data file is in a preset format, and an identifier of the target data file is the same as an identifier of the target SQL script; Determine the target data table of the data to be written into the target data file in the target subdatabase according to the identifier of the target data file and the mapping relationship between the identifier of the preset SQL script and the data table of the target subdatabase of the target subsystem; Performing a write operation on the target data table, wherein the write operation is used to write the data in the target data file; The performing a write operation to the target data table includes: Acquire a configuration file of the target sub-database, wherein the configuration file of the target sub-database includes: an access path of the target sub-database, a user name and a password required to access the target sub-database; According to the access path of the target sub-database, log in to the target sub-database using the user name and the password; Performing a write operation on the target data table; The performing a write operation to the target data table includes: Acquire data writing parameters, the data writing parameters including: an identifier of the target data file, an access path of the target data file, an identifier of whether the data in the target data file belongs to incremental data of the target data table, and a writing method; According to the data writing parameters, performing a write operation on the target data table to write the data in the target data file into the target data table; The performing a write operation to the target data table includes: If the write operation is interrupted and the data in the target data file belongs to the incremental data of the target data table, the data written into the target data table by executing the write operation is deleted from the target data table, and the write operation is re-executed on the target data table; The receiving the target data file from the first device of the source system includes: receiving a compressed target data file from the first device; Decompressing the compressed target data file to obtain the target data file; After performing the write operation to the target data table, the method further includes: After the data in the target data file is successfully written into the target data table, the target data file is deleted.

3. A data processing device, characterized in that: The e-commerce platform includes a source system and at least one subsystem, wherein the source system has a source database, and the subsystem has a sub-database. The apparatus is applied to a first device in the source system, and the apparatus includes: A first acquisition module is used to acquire data migration parameters, wherein the data migration parameters include: an identifier of a target SQL script corresponding to a target subsystem, wherein the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database, wherein the target SQL script includes extracting data of at least one column or at least one row in a data table in the source database; the target subsystem is any one of the at least one subsystem; A second acquisition module, configured to acquire the target SQL script according to the identifier of the target SQL script; An export module, used to export a target data file to be migrated to the target subsystem from the source database using the target SQL script, wherein the target data file is in a preset format and an identifier of the target data file is the same as an identifier of the target SQL script; a sending module, configured to send the target data file to a second device; the second device being a device of the target subsystem; the second device being configured to determine a target data table in the target subdatabase for the data to be written into the target data file according to an identifier of the target data file and a mapping relationship between an identifier of a preset SQL script and a data table of a target subdatabase of the target subsystem; and to perform a write operation on the target data table to write the data in the target data file into the target data table; The export module is specifically used to obtain a configuration file of the source database, wherein the configuration file of the source database includes: an access path of the source database, a user name and a password required to access the source database; log in to the source database using the user name and the password according to the access path of the source database; and use the target SQL script to export a target data file to be migrated to the target subsystem from the source database; A compression module, used for compressing the target data file to obtain a compressed target data file; The sending module is specifically configured to send the compressed target data file to the second device; The recording module is used to record the information of data export failure if the target data file to be migrated to the target subsystem is not successfully exported from the source database using the target SQL script.

4. A data processing device, characterized in that: The e-commerce platform includes a source system and at least one subsystem, wherein the source system has a source database, and the subsystem has a sub-database. The apparatus is applied to a second device of a target subsystem in the at least one subsystem, and the apparatus includes: a receiving module, configured to receive a target data file from a first device of the source system; the target data file is a data file to be migrated to the target subsystem, which is exported from the source database by the first device using a target SQL script; the target SQL script is used to indicate a method of extracting data to be migrated to the target subsystem from the source database; the target SQL script includes extracting data of at least one column or at least one row in a data table of the source database; the target data file is in a preset format, and an identifier of the target data file is the same as an identifier of the target SQL script; A determination module, configured to determine a target data table of the data to be written into the target data file in the target subdatabase according to an identifier of the target data file and a mapping relationship between an identifier of a preset SQL script and a data table of a target subdatabase of the target subsystem; An execution module, used for executing a write operation to the target data table, wherein the write operation is used for writing data in the target data file; The execution module is specifically used to obtain a configuration file of the target sub-database, wherein the configuration file of the target sub-database includes: an access path of the target sub-database, a user name and a password required to access the target sub-database; log in to the target sub-database using the user name and the password according to the access path of the target sub-database; and perform a write operation on the target data table; The execution module is specifically used to obtain data writing parameters, and execute a write operation to the target data table according to the data writing parameters, so as to write the data in the target data file into the target data table; wherein the data writing parameters include: an identifier of the target data file, an access path of the target data file, an identifier of whether the data in the target data file belongs to incremental data of the target data table, and a writing method; The execution module is specifically configured to delete the data written into the target data table by executing the write operation from the target data table if the write operation is interrupted and the data in the target data file belongs to the incremental data of the target data table, and re-execute the write operation to the target data table; The receiving module is specifically configured to receive the compressed target data file from the first device; A decompression module, used for decompressing the compressed target data file to obtain the target data file; The deleting module is used to delete the target data file after the data in the target data file is successfully written into the target data table.

5. An electronic device, characterized in that: include: at least one processor and memory; The memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 2.

6. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a processor, the method described in any one of claims 1 to 2 is implemented.

7. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 2 is implemented.

Images