Data processing method, apparatus and server

Abstract

The application provides a data processing method and device and a server, relates to data storage technology, and the method comprises the following steps: in response to a received data migration request, determining a target table for storing target data and an intermediate table corresponding to the target table. According to preset data mapping rule information, the preliminary data in the data table in each other server corresponding to the intermediate table is acquired respectively, and the intermediate table corresponding to each other server containing the preliminary data is obtained. According to preset integration rule information, the plurality of intermediate tables containing the preliminary data are integrated, and the integrated intermediate table containing the target data is obtained. According to the integrated intermediate table containing the target data, the target table containing the target data is obtained, so as to complete data migration. Through the preset data mapping rule information, the data corresponding to the fields in the intermediate table can be acquired from the plurality of other databases respectively, and the problem that the diversity of the migratable and integrated data source is low is solved.

Description

Technical Field

[0001] This application relates to data storage technology, and more particularly to a data processing method, apparatus, and server. Background Technology

[0002] Currently, with the increasing amount of data on various servers, it is necessary to migrate and integrate the data from multiple servers.

[0003] In existing technologies, when migrating and integrating data from multiple servers, since the data table structures of data tables containing data of the same business type are identical, data of the same business type from various servers can be migrated and integrated together.

[0004] However, existing technologies can only migrate and integrate data of the same business type from various servers, which means they can only support the migration and integration of data sources of the same business type, resulting in a low diversity of data sources that can be migrated and integrated. Summary of the Invention

[0005] This application provides a data processing method, apparatus, and server to address the technical problem of low diversity of data sources that can be migrated and integrated.

[0006] Firstly, this application provides a data processing method, including:

[0007] In response to a received data migration request, wherein the data migration request is used to instruct the acquisition of data from multiple other servers, to determine a target table for storing the target data, and an intermediate table corresponding to the target table; wherein the intermediate table has a preset correspondence with a data table in each of the other servers;

[0008] According to the preset data mapping rule information, the preliminary data in the data table of each other server corresponding to the intermediate table is obtained respectively, and the intermediate table containing the preliminary data is obtained for each other server; wherein, the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data table of other servers.

[0009] According to the preset integration rule information, multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing target data; and according to the integrated intermediate table containing target data, a target table containing target data is obtained to complete the data migration; wherein, the integration rule information is used to indicate the filtering rule information for the preliminary data.

[0010] Furthermore, according to preset integration rule information, multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing target data, including:

[0011] If it is determined that the preset primary key of each of the multiple intermediate tables corresponds to different preliminary data, then the priority of the primary key of each intermediate table is determined according to the preset primary key of each intermediate table containing preliminary data; wherein, the primary key represents the identifier of each record in the intermediate table.

[0012] Based on the preset integration rules, retain the initial data corresponding to the primary key with the highest priority;

[0013] Based on the initial data corresponding to the highest priority primary key, an integrated intermediate table containing the target data is obtained.

[0014] Further, the step of obtaining preliminary data from the data tables in each other server corresponding to the intermediate table according to preset data mapping rule information, to obtain an intermediate table containing the preliminary data corresponding to each other server, includes:

[0015] According to the preset data mapping rules, data acquisition instructions are sent to each of the other servers corresponding to the intermediate table, so that each of the other servers can acquire preliminary data from at least one data table corresponding to each other server according to the data acquisition instructions.

[0016] Receive the preliminary data sent by each other server corresponding to the intermediate table;

[0017] Based on the preliminary data sent by each other server corresponding to the intermediate table, an intermediate table containing the preliminary data is obtained for each other server.

[0018] Further, the step of obtaining preliminary data from the data tables in each other server corresponding to the intermediate table according to preset data mapping rule information, to obtain an intermediate table containing the preliminary data corresponding to each other server, includes:

[0019] Obtain at least one data table sent by each of the other servers;

[0020] In at least one data table sent by each of the other servers, preliminary data in the data table of each of the other servers corresponding to the intermediate table is obtained based on preset data mapping rule information.

[0021] Based on the preliminary data sent by each other server corresponding to the intermediate table, an intermediate table containing the preliminary data is obtained for each other server.

[0022] Further, the step of obtaining the target table containing the target data from the integrated intermediate table containing the target data includes:

[0023] Based on the pre-defined mapping relationship between the fields of the intermediate table and the fields of the target table, the target data of the intermediate table is stored in the target table.

[0024] Furthermore, the method also includes:

[0025] The system pre-acquires at least one data table sent by each of a plurality of other servers, as well as a plurality of target tables for storing the target data; wherein both the data table and the target table include a plurality of fields.

[0026] Determine the data table structure of each of the other data tables sent by the servers, and the data table structure of the target table;

[0027] Based on the data table structure of the target table, determine the primary key of the target table;

[0028] Based on the data table structure of each target table, an intermediate table corresponding to each target table is generated; wherein, there is a mapping relationship between the fields of the target table and the fields of the intermediate table;

[0029] For each of the other servers sending at least one data table, data mapping rule information between the intermediate table and the data table is generated based on the fields of the at least one data table sent by each of the other servers and the fields of the intermediate table corresponding to each of the other servers.

[0030] Further, determining the primary key of the target table based on its data table structure includes:

[0031] Based on the preset primary key information, the fields in the target table are analyzed for business meaning, and the fields that match the primary key information are confirmed as the primary keys of the target table.

[0032] Secondly, this application provides a data processing apparatus, comprising:

[0033] The first determining unit is configured to respond to a received data migration request, wherein the data migration request is configured to instruct the acquisition of data from multiple other servers, and to determine a target table for storing target data, and an intermediate table corresponding to the target table; the intermediate table has a preset correspondence with a data table in each of the other servers;

[0034] The second determining unit is used to obtain preliminary data from the data tables in each other server corresponding to the intermediate table according to preset data mapping rule information, so as to obtain an intermediate table containing the preliminary data corresponding to each other server; wherein, the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data tables in other servers.

[0035] The integration unit is used to integrate multiple intermediate tables containing preliminary data according to preset integration rules to obtain an integrated intermediate table containing the target data.

[0036] The third determining unit is used to obtain a target table containing the target data based on the integrated intermediate table containing the target data, so as to complete the data migration; wherein, the integration rule information is used to indicate the filtering rule information for the preliminary data.

[0037] Furthermore, the integration unit includes:

[0038] The first determining module is used to determine the priority of the preset primary key of each intermediate table if the preset primary key of each intermediate table corresponds to different preliminary data; wherein the primary key represents the identifier of each record in the intermediate table.

[0039] The retention module is used to retain the initial data corresponding to the primary key with the highest priority based on preset integration rules.

[0040] The integration module is used to obtain an integrated intermediate table containing the target data based on the preliminary data corresponding to the primary key with the highest priority.

[0041] Further, the second determining unit includes:

[0042] The sending module is used to send data acquisition instructions to each of the other servers corresponding to the intermediate table according to the preset data mapping rule information, so that each of the other servers can acquire preliminary data from at least one data table corresponding to each other server according to the data acquisition instructions.

[0043] A receiving module is used to receive the preliminary data sent by each other server corresponding to the intermediate table;

[0044] The second determining module is used to obtain an intermediate table containing the preliminary data corresponding to each other server based on the preliminary data sent by each other server corresponding to the intermediate table.

[0045] Further, the second determining unit includes:

[0046] The first acquisition module is used to acquire at least one data table sent by each of the other servers;

[0047] The second acquisition module is used to acquire preliminary data in each of the other servers' data tables corresponding to the intermediate table, based on preset data mapping rule information, from at least one data table sent by each of the other servers.

[0048] The third determining module is used to obtain an intermediate table containing the preliminary data corresponding to each other server based on the preliminary data sent by each other server corresponding to the intermediate table.

[0049] Furthermore, the third determining unit is specifically used for:

[0050] Based on the pre-defined mapping relationship between the fields of the intermediate table and the fields of the target table, the target data of the intermediate table is stored in the target table.

[0051] Furthermore, the device also includes:

[0052] The acquisition unit is configured to pre-acquire at least one data table sent by each of a plurality of other servers, and a plurality of target tables for storing the target data; wherein, both the data table and the target table include a plurality of fields;

[0053] The fourth determining unit is used to determine the data table structure of each of the other data tables sent by the servers, and the data table structure of the target table;

[0054] The fifth determining unit is used to determine the primary key of the target table based on the data table structure of the target table;

[0055] The first generation unit is used to generate an intermediate table corresponding to each target table based on the data table structure of each target table; wherein there is a mapping relationship between the fields of the target table and the fields of the intermediate table;

[0056] The second generation unit is configured to generate data mapping rule information between the intermediate table and the data table based on the fields of the at least one data table sent by each of the other servers and the fields of the intermediate table corresponding to each of the other servers.

[0057] Furthermore, the fifth determining unit is specifically used for:

[0058] Based on the preset primary key information, the fields in the target table are analyzed for business meaning, and the fields that match the primary key information are confirmed as the primary keys of the target table.

[0059] Thirdly, this application provides a server, including a memory and a processor, wherein the memory stores a computer program that can run on the processor, and the processor executes the computer program to implement the method described in the first aspect.

[0060] Fourthly, this application provides a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, are used to implement the method described in the first aspect.

[0061] Fifthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the method described in the first aspect.

[0062] This application provides a data processing method, apparatus, and server that, in response to a received data migration request, instructs the acquisition of data from multiple other servers, the determination of a target table for storing target data, and an intermediate table corresponding to the target table; each intermediate table has a preset correspondence with a data table in each of the other servers. According to preset data mapping rules, preliminary data from each data table in each of the other servers corresponding to the intermediate table is acquired, resulting in an intermediate table containing the preliminary data corresponding to each other server; wherein the preset data mapping rules characterize the mapping relationship between fields of the intermediate table and fields of data tables in other servers. According to preset integration rules, the multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing the target data; and based on the integrated intermediate table containing the target data, a target table containing the target data is obtained to complete the data migration; wherein the integration rules indicate filtering rules for the preliminary data. In this solution, upon receiving a data migration request, the target table and intermediate tables are first determined. Then, based on preset data mapping rules, the fields in the intermediate tables and the corresponding fields in other server tables are identified. Based on these fields, preliminary data is retrieved from each of the other server tables corresponding to the intermediate table's fields, resulting in an intermediate table containing the preliminary data for each other server. Next, according to preset integration rules, these multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing the target data. Since there is a correspondence between the intermediate and target tables, and based on the integrated intermediate table containing the target data, the target data from the intermediate tables can be stored in the target table, resulting in a target table containing the target data, thus completing the data migration. Therefore, by using preset data mapping rules, data corresponding to the fields in the intermediate tables can be retrieved from multiple other databases, completing the migration and integration of data from multiple other databases, thus solving the technical problem of low diversity of data sources that can be migrated and integrated. Attached Figure Description

[0063] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0064] Figure 1 A flowchart illustrating a data processing method provided in an embodiment of this application;

[0065] Figure 2 A flowchart illustrating another data processing method provided in an embodiment of this application;

[0066] Figure 3 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application;

[0067] Figure 4 This is a schematic diagram of another data processing apparatus provided in an embodiment of this application;

[0068] Figure 5 This is a schematic diagram of the structure of a server provided in an embodiment of this application.

[0069] The accompanying drawings have illustrated specific embodiments of this disclosure, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the concept in any way, but rather to illustrate the concepts of this disclosure to those skilled in the art through reference to particular embodiments. Detailed Implementation

[0070] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0071] In one example, when migrating and integrating data across multiple servers, since the data tables containing data of the same business type have the same structure, data of the same business type from each server can be migrated and integrated together. However, in existing technologies, because only data of the same business type from each server can be migrated and integrated together, it can only support the migration and integration of data sources of the same business type, resulting in a low diversity of data sources that can be migrated and integrated.

[0072] This application provides a data processing method, apparatus, and server, which aim to solve the above-mentioned technical problems in the prior art.

[0073] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will now be described with reference to the accompanying drawings.

[0074] Figure 1 This is a flowchart illustrating a data processing method provided in an embodiment of this application, as shown below. Figure 1 As shown, the method includes:

[0075] 101. In response to a received data migration request, wherein the data migration request is used to instruct the acquisition of data from multiple other servers, to determine a target table for storing the target data, and an intermediate table corresponding to the target table; the intermediate table has a preset correspondence with the data table in each of the other servers.

[0076] For example, the execution entity in this embodiment can be a server. The business structure of the data in the data tables of other servers is different from the business structure of the data in the target table of the server, that is, the fields of the data tables of other servers are different from the fields of the target table of the server. Therefore, in order to integrate data sources of multiple business types, after receiving the data migration request, the server determines the target table for storing the target data. Since there is a preset mapping relationship between the target table and the intermediate table, the intermediate table corresponding to the target table can be determined. The intermediate table has a preset correspondence with each data table in other servers. Specifically, if there are multiple data tables in other servers, each data table can correspond to one intermediate table, or multiple intermediate tables can correspond to the same intermediate table.

[0077] For example, data tables in other server A include data tables A1 and A2, data tables in other server B include data tables B1 and B2, and target tables include O1 and O2. Target table O1 corresponds to intermediate table M1, and target table O2 corresponds to intermediate table M2. In the first type of correspondence between intermediate tables and data tables in other servers, data table A1 corresponds to intermediate table M1, data table A2 corresponds to intermediate table M2, data table B1 corresponds to intermediate table M1, and data table B2 corresponds to intermediate table M2. Intermediate table M1 corresponding to data table A1 and intermediate table M1 corresponding to data table B1 both correspond to target table O1, and intermediate table M2 corresponding to data table A2 and intermediate table M2 corresponding to data table B2 both correspond to target table O2.

[0078] Alternatively, in the second type of correspondence between intermediate tables and data tables in other servers, data table A1 corresponds to intermediate table M1, data table A2 corresponds to intermediate table M1, data table B1 corresponds to intermediate table M1, data table B2 corresponds to intermediate table M2, and intermediate table M1 corresponding to data table A1, intermediate table M1 corresponding to data table A2, and intermediate table M1 corresponding to data table B1 all correspond to target table O1, while intermediate table M2 corresponding to data table B2 corresponds to target table O2.

[0079] 102. Based on the preset data mapping rule information, obtain the preliminary data in the data table of each other server corresponding to the intermediate table, and obtain the intermediate table containing the preliminary data corresponding to each other server; wherein, the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data table of other servers.

[0080] For example, since the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data tables in other servers, based on the preset data mapping rule information, the server can determine the corresponding fields in the data tables of each other server according to the fields of the intermediate table, and obtain the preliminary data corresponding to the corresponding fields in the data tables of each other server, thereby obtaining an intermediate table containing the preliminary data corresponding to each other server.

[0081] For example, let's say the target table is O1, and the target table O1 corresponds to an intermediate table M1. In the first type of correspondence between the intermediate table and data tables on other servers, the server can determine the corresponding fields in data table A1 on other server A based on the fields of intermediate table M1. Then, based on the corresponding fields in data table A1 on other server A, it retrieves the preliminary data corresponding to those fields in data table A1, resulting in an intermediate table M1 containing the preliminary data corresponding to other server A. Similarly, the server determines the corresponding fields in data table B1 on other server B based on the fields of intermediate table M1, and retrieves the preliminary data corresponding to those fields in data table B1 on other server B, resulting in an intermediate table M1 containing the preliminary data corresponding to other server B. Therefore, we obtain intermediate tables M1 containing preliminary data corresponding to other server A and other server B.

[0082] Alternatively, in the second type of correspondence between the intermediate table and data tables on other servers, the server can determine the corresponding fields in data table A1 and data table A2 on other server A based on the fields of the intermediate table M1. Based on the corresponding fields in data table A1 on other server A, the server obtains the preliminary data corresponding to the corresponding fields in data table A1, and the server also obtains the preliminary data corresponding to the corresponding fields in data table A2 on other server A. Based on the preliminary data corresponding to the corresponding fields in data table A1 and data table A2, an intermediate table M1 containing preliminary data corresponding to other server A is obtained. Furthermore, based on the fields of the intermediate table M1, the server determines the corresponding fields in data table B1 on other server B, and based on the corresponding fields in data table B1 on other server B, obtains the preliminary data corresponding to the corresponding fields in data table B1, resulting in an intermediate table M1 containing preliminary data corresponding to other server B. Therefore, an intermediate table M1 containing preliminary data corresponding to other server A and another intermediate table M1 containing preliminary data corresponding to other server B are obtained.

[0083] 103. Based on the preset integration rule information, integrate multiple intermediate tables containing preliminary data to obtain an integrated intermediate table containing target data; and based on the integrated intermediate table containing target data, obtain a target table containing target data to complete the data migration; wherein, the integration rule information is used to indicate the filtering rule information for the preliminary data.

[0084] For example, the integration rule information is preset information used to indicate filtering information for the same initial data. The filtering information can be a priority. Based on the preset integration rule information, the server integrates multiple intermediate tables containing initial data. If it is determined that the initial data corresponding to the same primary key differs in different tables on different servers, the server can retain the initial data corresponding to the highest priority primary key, determining this highest priority initial data as the integrated target data, thus obtaining the final intermediate table containing the target data. Finally, because there is a preset mapping relationship between the fields of the target table and the fields of the intermediate tables, the server can obtain the target table containing the target data based on the integrated intermediate table containing the target data, thereby completing the data migration.

[0085] In this embodiment, in response to a received data migration request, wherein the data migration request instructs the acquisition of data from multiple other servers, and the determination of a target table for storing the target data, and an intermediate table corresponding to the target table; the intermediate table has a preset correspondence with the data table in each of the other servers. According to preset data mapping rule information, preliminary data from the data table in each of the other servers corresponding to the intermediate table is acquired, resulting in an intermediate table containing the preliminary data corresponding to each other server; wherein the preset data mapping rule information characterizes the mapping relationship between the fields of the intermediate table and the fields of the data tables in other servers. According to preset integration rule information, the multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing the target data; and based on the integrated intermediate table containing the target data, a target table containing the target data is obtained to complete the data migration; wherein the integration rule information is used to instruct the filtering rule information for the preliminary data. In this solution, upon receiving a data migration request, the target table and intermediate tables are first determined. Then, based on preset data mapping rules, the fields in the intermediate tables and the corresponding fields in other server tables are identified. Based on these fields, preliminary data is retrieved from each of the other server tables corresponding to the intermediate table's fields, resulting in an intermediate table containing the preliminary data for each other server. Next, according to preset integration rules, these multiple intermediate tables containing preliminary data are integrated to obtain an integrated intermediate table containing the target data. Since there is a correspondence between the intermediate and target tables, and based on the integrated intermediate table containing the target data, the target data from the intermediate tables can be stored in the target table, resulting in a target table containing the target data, thus completing the data migration. Therefore, by using preset data mapping rules, data corresponding to the fields in the intermediate tables can be retrieved from multiple other databases, completing the migration and integration of data from multiple other databases, thus solving the technical problem of low diversity of data sources that can be migrated and integrated.

[0086] Figure 2 A flowchart illustrating another data processing method provided in this application embodiment is shown below. Figure 2 As shown, the method includes:

[0087] 201. Pre-acquire at least one data table sent by each of multiple other servers, and multiple target tables for storing target data; wherein both the data table and the target table include multiple fields.

[0088] For example, the server may pre-obtain at least one data table sent by each of a plurality of other servers, and a plurality of target tables for storing target data, wherein both the data tables and the target tables include a plurality of fields.

[0089] 202. Determine the data table structure of each other server's data table and the target table's data table structure.

[0090] For example, the data table structure refers to the business meaning, length, and code value range of fields. For instance, the business meaning of a field might refer to user information. The server can determine the data table structure of each data table sent by another server, as well as the data table structure of the target table.

[0091] 203. Determine the primary key of the target table based on its data table structure.

[0092] In one example, based on the preset primary key information, the fields in the target table are analyzed for their business meaning, and the fields that match the primary key information are confirmed to be the primary keys of the target table.

[0093] For example, the preset primary key information is pre-defined user information used to determine the primary key. For instance, the primary key information may include multiple user information entries; one user information entry can be set as the primary key of the target table, or two user information entries can be used together as the primary key. The server can perform business meaning analysis on the fields in the target table based on the preset primary key information, determine the meaning of each field, and compare each field with the preset primary key information to confirm that the field matching the primary key information is the primary key of the target table.

[0094] 204. Based on the data table structure of each target table, generate an intermediate table corresponding to each target table; wherein there is a mapping relationship between the fields of the target table and the fields of the intermediate table.

[0095] For example, the server can generate an intermediate table corresponding to each target table based on the data table structure of each target table. Therefore, there is a mapping relationship between the fields of the target table and the fields of the intermediate table. Specifically, the primary key of the target table is the same as the primary key of the intermediate table, and other fields are kept as consistent as possible, or there is a certain explicit mapping relationship between the fields of the target table and the fields of the intermediate table.

[0096] 205. For each other server sending at least one data table, generate data mapping rule information between the intermediate table and the data table based on the fields of the at least one data table sent by each other server and the fields of the intermediate table corresponding to each other server.

[0097] For example, firstly, if a user's editing operation is received, the server can determine the data source to be analyzed (i.e., other servers), identify all data tables in that data source that need to be migrated, and then each other server sends all data tables requiring migration to the server. After receiving the data tables from each other server, the server selects an intermediate table corresponding to the other server for each data table sent by the other server. Using the selected intermediate table as a reference, the server includes the fields of all data tables in the other server within a preset migration field mapping analysis scope. Since there is a mapping relationship between the fields of the intermediate table and the fields of the target table, the business meaning of the fields in the intermediate table can be determined. Based on the business meaning of the fields in the intermediate table and the business meaning of the fields in the at least one data table sent by the other server, data mapping rule information between the fields in the intermediate table and the fields in the other servers is generated, thus completing the mapping rule formulation from the data tables in the other servers to the intermediate table. The above steps are repeated to complete the data mapping rule information from a single data source to each intermediate table corresponding to the other server.

[0098] 206. In response to a received data migration request, wherein the data migration request is used to instruct the acquisition of data from multiple other servers, to determine a target table for storing the target data, and an intermediate table corresponding to the target table; the intermediate table has a preset correspondence with the data table in each of the other servers.

[0099] For example, this step can be referred to Figure 1 Step 101 in the text will not be repeated here.

[0100] 207. Based on the preset data mapping rule information, obtain the preliminary data in the data table of each other server corresponding to the intermediate table, and obtain the intermediate table containing the preliminary data corresponding to each other server; wherein, the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data table of other servers.

[0101] Step 207 includes two implementation methods:

[0102] The first implementation of step 207 is as follows: According to the preset data mapping rule information, a data acquisition instruction is sent to each other server corresponding to the intermediate table, so that each other server can acquire preliminary data from at least one data table corresponding to each other server according to the data acquisition instruction; the preliminary data sent by each other server corresponding to the intermediate table is received; and an intermediate table containing the preliminary data corresponding to each other server is obtained based on the preliminary data sent by each other server corresponding to the intermediate table.

[0103] The second implementation of step 207 is as follows: Obtain at least one data table sent by each other server; in each of the at least one data tables sent by each other server, based on the preset data mapping rule information, obtain the preliminary data in the data table of each other server corresponding to the intermediate table; according to the preliminary data sent by each other server corresponding to the intermediate table, obtain the intermediate table containing the preliminary data corresponding to each other server.

[0104] For example, the server can obtain preliminary data from the data tables of each other server corresponding to the intermediate table according to the preset data mapping rule information, and obtain an intermediate table containing preliminary data corresponding to each other server. The acquisition method can be that the server requests preliminary data from other servers, or the server can first obtain the data tables in other servers and then obtain the preliminary data in the data tables within the server.

[0105] In the method of obtaining preliminary data from other servers, the server sends a data retrieval instruction to each other server corresponding to the intermediate table according to preset data mapping rules. The data retrieval instruction includes the primary key and fields of the intermediate table. Since other servers also have preset data mapping rules, when each other server receives a data retrieval instruction, it retrieves preliminary data from at least one data table corresponding to itself, based on the instruction and the preset data mapping rules, and sends the retrieved preliminary data to the server. The server receives the preliminary data sent by each other server corresponding to the intermediate table and, based on this preliminary data, obtains the intermediate table containing the preliminary data corresponding to each other server.

[0106] Alternatively, in the method of first obtaining data tables from other servers and then obtaining preliminary data from the data tables within the server, the server first obtains at least one data table sent by each other server. In each of the at least one data tables sent by each other server, based on preset data mapping rule information, the server obtains the preliminary data in the data tables of each other server corresponding to the intermediate table, and obtains the intermediate table containing the preliminary data corresponding to each other server based on the preliminary data sent by each other server corresponding to the intermediate table.

[0107] 208. If it is determined that the preset primary key of each of the multiple intermediate tables corresponds to different preliminary data, then determine the priority of the preset primary key of each intermediate table; wherein, the primary key represents the identifier of each record in the intermediate table.

[0108] For example, each of the multiple intermediate tables includes multiple records, each record corresponds to multiple fields, and each record may include data corresponding to the multiple fields. Each record corresponds to a preset primary key, which represents the identifier of each record in the intermediate table. If the primary key of each of the multiple intermediate tables corresponds to different initial data, the priority of the preset primary key of each intermediate table containing the initial data is determined. Specifically, the priority may include good, better, poor, etc., and the priority level is preset, for example, "good" is better than "better," and "better" is better than "poor," without any restrictions.

[0109] 209. Based on the preset integration rule information, retain the preliminary data corresponding to the primary key with the highest priority; wherein, the integration rule information is used to indicate the filtering rule information for the preliminary data.

[0110] For example, the integration rule information is preset information used to indicate filtering information for the same initial data. The filtering information can be a priority. According to the preset integration rule information, the server can retain the initial data corresponding to the highest priority based on the primary key priority, supporting data from multiple data sources to be overwritten according to priority.

[0111] 210. Based on the initial data corresponding to the primary key with the highest priority, obtain the integrated intermediate table containing the target data.

[0112] For example, the server can determine the preliminary data corresponding to the highest priority as the integrated target data, and obtain the integrated intermediate table containing the target data based on the preliminary data corresponding to the primary key with the highest priority.

[0113] 211. Based on the pre-defined mapping relationship between the fields of the intermediate table and the fields of the target table, store the target data from the intermediate table into the target table to complete the data migration.

[0114] For example, since there is a preset mapping relationship between the fields of the target table and the fields of the intermediate table, the server can store the data corresponding to the fields of the intermediate table into the corresponding fields of the target table according to the fields of the integrated intermediate table containing the target data, so as to obtain the target table containing the target data.

[0115] In this embodiment, at least one data table sent by each of a plurality of other servers and a plurality of target tables for storing target data are pre-acquired; wherein both the data tables and the target tables include multiple fields. The data table structure of each data table sent by the other servers and the data table structure of the target tables are determined. The primary key of the target tables is determined based on the data table structure of the target tables. An intermediate table corresponding to each target table is generated based on the data table structure of each target table; wherein there is a mapping relationship between the fields of the target tables and the fields of the intermediate tables. For each data table sent by the other servers, data mapping rule information between the intermediate tables and the data tables is generated based on the fields of the at least one data table sent by the other servers and the fields of the intermediate tables corresponding to each other server. In response to a received data migration request, wherein the data migration request is used to instruct the acquisition of data from the plurality of other servers, the target tables for storing target data and the intermediate tables corresponding to the target tables are determined; a preset correspondence exists between the intermediate tables and the data tables in each of the other servers. Based on preset data mapping rules, preliminary data is retrieved from each data table on other servers corresponding to the intermediate table, resulting in an intermediate table containing the preliminary data for each other server. The preset data mapping rules represent the mapping relationship between fields in the intermediate table and fields in data tables on other servers. If it is determined that each intermediate table's preset primary key corresponds to different preliminary data, the priority of each intermediate table's preset primary key is determined; the primary key represents the identifier of each record in the intermediate table. Based on preset integration rules, the preliminary data corresponding to the highest priority primary key is retained; the integration rules indicate the filtering rules for the preliminary data. Based on the preliminary data corresponding to the highest priority primary key, an integrated intermediate table containing the target data is obtained. According to the preset mapping relationship between fields in the intermediate table and fields in the target table, the target data in the intermediate table is stored in the target table to complete the data migration. Therefore, through preset data mapping rules, data corresponding to fields in the intermediate table can be retrieved from multiple other databases, completing the migration and integration of data from multiple other databases, thus solving the technical problem of low diversity of data sources that can be migrated and integrated.

[0116] Figure 3 This is a schematic diagram of the structure of a data processing device provided in an embodiment of this application, as shown below. Figure 3 As shown, the device includes:

[0117] The first determining unit 31 is configured to respond to a received data migration request, wherein the data migration request is configured to instruct the acquisition of data from multiple other servers, and to determine a target table for storing the target data, and an intermediate table corresponding to the target table; the intermediate table has a preset correspondence with the data table in each of the other servers.

[0118] The second determining unit 32 is used to obtain preliminary data from the data tables in each other server corresponding to the intermediate table according to the preset data mapping rule information, so as to obtain an intermediate table containing preliminary data corresponding to each other server; wherein, the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data tables in other servers.

[0119] Integration unit 33 is used to integrate multiple intermediate tables containing preliminary data according to preset integration rule information to obtain an integrated intermediate table containing target data.

[0120] The third determining unit 34 is used to obtain a target table containing target data based on the integrated intermediate table containing target data, so as to complete the data migration; wherein, the integration rule information is used to indicate the filtering rule information for the preliminary data.

[0121] The apparatus in this embodiment can execute the technical solutions in the above method. Its specific implementation process and technical principles are the same, and will not be repeated here.

[0122] Figure 4 This is a schematic diagram of another data processing apparatus provided in an embodiment of this application. Figure 3 Based on the illustrated embodiments, as Figure 4 As shown, the integration unit 33 includes:

[0123] The first determining module 331 is used to determine the priority of the preset primary key of each intermediate table if the preset primary key of each intermediate table corresponds to different preliminary data; wherein the primary key represents the identifier of each record in the intermediate table.

[0124] The retention module 332 is used to retain the initial data corresponding to the primary key with the highest priority based on preset integration rule information.

[0125] Integration module 333 is used to obtain an integrated intermediate table containing the target data based on the preliminary data corresponding to the primary key with the highest priority.

[0126] In one example, the second determining unit 32 includes:

[0127] The sending module 321 is used to send data acquisition instructions to each other server corresponding to the intermediate table according to the preset data mapping rule information, so that each other server can acquire preliminary data from at least one data table corresponding to each other server according to the data acquisition instructions.

[0128] The receiving module 322 is used to receive preliminary data sent by each other server corresponding to the intermediate table.

[0129] The second determining module 323 is used to obtain an intermediate table containing preliminary data corresponding to each other server based on the preliminary data sent by each other server corresponding to the intermediate table.

[0130] In one example, the second determining unit 32 includes:

[0131] The first acquisition module 324 is used to acquire at least one data table sent by each other server.

[0132] The second acquisition module 325 is used to acquire preliminary data from at least one data table sent by each other server, based on preset data mapping rule information, the data in the data table of each other server corresponding to the intermediate table.

[0133] The third determining module 326 is used to obtain an intermediate table containing preliminary data corresponding to each other server based on the preliminary data sent by each other server corresponding to the intermediate table.

[0134] In one example, the third determining unit 34 is specifically used for:

[0135] Based on the pre-defined mapping relationship between the fields of the intermediate table and the fields of the target table, the target data of the intermediate table is stored in the target table.

[0136] In one example, the device also includes:

[0137] The acquisition unit 41 is used to pre-acquire at least one data table sent by each of the multiple other servers, and multiple target tables for storing target data; wherein both the data table and the target table include multiple fields.

[0138] The fourth determining unit 42 is used to determine the data table structure of each other server's data table and the data table structure of the target table.

[0139] The fifth determining unit 43 is used to determine the primary key of the target table based on the data table structure of the target table.

[0140] The first generation unit 44 is used to generate an intermediate table corresponding to each target table based on the data table structure of each target table; wherein there is a mapping relationship between the fields of the target table and the fields of the intermediate table.

[0141] The second generation unit 45 is used to generate data mapping rule information between the intermediate table and the data table based on the fields of the at least one data table sent by each other server and the fields of the intermediate table corresponding to each other server.

[0142] In one example, the fifth determining unit 43 is specifically used for:

[0143] Based on the preset primary key information, the fields in the target table are analyzed for business meaning to confirm that the fields matching the primary key information are the primary keys of the target table. The apparatus in this embodiment can execute the technical solution described above; its specific implementation process and technical principles are the same and will not be repeated here.

[0144] Figure 5 This application provides a schematic diagram of the structure of a server, as shown in the embodiment of the present application. Figure 5 As shown, the server includes: a memory 51 and a processor 52.

[0145] The memory 51 stores a computer program that can run on the processor 52.

[0146] Processor 52 is configured to perform the methods provided in the embodiments described above.

[0147] The server also includes a receiver 53 and a transmitter 54. The receiver 53 is used to receive instructions and data sent by external devices, and the transmitter 54 is used to send instructions and data to external devices.

[0148] This application also provides a non-transitory computer-readable storage medium, which, when the instructions in the storage medium are executed by the processor of a server, enables the server to perform the methods provided in the above embodiments.

[0149] This application also provides a computer program product, which includes: a computer program stored in a readable storage medium, at least one processor of the server can read the computer program from the readable storage medium, and the at least one processor executes the computer program to cause the server to perform the solution provided in any of the above embodiments.

[0150] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0151] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A data processing method, characterized by, The application comprises: in response to the received data migration request, wherein the data migration request is used to indicate the data from a plurality of other servers, determine the target table for storing target data, and the intermediate table corresponding to the target table; the intermediate table respectively exists a preset corresponding relationship between the data table in each other server; according to the preset data mapping rule information, respectively obtain the preliminary data in the data table in each other server corresponding to the intermediate table, obtain the intermediate table corresponding to each other server containing the preliminary data; wherein the preset data mapping rule information represents the mapping relationship between the fields of the intermediate table and the fields of the data table in the other server; according to the preset integration rule information, the plurality of intermediate tables containing preliminary data are integrated to obtain the integrated intermediate table containing target data; and according to the integrated intermediate table containing target data, the target table containing target data is obtained to complete the data migration; wherein the integration rule information is used to indicate the screening rule information for the preliminary data; the plurality of intermediate tables containing preliminary data are integrated to obtain the integrated intermediate table containing target data according to the preset integration rule information, comprising: if it is determined that the preset primary key of each intermediate table in the plurality of intermediate tables corresponds to different preliminary data, the priority of the primary key of each intermediate table is determined according to the preset primary key of each intermediate table containing preliminary data; wherein the primary key represents the identification of each record of the intermediate table; based on the preset integration rule information, the preliminary data corresponding to the primary key with the highest priority is retained; according to the preliminary data corresponding to the primary key with the highest priority, the integrated intermediate table containing target data is obtained.

2. The method of claim 1, wherein, the preliminary data in the data table in each other server corresponding to the intermediate table is obtained according to the preset data mapping rule information, and the intermediate table containing the preliminary data corresponding to each other server is obtained, comprising: according to the preset data mapping rule information, data acquisition instructions are respectively sent to each of the other servers corresponding to the intermediate table, so that each of the other servers respectively acquires preliminary data from at least one data table corresponding to each of the other servers according to the data acquisition instructions; the preliminary data sent by each of the other servers corresponding to the intermediate table is received; according to the preliminary data sent by each of the other servers corresponding to the intermediate table, the intermediate table containing the preliminary data corresponding to each of the other servers is obtained.

3. The method of claim 1, wherein, the preliminary data in the data table in each other server corresponding to the intermediate table is obtained according to the preset data mapping rule information, and the intermediate table containing the preliminary data corresponding to each other server is obtained, comprising: obtain at least one data table sent by each of the other servers; in at least one data table sent by each of the other servers, based on the preset data mapping rule information, the preliminary data in the data table in each other server corresponding to the intermediate table is obtained; According to the preliminary data sent by each other server corresponding to the intermediate table, an intermediate table containing the preliminary data corresponding to each other server is obtained.

4. The method of claim 1, wherein, The obtaining of the target table containing the target data according to the integrated intermediate table containing the target data comprises: According to a preset mapping relationship between fields of the intermediate table and fields of the target table, the target data of the intermediate table is stored into the target table.

5. The method according to any one of claims 1 to 4, characterized in that, The method further comprises: At least one data table sent by each other server in a plurality of other servers and a plurality of target tables for storing the target data are obtained in advance; wherein the data table and the target table each comprise a plurality of fields; A data table structure of the data table sent by each other server and a data table structure of the target table are determined; According to the data table structure of the target table, a primary key of the target table is determined; According to the data table structure of each target table, an intermediate table corresponding to each target table is generated; wherein a mapping relationship exists between fields of the target table and fields of the intermediate table; For at least one data table sent by each other server, data mapping rule information between the intermediate table and the data table is generated according to fields of at least one data table sent by each other server and fields of the intermediate table corresponding to each other server.

6. The method of claim 5, wherein, The determining of the primary key of the target table according to the data table structure of the target table comprises: According to preset primary key information, a field in the target table is analyzed in terms of business meaning, and a field conforming to the primary key information is confirmed as the primary key of the target table.

7. A data processing apparatus, characterized by, Comprise: A first determining unit is configured to determine a target table for storing target data and an intermediate table corresponding to the target table in response to a received data migration request, wherein the data migration request is used to indicate that data is acquired from a plurality of other servers; and a preset corresponding relationship exists between the intermediate table and a data table in each other server. A second determining unit is configured to acquire preliminary data in a data table in each other server corresponding to the intermediate table according to preset data mapping rule information, to obtain an intermediate table containing the preliminary data corresponding to each other server; wherein the preset data mapping rule information represents a mapping relationship between fields of the intermediate table and fields of the data table in the other server. An integrating unit is configured to integrate a plurality of intermediate tables containing preliminary data according to preset integration rule information, to obtain an integrated intermediate table containing target data. A third determining unit is configured to obtain a target table containing target data according to the integrated intermediate table containing the target data, to complete data migration; wherein the integration rule information is used to indicate a filtering rule information for the preliminary data. The integration unit is specifically configured to determine a priority of the primary key of each intermediate table if it is determined that the primary key of each intermediate table corresponds to different preliminary data, wherein the primary key represents an identifier of each record of the intermediate table; retain the preliminary data corresponding to the primary key with the highest priority based on preset integration rule information; and obtain an integrated intermediate table containing target data according to the preliminary data corresponding to the primary key with the highest priority.

8. A server, characterized by The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method in any one of claims 1-6.

10. A computer program product, characterised in that, The computer readable storage medium stores computer execution instructions, and the computer execution instructions are executed by the processor to implement the method in any one of claims 1-6.

Images