Database processing method and device

Abstract

The embodiment of the invention provides a database processing method and device, and the method comprises the steps: carrying out the clustering processing of a plurality of data tables based on the incidence relation among the plurality of data tables contained in a first database, and obtaining a plurality of data table sets; sorting the plurality of data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables contained in each data table set in the first set sequence; sorting data tables contained in each data table set in the first set sequence according to the evaluation indexes to obtain a second set sequence; and performing data table migration processing from the first database to the second database according to the second set sequence. By adopting the embodiment of the invention, the migration efficiency of data migration between the databases can be improved.

Description

Technical Field

[0001] This document relates to the field of data processing technology, and in particular to a database processing method and device. Background Art

[0002] With the continuous development of Internet technology and the continuous advancement of social information, the scale of database use has gradually expanded. From the simplest tables storing various data to large database systems capable of storing massive amounts of data, various types of databases have been widely used. The speed of database updates and iterations is constantly accelerating, which has led to an increasing demand for data migration between databases.

[0003] Currently, in the process of data migration between databases, SQL (Structured Query Language) statements are often used to complete data migration between databases. Data backup and data import are performed between databases through specific commands. There are many manual operations, resulting in low data migration efficiency. Summary of the invention

[0004] An objective of an embodiment of the present specification is to provide a database processing method and device to improve the migration efficiency of data migration between databases.

[0005] To solve the above technical problems, an embodiment of this specification is implemented as follows: In a first aspect, an embodiment of the present specification provides a database processing method, the method comprising: Based on the association relationship between the multiple data tables included in the first database, clustering the multiple data tables to obtain multiple data table sets; Sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence; Sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The data table migration process from the first database to the second database is performed according to the second set sequence.

[0006] The database processing method provided in this embodiment firstly clusters the multiple data tables based on the association relationship between the multiple data tables included in the first database to obtain multiple data table sets, and calculates the evaluation index of each data table according to the attribute information of the data tables included in each data table set in the first set sequence obtained by sorting the multiple data table sets. Secondly, sorts the data tables included in each data table set in the first set sequence with the help of the evaluation index to obtain a second set sequence. On the basis of sorting the multiple data table sets, sorts the data tables included in each data table set according to the evaluation index of each data table, so that not only the multiple data table sets have an arrangement order, but also the data tables included in each data table set have an arrangement order, which is helpful for the orderliness of subsequent data table migration. Finally, the data table migration from the first database to the second database is performed according to the second set sequence. In this way, the data table migration is performed through the second set sequence in which the data table sets have an arrangement order and the data tables have an arrangement order, so as to improve the convenience and comprehensiveness of the data table migration. In addition, no manual operation is involved in the data table migration process, so as to improve the automation of the data table migration between databases, and thus improve the efficiency of the data table migration.

[0007] In a second aspect, another embodiment of the present specification provides a database processing device, the device comprising: A clustering processing module, configured to perform clustering processing on the multiple data tables contained in the first database based on the association relationship between the multiple data tables, so as to obtain multiple data table sets; An index calculation module, used for sorting the multiple data table sets to obtain a first set sequence, and calculating the evaluation index of each data table according to the attribute information of the data table included in each data table set in the first set sequence; A sorting processing module, used for sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; A migration processing module is used to perform data table migration processing from the first database to the second database according to the second set sequence.

[0008] In a third aspect, another embodiment of the present specification provides a database processing device, the device comprising: a memory, a processor, and computer executable instructions stored in the memory and executable on the processor, the computer executable instructions, when executed by the processor, implement the steps of the database processing method as described in the first aspect above.

[0009] In a fourth aspect, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the steps of the database processing method as described in the first aspect above are implemented.

[0010] In a fifth aspect, another embodiment of the present specification provides a computer program product, wherein the computer program product includes a database processing program, and the database processing program is executed by a processor to implement the steps of the database processing method as described in the first aspect above. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] In order to more clearly illustrate one or more embodiments of the present specification or the technical solutions in the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present specification. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 A processing flow chart of a database processing method provided in one embodiment of this specification; Figure 2 A schematic diagram of a first table relationship diagram provided for one embodiment of this specification; Figure 3 A schematic diagram of a second table relationship diagram provided for one embodiment of this specification; Figure 4 A processing flow chart of a database processing method applied to a data table migration scenario provided by one embodiment of this specification; Figure 5 A schematic diagram of a database processing device provided in one embodiment of this specification; Figure 6 A schematic diagram of the structure of a database processing device provided in one embodiment of this specification. DETAILED DESCRIPTION

[0012] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will be combined with the drawings in one or more embodiments of this specification to clearly and completely describe the technical solutions in one or more embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this specification, not all of the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of this document.

[0013] An embodiment of a database processing method provided in this specification is as follows: The database processing method provided in this embodiment can be executed by a computer device, which can be a terminal or a server. The terminal can include a mobile phone, a laptop computer, an intelligent interactive device, etc., and the server can include an independent physical server, a server cluster composed of multiple servers, or a cloud server capable of cloud computing.

[0014] Reference Figure 1 The database processing method provided in this embodiment specifically includes the following steps S102 to S108.

[0015] Step S102: Based on the association relationship between the multiple data tables included in the first database, clustering processing is performed on the multiple data tables to obtain multiple data table sets.

[0016] The first database in this embodiment may include a source database, which refers to a database storing original data, and the first database may be any type of database; the association relationship may include an associated relationship and / or an unassociated relationship; the associated relationship may include a foreign key reference relationship between data tables, such as a foreign key reference relationship between data table a and data table b, where data table a references data table b through a foreign key, which means that data table a depends on data table b; the unassociated relationship may include that there is no association relationship between data tables, that is, there is no foreign key reference relationship; the first database may be deployed on a first server; each data table set in a plurality of data table sets refers to a set consisting of one or more data tables. This embodiment may be applied to a migration tool deployed on a remote server.

[0017] In specific implementation, before clustering multiple data tables to obtain multiple data table sets, in order to ensure successful data table migration from the first database to the second database, pre-detection processing may be performed on the first database and the second database in advance. If the pre-detection passes, clustering processing may be performed on the multiple data tables based on the association relationship between the multiple data tables contained in the first database to obtain multiple data table sets.

[0018] In the above-mentioned process of pre-detection processing on the first database and the second database, anomaly detection processing can be performed on the detection information corresponding to the detection items of each of the first database and the second database. If the anomaly detection passes, it is determined that the pre-detection passes; the detection items of each of the two may include at least one of the operating environment, database instance, database version, user permissions to the database, and database objects; the detection items of the first database and the second database can be the same or different; if the anomaly detection fails, the detection items that fail the anomaly detection can be repaired, and after the repair is successful, clustering processing of multiple data tables can be performed.

[0019] Before performing pre-detection processing on the first database and the second database, it may be necessary to obtain information required for the pre-detection processing from the first database and the second database, so channel information of the data acquisition channels of the first database and the second database can be configured. The channel information may include the IP (Internet Protocol), port, user ID and / or password of the first database and the second database.

[0020] After configuring the channel information of the data acquisition channel, the detection information corresponding to the detection items of the first database and the second database can be obtained through the data acquisition channel, such as obtaining the environmental information of the operating environment of the first database and the second database, the database instance status, and the database version information.

[0021] During the specific execution process, in the process of clustering the multiple data tables based on the association relationship between the multiple data tables included in the first database to obtain multiple data table sets, a first table relationship graph of the multiple data tables can be constructed based on the association relationship, the inflow quantity of each data table in the first table relationship graph is calculated, and the data table set consisting of the first candidate data table whose inflow quantity meets the preset quantity threshold is eliminated from the first table relationship graph; the second candidate data table corresponding to the first candidate data table is determined in the first table relationship graph, and the inflow quantity of the second candidate data table is updated; if the inflow quantity of the second candidate data table meets the preset quantity threshold, the data table set consisting of the first candidate data table and the data table set consisting of the second candidate data table are regarded as multiple data table sets; the inflow quantity, etc. are explained in detail below, and you can refer to it for reading.

[0022] In practical applications, the association relationship between the multiple data tables included in the first database has a great influence on the migration order. If a data table depends on other data tables, when the other data tables are not migrated, the data table may not be migrated due to its dependence on other data tables. In view of this, in order to improve the orderliness and effectiveness of data table migration; in an optional implementation manner provided by this embodiment, in the process of clustering the multiple data tables based on the association relationship between the multiple data tables included in the first database and obtaining multiple data table sets, the following operations are performed: Construct a first table relationship graph of the plurality of data tables based on the association relationship, remove the first data table from the first table relationship graph, and obtain a second table relationship graph; According to the first data table set composed of the first data table and the second table relationship diagram, a plurality of data table sets are determined.

[0023] The first data table may include an isolated data table, specifically a data table to which there is no connection in the first table relationship graph, that is, the first data table is not connected to other data tables in the first table relationship graph.

[0024] Specifically, in the process of constructing the first table relationship diagram of multiple data tables based on the association relationship, the connection direction between each two data tables can be determined based on the foreign key reference relationship between each two data tables in the multiple data tables, and each two data tables are connected according to the connection direction to obtain the first table relationship diagram; in the process of determining the connection direction between each two data tables based on the foreign key reference relationship between each two data tables in the multiple data tables, if the data table in each two data tables references the target data table through a foreign key, that is, the data table depends on the target data table, then the connection direction can be determined as the target data table pointing to the data table.

[0025] For example, the first database includes multiple data tables 1-4 and 6-9, and the first table relationship diagram constructed based on the association relationship between the multiple data tables is as follows: Figure 2 As shown, the association relationship between data table 1 and data table 3 is that data table 1 references data table 3 through a foreign key, data table 1 depends on data table 3, data table 3 depends on data table 2, data table 4 depends on data table 3, data table 3 depends on data table 6, data table 7 depends on data table 6, and data table 8 depends on data table 7; remove the first data table - data table 9 from the first table relationship diagram, and obtain the second table relationship diagram as shown Figure 3 shown.

[0026] Based on the determination of multiple data table sets according to the first data table set composed of the first data table and the second table relationship diagram, in an optional implementation provided by this embodiment, in the process of determining multiple data table sets according to the first data table set composed of the first data table and the second table relationship diagram, multiple data table sets may be determined according to the inflow quantity of each data table in the first data table set and the second table relationship diagram, and specifically the following operations may be performed: Calculate the inflow quantity of each data table in the second table relationship diagram, and filter out a second data table set consisting of second data tables whose inflow quantities meet a preset quantity threshold from the second table relationship diagram; Determine the third data table corresponding to the second data table in the second table relationship diagram, and update the inflow quantity of the third data table; If the inflow quantity of the third data table meets the preset quantity threshold, the third data table set consisting of the first data table set, the second data table set and the third data table is used as multiple data table sets.

[0027] The inflow quantity may include the number of other data tables pointing to each data table in the second table relationship diagram, that is, the number of other data tables referenced by each data table through foreign keys, that is, the number of other data tables that each data table depends on; for example Figure 3In the second table relationship diagram shown, the data tables pointing to data table 3 include data table 2 and data table 6. The number of other data tables pointing to data table 3 is 2, and the number of inflows into data table 3 is 2. The number of other data tables pointing to data table 1 is 1, and the inflow into data table 1 is 1, and the inflow into data table 6 is 0. The preset quantity threshold may be any preset quantity threshold, such as the preset quantity threshold is 0; the inflow quantity meeting the preset quantity threshold may be the inflow quantity 0. The third data table corresponding to the second data table may specifically be the third data table pointed to by the second data table in the second table relationship diagram, that is, the second data table points to the third data table in the second table relationship diagram.

[0028] Specifically, in the process of determining the third data table corresponding to the second data table in the second table relationship diagram, the third data table pointed to by the second data table can be determined in the second table relationship diagram; in the process of updating the inflow quantity of the third data table, the inflow quantity of the third data table can be updated according to the number of second data tables pointing to the third data table, and specifically, the difference between the current inflow quantity of the third data table and the number of second data tables pointing to the third data table can be calculated as the inflow quantity of the third data table; for example, in the second table relationship diagram, the third data tables pointed to by the second data table are determined to be: data table 3 pointed to by data table 2, data table 7 pointed to by data table 6, and data table 3, and the number of data tables pointing to data table 3 is 2, then the inflow quantity of data table 3 can be reduced by 2, and the number of data tables pointing to data table 7 is 1, then the inflow quantity of data table 7 can be reduced by 1.

[0029] For example, the second data table with an inflow quantity of 0, namely, the second data table set consisting of data table 2 and data table 6, is eliminated from the second table relationship diagram. In the second table relationship diagram, the third data tables pointed to by the second data table are determined to be: data table 3 pointed to by data table 2, data table 7 pointed to by data table 6, and data table 3. The inflow quantities of data table 3 and data table 7 are reduced by 2 and 1 respectively, and the inflow quantity of data table 3 is 0, and the inflow quantity of data table 7 is 0. Then data table 3 and data table 7 are the third data table set consisting of the third data table. Then, the inflow quantity of data table 8 pointed to by data table 7 is reduced by 1, and the inflow quantity of data table 8 is 0. Data table 8 is the target data table set consisting of the target data tables. The first data table set consisting of data table 9, the second data table set consisting of data table 2 and data table 6, the third data table set consisting of data table 3 and data table 7, and the target data table set consisting of data table 8 are taken as multiple data table sets.

[0030] It should be noted that the above process of using the third data table set consisting of the first data table set, the second data table set and the third data table as multiple data table sets if the inflow quantity of the third data table meets the preset quantity threshold can also be replaced by screening out the fourth data table whose inflow quantity meets the preset quantity threshold from the third data table, and if the fifth data table corresponding to the fourth data table is empty, the fourth data table set consisting of the first data table set, the second data table set and the fourth data table can be used as multiple data table sets; if the fifth data table is not empty, the inflow quantity of the fifth data table can be updated, and the sixth data table whose inflow quantity meets the preset quantity threshold can be screened out from the fifth data table, until the seventh data table corresponding to the sixth data table in the second table relationship diagram is empty, and the fourth data table set consisting of the first data table set, the second data table set, the fourth data table and the sixth data table set consisting of the sixth data table are used as multiple data table sets.

[0031] It should be supplemented that the above can also construct a first table relationship graph of multiple data tables based on the association relationship, remove the first data table from the first table relationship graph, obtain the second table relationship graph, and determine multiple data table sets according to the second table relationship graph; in the process of determining multiple data table sets according to the second table relationship graph, the inflow quantity of each data table in the second table relationship graph can be calculated, and the second data table set consisting of the second data tables whose inflow quantities meet the preset quantity threshold can be screened out from the second table relationship graph, and the third data table pointed to by the second data table is determined in the second table relationship graph, and the inflow quantity of the third data table is updated according to the quantity of the second data tables pointing to the third data table. If the inflow quantity of the third data table meets the preset quantity threshold, the third data table set consisting of the second data table set and the third data table is used as multiple data table sets.

[0032] Step S104: sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to the attribute information of the data tables included in each data table set in the first set sequence.

[0033] The first set sequence in this embodiment may include multiple data table sets arranged in order. The attribute information of the data tables contained in each data table set may include the number of foreign key references, the number of accesses, the data table level and / or the amount of table data of the data table; the number of foreign key references here may be the number of other data tables that reference the data table through foreign keys, specifically the number of other data tables that depend on the data table. The data table level can be obtained by presetting; the number of accesses can be obtained by tool scanning or by manual statistics, and the number of accesses can be the number of times the data table is accessed or operated. The evaluation index of each data table can be an index for evaluating the importance of each data table.

[0034] In specific implementation, in order to allow data tables that are dependent on other data tables to be migrated first, multiple data table sets can be sorted according to the generation order of each data table set in the multiple data table sets to obtain a first set sequence; the generation order here refers to the order of generating each data table set, that is, the generation order of each data table set determined according to the generation time of each data table set; therefore, since the data table set composed of the data tables that are dependent on other data tables is generated first, the data tables that are dependent on other data tables can be migrated first in the subsequent period through sorting here, thereby improving the effectiveness of data table migration.

[0035] In the process of calculating the evaluation index of each data table according to the attribute information of the data tables included in each data table set in the first set sequence, the evaluation index of each data table can be calculated according to the number of foreign key references, the number of accesses, the data table level and / or the table data volume of the data tables included in each data table set in the first set sequence; specifically, the evaluation index of each data table included in any data table set in the first set sequence can be obtained in the following manner: according to the attribute information of the current data table and the attribute information of the remaining data tables included in any (any one) data table set, the evaluation index of the current data table is calculated; the current data table may include the data table for which the evaluation index is calculated; for example, if any data table set contains 3 data tables, the evaluation index of the first data table can be calculated based on the attribute information of the first data table and the attribute information of the remaining 2 data tables.

[0036] For example, a data table set as shown in Table 1 below contains the number of foreign key references, access times, data table level, and table data volume of the data table:

[0037] Table 1 Among them, data table t1 is referenced by data tables t10 and t9 through foreign keys, so the number of foreign key references to data table t1 is 2; data table t3 is referenced by data table t5 through foreign keys, so the number of foreign key references to data table t3 is 1.

[0038] In addition, in order to improve the calculation efficiency of the evaluation indicators of each data table, an evaluation algorithm can be introduced to calculate the evaluation indicators. Specifically, the number of foreign key references, the number of accesses, the data table level and the amount of table data of the data tables contained in each data table set in the first set sequence can be input into the evaluation algorithm to calculate the evaluation indicators to obtain the evaluation indicators of each data table.

[0039] For example, evaluation algorithms include:

[0040] in, Represents the evaluation index of the i-th data table contained in each data table set; Represents the number of foreign key references of the i-th data table; Represents the minimum number of foreign key references among all the data tables contained in each data table set; Represents the maximum number of foreign key references among all the data tables contained in each data table set; Represents the number of times the i-th data table is accessed; Represents the minimum number of accesses among all the data tables contained in each data table set; Represents the maximum number of accesses among all the data tables contained in each data table set; is the preset parameter; Represents the table data volume of the i-th data table; represents the minimum logarithm of the sum of the logarithm of the table data volume of all data tables included in each data table set and the preset parameter c; represents the maximum logarithm of the summed logarithms; , , , represents the preset coefficient; Represents the table level of the i-th table.

[0041] The above-mentioned normalization of the number of foreign key references and the number of accesses facilitates the subsequent weighted calculation of each sub-evaluation indicator. Since the range of variation of the table data volume is large, the table data volume can be logarithmically scaled by logarithmic scaling and then normalized to reduce the impact of large-scale changes on the results of weighted calculation.

[0042] Step S106: sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence.

[0043] The above step calculates the evaluation index of each data table based on the attribute information of the data tables included in each data table set in the first set sequence. In this step, the data tables included in each data table set in the first set sequence are sorted according to the evaluation index to obtain the second set sequence.

[0044] During specific implementation, the data tables included in each data table set in the first set sequence can be sorted in descending order according to the evaluation index to obtain the second set sequence; in addition, the data tables included in the first data table set can also be sorted according to the evaluation index of the data tables included in the first data table set; the first data table set may include an isolated data table set, which may specifically be a set consisting of data tables that have no connection with other data tables, or may not include an isolated data table set; the first data table set may be a set consisting of the first data table or the isolated data table.

[0045] In practical applications, before performing data table migration from the first database to the second database, the operating environment of the second database is also crucial. The operating environment of the second database may affect the success rate of the data table migration. Therefore, there is a need to perform trustworthy verification on the operating environment of the second database, so as to improve the success rate of subsequent data table migration through trustworthy verification. In an optional implementation manner provided by this embodiment, after sorting the data tables included in each data table set in the first set sequence according to the evaluation index and obtaining the second set sequence for execution, the following operations are further performed: Calculating a trustworthy index of the operating environment according to the environment information of the operating environment of the second database; If the trustworthy index is greater than the preset index threshold, the following step S108 is executed to perform data table migration processing from the first database to the second database according to the second set sequence.

[0046] If the trust indicator is less than or equal to the preset indicator threshold, no processing is performed; the second database may include a target database, specifically a target database waiting for data table migration. The environment information of the operating environment may include hardware environment information, software environment information, network information and / or environment component information, such as hardware environment information such as motherboard, memory and / or network card; software environment information such as operating system, database and / or middleware; network information such as firewall and / or IP; environment component information such as libsemanage (security policy management tool). Trust indicators can be used to characterize the security level of the operating environment of the second database.

[0047] Specifically, in the process of calculating the trustworthy index of the operating environment based on the environmental information of the operating environment of the second database, the environmental information of the operating environment of the second database can be matched with the preset environmental parameters of the environmental evaluation items under the evaluation method to obtain the preset environmental parameters matched by the environmental information, and the trustworthy index of the operating environment of the second database is calculated according to the preset scores corresponding to the preset environmental parameters matched by the environmental information.

[0048] The environmental assessment items may include hardware environment items, software environment items, network items and / or environmental component items. The assessment method may be a first assessment method and / or a second assessment method, the first assessment method may be a whitelist assessment method, and the second assessment method may be a security level assessment method and / or a status assessment method.

[0049] Specifically, a first environment assessment item whose assessment method is the first assessment method and a second environment assessment item whose assessment method is the second assessment method can be determined in the environment assessment items corresponding to the operating environment of the second database; the assessment result of the environment information corresponding to the first environment assessment item can be calculated according to the matching result of the environment information corresponding to the first environment assessment item in the white list; the assessment result of the environment information corresponding to the second environment assessment item can be calculated according to the preset score corresponding to the preset environment parameter matched by the environment information corresponding to the second environment assessment item in the preset environment parameters of the environment assessment item; in the process of calculating the assessment result of the environment information corresponding to the first environment assessment item according to the matching result of the environment information corresponding to the first environment assessment item in the white list, if the environment information corresponding to the first environment assessment item is in the white list, the first preset score (for example, 1) can be used as the assessment result of the environment information corresponding to the first environment assessment item; if the environment information corresponding to the first environment assessment item is not in the white list, the second preset score (for example, 0) can be used as the assessment result of the environment information corresponding to the first environment assessment item.

[0050] For example, the preset environmental parameters corresponding to the environmental assessment items and the preset scores corresponding to the preset environmental parameters are shown in Table 2:

[0051] Table 2 The environment information of the operating environment of the second database is shown in Table 3 below:

[0052] Table 3 Assume that the CPU environment information c is in the whitelist c1, c2, c3, the motherboard m is not domestically produced; the storage s is not domestically produced; the memory d is not domestically produced; the network card n is not domestically produced; the operating system o is in the whitelist o1, o2, o3; the IP-xxxxx1 is in the whitelist xxxxx2; then the evaluation result of the CPU is 1, the evaluation result of the motherboard is 0.6, the evaluation result of the storage is 0.6, the evaluation result of the memory is 0.6, the evaluation result of the network card is 0.6, the evaluation result of the operating system is 1, the evaluation result of the IP is 1, and the evaluation result of libsemanage is 0.6. The trust index of the operating environment of the second database is calculated through the evaluation results of each environmental information.

[0053] In addition, after the above calculation obtains the evaluation results of the environmental information corresponding to the first environmental evaluation item and the evaluation results of the environmental information corresponding to the second environmental evaluation item, the evaluation results can also be input into the trusted indicator calculation algorithm for trusted indicator calculation to obtain the trusted indicator of the operating environment of the second database.

[0054] For example, the trusted indicator calculation algorithm includes:

[0055] Wherein, R represents the trust index of the operating environment of the second database; represents the evaluation result of the environmental information corresponding to the first environmental evaluation item, the evaluation result can be 0 or 1, and n represents the number of the first environmental evaluation items in the evaluation method; represents the evaluation result of the environmental information corresponding to the second environmental evaluation item; m represents the number of the second environmental evaluation items for which the evaluation method is used.

[0056] After obtaining the trustworthy index of the operating environment of the second database, no processing may be performed if the trustworthy index is less than or equal to a preset index threshold; the preset index threshold may be determined based on an actual business scenario.

[0057] Step S108: performing data table migration processing from the first database to the second database according to the second set sequence.

[0058] It should be noted that, in the present embodiment, the multiple data tables involved in the steps of clustering multiple data tables and sorting the data tables included in each data table set in the first set sequence, and the data tables included in each data table set may be merely table identifiers (such as table names) of the data tables, and the table identifier is used to characterize the uniqueness of the data table; the data table may also be a data table actually stored in the first database, that is, the data table stores actual data; in the case where the data table is a table identifier of a data table, before executing step S108, the data table may be exported from the first database according to the table identifiers of the data tables in the second set sequence, and the exported data tables may be migrated to the second database.

[0059] In actual applications, in addition to the second set sequence, there may be a first data table, that is, there is a data table that is not connected to other data tables, and the first data table may be more important, and there may be a need for priority migration. Therefore, in order to meet the diverse data table migration needs and improve the flexibility of data table migration; in an optional implementation provided by this embodiment, in the process of performing data table migration processing from the first database to the second database according to the second set sequence, the following operations are performed: According to the evaluation index of the first data table, the first data table is allocated to the data table set in the second set sequence to obtain a third set sequence; The data table migration process from the first database to the second database is performed according to the third set sequence.

[0060] The first data table may be a data table that has no connection with other data tables, that is, may be a data table that has no foreign key reference relationship with other data tables, that is, may be an isolated data table.

[0061] The calculation process of the evaluation index of the first data table here is similar to the calculation process of the evaluation index of each of the above data tables, which will not be repeated here; before the allocation processing is performed here, the first data table can be sorted according to the evaluation index of the first data table.

[0062] It should be noted that if the first data table set composed of the first data table is included in the second set sequence, then the first data table can be allocated to the remaining data table set in the second set sequence according to the evaluation index of the first data table included in the first data table set to obtain a third set sequence; the remaining data table set here refers to the remaining data table set in the second set sequence except the first data table set; the specific allocation process is similar to the allocation process described below, which can be read as a reference.

[0063] On this basis, in an optional implementation manner provided by this embodiment, in the process of allocating the first data table to the data table set in the second set sequence according to the evaluation index of the first data table, the following operations are performed: Compare the evaluation index of the first data table with the evaluation index of the last data table in the data table set; If the comparison result is that the evaluation index of the first data table is greater than the evaluation index of the last data table, the first data table is allocated to the data table set according to the evaluation index of the first data table.

[0064] The last data table included in the data table set refers to the data table arranged at the last position in the data table set.

[0065] Specifically, in the process of allocating the first data table to the data table set according to the evaluation index of the first data table, the allocation position of the first data table can be determined in the data table set according to the evaluation index of the first data table, and the first data table can be allocated to the allocation position in the data table set; in the process of allocating the first data table to the data table set in the second set sequence according to the evaluation index of the first data table, the evaluation index of the remaining data tables included in the first data table set composed of the first data table can also be compared with the evaluation index of the last data table of the current data table set in the second set sequence, and the remaining data tables can be allocated to the data table set according to the evaluation index of the first data table. The intermediate data table whose comparison result is that the evaluation index is greater than the evaluation index of the last data table is screened out, and the intermediate data table is allocated to the current data table set according to the evaluation index of the intermediate data table, and the evaluation index of the second remaining data table included in the first data table set is compared with the evaluation index of the last data table of the adjacent data table set of the current data table set. If the comparison result is that the evaluation index of the second remaining data table is less than or equal to the evaluation index of the last data table of the adjacent data table set, and the adjacent data table set is the last data table set in the second set sequence, then the second remaining data table can be written to the end of the second set sequence.

[0066] The remaining data tables contained in the first data table set here refer to other data tables in the first data table set except the data tables that have been allocated and processed; the adjacent data table set can be the next data table set of the current data table set, that is, it can be the next data table set; the second remaining data tables can be other data tables in the first data table set except the data tables that have been allocated and processed.

[0067] For example, the first data table set is sorted_isolation_list, and the evaluation index of the data tables included in the first data table set is first compared with the evaluation index of the last data table of the first data table set in the second set sequence, and the first screening data table whose evaluation index is greater than the evaluation index of the last data table of the first data table set is eliminated from the first data table set to obtain an intermediate data table set, and the first screening data table is assigned to the first data table set according to the evaluation index of the first screening data table, and then the evaluation index of the data tables included in the intermediate data table set is compared with the evaluation index of the last data table of the second data table set in the second set sequence, and the second screening data table whose evaluation index is greater than the evaluation index of the last data table of the second data table set is eliminated from the intermediate data table set to obtain the second intermediate data table set, and the second screening data table is assigned to the second data table set, and if the first data table set and the second data table set exist in the second set sequence, the second intermediate data table set is written to the last position in the second set sequence.

[0068] After obtaining the second set sequence, the data table sets contained in the second set sequence can be spliced ​​to obtain a data table sequence, and the data table migration process from the first database to the second database can be performed according to the data table sequence; the data table sequence can be a sequence composed of data tables arranged in order.

[0069] In the process of performing data table migration from the first database to the second database according to the data table sequence, the data table can be exported from the first database according to the data table sequence, and the exported data table can be migrated to the second database; after performing the data table migration from the first database to the second database, the data table migrated from the second database can be verified with the data table of the first database, and the verification can be a consistency verification to ensure the integrity of the migrated data.

[0070] On the basis of the third data table set composed of the second data table set and the third data table as multiple data table sets, in the process of performing data table migration processing from the first database to the second database according to the second set sequence, the first data table can be allocated to the data table set in the second set sequence according to the evaluation index of the first data table to obtain the third set sequence, and the data table migration processing from the first database to the second database can be performed according to the third set sequence; here, the first data table is allocated to the data table set in the second set sequence according to the evaluation index of the first data table to obtain the third set sequence. The specific implementation process is similar to the above and will not be repeated here.

[0071] It should be supplemented that each optional implementation method or each feasible execution method in steps S102 to S108 provided in this embodiment can be executed independently as needed, or can be combined and referenced with each other. At the same time, each specific execution step in each optional implementation method or each feasible execution method can also be executed independently, and this embodiment does not make specific limitations.

[0072] In summary, the database processing method provided in this embodiment firstly constructs a first table relationship graph of multiple data tables based on the association relationship, removes the first data table from the first table relationship graph, and obtains a second table relationship graph; calculates the inflow quantity of each data table in the second table relationship graph, and screens out a second data table set consisting of second data tables whose inflow quantities meet a preset quantity threshold from the second table relationship graph; determines a third data table corresponding to the second data table in the second table relationship graph, and updates the inflow quantity of the third data table; if the inflow quantity of the third data table meets the preset quantity threshold, then uses a third data table set consisting of the first data table set, the second data table set, and the third data table as multiple data table sets; Secondly, a first set sequence is obtained by sorting a plurality of data table sets in a generation order, an evaluation index of each data table is calculated according to the attribute information of the data tables contained in each data table set in the first set sequence, and the data tables contained in each data table set in the first set sequence are sorted according to the evaluation index to obtain a second set sequence; in this way, when the inflow quantity meets a preset quantity threshold, a corresponding data table set is generated, so that a data table set consisting of data tables that are dependent on other data tables is generated first, thereby making the first set sequence a set sequence constructed according to the dependency relationship between the data tables; within each data table set contained in the first set sequence, the data tables are sorted according to the evaluation index to improve the rationality and effectiveness of the arrangement of the data tables within each data table set; Finally, according to the evaluation index of the first data table included in the first data table set, the first data table is allocated to the data table set in the second set sequence to obtain a third set sequence, and the data table migration process from the first database to the second database is performed according to the third set sequence; on the basis of the first set sequence constructed according to the dependency relationship between the data tables, the data tables included in each data table set in the first set sequence are sorted, and the data table migration process is performed through the final second set sequence to improve the effectiveness of the data table migration and meet the diverse needs of the data table migration.

[0073] The following combination Figure 4 , taking the application of the database processing method provided in this embodiment in the data table migration scenario as an example, the database processing method provided in this embodiment is further explained. Figure 4 ,The database processing method applied to the data table migration scenario specifically includes the following steps.

[0074] Step S402: construct a first table relationship graph of the multiple data tables based on the association relationship between the multiple data tables included in the source database, remove the first data table from the first table relationship graph, and obtain a second table relationship graph.

[0075] The first data table may be a data table that has no connection with other data tables in the first table relationship graph, and the first data table may be an isolated data table.

[0076] Step S404, calculating the inflow quantity of each data table in the second table relationship diagram, and screening out from the second table relationship diagram a second data table set consisting of second data tables whose inflow quantities meet a preset quantity threshold.

[0077] Step S406, determining the third data table pointed to by the second data table in the second table relationship diagram, and updating the inflow quantity of the third data table.

[0078] Specifically, the inflow quantity of the third data table may be updated according to the quantity of the second data table pointing to the third data table.

[0079] Step S408: if the inflow quantity of the third data table meets the preset quantity threshold, the third data table set consisting of the second data table set and the third data table is used as a plurality of data table sets.

[0080] Step S410 , sorting multiple data table sets in a generation order to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence.

[0081] Step S412: sort the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence.

[0082] Step S414: according to the evaluation index of the first data table, the first data table is allocated to the data table set in the second set sequence to obtain a third set sequence.

[0083] Step S416: performing data table migration processing from the source database to the target database according to the third set sequence.

[0084] It should be noted that any one of steps S402 to S416 or any combination of multiple steps can be combined with any one of steps S102 to S108 to form a new implementation method according to the needs of implementation deployment; in addition, according to the needs of actual deployment, any one or multiple technical features can be selected from steps S402 to S416 and combined with any one or multiple technical features provided by steps S102 to S108 to form a new implementation method; or, any one or multiple technical features in steps S402 to S416 can be replaced with any one or multiple technical features provided by steps S102 to S108 to form a new implementation method according to the needs of actual deployment, which will not be repeated here.

[0085] Figure 5 A schematic diagram of a database processing device provided by an embodiment of the present invention is shown in FIG. Figure 5 As shown, the device comprises: A clustering processing module 502 is used to perform clustering processing on the multiple data tables contained in the first database based on the association relationship between the multiple data tables to obtain multiple data table sets; An index calculation module 504 is used to sort the multiple data table sets to obtain a first set sequence, and calculate an evaluation index for each data table according to the attribute information of the data table included in each data table set in the first set sequence; A sorting processing module 506 is used to sort the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The migration processing module 508 is used to perform data table migration processing from the first database to the second database according to the second set sequence.

[0086] The database processing device provided in this embodiment first runs a clustering processing module 502 to cluster the multiple data tables contained in a first database based on the association relationship between the multiple data tables to obtain multiple data table sets, runs an index calculation module 504 to sort the multiple data table sets to obtain a first set sequence, and calculates an evaluation index for each data table based on the attribute information of the data tables contained in each data table set in the first set sequence, then runs a sorting processing module 506 to sort the data tables contained in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence, and finally runs a migration processing module 508 to perform data table migration processing from the first database to the second database according to the second set sequence to improve the migration efficiency of data migration between databases.

[0087] The database processing device provided in one embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0088] Furthermore, an embodiment of the present specification also provides a database processing device, Figure 6 A schematic diagram of a database processing device provided in an embodiment of this specification is shown in FIG. Figure 6 As shown, the device includes: a memory 601, a processor 602, a bus 603 and a communication interface 604. The memory 601, the processor 602 and the communication interface 604 communicate through the bus 603, and the communication interface 604 may include an input and output interface, which includes but is not limited to a keyboard, a mouse, a display, a microphone, a loudspeaker, etc.

[0089] Figure 6 In the embodiment, the memory 601 stores computer executable instructions that can be run on the processor 602. When the computer executable instructions are executed by the processor 602, the following process is implemented: Based on the association relationship between the multiple data tables included in the first database, clustering the multiple data tables to obtain multiple data table sets; Sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence; Sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The data table migration process from the first database to the second database is performed according to the second set sequence.

[0090] The database processing device provided in this embodiment, through the cooperation of the memory 601, the processor 602, the bus 603 and the communication interface 604, clusters the multiple data tables contained in the first database based on the association relationship between the multiple data tables to obtain multiple data table sets; sorts the multiple data table sets to obtain a first set sequence, and calculates the evaluation index of each data table according to the attribute information of the data tables contained in each data table set in the first set sequence; sorts the data tables contained in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; and migrates the data tables from the first database to the second database according to the second set sequence to improve the migration efficiency of data migration between databases.

[0091] The database processing device provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0092] Furthermore, another embodiment of the present specification provides a computer-readable storage medium, wherein the computer-readable storage medium is used to store computer-executable instructions, and when the computer-executable instructions are executed by a processor, the following process is implemented: Based on the association relationship between the multiple data tables included in the first database, clustering the multiple data tables to obtain multiple data table sets; Sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence; Sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The data table migration process from the first database to the second database is performed according to the second set sequence.

[0093] The computer-readable storage medium provided in this embodiment clusters multiple data tables contained in a first database based on the association relationship between the multiple data tables to obtain multiple data table sets; sorts the multiple data table sets to obtain a first set sequence, and calculates an evaluation index for each data table based on the attribute information of the data tables contained in each data table set in the first set sequence; sorts the data tables contained in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; and migrates the data tables from the first database to the second database according to the second set sequence to improve the migration efficiency of data migration between databases.

[0094] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0095] A computer-readable storage medium provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0096] Furthermore, another embodiment of the present specification also provides a computer program product, which implements the following process when executed by a processor: Based on the association relationship between the multiple data tables included in the first database, clustering the multiple data tables to obtain multiple data table sets; Sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence; Sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The data table migration process from the first database to the second database is performed according to the second set sequence.

[0097] The computer program product provided in this embodiment clusters multiple data tables contained in a first database based on the association relationship between the multiple data tables to obtain multiple data table sets; sorts the multiple data table sets to obtain a first set sequence, and calculates an evaluation index for each data table based on the attribute information of the data tables contained in each data table set in the first set sequence; sorts the data tables contained in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; and migrates the data tables from the first database to the second database according to the second set sequence to improve the migration efficiency of data migration between databases.

[0098] A computer program product provided in an embodiment of this specification can implement each process in the aforementioned method embodiment and achieve the same functions and effects, which will not be repeated here.

[0099] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0100] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0101] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0102] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0103] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0104] Memory may include non-permanent storage in a computer-readable storage medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable storage medium.

[0105] Computer-readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer-readable storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable storage media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment or an embodiment combining software and hardware. Moreover, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program codes.

[0108] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A database processing method, characterized in that: The method comprises: Based on the association relationship between the multiple data tables included in the first database, clustering the multiple data tables to obtain multiple data table sets; Sorting the multiple data table sets to obtain a first set sequence, and calculating an evaluation index of each data table according to attribute information of the data tables included in each data table set in the first set sequence; Sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; The data table migration process from the first database to the second database is performed according to the second set sequence.

2. The database processing method according to claim 1, characterized in that: After the step of sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain the second set sequence is performed, the method further includes: Calculating a trust index of the operating environment according to the environment information of the operating environment of the second database; If the trust index is greater than a preset index threshold, the step of performing data table migration processing from the first database to the second database according to the second set sequence is executed.

3. The database processing method according to claim 1, characterized in that: The clustering process is performed on the multiple data tables based on the association relationship between the multiple data tables included in the first database to obtain multiple data table sets, including: Constructing a first table relationship graph of the plurality of data tables based on the association relationship, removing the first data table from the first table relationship graph, and obtaining a second table relationship graph; The multiple data table sets are determined according to a first data table set composed of the first data tables and a relationship diagram of the second tables.

4. The database processing method according to claim 3, characterized in that: The determining the plurality of data table sets according to the first data table set composed of the first data table and the second table relationship diagram comprises: Calculating the inflow quantity of each data table in the second table relationship graph, and screening out a second data table set consisting of second data tables whose inflow quantities meet a preset quantity threshold from the second table relationship graph; Determine a third data table corresponding to the second data table in the second table relationship diagram, and update the inflow quantity of the third data table; If the inflow quantity of the third data table meets the preset quantity threshold, a third data table set consisting of the first data table set, the second data table set and the third data table is used as the multiple data table sets.

5. The database processing method according to claim 1, characterized in that: The performing data table migration from the first database to the second database according to the second set sequence includes: According to the evaluation index of the first data table, the first data table is allocated to the data table set in the second set sequence to obtain a third set sequence; The data table migration process from the first database to the second database is performed according to the third set sequence.

6. The database processing method according to claim 5, characterized in that: The allocating the first data table to the data table set in the second set sequence according to the evaluation index of the first data table includes: Comparing the evaluation index of the first data table with the evaluation index of the last data table in the data table set; If the comparison result is that the evaluation index of the first data table is greater than the evaluation index of the last data table, the first data table is allocated to the data table set according to the evaluation index of the first data table.

7. A database processing device, characterized in that: The device comprises: A clustering processing module, configured to perform clustering processing on the multiple data tables contained in the first database based on the association relationship between the multiple data tables, so as to obtain multiple data table sets; An index calculation module, used for sorting the multiple data table sets to obtain a first set sequence, and calculating the evaluation index of each data table according to the attribute information of the data table included in each data table set in the first set sequence; A sorting processing module, used for sorting the data tables included in each data table set in the first set sequence according to the evaluation index to obtain a second set sequence; A migration processing module is used to perform data table migration processing from the first database to the second database according to the second set sequence.

8. A database processing device, characterized in that: The device includes a memory and a processor, wherein the memory stores computer executable instructions, and when the computer executable instructions are executed on the processor, the steps of the method described in any one of claims 1 to 6 can be implemented.

9. A computer-readable storage medium having computer-executable instructions stored therein, characterized in that: When the computer executable instructions are executed by a processor, the steps of the method described in any one of claims 1 to 6 can be implemented.

10. A computer program product, characterized in that The computer program product comprises a database processing program, and when the database processing program is executed by a processor, the steps of the method described in any one of claims 1 to 6 can be implemented.

Images