Database table data migration method and device

Abstract

This specification relates to the field of data processing technology, and specifically discloses a method and apparatus for migrating database table data. The method includes: obtaining a query result set to be migrated; the query result set includes table data and corresponding metadata; parsing the metadata to obtain data type information corresponding to each column of data in the table data; generating a read array based on the data type information; the read array includes a parsing object corresponding to each column of data; the parsing object encapsulates a process for parsing the table data; generating a write array based on the data type information; the write array includes a write object corresponding to each column of data; the write object encapsulates a process for setting the table data; using the read array to read and parse each column of data from the query result set; and using the write array to write the parsed data into the target database table. The above scheme can improve data migration efficiency.

Description

Technical Field

[0001] This specification relates to the field of data processing technology, and in particular to a method and device for migrating database table data. Background Art

[0002] In production and test environments, business requirements and resource transitions often require rapid migration of existing database and table data to a new database. There are many proven data migration solutions. The first approach can be performed using data migration tools. The second approach is to export the required data using a dump file and then import it into the target database. However, the first approach requires configuring the migration tool in the production environment, which imposes restrictions on operational permissions. Furthermore, it lacks flexibility and is not suitable for importing data from multiple tables. The second approach, migrating data using dump files, is time-consuming and can affect the real-time performance of production data.

[0003] To address the above issues, no effective solutions have been proposed so far. Summary of the Invention

[0004] The embodiments of this specification provide a method and apparatus for migrating database table data to solve the problem of low efficiency of data migration methods in the prior art.

[0005] This embodiment of the present invention provides a method for migrating database table data, including:

[0006] Obtaining a query result set to be migrated; the query result set includes table data and corresponding metadata; parsing the metadata to obtain data type information corresponding to each column of data in the table data;

[0007] Generate a read array based on the data type information; the read array includes parsing objects corresponding to the columns of data; the parsing objects encapsulate a process for parsing table data; generate a write array based on the data type information; the write array includes write objects corresponding to the columns of data; the write objects encapsulate a process for setting table data;

[0008] The read array is used to read each column of data from the query result set; and the write array is used to write each column of data obtained by parsing into a target database table.

[0009] In one embodiment, obtaining the query result set to be migrated includes:

[0010] receiving a data migration instruction, wherein the data migration instruction includes a query statement;

[0011] The query statement is executed to query table data from the source database table to obtain a query result set to be migrated.

[0012] In one embodiment, executing the query statement to query table data from a source database table includes:

[0013] Establish a connection with the source database through the JDBC API interface;

[0014] The query statement is executed to query table data from a source database table of the source database.

[0015] In one embodiment, the data type information includes at least one of the following: a date data type, a numeric data type, a character data type, and a string data type.

[0016] In one embodiment, the metadata is parsed to obtain data type information corresponding to each column of data in the table data, including:

[0017] The metadata is parsed to obtain data type information and column information corresponding to each column of data in the table data; the column information includes at least one of the following: column name, column size, and whether the column is empty.

[0018] In one embodiment, generating a read array based on the data type information includes:

[0019] Generate a parsing object corresponding to each column of data based on the data type information corresponding to each column of data;

[0020] The column identifiers and parsing objects corresponding to the columns of data are associated and stored in an array to obtain a read array.

[0021] In one embodiment, generating a write array based on the data type information includes:

[0022] Generate a write object corresponding to each column of data based on the data type information corresponding to each column of data;

[0023] The column identifiers corresponding to the columns of data and the write objects are associated and stored in an array to obtain a write array.

[0024] In one embodiment, reading each column of data from the query result set using the read array includes:

[0025] Based on the column identifier corresponding to each column of data, the corresponding parsing object in the read array is called to parse the each column of data.

[0026] In one embodiment, using the write array to write each column of data obtained by parsing into the target database table includes:

[0027] Based on the column identifier corresponding to each column of data obtained by parsing, the corresponding write object in the write array is called to set each column of data obtained by parsing, so as to write the data into the target database table.

[0028] In one embodiment, the query result set includes multiple query result sets, and each query result set carries a corresponding source database identifier and a target database identifier;

[0029] Accordingly, generating a read array based on the data type information includes:

[0030] A read array corresponding to each query result set is generated based on the data type information corresponding to each query result set; the read array carries a corresponding source database identifier and a target database identifier.

[0031] The embodiment of this specification also provides a database table data migration device, including:

[0032] An acquisition module is used to acquire a query result set to be migrated; the query result set includes table data and corresponding metadata; the metadata is parsed to obtain data type information corresponding to each column of data in the table data;

[0033] a generation module, configured to generate a read array based on the data type information; the read array including parsing objects corresponding to the respective columns of data; the parsing objects encapsulating a process for parsing table data; and a write array based on the data type information; the write array including write objects corresponding to the respective columns of data; the write objects encapsulating a process for setting table data.

[0034] The writing module is used to read each column of data from the query result set using the reading array; and write each column of data obtained by parsing into the target database table using the writing array.

[0035] In one embodiment, the acquisition module is specifically configured to:

[0036] receiving a data migration instruction, wherein the data migration instruction includes a query statement;

[0037] The query statement is executed to query table data from the source database table to obtain a query result set to be migrated.

[0038] An embodiment of this specification also provides a computer device, including a processor and a memory for storing processor-executable instructions, wherein when the processor executes the instructions, the steps of the database table data migration method described in any of the above embodiments are implemented.

[0039] An embodiment of this specification further provides a computer-readable storage medium having computer instructions stored thereon, which, when executed by a processor, implement the steps of the database table data migration method described in any of the above embodiments.

[0040] The embodiments of this specification also provide a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the database table data migration method described in any of the above embodiments.

[0041] In an embodiment of the present specification, a database table data migration method is provided. The method can obtain a query result set to be migrated, wherein the query result set includes table data and corresponding metadata, parse the metadata to obtain data type information corresponding to each column of data in the table data, generate a read array based on the data type information, the read array includes a parsing object corresponding to each column of data, and the parsing object encapsulates the process for parsing the table data. Based on the data type information, a write array is generated, wherein the write array includes a write object corresponding to each column of data, and the write object encapsulates the process for setting the table data. The read array is used to read each column of data from the query result set, and the write array is used to write the parsed column of data to the target database table. In this solution, when a query is first executed for a table migration data, the metadata in the query result set is parsed to generate a read array including parsing objects for reading the query result set. When parsing the query result set subsequently, the parsing is performed directly by quickly obtaining the parsing objects in the read array, so that when parsing each column of data, there is no need to determine which field is currently being parsed or which type to use. For data insertion, based on a similar principle, by parsing the metadata in the query result set, a write array is generated, including write objects that specify how to set the data type. When writing data, the corresponding write object is directly called, eliminating the need to determine which field is currently being written and how to set the data type when writing each column of data. Compared to the existing technology that reads and writes by determining the field type, the time complexity is always (number of field types × number of fields × total data volume). However, the time complexity of this solution is (number of field types × number of fields + total data volume), which is approximately equal to the total data volume in actual situations. This can significantly improve execution efficiency. By batch submitting and processing write data, the execution plan parsing on the database side only needs to be executed once, which can greatly improve the speed of database table migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The accompanying drawings described herein are used to provide a further understanding of this specification, constitute a part of this specification, and do not constitute a limitation of this specification. In the accompanying drawings:

[0043] Figure 1A schematic diagram showing an application scenario of a database table data migration method in an embodiment of this specification;

[0044] Figure 2 A flowchart of a method for migrating database table data in an embodiment of this specification is shown;

[0045] Figure 3 A flowchart of a method for migrating database table data in an embodiment of this specification is shown;

[0046] Figure 4 A schematic diagram of a database table data migration device in an embodiment of this specification is shown;

[0047] Figure 5 A schematic diagram of a computer device in an embodiment of the present specification is shown. DETAILED DESCRIPTION

[0048] The principles and spirit of this specification will be described below with reference to several exemplary embodiments. It should be understood that these embodiments are provided solely to enable those skilled in the art to better understand and implement this specification, and are not intended to limit the scope of this specification in any way. Rather, these embodiments are provided to make this specification more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0049] Those skilled in the art will appreciate that the embodiments of this specification may be implemented as a system, device, method, or computer program product. Therefore, the disclosure herein may be implemented in the following forms: entirely in hardware, entirely in software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0050] The embodiments of this specification provide a method for migrating database table data. Figure 1 Schematic diagram showing an application scenario of a database table data migration method in an embodiment of this specification. Figure 1 As shown, the method in this embodiment can be applied to a server. The server can be a single server, a server cluster, or a cloud server, etc. The specific composition is not limited in this application.

[0051] The server may receive a data migration instruction, which may include a query statement. The server may execute the query statement to query table data from a source database table and obtain a query result set to be migrated. The query result set includes the table data and corresponding metadata.

[0052] The server may parse the metadata to obtain data type information corresponding to each column of data in the table data.

[0053] The server may generate a read array based on the data type information. The read array includes parsing objects corresponding to each column of data. The parsing objects encapsulate a process for parsing table data.

[0054] In one embodiment, the server may obtain a corresponding parsing object based on the data type information of each column of data, and then the server may associate the parsing object corresponding to each column of data with the column identifier and store it in an array to obtain a read array.

[0055] The server may generate a write array based on the data type information. The write array includes write objects corresponding to the columns of data. The write objects encapsulate a process for setting table data.

[0056] In one embodiment, the server may obtain a corresponding write object based on the data type information of each column of data, and then the server may associate the write object corresponding to each column of data with the column identifier and store it in an array to obtain a write array.

[0057] The server may use the read array to read each column of data from the query result set, and may also use the write array to write each column of data obtained by parsing into the target database table.

[0058] In one embodiment, the server may call the corresponding parsing object in the read array based on the column identifier corresponding to each column of data to parse the column of data. The server may call the corresponding writing object in the write array based on the column identifier corresponding to each column of data obtained by parsing to set each column of data obtained by parsing, so as to write it into the target database table.

[0059] Figure 2 A flowchart of a method for migrating database table data in one embodiment of this specification is shown. Although this specification provides method operation steps or device structures as shown in the following embodiments or drawings, more or fewer operation steps or module units may be included in the method or device based on routine or no creative labor. In the steps or structures where there is no necessary causal relationship logically, the execution order of these steps or the module structure of the device is not limited to the execution order or module structure described in the embodiments of this specification and shown in the drawings. When the method or module structure is applied to an actual device or terminal product, it can be connected in accordance with the method or module structure shown in the embodiments or drawings for sequential execution or parallel execution (for example, a parallel processor or multi-threaded processing environment, or even a distributed processing environment).

[0060] Specifically, if Figure 2 As shown, a database table data migration method provided in one embodiment of this specification may include the following steps:

[0061] Step S201 , obtaining a query result set to be migrated; the query result set includes table data and corresponding metadata; parsing the metadata to obtain data type information corresponding to each column of data in the table data.

[0062] The method in this embodiment can be applied to a server. The server can obtain a query result set to be migrated. The query result set can include the queried table data and the corresponding source data. The server can parse the metadata corresponding to the query result set to obtain data type information corresponding to each column of data in the table data.

[0063] In some embodiments of the present specification, obtaining a query result set to be migrated may include: receiving a data migration instruction; the data migration instruction includes a query statement; and executing the query statement to query table data from a source database table to obtain the query result set to be migrated.

[0064] The server may receive a data migration instruction. The data migration instruction may include a query statement. The server may execute the query statement to query table data from a source database table and obtain a query result set to be migrated.

[0065] In some embodiments of the present specification, executing the query statement to query table data from the source database table may include: establishing a connection with the source database through a JDBC API interface; and executing the query statement to query table data from the source database table of the source database.

[0066] Specifically, the data migration instruction may also include a source database identifier and a target database identifier. The server may establish a connection with the source database via a JDBC API interface. After the connection is established, a query statement may be executed to query table data from a source database table of the source database.

[0067] In some embodiments of this specification, the data type information may include at least one of the following: a date data type, a numeric data type, a character data type, and a string data type. The data type of each column of data in the query result set may be one of the date data type, the numeric data type, the character data type, and the string data type.

[0068] In some embodiments of the present specification, parsing the metadata to obtain data type information corresponding to each column of data in the table data may include: parsing the metadata to obtain data type information and column information corresponding to each column of data in the table data; the column information includes at least one of the following: column name, column size, and whether the column is empty.

[0069] After parsing the metadata, you can also obtain column information including column name, column size, and whether the column is empty, so that when writing to the target database table, you can set the column information accordingly. You can set the corresponding column information at the same time as setting the data type.

[0070] Step S202: Generate a read array based on the data type information; the read array includes parsing objects corresponding to the columns of data; the parsing objects encapsulate a process for parsing table data; generate a write array based on the data type information; the write array includes write objects corresponding to the columns of data; the write objects encapsulate a process for setting table data.

[0071] After parsing the source data to obtain data type information, the server can generate a read array and a write array based on the data type information. The read array can include a parsing object corresponding to each column of data in the table data in the query result set. The parsing object can encapsulate the process for parsing the corresponding table data. The write array can include a write object corresponding to each column of data. The write object encapsulates the process for setting the table data.

[0072] Step S203 , using the read array to read each column of data from the query result set; and using the write array to write each column of data obtained by parsing into a target database table.

[0073] After obtaining the read array and write data, the server can read each column of data from the query result set to obtain the parsed data. The server can also use the write array to write the parsed data into the target database table.

[0074] In the above embodiment, when a query is first executed to migrate data for a table, the metadata in the query result set is parsed to generate a read array containing parsing objects for reading the query result set. Subsequent parsing of the query result set is performed directly by quickly obtaining the parsing objects in the read array. This eliminates the need to determine which field is currently being parsed and which type to use for each column of data. For data insertion, based on a similar principle, the metadata in the query result set is parsed to generate a write array containing write objects that specify how to set the data type. When writing data, the corresponding write objects are directly called, eliminating the need to determine which field is currently being parsed and how to set the data type for each column of data. Compared to the prior art method of reading and writing by determining the field type, which has a time complexity of (number of field types × number of fields × total data volume), the time complexity of this solution is (number of field types × number of fields + total data volume), which is approximately equal to the total data volume in actual situations. This significantly improves execution efficiency. By batch-submitting and processing the written data, the database-side execution plan parsing only needs to be executed once, significantly improving the speed of database table migration.

[0075] In some embodiments of the present specification, generating a read array based on the data type information may include: generating a parsing object corresponding to each column of data based on the data type information corresponding to each column of data; and associating the column identifier and the parsing object corresponding to each column of data and storing them in an array to obtain a read array.

[0076] Specifically, the server may generate a parsing object corresponding to each column of data based on the data type information corresponding to each column of data.

[0077] In one embodiment, a server may store a parsing object library, which may store multiple parsing objects and the corresponding relationship between each of the multiple parsing objects and data type information. After obtaining the type information corresponding to each column of data, the corresponding parsing object can be queried from the parsing object library.

[0078] After obtaining the parsing object corresponding to each column of data, the server can associate the column identifier corresponding to each column of data with the parsing object and store it in an array to obtain a read array. In this way, a parsing array corresponding to the query result set can be generated.

[0079] In some embodiments of the present specification, generating a write array based on the data type information may include: generating a write object corresponding to each column of data based on the data type information corresponding to each column of data; associating the column identifier and the write object corresponding to each column of data and storing them in an array to obtain a write array.

[0080] Specifically, the server may generate a write object corresponding to each column of data based on the data type information corresponding to each column of data.

[0081] In one embodiment, a server may store a write object library, which may store multiple write objects and a correspondence between each of the multiple write objects and data type information. After obtaining the type information corresponding to each column of data, the corresponding write object may be queried from the write object library.

[0082] After obtaining the write object corresponding to each column of data, the server can associate the column identifier corresponding to each column of data with the write object and store it in an array to obtain a write array. In this way, a write array corresponding to the query result set can be generated.

[0083] In some embodiments of the present specification, reading each column of data from the query result set using the read array may include: calling a corresponding parsing object in the read array to parse each column of data based on a column identifier corresponding to each column of data.

[0084] Specifically, when reading data from a query result set, the corresponding parsing object in the read array can be called based on the column identifier corresponding to each column of data in the query result set to parse the data in each column, thereby achieving the reading of each column of data in the query result set. In other words, the column index constructed by the column identifier can be used to locate the required parsing object (quickly locating the parsing object based on the read array subscript). In this way, the parsing object can be quickly located to parse the column data, thereby achieving the reading of the query result set.

[0085] In some embodiments of the present specification, using the write array to write each column of data obtained by parsing into the target database table may include: calling the corresponding write object in the write array based on the column identifier corresponding to each column of data obtained by parsing to set each column of data obtained by parsing so as to write into the target database table.

[0086] Specifically, when writing data from a query result set, the corresponding write object in the write array can be called based on the column identifier corresponding to each column of data obtained by parsing to set the parsed column data for writing to the target database table. In other words, the desired write object can be located using the column index constructed from the column identifier (quickly locating the write object based on the write array subscript). This method allows for quick locating of the write object, setting the column data type and value, and thus completing the writing of the query result set.

[0087] In some embodiments of the present specification, the query result set may include multiple query result sets, each of the multiple query result sets carries a corresponding source database identifier and a target database identifier; accordingly, generating a read array based on the data type information may include: generating a read array corresponding to each query result set based on the data type information corresponding to each query result set; the read array carries a corresponding source database identifier and a target database identifier.

[0088] Data migration can be performed in batches. Specifically, a server can receive a data migration instruction, which can include multiple query statements, each of which corresponds to a source database identifier and a target database identifier. When executing a query statement, the server can connect to the source database corresponding to the source database identifier corresponding to the query statement and execute the query statement.

[0089] After obtaining the query result sets corresponding to each of the multiple query statements, metadata corresponding to each query result set can be parsed to obtain data type information corresponding to each query result set. Based on the data type information corresponding to each query result set, read data and write arrays corresponding to each query result set can be generated. The read arrays and write arrays can carry corresponding source database identifiers and target database identifiers.

[0090] When performing data migration, the server can use the read array and write array corresponding to each query result set to read and write the corresponding query result set, realizing data migration of multiple query result sets. In this way, data can be transferred in batches and multi-threaded parallel processing can be performed when necessary.

[0091] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on the differences from other embodiments. For details, please refer to the description of the aforementioned related processing embodiments, and no further description is given here.

[0092] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0093] The above method is described below with reference to a specific embodiment. However, it should be noted that this specific embodiment is only for better illustrating this specification and does not constitute an improper limitation to this specification.

[0094] This specific embodiment provides a method for migrating database table data. This method primarily uses JDBC to handle data migration, effectively accommodating business needs. It allows for flexible import of single-table and multi-table joint query data, and boasts high scalability. Furthermore, by pre-setting the field read and write methods, import speed is significantly improved.

[0095] Please refer to Figure 3 , shows a flow chart of the database table data migration method in this specific embodiment. Figure 3 As shown, you can query the source table data to obtain a query result set ResultSet. The query result set ResultSet can include metadata metaData. The metadata metaData can be parsed to obtain the field type information corresponding to each list data.

[0096] Based on the table field type information, you can generate a readerList array (readerList) and a writerList array (writerList). The readerList array (readerList) can set the field's reader class based on the field type information to implement field parsing. The writerList array (writerList) can set the field's writer class based on the field type information to implement field writing. The reader class and writer class are also referred to as parsing objects and writing objects. The reader class can be used to obtain data values. The writer class can be used to set data values, implementing batch data insertion into the target table.

[0097] In this specific embodiment, when a query for migrating data of a certain table is executed for the first time, an array of field resolution objects used for reading the result set is generated by parsing the result set metadata, and the fields are arranged in order. In subsequent result set parsing, the field resolution object is quickly obtained directly through the index of the current field (by counting), eliminating the need for the subsequent judgment operation cost of each row of data parsing to determine which field is currently and which type needs to be used for parsing. For data insertion, a similar principle is adopted, and an array of objects for how to set the data type is generated for the first time. When a specific result set is set, the object used is directly located by the column index (based on the array subscript to quickly locate the object), eliminating the need for the subsequent judgment operation cost of each row of data parsing to determine which field is currently and which type needs to be used for parsing.

[0098] This solution allows you to quickly retrieve and set table field types for the final data write. First, query the ResultSet's metaData property to retrieve the number and types of table fields. Then, for each table field, set the readerList and writerList objects. Each data type has a corresponding implementation class for reading and precompiling field values. This eliminates the need to determine the type of each field when reading a ResultSet row; the implementation class can be directly called to retrieve the field value. Similarly, when writing data, precompiling data does not require determining the value type; the corresponding implementation class can be directly called. While reading and writing data based on field type determination takes a time complexity of (number of field types × number of fields × total data volume), this method's time complexity is (number of field types × number of fields + total data volume), which is approximately equal to the total data volume in real-world scenarios, significantly improving execution efficiency. Finally, by batch-submitting data for write processing, the database only needs to perform execution plan parsing once, significantly improving database write speed.

[0099] Based on the same inventive concept, the embodiments of this specification also provide a database table data migration device, as described in the following embodiments. Since the principle of solving the problem by the database table data migration device is similar to that of the database table data migration method, the implementation of the database table data migration device can refer to the implementation of the database table data migration method, and the repeated parts will not be repeated. As used below, the term "unit" or "module" can be a combination of software and / or hardware that implements the predetermined function. Although the device described in the following embodiments is preferably implemented in software, implementation by hardware, or a combination of software and hardware, is also possible and conceivable. Figure 4 This is a structural block diagram of a database table data migration device according to an embodiment of this specification. Figure 4 As shown, it includes: an acquisition module 401, a generation module 402 and a writing module 403. The structure is described below.

[0100] The acquisition module 401 is used to acquire a query result set to be migrated; the query result set includes table data and corresponding metadata; the metadata is parsed to obtain data type information corresponding to each column of data in the table data.

[0101] The generation module 402 is used to generate a read array based on the data type information; the read array includes the parsing objects corresponding to the columns of data; the parsing objects encapsulate the process for parsing the table data; and generate a write array based on the data type information; the write array includes the write objects corresponding to the columns of data; the write objects encapsulate the process for setting the table data.

[0102] The writing module 403 is used to read each column of data from the query result set using the read array; and write each column of data obtained by parsing into the target database table using the write array.

[0103] In some embodiments of this specification, the acquisition module may specifically include:

[0104] A receiving unit, configured to receive a data migration instruction, wherein the data migration instruction includes a query statement;

[0105] The execution unit is configured to execute the query statement to query table data from a source database table and obtain a query result set to be migrated.

[0106] In some embodiments of this specification, the execution unit may be specifically configured to:

[0107] Establish a connection with the source database through the JDBC API interface;

[0108] The query statement is executed to query table data from a source database table of the source database.

[0109] In some embodiments of the present specification, the data type information includes at least one of the following: a date data type, a numeric data type, a character data type, and a string data type.

[0110] In some embodiments of this specification, the acquisition module further includes:

[0111] The parsing unit is used to parse the metadata to obtain data type information and column information corresponding to each column of data in the table data; the column information includes at least one of the following: column name, column size, and whether the column is empty.

[0112] In some embodiments of this specification, the generation module may be specifically used to:

[0113] Generate a parsing object corresponding to each column of data based on the data type information corresponding to each column of data;

[0114] The column identifiers and parsing objects corresponding to the columns of data are associated and stored in an array to obtain a read array.

[0115] In some embodiments of this specification, the generation module may be specifically used to:

[0116] Generate a write object corresponding to each column of data based on the data type information corresponding to each column of data;

[0117] The column identifiers corresponding to the columns of data and the write objects are associated and stored in an array to obtain a write array.

[0118] In some embodiments of this specification, the writing module may be specifically used to:

[0119] Based on the column identifier corresponding to each column of data, the corresponding parsing object in the read array is called to parse the each column of data.

[0120] In some embodiments of this specification, the writing module may also be specifically used to:

[0121] Based on the column identifier corresponding to each column of data obtained by parsing, the corresponding write object in the write array is called to set each column of data obtained by parsing, so as to write the data into the target database table.

[0122] In some embodiments of the present specification, the query result set includes multiple query result sets, and each query result set carries a corresponding source database identifier and a target database identifier;

[0123] Accordingly, the generation module can be used specifically for:

[0124] A read array corresponding to each query result set is generated based on the data type information corresponding to each query result set; the read array carries a corresponding source database identifier and a target database identifier.

[0125] From the above description, it can be seen that the embodiments of this specification achieve the following technical effects: when a query for migrating data for a table is executed for the first time, a read array including a parsing object for reading the query result set is generated by parsing the metadata in the query result set. When parsing the query result set later, the parsing is performed directly by quickly obtaining the parsing object in the read array, so that when parsing each column of data later, there is no need to determine which field is currently and which type to use for parsing. For data insertion, based on a similar principle, a write array including a write object for setting the data type is generated by parsing the metadata in the query result set. When writing data, the corresponding write object is directly called, so that when writing each column of data later, there is no need to determine which field is currently and how to set the written data type. Compared with the prior art of reading and writing by judging the field type, the time complexity is always the number of field types × the number of fields × the total amount of data. However, the time complexity of this solution is (number of field types × the number of fields + the total amount of data), which is approximately equal to the total amount of data in actual situations, which can greatly improve execution efficiency. By batch submitting and processing the written data, the execution plan parsing on the database side only needs to be executed once, which can greatly improve the speed of database table migration.

[0126] This specification also provides a computer device. Figure 5The diagram shows the structure of a computer device based on the database table data migration method provided in an embodiment of this specification. The computer device may include an input device 51, a processor 52, and a memory 53. The memory 53 is used to store processor-executable instructions. When the processor 52 executes these instructions, the steps of the database table data migration method described in any of the above embodiments are implemented.

[0127] In this embodiment, the input device can specifically be one of the primary devices for exchanging information between a user and a computer system. The input device can include a keyboard, mouse, camera, scanner, light pen, handwriting input tablet, voice input device, etc.; the input device is used to input raw data and programs for processing these data into the computer. The input device can also receive data transmitted from other modules, units, and devices. The processor can be implemented in any appropriate manner. For example, the processor can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (such as software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers, etc. The memory can specifically be a memory device used to store information in modern information technology. The memory can include multiple levels. In digital systems, anything that can store binary data can be considered a memory device. In integrated circuits, a circuit with storage functionality that does not have a physical form is also called a memory device, such as a RAM or FIFO. In systems, a physical storage device is also called a memory device, such as a memory stick or a TF card.

[0128] In this embodiment, the specific functions and effects achieved by the computer device can be explained in comparison with other embodiments and will not be repeated here.

[0129] In an embodiment of the present specification, a computer storage medium based on the database table data migration method is further provided. The computer storage medium stores computer program instructions, and when the computer program instructions are executed by a processor, the steps of the database table data migration method described in any of the above embodiments are implemented.

[0130] In this embodiment, the storage medium includes, but is not limited to, random access memory (RAM), read-only memory (ROM), cache, hard disk drive (HDD), or memory card. The memory can be used to store computer program instructions. The network communication unit can be an interface configured in accordance with the standards specified by the communication protocol for network connection communication.

[0131] In this embodiment, the functions and effects specifically implemented by the program instructions stored in the computer storage medium can be explained in comparison with other embodiments and will not be repeated here.

[0132] An embodiment of this specification also provides a computer program product, including a computer program / instruction, which, when executed by a processor, implements the steps of the application deployment method based on a low-code platform described in any of the above embodiments.

[0133] Obviously, those skilled in the art should understand that the various modules or steps of the above-mentioned embodiments of this specification can be implemented using a general-purpose computing device, they can be concentrated on a single computing device, or distributed across a network composed of multiple computing devices. Alternatively, they can be implemented using program code executable by the computing device, so that they can be stored in a storage device and executed by the computing device. In some cases, the steps shown or described can be performed in a different order than herein, or they can be made into separate integrated circuit modules, or multiple modules or steps can be made into a single integrated circuit module for implementation. Thus, the embodiments of this specification are not limited to any specific combination of hardware and software.

[0134] It should be understood that the above description is intended to be illustrative and not limiting. Numerous embodiments and applications beyond the examples provided will be readily apparent to those skilled in the art upon reading the above description. Therefore, the scope of this specification should not be determined with reference to the above description, but rather with reference to the preceding claims, along with the full scope of equivalents to which such claims are entitled.

[0135] The above description is merely a preferred embodiment of this specification and is not intended to limit this specification. Those skilled in the art will readily appreciate that various modifications and variations to the embodiments of this specification are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of this specification shall be within the scope of protection of this specification.

Claims

1. A database table data migration method, characterized in that: include: Get the query result set to be migrated; The query result set includes table data and corresponding metadata; Parsing the metadata to obtain data type information corresponding to each column of data in the table data; Generate a read array based on the data type information; the read array includes parsing objects corresponding to the columns of data; the parsing objects encapsulate a process for parsing table data; generate a write array based on the data type information; the write array includes write objects corresponding to the columns of data; the write objects encapsulate a process for setting table data; The read array is used to read each column of data from the query result set; and the write array is used to write each column of data obtained by parsing into a target database table.

2. The database table data migration method according to claim 1, characterized in that: Get the query result set to be migrated, including: receiving a data migration instruction, wherein the data migration instruction includes a query statement; The query statement is executed to query table data from the source database table to obtain a query result set to be migrated.

3. The database table data migration method according to claim 2, characterized in that: Executing the query statement to query table data from the source database table includes: Establish a connection with the source database through the JDBC API interface; The query statement is executed to query table data from a source database table of the source database.

4. The database table data migration method according to claim 1, characterized in that: The data type information includes at least one of the following: a date data type, a number data type, a character data type, and a string data type.

5. The database table data migration method according to claim 1, characterized in that: Parse the metadata to obtain data type information corresponding to each column of data in the table data, including: The metadata is parsed to obtain data type information and column information corresponding to each column of data in the table data; the column information includes at least one of the following: column name, column size, and whether the column is empty.

6. The database table data migration method according to claim 1, characterized in that: Generating a read array based on the data type information includes: Generate a parsing object corresponding to each column of data based on the data type information corresponding to each column of data; The column identifiers and parsing objects corresponding to the columns of data are associated and stored in an array to obtain a read array.

7. The database table data migration method according to claim 1, characterized in that: Generating a write array based on the data type information includes: Generate a write object corresponding to each column of data based on the data type information corresponding to each column of data; The column identifiers corresponding to the columns of data and the write objects are associated and stored in an array to obtain a write array.

8. The database table data migration method according to claim 6, characterized in that: Reading each column of data from the query result set using the read array includes: Based on the column identifier corresponding to each column of data, the corresponding parsing object in the read array is called to parse the each column of data.

9. The database table data migration method according to claim 7, characterized in that: The write array is used to write each column of data obtained by parsing into the target database table, including: Based on the column identifier corresponding to each column of data obtained by parsing, the corresponding write object in the write array is called to set each column of data obtained by parsing, so as to write the data into the target database table.

10. The database table data migration method according to claim 1, characterized in that: The query result set includes multiple query result sets, each of the multiple query result sets carries a corresponding source database identifier and a target database identifier; Accordingly, generating a read array based on the data type information includes: A read array corresponding to each query result set is generated based on the data type information corresponding to each query result set; the read array carries a corresponding source database identifier and a target database identifier.

11. A database table data migration device, characterized in that: include: The acquisition module is used to obtain the query result set to be migrated; The query result set includes table data and corresponding metadata; Parsing the metadata to obtain data type information corresponding to each column of data in the table data; a generation module, configured to generate a read array based on the data type information; the read array including parsing objects corresponding to the respective columns of data; the parsing objects encapsulating a process for parsing table data; and a write array based on the data type information; the write array including write objects corresponding to the respective columns of data; the write objects encapsulating a process for setting table data. The writing module is used to read each column of data from the query result set using the reading array; and write each column of data obtained by parsing into the target database table using the writing array.

12. The database table data migration device according to claim 11, characterized in that: The acquisition module is specifically used for: receiving a data migration instruction, wherein the data migration instruction includes a query statement; The query statement is executed to query table data from the source database table to obtain a query result set to be migrated.

13. A computer device, characterized in that: The method comprises a processor and a memory for storing processor-executable instructions, wherein the processor implements the steps of the method according to any one of claims 1 to 10 when executing the instructions.

14. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

15. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 10 are implemented.

Images