Data migration method and system from multi-relational database to non-relational database

Abstract

The present invention provides a data migration method and system from a multi-relational database to a non-relational database, wherein the method comprises constructing a multi-column tree structure about first structure data in a first database, wherein the tree structure comprises at least a volume structure mapping and a storage pool mapping; the volume structure mapping comprises first mapping data and sub-volume information, and the storage pool mapping comprises second mapping data and storage pool data; allocating mapping space such as the volume structure mapping and the storage pool mapping to the second database; writing any first sub-volume into the mapping space, and after writing a value into the metadata of the second sub-volume of the mapping space, deleting the original mapping corresponding to the newly added volume in the volume structure mapping. The present invention realizes data migration from a multi-relational database to a non-relational database as well as backup migration and backup encryption during data migration.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a data migration system from a multi-relational database to a non-relational database. Background Art

[0002] At present, with the development of computer-related technologies, information systems are applied in many fields. With the development of information systems, information systems are also constantly upgrading. As the storage medium in the information system, the database plays a vital role, so the upgrade or replacement of the database will have a significant impact on the information system.

[0003] During the database replacement process, due to the differences in data formats, data types, and data syntax between different databases, it will greatly increase the difficulty of migrating schemas and data between databases. Whether the schema and data in the source data can be completely and correctly migrated to another database becomes a key factor in determining whether the information system can be updated and replaced.

[0004] In particular, a multi-relational database is migrated to a non-relational database. In view of this, a method and system for migrating data from a multi-relational database to a non-relational database are provided. Summary of the invention

[0005] The present invention provides a method and system for data migration from a multi-relational database to a non-relational database, which realizes data migration from a multi-relational database to a non-relational database and backup migration and backup encryption during the data migration process.

[0006] A first aspect of the present invention provides a method for converting a multi-relational database to a non-relational database, comprising the following steps:

[0007] S1. Obtaining first structural data from a first database;

[0008] S2. Construct a multi-column tree structure about the first structure data, wherein the multi-column tree structure at least includes a volume mapping tree and a storage pool mapping tree, wherein the volume mapping tree includes first mapping data and sub-volume data, wherein the storage pool mapping includes second mapping data and storage pool data, wherein the storage pool data includes a plurality of volume mapping trees and mappings set accordingly;

[0009] S3, obtaining second structure data of the second database, and allocating a mapping space required for executing the migration to the second structure data;

[0010] S4, when writing any first sub-volume from the volume mapping tree to the mapping space, constructing a second sub-volume corresponding to the first sub-volume in the mapping space;

[0011] S5, writing the metadata of the first volume to the second volume, obtaining the first mapping data of the first volume in the volume mapping tree, and writing the first mapping data and the second mapping data to a preset position in the metadata of the second volume after the second structure data is completely written to the first volume;

[0012] S6, deleting the first mapping data corresponding to the first volume in the volume mapping tree, and updating the volume mapping tree;

[0013] S6, deleting the second mapping data corresponding to the first volume in the storage pool mapping, and updating the storage pool mapping tree;

[0014] S7, repeatedly executing S4-S6 until any of the volume mapping trees does not contain any of the first mapping data, and deleting the volume mapping tree;

[0015] S8. Repeat S4-S7 until any storage pool mapping does not include a volume mapping tree, and then delete the storage pool mapping.

[0016] S9. Repeat S4-S8 until the multi-column tree structure does not include any mapping data.

[0017] Furthermore, the volume mapping tree is specifically configured as follows:

[0018] The first mapping data includes at least a first mapping and a second mapping;

[0019] The volume data includes a main volume and multiple sub-volumes. A first mapping is set from the main volume to each of the sub-volumes, and a second mapping is set between each of the sub-volumes. If the sub-volume is set to a next-level volume, the mapping of the sub-volume to the next-level volume and the mapping between the next-level volumes are sequentially added and stored in the first mapping data.

[0020] Furthermore, the storage pool mapping is specifically set to:

[0021] The second mapping data at least includes a third mapping and a fourth mapping,

[0022] The third mapping is a mapping index value stored in the same volume mapping tree between different storage pools, and the third mapping includes index values ​​for the first mapping and the second mapping, and the fourth mapping is an index value for multiple volume mapping trees in the same storage pool;

[0023] The storage pool data is a set of all data written to the storage pool.

[0024] Further, when performing migration, partial migration or complete migration is performed in the multi-column tree structure according to the migration requirements;

[0025] If the migration requirement is partial migration, executing the designated sub-volumes in the multi-column tree structure until the migration is completed;

[0026] If the migration requirement is a complete migration of the volume, the process ends at S7;

[0027] If the migration requirement is a complete migration of the storage pool, the process ends at S8;

[0028] If the migration requirement is a complete migration of the first database, the process ends at S9;

[0029] When performing the partial migration or the complete migration, the migration is performed as a random migration or a sequential migration.

[0030] Further, when any step of S4-S9 is completed, the following steps are performed:

[0031] Establishing a backup area in the first database and / or the second database;

[0032] Writing metadata of the specified first volume in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume;

[0033] After restoring the value of the second volume written to the preset position according to the first mapping data and the second mapping data;

[0034] The error rates of the restored second volume and the backup volume in the backup area are checked. If the error rate is higher than a preset threshold, the backup volume is written into the tree structure and all steps before S4-S9 are executed accordingly.

[0035] Furthermore, the specific steps of executing the migration requirement and writing the first mapping data and the second mapping data into a preset position in the metadata of the second volume are:

[0036] According to the writing of the first mapping data and the second mapping data, a first array of volume mapping data of the multi-column tree structure and a second array of storage pool mapping data are obtained;

[0037] After encrypting the first array and the second array, encapsulate them into the second volume along with the first mapping data and the second mapping data;

[0038] An encryption log corresponding to the first array and the second array is generated, wherein the encryption log includes an encryption key.

[0039] Furthermore, the first array is

[0040] After any of the first mapping data is deleted, a placeholder value is written to the position of the mapping data in the volume mapping tree to form a merged array with the value of the position of the first mapping data;

[0041] The second array is

[0042] After any of the second mapping data is deleted, a placeholder value is written to the position of the mapping data in the storage pool mapping tree to form a merged array with the value of the position with the second mapping data.

[0043] A second aspect of the present invention provides a data migration system from a multi-relational database to a non-relational database, comprising:

[0044] A first acquisition module is configured to obtain first structural data and migration requirements of a first database;

[0045] A second acquisition module is configured to obtain second structure data and a mapping space of a second database;

[0046] a migration module, configured to construct a multi-column tree structure about the first structure data, the multi-column tree structure at least comprising a volume mapping tree and a storage pool mapping tree, the volume mapping tree comprising first mapping data and sub-volume data, the storage pool mapping comprising second mapping data and storage pool data, wherein the storage pool data comprises a plurality of volume mapping trees and mappings set corresponding thereto, and configured to migrate the first sub-volume specified in the multi-column tree structure to the mapping space according to the migration requirement;

[0047] The backup module is configured to, when any migration requirement is executed and any first volume specified in the migration requirement is completed, establish a backup area in the first database and / or the second database, write metadata of the first volume specified in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume, and then restore the value of the second volume written to the preset position according to the first mapping data and the second mapping data, check the error rate of the restored second volume and the backup volume in the backup area, and if the error rate is higher than a preset threshold, write the backup volume into the tree structure and then execute the steps before the migration requirement is executed;

[0048] The encryption log module is configured to obtain a first array of volume mapping data of the multi-column tree structure and a second array of storage pool mapping data according to the writing of the first mapping data and the second mapping data; encrypt the first array and the second array and then package them into the second volume along with the first mapping data and the second mapping data; generate an encryption log corresponding to the first array and the second array, wherein the encryption log includes an encryption key.

[0049] According to a third aspect of the present invention, there is provided an electronic device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for migrating data from a multi-relational database to a non-relational database according to the first aspect is implemented.

[0050] A fourth aspect of the present invention provides a storage medium having a computer program stored thereon, which, when executed by a processor, is suitable for implementing the data migration method from a multi-relational database to a non-relational database as in the first aspect.

[0051] The present invention has the following advantages over the prior art:

[0052] 1. The present invention realizes data migration from the table structure in a multi-relational database to the non-table structure of a non-relational database, establishes a mapping tree based on the table structure in the first database, migrates data based on the mapping tree, and migrates the mapping relationship in the table structure as a data package to the non-relational database. In the process, the dynamic change of the mapping relationship is used as a dynamic key to encrypt and package each migrated data. The dynamic key also records the log status of the migrated data to ensure data security, thereby realizing the log record and backup encryption of the migration process.

[0053] 2. The present invention constructs an encryption log for migration, wherein the storage of the encryption log includes an encryption key for migration. As a dual authentication method, the generation of the encryption key adopts a general key rule and changes dynamically with the length of the value of the first mapping data and / or the second mapping data in the migration requirement. As an example, when migration is performed according to the migration requirement, the initial key is distributed to the first database and the second database. Then, according to the value deleted in any mapping tree during the migration, the first database deletes the value corresponding to the mapping tree position in the initial key, and the second database writes the first or last value to the initial key to obtain the first initial key and the second initial key. The first initial key and the second initial key are combined to obtain the encryption key for encrypting the first array and the second array. The encryption key also serves the encryption requirement of the migration log of the migration requirement. It should be noted that if the first database and the second database are not the same holders, the information about the encryption key obtained by both is asymmetric information. The second database can only obtain the first sub-volume to be migrated, and cannot obtain its location in the volume mapping tree or storage pool mapping tree in the first database. Therefore, the first initial key is black box information for the second database, and the first database needs to verify the identity of the second database before decryption can be achieved, thereby achieving dual authentication. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a structural diagram of the system provided in the second embodiment disclosed in the present invention;

[0055] Figure 2 It is a schematic diagram of the structure of the electronic device provided in the third embodiment disclosed in the present invention. DETAILED DESCRIPTION

[0056] In order to be able to understand the features and technical contents of the embodiments of the present disclosure in more detail, the implementation of the embodiments of the present disclosure is described in detail below in conjunction with the accompanying drawings. The attached drawings are for reference only and are not used to limit the embodiments of the present disclosure. In the following technical description, for the convenience of explanation, a full understanding of the disclosed embodiments is provided through multiple details. However, one or more embodiments can still be implemented without these details. In other cases, to simplify the drawings, well-known structures and devices can be simplified for display.

[0057] Embodiment 1

[0058] The present disclosure provides a method for migrating data from a multi-relational database to a non-relational database, comprising the following steps:

[0059] S1. Obtain first structural data of a first database. In the embodiment of the present disclosure, the first database is a multi-relational database.

[0060] S2. Construct a multi-column tree structure about the first structure data, wherein the multi-column tree structure at least includes a volume mapping tree and a storage pool mapping tree, wherein the volume mapping tree includes first mapping data and sub-volume data, wherein the storage pool mapping includes second mapping data and storage pool data, wherein the storage pool data includes a plurality of volume mapping trees and mappings set accordingly;

[0061] S3. Obtain second structural data of a second database, and allocate mapping space required for performing migration to the second structural data. In the embodiment of the present disclosure, the second database is a non-relational database, and the second structural data is storage pool distribution data of the database, indicating the occupancy and distribution of each storage pool in the second database.

[0062] S4, when writing any first sub-volume from the volume mapping tree to the mapping space, constructing a second sub-volume corresponding to the first sub-volume in the mapping space;

[0063] S5, writing the metadata of the first volume to the second volume, obtaining the first mapping data of the first volume in the volume mapping tree, and writing the first mapping data and the second mapping data to a preset position in the metadata of the second volume after the second structure data is completely written to the first volume;

[0064] S6, deleting the first mapping data corresponding to the first volume in the volume mapping tree, and updating the volume mapping tree;

[0065] S6, deleting the second mapping data corresponding to the first volume in the storage pool mapping, and updating the storage pool mapping tree;

[0066] S7, repeatedly executing S4-S6 until any of the volume mapping trees does not contain any of the first mapping data, and deleting the volume mapping tree;

[0067] S8. Repeat S4-S7 until any storage pool mapping does not include a volume mapping tree, and then delete the storage pool mapping.

[0068] S9. Repeat S4-S8 until the multi-column tree structure does not include any mapping data.

[0069] It should be noted that the multi-column tree structure in the embodiment of the present disclosure is a tree structure established based on the table or key value of the first database, and the first structural data of the database is used to form a storage table mapped as a multi-level branch in the manner of database-storage pool-volume, and all the steps of the embodiment of the present disclosure are executed based on the storage table, and the migration completion part is indicated by deleting the mapping.

[0070] As a preferred method in the embodiment of the present disclosure, when performing migration, partial migration or complete migration is performed in the multi-column tree structure according to the migration requirements, that is, the specific sequence of steps performed in the embodiment of the present disclosure is determined according to the migration requirements, and is set as follows:

[0071] If the migration requirement is partial migration, executing the designated sub-volumes in the multi-column tree structure until the migration is completed;

[0072] If the migration requirement is a complete migration of the volume, the process ends at S7;

[0073] If the migration requirement is a complete migration of the storage pool, the process ends at S8;

[0074] If the migration requirement is a complete migration of the first database, the process ends at S9;

[0075] When performing the partial migration or the complete migration, the migration is performed as a random migration or a sequential migration.

[0076] As a preferred method for adapting S2 in the embodiment of the present disclosure, the volume mapping tree is specifically configured as follows:

[0077] The first mapping data includes at least a first mapping and a second mapping;

[0078] The volume data includes a main volume and multiple sub-volumes. The main volume sets a first mapping to each sub-volume, and each sub-volume sets a second mapping. If the sub-volume sets a lower-level volume, the mapping of the sub-volume to the lower-level volume and the mapping between the lower-level volumes are sequentially added and stored in the first mapping data. The first mapping is used to index the main volume to all the sub-volumes it includes, and the second mapping is used to index to all the sub-volumes belonging to the same main volume according to a certain sub-volume.

[0079] As a preferred method for adapting S2 in the embodiment of the present disclosure, the storage pool mapping is specifically set to:

[0080] The second mapping data at least includes a third mapping and a fourth mapping,

[0081] The third mapping is a mapping index value stored in the same volume mapping tree between different storage pools, and the third mapping includes index values ​​for the first mapping and the second mapping. The fourth mapping is an index value between multiple volume mapping trees in the same storage pool, wherein the second mapping is used to index a volume mapping tree of the storage pool to the location where the volume mapping tree is stored in other volumes, and the fourth mapping is used to retrieve correlation mappings between multiple volume mapping trees.

[0082] It should be noted that the storage pool mapping tree also includes the storage mapping of a single first volume. In this case, the storage pool mapping tree distributes the volumes in the same volume mapping tree to different storage pools, and when there is no need to store all the data in a volume mapping tree, the storage pool mapping tree retains mappings with a smaller total amount. For example, after a single volume corresponding to a volume mapping tree is stored in a storage pool, only the mapping of the volume mapping tree or the mapping of the single volume is saved separately.

[0083] The storage pool data is a set of all data written to the storage pool.

[0084] As a preferred embodiment of the present disclosure, when any step S4-S9 is completed, the following steps are performed:

[0085] Establishing a backup area in the first database and / or the second database;

[0086] Writing metadata of the specified first volume in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume;

[0087] After restoring the value of the second volume written to the preset position according to the first mapping data and the second mapping data;

[0088] The error rates of the restored second volume and the backup volume in the backup area are checked. If the error rate is higher than a preset threshold, the backup volume is written into the tree structure and all steps before S4-S9 are executed accordingly.

[0089] Furthermore, the specific steps of executing the migration requirement and writing the first mapping data and the second mapping data into a preset position in the metadata of the second volume are:

[0090] According to the writing of the first mapping data and the second mapping data, a first array of volume mapping data of the multi-column tree structure and a second array of storage pool mapping data are obtained;

[0091] After encrypting the first array and the second array, encapsulate them into the second volume along with the first mapping data and the second mapping data;

[0092] An encryption log corresponding to the first array and the second array is generated, wherein the encryption log includes an encryption key.

[0093] As a further preferred embodiment of the present disclosure, in the embodiment of the present disclosure, the first array is a placeholder value written to the position of the mapping data in the volume mapping tree after any of the first mapping data is deleted, forming a merged array with the value of the position of the first mapping data; the second array is a placeholder value written to the position of the mapping data in the storage pool mapping tree after any of the second mapping data is deleted, forming a merged array with the value of the position of the second mapping data.

[0094] Specifically, an example is given, such as the construction of the first array in the embodiment of the present disclosure, taking binary as an example, if any first sub-volume in any volume mapping tree is migrated, a placeholder value 0 is written to the sub-volume at the generation position of the volume mapping tree, and the entire first mapping data in the volume mapping tree and the placeholder value are obtained. According to the binary value of the initial position of each first mapping data when the volume mapping tree is generated, a binary merged array is obtained, and after encryption with a key, a ciphertext with the same or different format as the original data is generated, and the ciphertext is packaged into the second sub-volume along with the written first mapping data. In addition, for the first mapping and the second mapping in the first mapping data, the two belong to mappings of different levels. When used for the generation of the first array, the array is arranged in the mapping priority order between the first mapping and the second mapping, wherein the priority of the first mapping is higher than the priority of the second mapping, and similarly, the priority of the second mapping is higher than the mapping of other sub-volumes, and the preferred method for adapting S2 in the embodiment of the present disclosure is explained.

[0095] Specifically, the construction of the second array is similar to the construction of the first array, and will not be repeated here.

[0096] If, in the embodiment of the present disclosure, the executed migration requirement is migrated in a sequential migration manner, since consecutive identical placeholder values ​​will result in consecutive identical ciphertexts after encryption, the recorded data information will directly disclose the migration rule of the migration requirement in the ciphertext. Therefore, in the embodiment of the present disclosure, the value of the array is changed by generating data in adjacent positions or interval adjacent positions. In the embodiment of the present disclosure, binary is still used as an example. Each time a placeholder value is generated, a progressive value 1 is written to the binary value of the adjacent or interval adjacent first mapping data of the first volume. It should be noted that the progressive value can be set according to actual needs. Afterwards, the values ​​continuously obtained after each generation of the array will be confused with other first mapping data written with the progressive value in the array and lose continuity, so as to avoid leakage of migration requirement information.

[0097] It should be noted that the first array and the second array in the embodiment of the present disclosure are not limited to binary. The standard system and data format in the field can be used as the array generation method of the embodiment of the present disclosure. Taking decimal and hexadecimal as examples, similarly, if it is a sequential migration, each time a placeholder value is generated, a progressive value is written to its adjacent position or the interval adjacent position. It should be noted that the progressive value in the embodiment of the present disclosure can always advance the first mapping data to the upper position by 1.

[0098] It should also be noted that if the data format is a non-standard base format, there is no need to consider the progression during sequential migration. Accordingly, the generation of placeholder values ​​can also be dynamic, and the setting of the dynamically generated placeholder values ​​is always smaller than the byte length of the first mapping data and / or the second mapping data.

[0099] In the embodiment of the present disclosure, the storage of the encrypted log includes an encryption key used for migration. As a dual authentication method, the generation of the encryption key adopts a general key rule and changes dynamically with the length of the value of the first mapping data and / or the second mapping data in the migration requirement. As an example, when migrating according to the migration requirement, the initial key is distributed to the first database and the second database. Then, according to the value deleted in any mapping tree during the migration, the first database deletes the value corresponding to the mapping tree position in the initial key, and the second database writes the first or last value to the initial key to obtain the first initial key and the second initial key. The first initial key and the second initial key are combined to obtain the encryption key for encrypting the first array and the second array. The encryption key also serves the encryption requirement of the migration log of the migration requirement. It should be noted that if the first database and the second database are not the same holders, the information about the encryption key obtained by both is asymmetric information. The second database can only obtain the first sub-volume to be migrated, and cannot obtain its location in the volume mapping tree or storage pool mapping tree in the first database. Therefore, the first initial key is black box information for the second database, and the first database needs to verify the identity of the second database before decryption can be achieved, thereby realizing the dual authentication in the embodiment of the present disclosure.

[0100] In the embodiment of the present disclosure, the encrypted log also includes the timestamp when the migration requirement is executed. Recording the migration time is a common technical means in the art and will not be described in detail here.

[0101] The disclosed embodiment realizes data migration from a table structure in a multi-relational database to a non-table structure in a non-relational database, establishes a mapping tree based on the table structure in a first database, migrates data based on the mapping tree, migrates the mapping relationship in the table structure as a data package to the non-relational database, and in the process uses the dynamic changes of the mapping relationship as a dynamic key to encrypt and package each migrated data. The dynamic key also records the log status of the migrated data to ensure data security, thereby realizing log recording and backup encryption of the migration process.

[0102] Embodiment 2

[0103] See also Figure 1 The embodiment of the present disclosure provides a data migration system from a multi-relational database to a non-relational database, including a first acquisition module, a second acquisition module, a migration module, a backup module and an encryption log module.

[0104] The first acquisition module in the embodiment of the present disclosure is configured to obtain first structural data and migration requirements of the first database.

[0105] The second acquisition module is configured to obtain second structure data and mapping space of the second database.

[0106] a migration module, configured to construct a multi-column tree structure about the first structure data, the multi-column tree structure at least comprising a volume mapping tree and a storage pool mapping tree, the volume mapping tree comprising first mapping data and sub-volume data, the storage pool mapping comprising second mapping data and storage pool data, wherein the storage pool data comprises a plurality of volume mapping trees and mappings set corresponding thereto, and configured to migrate the first sub-volume specified in the multi-column tree structure to the mapping space according to the migration requirement;

[0107] The backup module is configured to, when any migration requirement is executed and any first volume specified in the migration requirement is completed, establish a backup area in the first database and / or the second database, write metadata of the first volume specified in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume, and then restore the value of the second volume written to the preset position according to the first mapping data and the second mapping data, check the error rate of the restored second volume and the backup volume in the backup area, and if the error rate is higher than a preset threshold, write the backup volume into the tree structure and then execute the steps before the migration requirement is executed;

[0108] The encryption log module is configured to obtain a first array of volume mapping data of the multi-column tree structure and a second array of storage pool mapping data according to the writing of the first mapping data and the second mapping data; encrypt the first array and the second array and then package them into the second volume along with the first mapping data and the second mapping data; generate an encryption log corresponding to the first array and the second array, wherein the encryption log includes an encryption key.

[0109] Specifically, the steps performed by the migration module when configured in the embodiment of the present disclosure have been described in detail in the first embodiment, and will not be repeated here for the sake of brevity.

[0110] The disclosed embodiment realizes data migration from a table structure in a multi-relational database to a non-table structure in a non-relational database, establishes a mapping tree based on the table structure in a first database, migrates data based on the mapping tree, migrates the mapping relationship in the table structure as a data package to the non-relational database, and in the process uses the dynamic changes of the mapping relationship as a dynamic key to encrypt and package each migrated data. The dynamic key also records the log status of the migrated data to ensure data security, thereby realizing log recording and backup encryption of the migration process.

[0111] Embodiment 3

[0112] See also Figure 2 An embodiment of the present disclosure provides an electronic device, including a processor (processor) 30 and a memory (memory) 31.

[0113] Optionally, the device may further include a communication interface (CommunicationInterface) 32 and a bus 33. The processor 30, the communication interface 32, and the memory 31 may communicate with each other through the bus 33. The communication interface 32 may be used for information transmission. The processor 30 may call the logic instructions in the memory 31 to execute the data migration method from a multi-relational database to a non-relational database as provided in the first embodiment of the present disclosure.

[0114] The embodiment of the present disclosure also provides a storage medium storing computer executable instructions, wherein the computer executable instructions are configured as a method for migrating data from a multi-relational database to a non-relational database as provided in the first embodiment of the present disclosure.

[0115] The above-mentioned storage medium may be a transient computer-readable storage medium or a non-transient computer-readable storage medium. Non-transient storage media include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes, and may also be a transient storage medium.

[0116] The above description and accompanying drawings fully illustrate the embodiments of the present disclosure so that those skilled in the art can practice them. Other embodiments may include structural, logical, electrical, process and other changes. The embodiments represent possible changes only. Unless explicitly required, separate components and functions are optional, and the order of operation may vary. The parts and features of some embodiments may be included in or replace the parts and features of other embodiments. Moreover, the words used in this application are only used to describe the embodiments and are not used to limit the claims. As used in the description of the embodiments and claims, unless the context clearly indicates, the singular forms of "a", "an" and "the" are intended to include plural forms as well. Similarly, the term "and / or" as used in this application refers to any and all possible combinations of listings containing one or more associated ones. In addition, when used in the present application, the term "comprise" and its variants "comprises" and / or comprising refer to the presence of stated features, wholes, steps, operations, elements, and / or components, but do not exclude the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof. In the absence of further restrictions, the elements defined by the sentence "comprising a ..." do not exclude the presence of other identical elements in the process, method or device comprising the elements. In this article, each embodiment may focus on the differences from other embodiments, and the same and similar parts between the various embodiments may refer to each other. For the methods, products, etc. disclosed in the embodiments, if they correspond to the method part disclosed in the embodiments, then the relevant parts can refer to the description of the method part.

Claims

1. A method for data migration from a multi-relational database to a non-relational database, characterized in that: The steps include: S1. Obtaining first structural data from a first database; S2. Construct a multi-column tree structure about the first structure data, wherein the multi-column tree structure at least includes a volume mapping tree and a storage pool mapping tree, wherein the volume mapping tree includes first mapping data and sub-volume data, wherein the storage pool mapping tree includes second mapping data and storage pool data, wherein the storage pool data includes a plurality of volume mapping trees and mappings set corresponding to the volume mapping trees; S3, obtaining second structure data of the second database, and allocating a mapping space required for executing the migration to the second structure data; S4, when writing any first sub-volume from the volume mapping tree to the mapping space, constructing a second sub-volume corresponding to the first sub-volume in the mapping space; S5, writing the metadata of the first volume to the second volume, obtaining the first mapping data of the first volume in the volume mapping tree, and writing the first mapping data and the second mapping data to a preset position in the metadata of the second volume after the second structure data is completely written to the first volume; The first mapping data includes at least a first mapping and a second mapping; The volume data includes a main volume and multiple sub-volumes, a first mapping is set from the main volume to each of the sub-volumes, and a second mapping is set between each of the sub-volumes. If the sub-volume is set as a next-level volume, the mapping from the sub-volume to the next-level volume and the mapping between the next-level volumes are sequentially added and stored in the first mapping data; The second mapping data at least includes a third mapping and a fourth mapping, The third mapping is a mapping index value stored in the same volume mapping tree between different storage pools, and the third mapping includes index values ​​for the first mapping and the second mapping, and the fourth mapping is an index value between multiple volume mapping trees in the same storage pool; S6, deleting the first mapping data corresponding to the first volume in the volume mapping tree, and updating the volume mapping tree; Deleting the second mapping data corresponding to the first volume in the storage pool mapping tree, and updating the storage pool mapping tree; S7, repeatedly executing S4-S6 until any of the volume mapping trees does not contain any of the first mapping data, and deleting the volume mapping tree; S8, repeatedly executing S4-S7 until any of the storage pool mapping trees does not include a volume mapping tree, deleting the storage pool mapping tree; S9. Repeat S4-S8 until the multi-column tree structure does not include any mapping data.

2. The method for data migration from a multi-relational database to a non-relational database according to claim 1, characterized in that: The storage pool data is a set of all data written to the storage pool.

3. The method for data migration from a multi-relational database to a non-relational database according to claim 2, characterized in that: When performing migration, partial migration or complete migration is performed in the multi-column tree structure according to migration requirements; If the migration requirement is partial migration, executing the designated sub-volumes in the multi-column tree structure until the migration is completed; If the migration requirement is a complete migration of the volume, the process ends at S7; If the migration requirement is a complete migration of the storage pool, the process ends at S8; If the migration requirement is a complete migration of the first database, the process ends at S9; When performing the partial migration or the complete migration, the migration is performed as a random migration or a sequential migration.

4. The method for data migration from a multi-relational database to a non-relational database according to claim 3, characterized in that: When any step of S4-S9 is completed, the following steps are performed: Establishing a backup area in the first database and / or the second database; Writing metadata of the specified first volume in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume; After restoring the value of the second volume written to the preset position according to the first mapping data and the second mapping data; The error rates of the restored second volume and the backup volume in the backup area are checked. If the error rate is higher than a preset threshold, the backup volume is written into the tree structure and all steps before S4-S9 are executed accordingly.

5. The method for data migration from a multi-relational database to a non-relational database according to claim 3 or 4, characterized in that: The specific steps of executing the migration requirement and writing the first mapping data and the second mapping data into a preset position in the metadata of the second volume are: According to the writing of the first mapping data and the second mapping data, a first array of the volume mapping tree and a second array of the storage pool mapping tree of the multi-column tree structure are obtained; After encrypting the first array and the second array, encapsulate them into the second volume along with the first mapping data and the second mapping data; An encryption log corresponding to the first array and the second array is generated, wherein the encryption log includes an encryption key.

6. The method for data migration from a multi-relational database to a non-relational database according to claim 5, characterized in that: The first array is After any of the first mapping data is deleted, a placeholder value is written to the position of the mapping data in the volume mapping tree to form a merged array with the value of the position of the first mapping data; The second array is After any of the second mapping data is deleted, a placeholder value is written to the position of the mapping data in the storage pool mapping tree to form a merged array with the value of the position with the second mapping data.

7. A data migration system from a multi-relational database to a non-relational database, characterized in that: include: A first acquisition module is configured to obtain first structural data and migration requirements of a first database; A second acquisition module is configured to obtain second structure data and a mapping space of a second database; a migration module, configured to construct a multi-column tree structure about the first structure data, the multi-column tree structure at least comprising a volume mapping tree and a storage pool mapping tree, the volume mapping tree comprising first mapping data and sub-volume data, the storage pool mapping tree comprising second mapping data and storage pool data, wherein the storage pool data comprises a plurality of volume mapping trees and mappings set corresponding thereto, and configured to migrate the first sub-volume specified in the multi-column tree structure to the mapping space according to the migration requirement; The first mapping data includes at least a first mapping and a second mapping; The volume data includes a main volume and multiple sub-volumes, a first mapping is set from the main volume to each of the sub-volumes, and a second mapping is set between each of the sub-volumes. If the sub-volume is set as a next-level volume, the mapping from the sub-volume to the next-level volume and the mapping between the next-level volumes are sequentially added and stored in the first mapping data; The second mapping data at least includes a third mapping and a fourth mapping, The third mapping is a mapping index value stored in the same volume mapping tree between different storage pools, and the third mapping includes index values ​​for the first mapping and the second mapping, and the fourth mapping is an index value between multiple volume mapping trees in the same storage pool; The backup module is configured to, when any migration requirement is executed and the execution of any first volume specified in the migration requirement is completed, establish a backup area in the first database and / or the second database, write metadata of the first volume specified in the migration requirement and first mapping data and second mapping data corresponding to the first volume into the backup area to obtain a backup volume, and then restore the value of the second volume written to the preset position according to the first mapping data and the second mapping data, check the error rate of the restored second volume and the backup volume in the backup area, and if the error rate is higher than a preset threshold, write the backup volume into the tree structure and then execute the steps before the migration requirement is executed; The encryption log module is configured to obtain a first array of the volume mapping tree of the multi-column tree structure and a second array of the storage pool mapping tree according to the writing of the first mapping data and the second mapping data; encrypt the first array and the second array and then package them into the second volume along with the first mapping data and the second mapping data; generate an encryption log corresponding to the first array and the second array, wherein the encryption log includes an encryption key.

8. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the program, the method for migrating data from a multi-relational database to a non-relational database as claimed in any one of claims 1 to 6 is implemented.

9. A storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it is suitable for implementing the data migration method from a multi-relational database to a non-relational database as described in any one of claims 1 to 6.

Images