Data migration method and device for new and old databases and storage medium thereof

Through the data migration method of new and old databases, database format differences and field conflicts are handled, and compatibility issues caused by data format changes in software development are solved, data consistency and automated migration are achieved, and maintenance costs and user experience impact are reduced.

CN118964326AInactive Publication Date: 2024-11-15GUANGZHOU SENYANG ELECTRONIC TECH CO LTD

Patent Information

Application Number
CN202410988116.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-15
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

During the software development process, function iteration leads to changes in data format, resulting in compatibility issues between old and new databases, resulting in loss or abnormal user configuration, increasing maintenance costs and user experience impact.

Method used

Through a data migration method for new and old databases, it includes checking the database data format, comparing tables and fields, processing row data, using the default values ​​of the new table to process fields that do not exist in the old table, automating migration steps, and ensuring data consistency , and handle conflicts during the write process.

Benefits of technology

Reduces development and maintenance workload, reduces compatibility issues due to version differences, ensures that the migrated data is consistent with the new database requirements, avoids data inconsistency and configuration loss, and improves user experience and automation.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118964326A_ABST
    Figure CN118964326A_ABST
Patent Text Reader

Abstract

The invention discloses a data migration method of a new database and an old database, which comprises the following steps of: checking whether a database data format needs to be upgraded or not before software upgrading or first starting, simultaneously starting the new database and the old database, comparing new and old database files, identifying a table and a field which need to be migrated, and performing data migration in a field information processing stage. According to the data migration method and device, the'AND 'operation of the fields is executed, only the same name fields existing in the two databases are reserved, compatibility processing on different field types is reduced, in the line data processing stage, data are processed line by line, new fields are filled with default values, old fields are directly copied, the complexity of data migration is simplified, and the data migration efficiency is improved. The compatibility problem is reduced, in the new line data writing stage, conflicts in the data writing process are processed, for example, data are covered or newly added when a primary key or a unique key conflicts, and the consistency of the data is ensured; by repeatedly executing field and row data processing steps, it is ensured that data migration of all tables can reduce the compatibility problem.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of data processing, and in particular to a method and device for migrating data from a new database to an old database, and a storage medium thereof. Background Art

[0002] In software development, in order to meet various ever-changing needs, parameters often increase or decrease during the process of functional iteration and upgrading, which directly affects the data format used in the program. Programmers have to spend time considering the data format of historical versions and do compatibility processing, which will make compatible codes more and more and maintenance costs increase over time.

[0003] If compatibility is not handled properly, the configuration used daily by users of the old software will be lost, incomplete or abnormal. When the printing software is upgraded, recalibration or resetting of printing parameters will often be required, which will greatly affect the user experience. Summary of the invention

[0004] In order to solve the above-mentioned technical problems, the present invention provides a data migration method between a new and old databases.

[0005] The technical solution of the present invention is achieved in this way:

[0006] A method for migrating data between a new and old database comprises the following steps:

[0007] S1. Before upgrading the software or starting it for the first time, check whether the database data format needs to be upgraded, and start both the old and new databases at the same time;

[0008] S2, comparison of database and table;

[0009] S2-1, compare multiple database files one by one according to the same file name;

[0010] S2-2, determine that the original database is database A, and the new database is database B;

[0011] S2-3, obtain the tables and their field information with the same name in database A and database B;

[0012] S3, field information processing, compare the fields in Table A and Table B, perform "and" operations on the fields, and retain the fields with the same name;

[0013] S4, row data processing, processing row data in table A row by row;

[0014] If a field in table A does not exist in table B, it will be processed according to the default value of the field in table B.

[0015] If the field exists in both table A and table B, copy the original row value in table A;

[0016] If the field in table A does not exist in table B, ignore the row of data;

[0017] S5. Write new row data, write the processed row data into table B, and handle data conflicts during the writing process;

[0018] Among them, if there is a primary key or unique key conflict in table B, the row data is overwritten;

[0019] If there is no conflict, add new row data;

[0020] S6. After the table data migration is completed, steps S3 to S5 are repeated to complete the data migration of all data tables with the same name.

[0021] S7. Database file processing: after completing the data migration of all tables, delete the db file of the old database A;

[0022] S8, subsequent database file processing, continue to process the next database db file with the same name, and repeat steps S2 to S7;

[0023] S9. Complete the data migration of all database files.

[0024] A data migration device based on the above method includes a processor and a memory; wherein:

[0025] The memory is used to store program instructions;

[0026] The processor can perform the following operations after calling the program instructions stored in the memory:

[0027] From multiple database DB files, compare the databases with the same file name one by one;

[0028] Wherein, the multiple database DB files include a first database DB file and a second database DB file;

[0029] For the first database DB file and the second database DB file, obtaining at least one table corresponding to each database DB file in the first database DB file and the second database DB file;

[0030] Merging all columns of each table in the at least one table, and if the number of the merged columns is greater than a preset number, removing some of the merged columns to obtain final columns;

[0031] For the final column, the data in the first database DB file is migrated to the second database DB file.

[0032] A storage medium based on the above method, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the processor is caused to execute the data migration method of the new and old databases;

[0033] It also includes a chip, which is coupled to the memory and is used to execute the computer program stored in the memory to execute the data migration method between the new and old databases.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] 1. The present invention uses the default values ​​of the new table to process the fields that do not exist in the old table during the migration process. Developers do not need to write additional compatible code for the old data format, thereby reducing the workload of development and maintenance. The migrated data follows the format of the new database, and there is no need to write specific processing logic for different versions of data formats.

[0036] 2. Through automated migration steps, human intervention is reduced and compatibility issues caused by version differences are reduced;

[0037] 3. By comparing and processing field and row data, it ensures that the migrated data is consistent with the requirements of the new database, avoiding data inconsistency issues. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is a flow chart of a method for migrating data between a new database and an old database according to Example 1. DETAILED DESCRIPTION

[0039] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0040] The terms used in this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The singular forms of "a", "said" and "the" used in this application and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used in this article refers to and includes any or all possible combinations of one or more corresponding listed items.

[0041] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0042] Example 1

[0043] like Figure 1 As shown, a method for migrating data from a new database to an old database includes the following steps:

[0044] S1. Before upgrading the software or starting it for the first time, check whether the database data format needs to be upgraded, and start both the old and new databases at the same time;

[0045] S2, comparison of database and table;

[0046] S2-1, compare multiple database files one by one according to the same file name;

[0047] S2-2, determine that the original database is database A, and the new database is database B;

[0048] S2-3, obtain the tables and their field information with the same name in database A and database B;

[0049] Furthermore, a version control mechanism is introduced in S2 to identify the different versions of databases A and B and adjust the migration strategy accordingly;

[0050] The specific process of adjusting the migration strategy is as follows:

[0051] 1. Determine database versions and differences:

[0052] 1) Version detection: Obtain the version information of database A and B by querying the database system table or using the tools provided by the database;

[0053] 2) Difference analysis: Compare the version numbers of the two databases to determine the differences between them. If the version numbers are A_ver and B_ver, the difference can be expressed as: Difference = |A ver -B ver |;

[0054] 3) Compatibility check: Determine the compatibility between the two databases based on the differences. If there are major version differences, additional migration steps or adapters may be required.

[0055] 2. Adjust the migration strategy:

[0056] 1) Migration path planning: If there are multiple version differences, plan a gradual migration path, for example, from A to the intermediate version C, and then to B;

[0057] 2) Data structure mapping: Map the data structure of the old database to the data structure of the new database, including field type, length, etc.;

[0058] 3) Migration script generation: Generate migration scripts or use migration tools based on the mapping results. These scripts will contain data conversion logic.

[0059] 4) Data type conversion: For incompatible data types, apply the conversion formula. For example, if you need to convert the date from YYYYMMDD format to YYYY-MM-DD format, you can use the following formula:

[0060] NewDate=SUBSTRING(OldDate,1,4)+″-″+SUBSTRING(OldDate,5,2)+″-″+SUBSTRING(OldDate,7,2)

[0061] The formula works as follows:

[0062] SUBSTRING(OldDate, 1, 4) extracts the year part from the original date string OldDate, starting from the first character and taking 4 characters;

[0063] SUBSTRING(OldDate, 5, 2) extracts the month part, starting from the 5th character and taking 2 characters;

[0064] SUBSTRING(OldDate, 7, 2) extracts the date part, starting from the 7th character and taking 2 characters;

[0065] Then connect these three parts with a hyphen "-" to form a new date format YYYY-MM-DD;

[0066] 5) Index and constraint processing: Identify and migrate database constraints such as indexes, primary keys, foreign keys, etc. These constraints may need to be recreated to adapt to the structure of the new database;

[0067] 6) Migration simulation: Before the actual migration, a simulated migration is performed to verify the correctness of the migration script and logic;

[0068] 7) Performance evaluation: Evaluate the performance of the migration process, such as data migration speed and resource consumption. Use the following formula to evaluate the migration rate:

[0069] 8) Error handling: Design error handling mechanisms to capture and resolve data inconsistencies or conversion errors during the migration process;

[0070] 9) User-defined functions (UDF): Write user-defined functions to handle specific data migration logic;

[0071] 10) Execute the migration: After determining the migration strategy and script, execute the migration process and monitor the migration progress and system performance;

[0072] 11) Data verification: After the migration is completed, data verification is performed to ensure the integrity and accuracy of the data;

[0073] 12) Optimization and adjustment: Optimize and adjust the migration strategy based on the performance data and logs collected during the migration process;

[0074] S3, field information processing, compare the fields in Table A and Table B, perform "and" operations on the fields, and retain the fields with the same name;

[0075] S4, row data processing, processing row data in table A row by row;

[0076] If a field in table A does not exist in table B, it will be processed according to the default value of the field in table B.

[0077] If the field exists in both table A and table B, copy the original row value in table A;

[0078] If the field in table A does not exist in table B, ignore the row of data;

[0079] S5. Write new row data, write the processed row data into table B, and handle data conflicts during the writing process;

[0080] Among them, if there is a primary key or unique key conflict in table B, the row data is overwritten;

[0081] If there is no conflict, add new row data;

[0082] S6. After the table data migration is completed, steps S3 to S5 are repeated to complete the data migration of all data tables with the same name.

[0083] S7. Database file processing: after completing the data migration of all tables, delete the db file of the old database A;

[0084] S8, subsequent database file processing, continue to process the next database db file with the same name, and repeat steps S2 to S7;

[0085] S9. Complete the data migration of all database files.

[0086] In this embodiment, by checking whether the database data format needs to be upgraded before the software is upgraded or started for the first time, and opening the new and old databases at the same time, then by comparing the new and old database files, the tables and fields that need to be migrated are identified. In the field information processing stage, the "AND" operation of the field is performed, and only the fields with the same name that exist in the two databases are retained, which reduces the compatibility processing of different field types. In the row data processing stage, the data is processed row by row, and the new fields are filled with default values, and the old fields are directly copied, which simplifies the complexity of data migration and reduces the occurrence of compatibility problems. In the new row data writing stage, we handle conflicts in the data writing process, such as overwriting data or adding new data when the primary key or unique key conflicts, to ensure data consistency;

[0087] By repeating the field and row data processing steps, we ensure that data migration of all tables can reduce compatibility issues. After completing the data migration of all tables, we delete the old database files and perform a final check to ensure the integrity and accuracy of the data migration and avoid configuration loss due to compatibility issues. This not only reduces the compatibility complexity of historical versions of the software and avoids the decline in user experience due to compatibility issues, but also improves the automation and accuracy of data migration.

[0088] Example 2

[0089] This embodiment provides a data migration device based on the above method, including a processor and a memory; wherein:

[0090] The memory is used to store program instructions;

[0091] The processor can perform the following operations after calling the program instructions stored in the memory:

[0092] From multiple database DB files, compare the databases with the same file name one by one;

[0093] Wherein, the multiple database DB files include a first database DB file and a second database DB file;

[0094] For the first database DB file and the second database DB file, obtaining at least one table corresponding to each database DB file in the first database DB file and the second database DB file;

[0095] Merging all columns of each table in the at least one table, and if the number of the merged columns is greater than a preset number, removing some of the merged columns to obtain final columns;

[0096] For the final column, the data in the first database DB file is migrated to the second database DB file.

[0097] A storage medium based on the above method, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the processor is caused to execute the data migration method of the new and old databases;

[0098] It also includes a chip, which is coupled to the memory and is used to execute the computer program stored in the memory to execute the data migration method between the new and old databases.

Claims

1. A method for data migration between a new and old database, characterized in that: The following steps are involved: S1. Before upgrading the software or starting it for the first time, check whether the database data format needs to be upgraded, and start both the old and new databases at the same time; S2, comparison of database and table; S3, field information processing, comparing the fields in Table A and Table B; S4, row data processing, processing row data in table A row by row; S5. Write new row data, write the processed row data into table B, and handle data conflicts during the writing process; S6. After the table data migration is completed, steps S3 to S5 are repeated to complete the data migration of all data tables with the same name. S7. Database file processing: after completing the data migration of all tables, delete the db file of the old database A; S8, subsequent database file processing, continue to process the next database db file with the same name, and repeat steps S2 to S7; S9. Complete the data migration of all database files.

2. The data migration method of the new and old databases according to claim 1 is characterized by: The S2 further includes: S2-1, compare multiple database files one by one according to the same file name; S2-2, determine that the original database is database A, and the new database is database B; S2-3. Obtain the tables and their field information with the same name in database A and database B.

3. The data migration method of the new and old databases according to claim 1 is characterized in that: When processing fields in S3, an "AND" operation is performed on the fields, and fields with the same name are retained.

4. The method for data migration between new and old databases according to claim 1, characterized in that: In S4, if a field in table A does not exist in table B, it is processed according to the default value of the field in table B; If the field exists in both table A and table B, copy the original row value in table A; if the field in table A does not exist in table B, ignore the row data.

5. The method for data migration between new and old databases according to claim 1, characterized in that: In S5, if there is a primary key or unique key conflict in table B, the row data is overwritten; if there is no conflict, new row data is added.

6. The method for data migration between new and old databases according to claim 1, characterized in that: The version control mechanism is introduced in S2 to identify the different versions of databases A and B and adjust the migration strategy accordingly, specifically:

1. Determine the database version and difference; 2. Adjust the migration strategy.

7. The method for data migration between new and old databases according to claim 6, characterized in that: The process of determining database versions and differences is as follows: 1) Version detection: Obtain the version information of database A and B by querying the database system table or using the tools provided by the database; 2) Difference analysis: compare the version numbers of the two databases to determine the differences between them; 3) Compatibility check: Determine the compatibility between the two databases based on the differences.

8. The method for data migration between new and old databases according to claim 6, characterized in that: The process of adjusting the migration strategy is as follows: 1) Migration path planning: If there are multiple version differences, plan a gradual migration path, for example, from A to the intermediate version C, and then to B; 2) Data structure mapping: Map the data structure of the old database to the data structure of the new database, including field type, length, etc.; 3) Migration script generation: Generate migration scripts or use migration tools based on the mapping results. These scripts will contain data conversion logic. 4) Data type conversion: For incompatible data types, apply the conversion formula: 5) Index and constraint processing: Identify and migrate database constraints; 6) Migration simulation: Before the actual migration, a simulated migration is performed to verify the correctness of the migration script and logic; 7) Performance evaluation: Evaluate the performance during the migration process; 8) Error handling: Design error handling mechanisms to capture and resolve data inconsistencies or conversion errors during the migration process; 9) User-defined functions (UDF): Write user-defined functions to handle specific data migration logic; 10) Execute the migration: After determining the migration strategy and script, execute the migration process and monitor the migration progress and system performance; 11) Data verification: After the migration is completed, data verification is performed to ensure the integrity and accuracy of the data; 12) Optimization and adjustment: Optimize and adjust the migration strategy based on the performance data and logs collected during the migration process.

9. A data migration device based on the method according to any one of claims 1 to 8, characterized in that: comprising a processor and a memory; wherein, The memory is used to store program instructions; The processor can perform the following operations after calling the program instructions stored in the memory: From multiple database DB files, compare the databases with the same file name one by one; Wherein, the multiple database DB files include a first database DB file and a second database DB file; For the first database DB file and the second database DB file, obtaining at least one table corresponding to each database DB file in the first database DB file and the second database DB file; Merging all columns of each table in the at least one table, and if the number of the merged columns is greater than a preset number, removing some of the merged columns to obtain final columns; For the final column, the data in the first database DB file is migrated to the second database DB file.

10. A storage medium based on the method according to any one of claims 1 to 8, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the processor executes the method for migrating data from the new database to the old database; It also includes a chip, which is coupled to the memory and is used to execute the computer program stored in the memory to execute the data migration method between the new and old databases.

Citation Information

Patent Citations

  • Data heterogeneous migration visual analysis method based on ARM kernel

    CN114077600A

  • Database migration processing method and system

    CN117349267A

  • Data migration method and system based on heterogeneous database

    CN117971809A

Cited By

  • Compliance specification matching application method

    CN119918943A