Data Migration Method, Device, Electronic Device and Storage Medium

The method optimizes data migration by scaling strategies based on data volume, ensuring minimal disruption and maintaining application functionality during the process.

CN117785839BActive Publication Date: 2025-07-15BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202311744652.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2023-12-18
Publication Date
2025-07-15
Estimated Expiration
2043-12-18

AI Technical Summary

Technical Problem

In the prior art, data migration solutions lead to user interaction blockage and poor user experience, especially when large data migration is not allowed to use APP for a long time, resulting in user churn.

Method used

According to the scale of the data to be migrated, the data is migrated from the first database to the second database by overall migration or batch migration, and the main thread access of the application is monitored during the migration process, exception signals are generated to prevent lags, and to ensure the continuity of user interaction.

Benefits of technology

By optimizing the migration methods and monitoring mechanism, the impact of data migration on users is reduced, the user experience is improved, long-term blockage is avoided, and the risk of user churn is reduced.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117785839B_ABST
    Figure CN117785839B_ABST
Patent Text Reader

Abstract

The present disclosure relates to the technical field of data migration, and particularly to a data migration method, apparatus, electronic device, and storage medium. The specific implementation solution is as follows: Determine the migration method of the data to be migrated according to the data scale of the data to be migrated; According to the determined migration method, migrate the data to be migrated from the first database to the second database as a whole, or migrate the data to be migrated from the first database to the second database in batches. The present disclosure combines the methods of overall migration and batch migration, determines the migration method according to the data scale, and avoids blocking users from using the APP due to long-term data migration, resulting in poor user experience.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the technical field of data migration, and in particular, to a data migration method, apparatus, electronic device, and storage medium. Background Art

[0002] As the functions of an APP (Application) are increasing, the need to maintain APP modules is also increasing. However, since the APP needs to be quickly verified and launched now, the complexity of the APP is not considered during the APP design, and different types of data are not classified and stored. Instead, a unified SQLite database is used for storage. However, the business logic of the APP adopts a modular design. The modularization of business logic requires separating different business functions and data to form an independent module, making it more convenient for each to share functions and data. Therefore, many large-scale APPs have the requirement to split the unified SQLite database, which involves the problem of data migration.

[0003] Most of the existing data migration solutions are in the form of blocking user interaction and forcing data upgrade to complete the data migration work. The amount of data to be migrated and the data migration time increase exponentially. That is to say, the larger the amount of data, the longer the user waits. If the user has a large amount of data, the user interaction will be blocked for a long time, and the user cannot use the APP during the data migration process, resulting in poor user experience and user loss. Summary of the Invention

[0004] The present disclosure provides a data migration method, apparatus, electronic device, and storage medium.

[0005] According to one aspect of the present disclosure, a data migration method is provided, including:

[0006] Determining a migration method for the data to be migrated according to the data scale of the data to be migrated; wherein, the migration method includes overall migration and batch migration;

[0007] According to the determined migration method, migrating the data to be migrated from the first database to the second database as a whole, or migrating the data to be migrated from the first database to the second database in batches.

[0008] According to another aspect of the present disclosure, a data migration apparatus is provided, including:

[0009] A determination module configured to determine a migration method for the data to be migrated according to the data scale of the data to be migrated; wherein, the migration method includes overall migration and batch migration;

[0010] A data migration module, configured to migrate the data to be migrated from the first database to the second database as a whole or in batches from the first database to the second database according to the determined migration method.

[0011] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0012] At least one processor; and

[0013] A memory communicatively connected to the at least one processor; wherein,

[0014] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any one of the above technical solutions.

[0015] According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the method described in any one of the above technical solutions.

[0016] According to a fifth aspect of the present disclosure, there is provided a computer program product, including a computer program, and the computer program implements the method described in any one of the above technical solutions when executed by a processor.

[0017] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] The drawings are used to better understand the solution and do not constitute a limitation to the present disclosure. Among them:

[0019] Figure 1 is a schematic diagram of the steps of the data migration method in an embodiment of the present disclosure;

[0020] Figure 2 is a schematic block diagram of the data migration device in an embodiment of the present disclosure;

[0021] Figure 3 is a structural diagram of a client system applying the data migration method in an embodiment of the present disclosure;

[0022] Figure 4 is a block diagram of an electronic device for implementing the data migration method in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] The following describes exemplary embodiments of the present disclosure with reference to the accompanying drawings. Various details of the embodiments of the present disclosure are included to facilitate understanding, and they should be considered merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, descriptions of well-known functions and structures are omitted in the following description for clarity and conciseness.

[0024] There are two existing client data migration solutions:

[0025] (1) Forced deletion method: Directly delete the historical data used in the local sandbox through remote control or by releasing a new version of the APP, and then log out of the APP.

[0026] (2) Forced data copying method: When the APP is upgraded to a new version, first block the user operation, and the data upgrade can be completed only after the data migration is completed.

[0027] The above solutions complete the data migration work by blocking the user interaction and forcing the data upgrade. The amount of data to be migrated and the time for data migration increase exponentially, that is, the larger the amount of data, the longer the user waits. If the user has a large amount of data, the user interaction will be blocked for a long time, and the user cannot use the APP during the data migration process, resulting in a poor user experience and user loss.

[0028] In view of the above technical problems, the present disclosure provides a data migration method, as Figure 1 shown, including:

[0029] Step S101, determine the migration method of the data to be migrated according to the data scale of the data to be migrated; wherein, the migration methods include overall migration and batch migration.

[0030] Step S102, according to the determined migration method, migrate the data to be migrated from the first database to the second database as a whole, or migrate the data to be migrated from the first database to the second database in batches.

[0031] Specifically, the data scale may include the number of data to be migrated and the volume of the data. For the data to be migrated with a small data scale, the overall migration method can be adopted, that is, the data to be migrated is migrated from the old database to the new database at one time. For the data to be migrated with a large data scale, the batch migration method can be adopted, and the data to be migrated is migrated from the old database to the new database in batches. According to the time required for data migration and combining the two different migration methods, it is possible to avoid blocking the user from using the APP due to long-term data migration and causing a poor user experience.

[0032] As an alternative implementation, in step S101, determining the migration method of the data to be migrated according to the data scale of the data to be migrated includes:

[0033] Judging whether the number of data to be migrated reaches a first preset quantity, and judging whether the volume of the data to be migrated reaches a preset volume;

[0034] In response to the number of data to be migrated not reaching the first preset quantity and the volume of the data to be migrated not reaching the preset volume, determining that the migration method of the data to be migrated is overall migration;

[0035] In response to the number of data to be migrated reaching the first preset quantity and / or the volume of the data to be migrated reaching the preset volume, determining that the migration method of the data to be migrated is batch migration.

[0036] Exemplarily, when the number of data to be migrated is less than 300,000 and the volume of the data to be migrated is less than 50 MB, it is judged that the scale of the data to be migrated is small and the time consumed for data migration is less, so the overall migration method can be adopted; when the number of data to be migrated reaches 300,000, or the volume of the data to be migrated reaches 50 MB, or the number reaches 300,000 and the volume reaches 50 MB at the same time, it is judged that the scale of the data to be migrated is large and the time consumed for data migration is more, so the batch migration method can be adopted to avoid the situation that the user cannot use the APP due to long-time data migration, resulting in poor user experience.

[0037] As an alternative implementation, migrating the data to be migrated from the first database to the second database as a whole includes:

[0038] Creating a new table in the second database and importing the content of the old table in the first database into the new table in the second database.

[0039] Specifically, in this embodiment, an implementation method for overall data migration is provided. A data migration transaction is started. In the data migration transaction, the method of "INSERT INTO (insert) new table name SELECT FROM (select from) old table name" is used to import the content of the old table into the new table structure. Since the data scale is small, the time consumed for overall data migration is also relatively short and can usually be completed within a few milliseconds. Therefore, users usually cannot perceive it and it will not affect the user's use of the APP.

[0040] As an alternative implementation, before creating a new table in the second database and importing the content of the old table in the first database into the new table in the second database, it further includes:

[0041] Attaching the first database and the second database so that the application program can access the first database and the second database simultaneously during the data migration process.

[0042] In this embodiment, while performing an overall migration of the data, the first database and the second database are also associated, and a connection is established between the first database and the second database, so that during the data migration process, the application program can still access the old database (the first database) and can also access the new database (the second database), and user interaction will not be blocked due to data migration.

[0043] As an optional implementation manner, migrating the data to be migrated from the first database to the second database in batches includes:

[0044] Associating the first database and the second database;

[0045] Creating a new view according to the data result set after merging the data of the first database and the second database; wherein, the new view is used to access the first database and the second database;

[0046] Dividing the data to be migrated into multiple data blocks according to a second preset quantity, and sequentially importing them from the first database into the second database until all the data to be migrated are imported into the second database.

[0047] Specifically, the "Union on" SQL (Structured Query Language, database language) operator can be used to merge the data in multiple tables into a result set and automatically remove duplicates. By creating a new view (View), during the data migration process, the data in the first database or the second database can be accessed arbitrarily. After creating the new view, start the APP background task and start the data migration transaction. The data to be migrated can be gradually imported from the first database into the second database at 2000 pieces of data per batch. After the data migration is completed, disassociate (DETACH) the first database and the second database, and delete the first database.

[0048] As an optional implementation manner, in step S102, while migrating the data to be migrated from the first database to the second database as a whole or in batches from the first database to the second database, it further includes:

[0049] Monitoring whether the main thread of the application program accesses the database, and generating an exception signal in response to the main thread accessing the database.

[0050] Specifically, whether it is a whole migration or a batch migration, during the data migration process, it is possible to monitor whether the main thread of the APP is accessing the database. The part that users perceive the most about the product is the UI (User Interface), and the main thread of the application is needed for UI refreshing. If the main thread is occupied to access the database during the data migration process, it will cause the APP to freeze. Therefore, when it is monitored that the main thread is accessing the database, an exception signal can be generated and fed back to the upper-layer APP caller to prevent the main thread from accessing the database and prevent the APP from freezing.

[0051] The present disclosure also provides a data migration device 200, as Figure 2 shown, including:

[0052] A determination module 201, configured to determine the migration method of the data to be migrated according to the data scale of the data to be migrated; wherein, the migration methods include whole migration and batch migration;

[0053] A data migration module 202, configured to migrate the data to be migrated from the first database to the second database as a whole or in batches according to the determined migration method.

[0054] Specifically, the data scale may include the number of data to be migrated and the volume of the data. For the data to be migrated with a small data scale, a whole migration method can be adopted, that is, the data to be migrated is migrated from the old database to the new database at one time. For the data to be migrated with a large data scale, a batch migration method can be adopted, and the data to be migrated is migrated from the old database to the new database in batches. According to the time required for data migration and combining the two different migration methods, it is possible to avoid blocking the user from using the APP due to long-term data migration and causing a poor user experience.

[0055] As an optional implementation manner, the determination module 201 determines the migration method of the data to be migrated according to the data scale of the data to be migrated, including:

[0056] Judging whether the number of data to be migrated reaches a first preset quantity, and judging whether the volume of the data to be migrated reaches a preset volume;

[0057] In response to the number of data to be migrated not reaching the first preset quantity and the volume of the data to be migrated not reaching the preset volume, determining that the migration method of the data to be migrated is whole migration;

[0058] In response to the number of data to be migrated reaching the first preset quantity and / or the volume of the data to be migrated reaching the preset volume, determining that the migration method of the data to be migrated is batch migration.

[0059] Exemplarily, when the number of data to be migrated is less than 300,000 and the volume of the data to be migrated is less than 50MB, it is determined that the scale of the data to be migrated is small and the time consumed for data migration is less. Therefore, the overall migration method can be adopted; when the number of data to be migrated reaches 300,000, or the volume of the data to be migrated reaches 50MB, or the number reaches 300,000 and the volume reaches 50MB at the same time, it is determined that the scale of the data to be migrated is large and the time consumed for data migration is more. Therefore, the batch migration method can be adopted to avoid the situation that the user cannot use the APP due to long-term data migration, resulting in poor user experience.

[0060] As an optional implementation manner, the data migration module migrating the data to be migrated from the first database to the second database as a whole includes:

[0061] Create a new table in the second database and import the content of the old table in the first database into the new table in the second database.

[0062] Specifically, in this embodiment, an implementation method for overall data migration is provided. Start a data migration transaction. In the data migration transaction, use the method of "INSERT INTO (insert) new table name SELECT FROM (select from) old table name" to import the content of the old table into the new table structure. Since the time consumed for overall data migration is relatively short when the data scale is small, usually it can be completed within a few milliseconds, so users usually cannot perceive it and it will not affect the user's use of the APP.

[0063] As an optional implementation manner, the device further includes:

[0064] An association module, configured to associate the first database and the second database before the data migration module creates a new table in the second database and imports the content of the old table in the first database into the new table in the second database, so that the application program can access the first database and the second database simultaneously during the data migration process.

[0065] In this embodiment, while migrating the data as a whole, the first database and the second database are also associated, and a connection is established between the first database and the second database, so that during the data migration process, the application program can still access the old database (the first database) and can also access the new database (the second database), and the user interaction will not be blocked due to data migration.

[0066] As an optional implementation manner, the data migration module migrating the data to be migrated from the first database to the second database in batches includes:

[0067] Associate the first database and the second database;

[0068] Create a new view based on the data result set after merging data from the first database and the second database; wherein, the new view is used to access the first database and the second database.

[0069] Divide the data to be migrated into multiple data blocks according to the second preset quantity, and import them into the second database from the first database in sequence until all the data to be migrated are imported into the second database.

[0070] Specifically, through the "Union on" SQL operator, the data in multiple tables can be merged into a result set and duplicates can be automatically removed. By creating a new view (View), during the data migration process, the data in the first database or the second database can be accessed arbitrarily. After creating the new view, start the APP background task and start the data migration transaction. The data to be migrated can be gradually imported from the first database into the second database in batches of 2000 data per batch. After the data migration is completed, disassociate (DETACH) the first database and the second database, and delete the first database.

[0071] As an optional implementation manner, the device further includes:

[0072] As Figure 3 shown, a monitoring module 301 is configured to monitor whether the main thread of the application accesses the database while the data migration module migrates the data to be migrated from the first database to the second database as a whole or in batches from the first database to the second database, and generate an exception signal in response to the main thread accessing the database.

[0073] Specifically, whether it is a whole migration or a batch migration, during the data migration process, it is possible to monitor whether the main thread of the APP is accessing the database. The UI is what users perceive the most about the product, and the main thread of the application is needed for UI refreshing. If the main thread is occupied to access the database during the data migration process, it will cause the APP to freeze. Therefore, when it is monitored that the main thread accesses the database, an exception signal can be generated and fed back to the upper-layer APP caller to prevent the main thread from accessing the database and prevent the APP from freezing.

[0074] As an optional implementation manner, as Figure 3As shown, a small traffic control module 302 can also be set on the client side, which is used to manage the permission of user data migration during the test phase, allowing a small number of users to perform data migration, so as to test the data migration process. The small traffic control module 302 accesses the first database and the second database through the database access adapter 303. The additional database module 304 can associate the first database 305 and the second database 306 to achieve table connection between different databases, so that users can access the first database and the second database simultaneously. The server interface can be used to control the small traffic and grant the permission of data migration to some active users. Users need to obtain the small traffic control permission before they can perform data migration. At the same time, the migration report, such as the migration time-consuming report, the main thread monitoring report, the data log report, etc., can also be displayed through the small traffic control supervision platform 307.

[0075] In the technical solution of the present disclosure, the acquisition, storage, and application of the user's personal information involved all comply with the provisions of relevant laws and regulations and do not violate public order and good customs.

[0076] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0077] Figure 4 FIG. shows a schematic block diagram of an exemplary electronic device 400 that can be used to implement the embodiments of the present disclosure. The electronic device is intended to represent various forms of digital computers, such as, a laptop computer, a desktop computer, a workbench, a personal digital assistant, a server, a blade server, a mainframe computer, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as, a personal digital processor, a cellular phone, a smart phone, a wearable device, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0078] As Figure 4 shown, the device 400 includes a computing unit 401, which can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 402 or the computer program loaded from the storage unit 408 into the random access memory (RAM) 403. In the RAM 403, various programs and data required for the operation of the device 400 can also be stored. The computing unit 401, the ROM 402, and the RAM 403 are connected to each other through a bus 404. The input / output (I / O) interface 405 is also connected to the bus 404.

[0079] Multiple components in device 400 are connected to I / O interface 405, including: an input unit 406, such as a keyboard, a mouse, etc.; an output unit 407, such as various types of displays, speakers, etc.; a storage unit 408, such as a disk, an optical disc, etc.; and a communication unit 409, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 409 allows device 400 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0080] The computing unit 401 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 401 include but are not limited to a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning objective function algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 401 executes the various methods and processes described above, such as the data migration method. For example, in some embodiments, the data migration method can be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as the storage unit 408. In some embodiments, part or all of the computer program can be loaded and / or installed onto device 400 via the ROM 402 and / or the communication unit 409. When the computer program is loaded into the RAM 403 and executed by the computing unit 401, one or more steps of the data migration method described above can be executed. Alternatively, in other embodiments, the computing unit 401 can be configured to execute the data migration method in any other suitable manner (e.g., by means of firmware).

[0081] Various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special or general-purpose programmable processor, and can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0082] The program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, a special purpose computer, or other programmable data processing apparatus, such that the program codes, when executed by the processor or controller, cause the functions / operations specified in the flowchart and / or block diagram to be implemented. The program code may execute entirely on the machine, partly on the machine, as a stand-alone software package partly on the machine and partly on a remote machine, or entirely on the remote machine or server.

[0083] In the context of the present disclosure, a machine-readable medium may be a tangible medium that can contain, or store a program for use by or in connection with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronics, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of a machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0084] In order to provide interaction with a user, the systems and techniques described herein may be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0085] The systems and techniques described herein can be implemented in a computing system including backend components (e.g., as a data server), or a computing system including middleware components (e.g., an application server), or a computing system including frontend components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with an implementation of the systems and techniques described herein), or a computing system including any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected to each other by digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), and the Internet.

[0086] A computer system can include a client and a server. The client and the server are generally remote from each other and typically interact through a communication network. The client - server relationship is generated by computer programs running on the respective computers and having a client - server relationship with each other. The server can be a cloud server, can also be a server of a distributed system, or a server incorporating a blockchain.

[0087] It should be understood that various forms of the processes shown above can be used, steps can be reordered, added, or deleted. For example, the steps recited in this disclosure can be executed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of this disclosure can be achieved, and this is not limited herein.

[0088] The above - mentioned specific embodiments do not constitute a limitation on the protection scope of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub - combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the protection scope of this disclosure.

Claims

1. A data migration method, comprising: Determining a migration method for the data to be migrated according to the data scale of the data to be migrated; wherein, the migration method includes overall migration and batch migration; If the migration method is overall migration, creating a new table in the second database and importing the content of the old table in the first database into the new table in the second database; If the migration method is batch migration, associating the first database and the second database; after merging the data of the first database and the second database using a union operator to obtain a data result set and removing duplicates, creating a new view; wherein, the new view is used to access the first database and the second database; dividing the data to be migrated into multiple data blocks according to a second preset quantity, and sequentially importing the data to be migrated from the first database into the second database until all the data to be migrated is imported into the second database; While migrating the data to be migrated from the first database to the second database as a whole or migrating the data to be migrated from the first database to the second database in batches, monitoring whether the main thread of the application program accesses the database, and in response to the main thread accessing the database, generating an exception signal to prevent the main thread from accessing the database.

2. The method according to claim 1, wherein, The determining the migration method for the data to be migrated according to the data scale of the data to be migrated includes: Judging whether the number of the data to be migrated reaches a first preset quantity and judging whether the volume of the data to be migrated reaches a preset volume; In response to the number of the data to be migrated not reaching the first preset quantity and the volume of the data to be migrated not reaching the preset volume, determining the migration method for the data to be migrated as overall migration; In response to the number of the data to be migrated reaching the first preset quantity and / or the volume of the data to be migrated reaching the preset volume, determining the migration method for the data to be migrated as batch migration.

3. The method according to claim 1, wherein Before creating a new table in the second database and importing the content of the old table in the first database into the new table in the second database, it further includes: Associating the first database and the second database so that the application program can access the first database and the second database simultaneously during the data migration process.

4. The method according to claim 1, wherein After dividing the data to be migrated into multiple batches according to a second preset quantity and sequentially importing the data to be migrated from the first database into the second database until all the data to be migrated is imported into the second database, it further includes: Disassociating the first database and the second database and deleting the first database.

5. A data migration device, comprising: A determining module configured to determine a migration method for the data to be migrated according to the data scale of the data to be migrated; wherein, the migration method includes overall migration and batch migration; A data migration module, configured to create a new table in the second database and import the content of the old table in the first database into the new table in the second database if the migration method is overall migration; if the migration method is batch migration, associate the first database and the second database; after performing a union operation on the first database and the second database to merge data to obtain a data result set and deduplicate it, create a new view; wherein, the new view is used to access the first database and the second database; divide the data to be migrated into multiple data blocks according to a second preset quantity, and sequentially import them from the first database into the second database until all the data to be migrated is imported into the second database; A monitoring module, configured to monitor whether the main thread of the application program accesses the database while the data migration module migrates the data to be migrated from the first database to the second database as a whole or migrates the data to be migrated from the first database to the second database in batches; In response to the main thread accessing the database, the monitoring module generates an exception signal to prevent the main thread from accessing the database.

6. The apparatus according to claim 5, wherein, The determination module determines the migration method of the data to be migrated according to the data scale of the data to be migrated, including: Judging whether the number of the data to be migrated reaches a first preset quantity, and judging whether the volume of the data to be migrated reaches a preset volume; In response to the number of the data to be migrated not reaching the first preset quantity and the volume of the data to be migrated not reaching the preset volume, determining that the migration method of the data to be migrated is overall migration; In response to the number of the data to be migrated reaching the first preset quantity, and / or the volume of the data to be migrated reaching the preset volume, determining that the migration method of the data to be migrated is batch migration.

7. The apparatus according to claim 5, wherein Further comprising: An association module, configured to associate the first database and the second database before the data migration module creates a new table in the second database and imports the content of the old table in the first database into the new table in the second database, so that the application program can access the first database and the second database simultaneously during the data migration process.

8. The device according to claim 5, wherein, After the data migration module divides the data to be migrated into multiple batches according to a second preset quantity and sequentially imports them from the first database into the second database until all the data to be migrated is imported into the second database, it further includes: Disassociate the first database and the second database, and delete the first database.

9. An electronic device, comprising: At least one processor; And A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method according to any one of claims 1-4.

10. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to execute the method according to any one of claims 1-4.

11. A computer program product comprising a computer program which, when executed by a processor, implements the method according to any one of claims 1 - 4.

Citation Information

Patent Citations

  • Lightweight concurrent migration method of database

    CN112783859A

  • Method, system and equipment for batch migration of big data tables between databases and medium

    CN114153813A