Data migration method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a data migration method and device, electronic equipment and a storage medium. The method comprises the steps of determining an abnormal object in a target machine room in response to an abnormal state of the target machine room in a process of executing data migration; determining a replacement object matched with the abnormal object in a pre-constructed disaster recovery machine room corresponding to the target machine room; replacing the abnormal object with a replacement object, and executing a data migration process based on the replacement object; the abnormal object comprises at least one of a first database cluster in the target machine room, a first collection library and a first migration terminal used for migrating data in the first database cluster to the first collection library, the first database cluster is a source end of the data migration process, and the first collection library is a target end of the data migration process; the replacement object comprises at least one of a second database cluster in the disaster recovery machine room, a second collection library and a second migration terminal used for migrating data in the second database cluster to the second collection library.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of database technology, and in particular to a data migration method, device, electronic device and storage medium. Background Art

[0002] With the advent of the digital age, the amount of data has gradually increased. Distributed databases have emerged in this context to achieve efficient data storage and processing.

[0003] In the prior art, in order to uniformly process the data of different databases in the target computer room according to the actual business needs, the data in each database in the database cluster is usually migrated to the same collection library of the target computer room to unify and organize the data from different databases. However, since the target computer room may have a fault during the data migration process, the aggregation fails, affecting business processing. Summary of the invention

[0004] The embodiments of the present invention provide a data migration method, device, electronic device and storage medium to ensure that the data migration process can be effectively executed, thereby achieving the purpose of improving the stability of the data aggregation process and avoiding affecting business processing.

[0005] According to one aspect of the present invention, there is provided a data migration method, comprising:

[0006] In response to an abnormal state of the target computer room during data migration, determining an abnormal object in the target computer room;

[0007] In a pre-built disaster recovery computer room corresponding to the target computer room, determining a replacement object that matches the abnormal object;

[0008] replacing the abnormal object with the replacement object, and performing the data migration process based on the replacement object;

[0009] Among them, the abnormal object includes at least one of a first database cluster, a first collection library and a first migration terminal used to migrate data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; the replacement object includes at least one of a second database cluster, a second collection library and a second migration terminal used to migrate data in the second database cluster to the second collection library in the disaster recovery computer room.

[0010] According to another aspect of the present invention, there is provided a data migration device, the device comprising:

[0011] an abnormal object determination module, configured to determine an abnormal object in the target computer room in response to an abnormal state of the target computer room during data migration;

[0012] A replacement object determination module, used to determine a replacement object matching the abnormal object in a pre-built disaster recovery computer room corresponding to the target computer room;

[0013] A data migration module, used to replace the abnormal object with the replacement object, and perform the data migration process based on the replacement object;

[0014] Among them, the abnormal object includes at least one of a first database cluster, a first collection library and a first migration terminal used to migrate data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; the replacement object includes at least one of a second database cluster, a second collection library and a second migration terminal used to migrate data in the second database cluster to the second collection library in the disaster recovery computer room.

[0015] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the data migration method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data migration method described in any embodiment of the present invention when executed.

[0020] The technical solution of the embodiment of the present invention is to determine the abnormal object in the target computer room in response to the abnormal state of the target computer room during the data migration process; wherein the abnormal object includes at least one of the first database cluster, the first collection library and the first migration terminal for migrating the data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; by determining the replacement object matching the abnormal object in the pre-built disaster recovery computer room corresponding to the target computer room; wherein the replacement object includes at least one of the second database cluster, the second collection library and the second migration terminal for migrating the data in the second database cluster to the second collection library in the disaster recovery computer room. Finally, the abnormal object is replaced with the replacement object, and the data migration process is performed based on the replacement object; thereby, through the replacement object in the disaster recovery computer room, it is ensured that the data migration process can be effectively executed, the stability of the data aggregation process is improved, and the impact on business processing is avoided.

[0021] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 is a flow chart of a data migration method provided according to an embodiment of the present invention;

[0024] Figure 2 is a schematic diagram of a data migration process provided according to an embodiment of the present invention;

[0025] Figure 3 is a flow chart of a data migration method provided according to an embodiment of the present invention;

[0026] Figure 4 is a structural schematic diagram of a data migration device provided according to an embodiment of the present invention;

[0027] Figure 5 It is a structural schematic diagram of an electronic device for implementing the data migration method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "etc." and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0030] It should be noted that the collection, collection, updating, analysis, processing, use, transmission, storage and other aspects of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken for user personal information to prevent illegal access to user personal information data and maintain the security of user personal information and network security.

[0031] Figure 1 This is a flow chart of a data migration method provided according to an embodiment of the present invention. This embodiment is applicable to the case where business data in a database cluster is migrated to a collection library to aggregate the business data. The method can be executed by a data migration device, which can be implemented in the form of hardware and / or software.

[0032] like Figure 1 As shown, the method of this embodiment may specifically include:

[0033] S110 . In response to an abnormal state of the target computer room during data migration, determining an abnormal object in the target computer room.

[0034] In order to clearly and accurately describe the data migration process, please refer to Figure 2. The target computer room includes a first database cluster, a first collection library, and a first migration terminal for migrating data in the first database cluster to the first collection library. The first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process. The disaster recovery computer room is a backup computer room used to recover the target business of the target computer room when a failure occurs in the target computer room. The disaster recovery computer room includes a second database cluster, a second collection library, and a second migration terminal for migrating data in the second database cluster to the second collection library. The second database cluster is synchronized with the data in the first database cluster, and the second collection library is synchronized with the data in the first collection library.

[0035] Replication and disaster recovery switching of different computer rooms are achieved through DRSP (database replication and disaster recovery switching) in different computer rooms. Each computer room aggregates the data in the database cluster to the collection library through the migration terminal. The abnormal state includes a fault state caused by a fault in at least one of the first database cluster, the first collection library and the first migration terminal in the target computer room; it may also include a non-working state of the target computer room caused by a pre-set work plan and a disaster recovery pre-drill state.

[0036] In this embodiment, an object in an abnormal state in the target computer room may be regarded as an abnormal object. The target computer room includes at least one of a first database cluster, a first collection library, and a first migration terminal for migrating data in the first database cluster to the first collection library.

[0037] In a specific implementation, different abnormal objects can be determined for different abnormal states. Exemplarily, when it is detected that the abnormal state is a fault state caused by a fault in at least one of the first database cluster, the first collection library, and the first migration terminal in the target computer room, the object with the fault can be used as an abnormal object. For example, if the first collection library and the first database cluster fail, both the first database cluster and the first collection library can be used as abnormal objects. When it is detected that the abnormal state is a non-working state caused by a pre-set work plan, the object that is planned to be in a non-working state in the work plan can be used as an abnormal object. For example, if the first database cluster needs to be replaced, the first database cluster is an abnormal object. When it is detected that the abnormal state is a disaster recovery pre-drill state, since the pre-drill is to verify the high availability of the disaster recovery computer room, the target computer room as a whole can be used as an abnormal object, that is, the first database cluster, the first collection library, and the first migration terminal are all abnormal objects.

[0038] S120: Determine a replacement object that matches the abnormal object in a pre-built disaster recovery computer room corresponding to the target computer room.

[0039] In this embodiment, a replacement object corresponding to the abnormal object can be determined in the disaster recovery room. The replacement object is an object in the disaster recovery room that is used to replace the abnormal object during data migration. The replacement object includes at least one of a second database cluster in the disaster recovery room, a second collection library, and a second migration terminal for migrating data in the second database cluster to the second collection library.

[0040] Specifically, the replacement object is an object in the disaster recovery room that has the same function as the abnormal object during the data migration process. Exemplarily, if the abnormal object is the first database cluster, the replacement object is the second database cluster; if the abnormal object is the first collection library, the replacement object is the second collection library; if the abnormal object is the first migration terminal, the replacement object is the second migration terminal. If the abnormal objects are the first database cluster and the first collection library, the replacement objects may be the second database cluster and the second collection library.

[0041] S130: Replace the abnormal object with a replacement object, and perform a data migration process based on the replacement object.

[0042] In order to ensure that the data migration process can be executed normally, the abnormal object can be replaced with a replacement object, and the data migration process can be continued through the replacement object. Exemplarily, the data migration process in the target computer room is: the process of migrating the data in the first database cluster to the first collection library through the first migration terminal abnormal object. When the abnormal object is the first database cluster, the replacement object is the second database cluster. After the second database cluster replaces the first database cluster, the data migration process is: migrating the data in the second database cluster to the first collection library through the first migration terminal. It should be noted that since the second database cluster is a cluster used for disaster recovery, the data in the first database cluster is synchronized with the data in the second database cluster.

[0043] Optionally, when executing the data migration process based on the replacement object, the data interruption point of the abnormal object in the abnormal state during the data migration process can be determined, that is, the information of the last data migrated in the original data migration process; based on the data interruption point, the starting point information of the replacement object is determined, and the data migration process is continued for the data corresponding to the starting point information, thereby avoiding the process of repeated data migration.

[0044] In this embodiment, the abnormal state includes a non-working state of the target computer room caused by a pre-set work plan, the work plan includes a migration plan for the target computer room, a replacement plan for the first database cluster, and a replacement plan for the first collection library; the abnormal object is replaced with a replacement object, and a data migration process is performed based on the replacement object, including: when the first collection library receives all business data generated by the first database cluster before the business stop time corresponding to the work plan, the abnormal object is replaced with the replacement object; based on the replacement object, a data migration operation is performed on the business data generated after the business stop time.

[0045] It should be noted that for a planned replacement operation, the business can be stopped in advance according to the work plan, and the business stop time can be recorded. Based on the business stop time, the data migration operation is performed.

[0046] Optionally, when it is detected that the target computer room is in a non-working state, it can be determined whether the first collection library has received all the business data generated by the first database cluster; if so, the abnormal object can be directly replaced with the replacement object, and the data migration operation of the business data generated after the business stop time can be continued. If not, it is necessary to ensure that the first collection library receives all the business data generated before the business stop time, replace the abnormal object with the replacement object, and continue the data migration operation.

[0047] Specifically, the method for determining whether the first collection library has received all the business data generated by the first database cluster may be: determining the sending time in the first database cluster corresponding to the latest data received by the first collection library, if the sending time is later than or equal to the business stop time, it means that the first collection library has received all the business data generated by the first database cluster; otherwise, it means that the first collection library has not received all the business data. The method for determining the sending time may be: calling the reset interface of the first migration terminal, obtaining the playback delay pre-recorded on the first migration terminal, and determining the sending time of the latest data based on the playback delay. The playback delay is the delay generated when the first database cluster migrates data to the first collection library.

[0048] Furthermore, when the target computer room is in working state, the replacement object can be replaced with the object in the target computer room to complete the switchback operation; and after the switchback operation, the data migration operation can be continued.

[0049] In a specific implementation, at least one of the first database cluster, the first collection library, and the first migration terminal may be replaced, thereby achieving replacement of any object in the target computer room, as well as the entire computer room, to ensure that the data migration operation is not affected when executing the work plan.

[0050] In this embodiment, the abnormal state includes a disaster recovery pre-drill state; after the abnormal object is replaced with a replacement object and a data migration process is performed based on the replacement object, it also includes: in response to the pre-drill end state of the disaster recovery computer room, stopping the data migration process, and determining the amount of migration data generated by the second migration terminal and the amount of playback data generated by the second collection library after the replacement object performs the data migration process; when the amount of migration data matches the amount of playback data, based on the synchronization site information recorded at the start time of the drill corresponding to the pre-stored disaster recovery pre-drill state, data reverse rollback processing is performed on the second database cluster and the second collection library to restore the disaster recovery computer room to the state corresponding to the start time of the drill.

[0051] It should be noted that the disaster recovery pre-drill is used to simulate the scenario where the entire target computer room is unavailable, and to verify whether the disaster recovery computer room can effectively and promptly respond to the overall failure of the target computer room. The second migration terminal in the disaster recovery computer room creates cold standby tasks that correspond one to one with the local tasks in the target computer room in advance, and periodically obtains the execution site information and delay information of the local tasks in a task-by-task manner through a script; wherein the delay information includes: the delay between the first database cluster and the first collection library, the delay between the first database cluster and the second database cluster, and the delay between the first collection library and the second collection library. In addition, the execution site information and delay information are written to the second database cluster. Exemplarily, the execution site information and delay information can be obtained every 2 seconds.

[0052] In a specific implementation, when it is detected that the disaster recovery room is in the disaster recovery pre-drill state, the network can be disconnected, and the synchronization location information can be determined based on the delay information and the execution location information. In addition, the data corresponding to the synchronization location information is used as the starting data by the disaster recovery room to continue the data migration operation.

[0053] After the disaster recovery pre-drill is completed, when it is detected that the disaster recovery computer room is in the pre-drill end state, the data migration process of the disaster recovery computer room can be stopped, and the amount of migration data generated by the second migration terminal in the disaster recovery pre-drill state and the amount of playback data received by the second collection library are determined. If the amount of migration data matches the amount of playback data, it means that there is no data omission in the migration process, the disaster recovery computer room can accurately respond to the failure of the target computer room, and the drill is successful. If the amount of migration data does not match the amount of playback data, it means that there is an error in the migration process, the disaster recovery computer room cannot respond to the failure of the target computer room, and the drill fails.

[0054] Furthermore, if the drill is successful, in order to enable the target computer room to work normally according to the state before the drill, the second database cluster and the second collection library are subjected to data reverse rollback based on the synchronization site information recorded at the drill start time corresponding to the pre-stored disaster recovery pre-drill state, so that the disaster recovery computer room is restored to the state corresponding to the drill start time. In addition, the migration task corresponding to the second migration terminal is reset to restore to the state before the drill.

[0055] In this embodiment, by determining the amount of migrated data and the amount of replayed data, the drill result of the disaster recovery computer room can be effectively determined; and after the drill is completed, a reverse rollback is performed so that the drill operation does not affect the normal operation of the target computer room.

[0056] In order to facilitate the management of the data migration process, a graphical management interface can be provided to display the data volume of the first database cluster in the target computer room and the data volume of the second database cluster in the disaster recovery computer room. The interface can also display the business processing speed. Through the graphical management interface, business personnel can intuitively view the business combing situation.

[0057] The technical solution of the embodiment of the present invention is to determine the abnormal object in the target computer room in response to the abnormal state of the target computer room during the data migration process; wherein the abnormal object includes at least one of the first database cluster, the first collection library and the first migration terminal for migrating the data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; by determining the replacement object matching the abnormal object in the pre-built disaster recovery computer room corresponding to the target computer room; wherein the replacement object includes at least one of the second database cluster, the second collection library and the second migration terminal for migrating the data in the second database cluster to the second collection library in the disaster recovery computer room. Finally, the abnormal object is replaced with the replacement object, and the data migration process is performed based on the replacement object; thereby, through the replacement object in the disaster recovery computer room, it is ensured that the data migration process can be effectively executed, the stability of the data aggregation process is improved, and the impact on business processing is avoided.

[0058] Figure 3It is a flowchart of a data migration method provided according to an embodiment of the present invention. In this embodiment, based on the above embodiment, optionally, the abnormal state includes a fault state caused by a fault in at least one of the first database cluster, the first collection library and the first migration terminal in the target computer room; the data migration process is performed based on the replacement object, including: based on the predetermined playback delay between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, determining the starting point information of the replacement object in executing the data migration process; and performing the data migration process based on the starting point information and the replacement object. Among them, the explanations of the terms that are the same or corresponding to the above embodiments are not repeated here. Figure 3 As shown, the method includes:

[0059] S210 . In response to an abnormal state of the target computer room during data migration, determine an abnormal object in the target computer room.

[0060] Optionally, the abnormal state includes a fault state caused by a fault in at least one of the first database cluster, the first collection library, and the first migration terminal in the target computer room. The abnormal object can be determined by detecting the working states of the first database cluster, the first collection library, and the first migration terminal in the target computer room. Exemplarily, the fault may include power failure, network disconnection, and cluster downtime.

[0061] S220: Determine a replacement object that matches the abnormal object in a pre-built disaster recovery computer room corresponding to the target computer room.

[0062] S230. Replace the abnormal object with a replacement object, and determine the starting position information of the replacement object in executing the data migration process based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state; and execute the data migration process based on the starting position information and the replacement object.

[0063] The playback delay is the delay generated in the process of migrating the data in the first database cluster to the first collection library, that is, the time difference between the time when the data is sent in the first database cluster and the time when the data is stored in the first collection library. The starting position information is the position information of the starting migration data when the replacement object performs the data migration process after the replacement object replaces the abnormal object. Exemplarily, the starting position information can be a timestamp and a data identifier.

[0064] In this embodiment, the fault state may include at least one of a collection library failure state caused by a fault in the first collection library in the target computer room, a cluster failure state caused by a fault in the first database cluster in the target computer room, and a migration failure state caused by a fault in the first migration terminal in the target computer room.

[0065] In a specific implementation, the abnormal object can be determined by detecting the working conditions of the first database cluster, the first collection library and the first migration terminal in the target computer room. In addition, during the data migration process, the playback delay between the first database cluster and the first collection library can be detected, and the playback delay can be recorded in the second database cluster. Furthermore, the first delay between the first database cluster and the second database cluster, and the second delay between the first collection library and the second collection library can be recorded in the second database cluster, so as to facilitate the determination of the starting point information.

[0066] In order to improve the accuracy of determining the starting position information, the playback delay, the first delay and the second delay may be periodically detected, and the playback delay, the first delay and the second delay may be updated based on the detection results. Alternatively, the first delay, the playback delay and the second delay may be collected in sequence each time the data synchronization between the first database cluster and the second database cluster, the data migration between the first database cluster and the first collection library, and the data synchronization between the first collection library and the second collection library is performed.

[0067] In a specific implementation, when the fault state is a cluster fault state caused by a fault in the first database cluster in the target computer room, the abnormal object is the first database cluster and the replacement object is the second database cluster. Based on the predetermined playback delay between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the implementation method of determining the starting point information of the replacement object in the data migration process includes: determining the first point time corresponding to the received data of the first collection library at the cluster fault time based on the playback delay and the cluster fault time corresponding to the cluster fault state; determining the second point time corresponding to the received data of the second database cluster at the cluster fault time based on the predetermined cluster fault time and the first delay between the first database cluster and the second database cluster; and when the first point time is earlier than the second point time, using the first point time as the starting point information of the data migration process of the second database cluster.

[0068] In this embodiment, the time obtained by subtracting the playback delay from the cluster failure time is used as the first point time. The time obtained by subtracting the first delay from the cluster failure time is used as the second point time. The first point time is used to reflect the sending time in the first database cluster corresponding to the received data received by the first collection library at the cluster failure time. The second point time is used to reflect the sending time in the first database cluster corresponding to the received data of the second database cluster at the cluster failure time. Exemplarily, the cluster failure time may be t1, the playback delay is p, and the first delay is sd, then the first point time is t1-p=Tx1; the second point time is t1-sd=Ty1.

[0069] It should be noted that after the failure of the first database cluster, all newly generated business data are stored in the second database cluster, and the second database cluster can replace the first database cluster to continue to perform data migration operations. Before performing the migration operation through the second database cluster, if the first point time is earlier than the second point time, it means that the amount of data synchronized from the first database cluster to the second database cluster is greater than the amount of data migrated from the first database cluster to the first collection library. In order to avoid data omissions during the data migration process, the differential data can be rolled back, that is, the first point time is used as the starting point information of the data migration process performed by the second database cluster. After replacing the first database cluster with the second database cluster, the data corresponding to the starting point information is used as the starting data, and the second database cluster continues to send data to the first collection library from the starting data to complete the data migration process, thereby ensuring that the data sent from the first database cluster between the first point time and the second point time can be effectively transmitted to the first collection library.

[0070] Furthermore, if the first point time is not earlier than the second point time, it means that the amount of data synchronized from the first database cluster to the second database cluster is less than or equal to the amount of data migrated from the first database cluster to the first collection library. In this case, the data in the second database cluster can be compared and corrected with the data in the first database cluster to synchronize and align the data in the first database cluster and the second database cluster, and the second point time can be used as the starting point information to ensure that the data is not migrated repeatedly when migrating data through the second database cluster.

[0071] In this embodiment, the fault state may be a collection library fault state caused by a fault in the first collection library in the target computer room. When the fault state is a collection library fault state, the abnormal object is the first collection library and the replacement object is the second collection library. Based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the specific implementation method of determining the starting point information of the replacement object in executing the data migration process is as follows: based on the collection library failure time corresponding to the collection library failure state, the playback delay and the predetermined second delay between the first collection library and the second collection library, determine the third point time corresponding to the received data of the second collection library at the collection library failure time; use the third point time as the starting point information of the first database cluster executing the data migration process after the second collection library replaces the first collection library.

[0072] It should be noted that when a failure occurs in the first collection library, the second collection library can be used as a replacement object, and the second collection library replaces the first collection library to continue to perform the data migration operation.

[0073] Optionally, the status of the first collection library can be detected. When the first collection library is detected to be down, the first migration terminal is required to promptly and proactively switch to the second collection library to migrate the data in the first database cluster to the second collection library. Before switching to the second collection library, the third point time can be determined. The third point time is determined as follows:

[0074] t2-p-td=Tx2

[0075] Among them, Tx2 is the third point time, t2 is the collection library failure time, p is the pre-determined playback delay, and td is the second delay.

[0076] In a specific implementation, the third point time can be used as the starting point information after the second collection library replaces the first collection library. The first database cluster performs differential data rollback based on the third point time to migrate the data to the second collection library through the first migration terminal according to the starting point information.

[0077] This embodiment provides a method for performing data migration operations when a collection library is in a fault state, thereby avoiding the impact on data migration due to the collection library failure and improving the stability of the data migration process.

[0078] In addition, the fault state may be a migration fault state caused by a fault in the first migration terminal in the target computer room. When the fault state is a migration fault state, the abnormal object is the first migration terminal and the replacement object is the second migration terminal. Based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the starting position information of the replacement object in executing the data migration process is determined, including: based on the migration failure time and the playback delay corresponding to the migration fault state, determining the fourth position time corresponding to the received data of the first collection library at the migration failure time; using the fourth position time as the starting position information of the first database cluster executing the data migration process after the second migration terminal replaces the first migration terminal.

[0079] It should be noted that after the first migration terminal fails, the first migration terminal may be replaced with a second migration terminal so as to migrate the data in the first database cluster to the first collection library through the second migration terminal.

[0080] In a specific implementation, the fourth point time may be determined by:

[0081] t3-p=Tx3

[0082] Among them, Tx3 is the fourth point time, t3 is the migration fault time, and p is the playback delay.

[0083] In this embodiment, the fourth location time can be used as the starting location information. The first migration terminal is replaced with the second migration terminal, and during the data migration process, the data of the first database cluster corresponding to the starting location information is migrated to the first collection library through the second migration terminal, thereby ensuring that in the event of a failure of the first migration terminal, the migration of the data of the first database cluster to the first collection library is not affected.

[0084] In this embodiment, the first database cluster, the first collection library and the first migration terminal in the target computer room may all fail. For this situation where the target computer room fails as a whole, the disaster recovery computer room can replace the entire target computer room. Specifically, when the target computer room works normally, the first migration terminal regularly records the collection site, playback delay, first delay and second delay, and records the collection site, playback delay, first delay and second delay to the second database cluster. When the target computer room fails as a whole, the computer room failure time t4 is recorded, and based on the playback delay and the computer room failure time, the fifth site time corresponding to the received data of the second collection library at the time of the computer room failure and the sixth site time corresponding to the received data of the second database cluster at the time of the computer room failure are determined. The fifth site time is determined as follows: t4-p-td=Tx4; the sixth site time is determined as follows: t4-sd=Ty4. If Tx4 is earlier than Ty4, Tx4 will be used as the starting point information; otherwise, it means that there are more data in the second collection library than in the second database cluster. It is necessary to synchronize the data in the second database cluster with the data in the second collection library, and then use the sixth point time as the starting point information to migrate the data in the second database cluster to the second collection library through the second migration terminal.

[0085] This embodiment utilizes a disaster recovery computer room to improve the availability of data migration and comprehensively considers unpredictable fault switching scenarios, thereby ensuring that data migration operations can be quickly continued in the event of a failure, ensuring that no data is lost, and safeguarding the data migration process.

[0086] Figure 4 1 is a schematic diagram of the structure of a data migration device provided according to an embodiment of the present invention, and the device is used to execute the data migration method provided in any of the above embodiments. The device and the data migration method of the above embodiments belong to the same inventive concept, and the details not described in detail in the embodiment of the data migration device can refer to the embodiment of the above data migration method. Figure 4 As shown, the device comprises:

[0087] The abnormal object determination module 10 is used to determine the abnormal object in the target computer room in response to the abnormal state of the target computer room during the data migration process;

[0088] A replacement object determination module 11 is used to determine a replacement object that matches the abnormal object in a pre-built disaster recovery computer room corresponding to the target computer room;

[0089] A data migration module 12, used to replace the abnormal object with a replacement object and perform a data migration process based on the replacement object;

[0090] Among them, the abnormal objects include at least one of the first database cluster, the first collection library and the first migration terminal for migrating data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; the replacement objects include at least one of the second database cluster, the second collection library and the second migration terminal for migrating data in the second database cluster to the second collection library in the disaster recovery computer room.

[0091] Based on any optional technical solution in the embodiment of the present invention, optionally, the abnormal state includes a fault state caused by a fault in at least one of the first database cluster, the first collection library, and the first migration terminal in the target computer room; the data migration module 12 includes:

[0092] A first information determination submodule, configured to determine the starting position information of the replacement object in the data migration process based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state;

[0093] The first data migration submodule is used to perform a data migration process based on the starting point information and the replacement object.

[0094] Based on any optional technical solution in the embodiment of the present invention, optionally, the fault state is a cluster fault state caused by a fault in the first database cluster in the target computer room, the abnormal object is the first database cluster, and the replacement object is the second database cluster;

[0095] The first information determination submodule includes:

[0096] A first point time determination unit is used to determine a first point time corresponding to the received data of the first collection library at the cluster failure moment based on the playback delay and the cluster failure moment corresponding to the cluster failure state;

[0097] A second location time determination unit, configured to determine a second location time corresponding to received data of the second database cluster corresponding to the cluster failure moment based on a predetermined cluster failure moment and a first time delay between the first database cluster and the second database cluster;

[0098] The first information determining unit is used to use the first point time as the starting point information for the second database cluster to perform a data migration process when the first point time is earlier than the second point time.

[0099] On the basis of any optional technical solution in the embodiment of the present invention, optionally, the fault state is a collection library fault state caused by a fault in a first collection library in a target computer room, the abnormal object is the first collection library, and the replacement object is the second collection library;

[0100] The first information determination submodule includes:

[0101] A third point time determination unit is used to determine a third point time corresponding to the received data of the second collection library at the collection library failure moment based on the collection library failure moment corresponding to the collection library failure state, the playback delay, and a predetermined second delay between the first collection library and the second collection library;

[0102] The second information determining unit is used to use the third site time as the starting site information of the first database cluster executing the data migration process after the second collection library replaces the first collection library.

[0103] On the basis of any optional technical solution in the embodiment of the present invention, optionally, the fault state is a migration fault state caused by a fault in a first migration terminal in a target computer room, the abnormal object is the first migration terminal, and the replacement object is the second migration terminal;

[0104] The first information determination submodule includes:

[0105] A fourth location time determination unit, configured to determine a fourth location time corresponding to the received data of the first collection library at the migration failure time based on the migration failure time corresponding to the migration failure state and the playback delay;

[0106] The third information determining unit is configured to use the fourth location time as the starting location information of the first database cluster executing the data migration process after the second migration terminal replaces the first migration terminal.

[0107] Based on any optional technical solution in the embodiment of the present invention, optionally, the abnormal state includes a non-working state of the target computer room caused by a preset work plan, and the work plan includes a migration plan for the target computer room, a replacement plan for the first database cluster, and a replacement plan for the first collection library;

[0108] The data migration module 12 includes:

[0109] A replacement submodule, configured to replace the abnormal object with a replacement object when the first collection library receives all the business data generated by the first database cluster before the business stop time corresponding to the work plan;

[0110] The second data migration submodule is used to perform data migration operations on the business data generated after the business stop time based on the replacement object.

[0111] Based on any optional technical solution in the embodiment of the present invention, optionally, the abnormal state includes a disaster recovery pre-drill state; and further includes:

[0112] A data volume determination module is used to stop the data migration process in response to the end state of the pre-drill of the disaster recovery computer room after replacing the abnormal object with the replacement object and executing the data migration process based on the replacement object, and determine the amount of migration data generated by the second migration terminal and the amount of playback data generated by the second collection library after the replacement object executes the data migration process;

[0113] The reverse rollback processing module is used to perform data reverse rollback processing on the second database cluster and the second collection library based on the synchronization site information recorded at the start time of the drill corresponding to the pre-stored disaster recovery pre-drill state when the amount of migrated data matches the amount of replayed data, so as to restore the disaster recovery computer room to the state corresponding to the start time of the drill.

[0114] The technical solution of the embodiment of the present invention is to determine the abnormal object in the target computer room in response to the abnormal state of the target computer room during the data migration process; wherein the abnormal object includes at least one of the first database cluster, the first collection library and the first migration terminal for migrating the data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; by determining the replacement object matching the abnormal object in the pre-built disaster recovery computer room corresponding to the target computer room; wherein the replacement object includes at least one of the second database cluster, the second collection library and the second migration terminal for migrating the data in the second database cluster to the second collection library in the disaster recovery computer room. Finally, the abnormal object is replaced with the replacement object, and the data migration process is performed based on the replacement object; thereby, through the replacement object in the disaster recovery computer room, it is ensured that the data migration process can be effectively executed, the stability of the data aggregation process is improved, and the impact on business processing is avoided.

[0115] It is worth noting that in the embodiment of the above-mentioned data migration device, the various units and modules included are only divided according to functional logic, but are not limited to the above-mentioned division, as long as the corresponding functions can be achieved; in addition, the specific names of the functional units are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of the present invention.

[0116] Figure 5Schematic diagram of the structure of an electronic device for implementing the data migration method of an embodiment of the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0117] like Figure 5 As shown, the electronic device 20 includes at least one processor 21, and a memory connected to the at least one processor 21, such as a read-only memory (ROM) 22, a random access memory (RAM) 23, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 21 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 22 or the computer program loaded from the storage unit 28 to the random access memory (RAM) 23. In RAM23, various programs and data required for the operation of the electronic device 20 can also be stored. The processor 21, ROM22 and RAM23 are connected to each other through a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0118] A number of components in the electronic device 20 are connected to the I / O interface 25, including: an input unit 26, such as a keyboard, a mouse, etc.; an output unit 27, such as various types of displays, speakers, etc.; a storage unit 28, such as a disk, an optical disk, etc.; and a communication unit 29, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 29 allows the electronic device 20 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0119] The processor 21 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 21 executes the various methods and processes described above, such as the data migration method.

[0120] In some embodiments, the data migration method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 20 via the ROM 22 and / or the communication unit 29. When the computer program is loaded into the RAM 23 and executed by the processor 21, one or more steps of the data migration method described above may be performed. Alternatively, in other embodiments, the processor 21 may be configured to perform the data migration method in any other appropriate manner (e.g., by means of firmware).

[0121] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0122] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0123] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0124] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0125] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0126] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0127] This embodiment also provides a computer program product, including a computer program, which, when executed by a processor, implements the data migration method provided in any embodiment of the present application.

[0128] In the process of implementation, the computer program product can be written in one or more programming languages ​​or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

[0129] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0130] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A data migration method, characterized in that: include: In response to an abnormal state of the target computer room during data migration, determining an abnormal object in the target computer room; In a pre-built disaster recovery computer room corresponding to the target computer room, determining a replacement object that matches the abnormal object; replacing the abnormal object with the replacement object, and performing the data migration process based on the replacement object; Among them, the abnormal object includes at least one of a first database cluster, a first collection library and a first migration terminal used to migrate data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; the replacement object includes at least one of a second database cluster, a second collection library and a second migration terminal used to migrate data in the second database cluster to the second collection library in the disaster recovery computer room.

2. The method according to claim 1, characterized in that The abnormal state includes a fault state caused by a fault in at least one of the first database cluster, the first collection library, and the first migration terminal in the target computer room; The performing the data migration process based on the replacement object includes: Determine the starting position information of the replacement object in the data migration process based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state; Based on the starting point information and the replacement object, the data migration process is performed.

3. The method according to claim 2, characterized in that The fault state is a cluster fault state caused by a fault in the first database cluster in the target computer room, the abnormal object is the first database cluster, and the replacement object is the second database cluster; The determining, based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the starting position information of the replacement object in executing the data migration process includes: Based on the playback delay and the cluster failure time corresponding to the cluster failure state, determining a first point time corresponding to the received data of the first collection library at the cluster failure time; Determine, based on the predetermined cluster failure moment and a first delay between the first database cluster and the second database cluster, a second location time corresponding to the received data of the second database cluster corresponding to the cluster failure moment; In the case where the first site time is earlier than the second site time, the first site time is used as the starting site information for the second database cluster to perform a data migration process.

4. The method according to claim 2, characterized in that: The fault state is a collection library fault state caused by a fault in the first collection library in the target computer room, the abnormal object is the first collection library, and the replacement object is the second collection library; The determining, based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the starting position information of the replacement object in executing the data migration process includes: Determine, based on the collection library failure moment corresponding to the collection library failure state, the playback delay, and a predetermined second delay between the first collection library and the second collection library, a third location time corresponding to the received data of the second collection library at the collection library failure moment; After the third site time is used as the starting site information of the data migration process performed by the first database cluster after the second collection library replaces the first collection library.

5. The method according to claim 2, characterized in that: The fault state is a migration fault state caused by a fault in the first migration terminal in the target computer room, the abnormal object is the first migration terminal, and the replacement object is the second migration terminal; The determining, based on the predetermined playback delay generated between the first database cluster and the first collection library during the data migration process and the fault time corresponding to the fault state, the starting position information of the replacement object in executing the data migration process includes: Determine, based on the migration failure moment corresponding to the migration failure state and the playback delay, a fourth location time corresponding to the data received by the first collection repository at the migration failure moment; The fourth location time is used as the starting location information of the first database cluster executing the data migration process after the second migration terminal replaces the first migration terminal.

6. The method according to claim 1, characterized in that The abnormal state includes a non-working state of the target computer room caused by a preset work plan, wherein the work plan includes a migration plan for the target computer room, a replacement plan for the first database cluster, and a replacement plan for the first collection library; The replacing the abnormal object with the replacement object and performing the data migration process based on the replacement object includes: When the first collection repository receives all the business data generated by the first database cluster before the business stop time corresponding to the work plan, the abnormal object is replaced by the replacement object; Based on the replacement object, a data migration operation is performed on the service data generated after the service stop time.

7. The method according to claim 1, characterized in that The abnormal state includes a disaster recovery pre-drill state; After replacing the abnormal object with the replacement object and performing the data migration process based on the replacement object, the process further includes: In response to the end state of the pre-drill of the disaster recovery computer room, the data migration process is stopped, and the amount of migration data generated by the second migration terminal and the amount of playback data generated by the second collection library are determined after the replacement object executes the data migration process; When the amount of migrated data matches the amount of replayed data, based on the synchronization site information recorded at the start time of the drill corresponding to the pre-stored disaster recovery pre-drill state, data reverse rollback processing is performed on the second database cluster and the second collection library to restore the disaster recovery computer room to the state corresponding to the start time of the drill.

8. A data migration device, characterized in that: include: an abnormal object determination module, configured to determine an abnormal object in the target computer room in response to an abnormal state of the target computer room during data migration; A replacement object determination module, used to determine a replacement object matching the abnormal object in a pre-built disaster recovery computer room corresponding to the target computer room; A data migration module, used to replace the abnormal object with the replacement object, and perform the data migration process based on the replacement object; Among them, the abnormal object includes at least one of a first database cluster, a first collection library and a first migration terminal used to migrate data in the first database cluster to the first collection library in the target computer room, the first database cluster is the source end of the data migration process, and the first collection library is the target end of the data migration process; the replacement object includes at least one of a second database cluster, a second collection library and a second migration terminal used to migrate data in the second database cluster to the second collection library in the disaster recovery computer room.

9. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can perform the data migration method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the data migration method according to any one of claims 1 to 7 when executed.

Images