Data migration method and device

Abstract

The invention provides a data migration method and device, and the method comprises the steps: responding to a condition that a target database is in an initialization state, and judging whether a target instance of a running target scheme is initialized or not; in response to the condition that the target instance is not initialized, reading scheme configuration information of the target scheme; according to the scheme configuration information, generating a to-be-processed task queue of the target scheme to complete initialization of a target instance; and processing the plurality of to-be-processed timed tasks by adopting the target instance so as to realize data migration of the to-be-migrated data, so that the data migration process is decomposed into the plurality of to-be-processed timed tasks, and the to-be-processed timed tasks can be gradually processed according to the sequence in the to-be-processed task queue, so that the problem that the amount of data involved during migration is too large cannot be caused, and the migration efficiency of the to-be-migrated data is improved. And meanwhile, the data migration of the to-be-migrated data is automatically and orderly completed, so that the data migration efficiency and accuracy are improved, and the possibility of human intervention and errors is reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data migration method and device. Background Art

[0002] As the system goes online for longer, more and more data will accumulate in the system, which may cause performance problems for the system. Therefore, some historical data in the system needs to be migrated.

[0003] At present, data migration is usually performed by writing scripts and manually scheduling them regularly. Since the manual scheduling method is cumbersome, the interval between two data migrations is usually longer. However, the longer the interval between two data migrations, the larger the amount of data involved in the migration, resulting in a longer data migration time each time. In addition, the data migration efficiency using manual scheduling is low, which may affect the normal use of system data. Summary of the invention

[0004] The present disclosure provides a data migration method and device to solve one of the technical problems in the related art at least to a certain extent. The technical solution of the present disclosure is as follows:

[0005] According to a first aspect of an embodiment of the present disclosure, a data migration method is provided, comprising: in response to a target database being in an initialization state, determining whether a target instance running a target solution has completed initialization; wherein the target solution is used to migrate data to be migrated in the target database; in response to the target instance not completing initialization, reading solution configuration information of the target solution; based on the solution configuration information, generating a queue of pending tasks of the target solution to complete the initialization of the target instance; wherein the task queue includes a plurality of pending scheduled tasks; and using the target instance to process the plurality of pending scheduled tasks to implement data migration of the data to be migrated.

[0006] According to a second aspect of an embodiment of the present disclosure, a data migration device is provided, comprising: a judgment module, for judging whether a target instance running a target solution has completed initialization in response to a target database being in an initialization state; wherein the target solution is used to migrate data to be migrated in the target database; a reading module, for reading solution configuration information of the target solution in response to the target instance not completing initialization; a generating module, for generating a queue of pending tasks of the target solution according to the solution configuration information to complete the initialization of the target instance; wherein the task queue includes a plurality of pending scheduled tasks; and a processing module, for processing the plurality of pending scheduled tasks using the target instance to realize data migration of the data to be migrated.

[0007] According to a third aspect of an embodiment of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to execute the instructions to implement the data migration method as described in the embodiment of the first aspect of the present disclosure.

[0008] According to a fourth aspect of an embodiment of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device can execute the data migration method as described in the embodiment of the first aspect of the present disclosure.

[0009] According to a fifth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising: a computer program, wherein when the computer program is executed by a processor, the data migration method as described in the embodiment of the first aspect of the present disclosure is implemented.

[0010] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0011] In the present technical scheme, in response to the target database being in the initialization state, it is determined whether the target instance of the target scheme has completed initialization; wherein, the target scheme is used to migrate the data to be migrated in the target database; in response to the target instance not completing initialization, the scheme configuration information of the target scheme is read; according to the scheme configuration information, a queue of pending tasks of the target scheme is generated to complete the initialization of the target instance; wherein, the task queue includes multiple pending scheduled tasks; the target instance is used to process the multiple pending scheduled tasks to realize the data migration of the data to be migrated; thus, by checking the initialization state of the target instance, and in the case that the target instance has not completed initialization, the queue of pending tasks of the target scheme is generated according to the scheme configuration information of the target scheme, so that the target instance is in a correct and ready state before data migration, and the data migration process is decomposed into multiple pending scheduled tasks, and the pending scheduled tasks will be processed step by step according to the order in the queue of pending tasks, so that even if the interval between two data migrations is long, it will not cause migration. The problem of too much data involved in the process is solved, and the data migration of the data to be migrated is completed automatically and orderly, which not only improves the efficiency and accuracy of data migration, but also reduces the possibility of human intervention and errors; before executing each process node of the target solution, it is judged whether the user's solution control instruction is received, which improves the controllability of data migration, and when the solution control instruction is not received, the pending scheduled tasks corresponding to the current process node are accurately obtained from multiple pending scheduled tasks, which realizes the pertinence and accuracy of data migration and avoids unnecessary waste of resources; then, it is judged whether the current process node is a process node for batch submission of transactions, which helps to adopt corresponding processing strategies according to different types of process nodes, and improves the efficiency of data migration. Furthermore, when the current process node is a process node for batch submission of transactions, the target instance is used to perform at least one batch processing, which realizes the batch and efficiency of data migration, so that a large amount of data can be processed quickly and accurately, while improving the reliability and efficiency of data migration, and enhancing the controllability and flexibility of the entire migration process.

[0012] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0014] Figure 1 is a flowchart of a data migration method shown in the first embodiment of the present disclosure;

[0015] Figure 2 is a flow chart of a data migration method shown in the second embodiment of the present disclosure;

[0016] Figure 3 is a flowchart of a data migration method shown in the third embodiment of the present disclosure;

[0017] Figure 4 is a flowchart of a data migration method shown in the fourth embodiment of the present disclosure;

[0018] Figure 5 is a schematic diagram of a process of initializing the current instance shown in an embodiment of the present disclosure;

[0019] Figure 6 It is a schematic diagram of the creation process of a scheduled task to be processed shown in an embodiment of the present disclosure;

[0020] Figure 7 is a schematic diagram of the processing flow of the target solution shown in the embodiment of the present disclosure;

[0021] Figure 8 is a structural schematic diagram of a data migration device shown in the fifth embodiment of the present disclosure;

[0022] Fig. 9 It is a schematic structural diagram of an electronic device shown in an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0023] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.

[0025] It should be noted that in the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information are all carried out with the user's consent, and are in compliance with relevant laws and regulations and do not violate public order and good morals.

[0026] The data migration method and device according to the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0027] Figure 1 It is a flowchart of the data migration method shown in the first embodiment of the present disclosure.

[0028] like Figure 1 As shown, the data migration method includes the following steps:

[0029] Step 101 , in response to a target database being in an initialization state, determining whether a target instance running a target solution has completed initialization.

[0030] The target solution is used to migrate the data to be migrated in the target database.

[0031] In order to avoid unnecessary waste of resources and time consumption, as a possible implementation method, by querying the status of the target database (such as REDIS), it is quickly determined whether it is necessary to execute the initialization action of the target instance running the target solution.

[0032] As an example, when the target database is in an initialization state, it is determined whether the target instance running the target solution has completed initialization, wherein the target solution is used to migrate the data to be migrated in the target database.

[0033] As another example, when the target database is not in an initialized state, for example, the target database is in a completed state, it means that the relevant data migration or solution process configuration has been completed, and there is no need to initialize the target instance of the target solution.

[0034] Step 102: In response to the target instance not completing initialization, read the solution configuration information of the target solution.

[0035] In order to perform data migration according to actual user needs and improve the flexibility of data migration, as an example, when the target instance is initialized, the solution configuration information of the target solution is read from the target database. It should be noted that the solution configuration information is obtained by the user through configuration on the visual configuration page according to needs.

[0036] Step 103: Generate a to-be-processed task queue of the target solution according to the solution configuration information to complete the initialization of the target instance.

[0037] The task queue includes multiple scheduled tasks to be processed.

[0038] In order to improve the accuracy and efficiency of data migration, as an example, the solution configuration information is read and parsed, and then a plurality of pending scheduled tasks are created according to the solution configuration information, and the plurality of pending scheduled tasks are written into a pending task queue.

[0039] It should be noted that in order to ensure that the initialization of the target instance is re-executed when the solution configuration information is updated, the state of the target database is updated from the initialization state to the completion state when the target instance initialization is completed, and when the solution configuration information is updated, the state of the target database is updated from the completion state to the initialization state.

[0040] Step 104 , using the target instance to process the multiple scheduled tasks to be processed, so as to achieve data migration of the data to be migrated.

[0041] Furthermore, based on the target instance, multiple pending scheduled tasks are automatically executed, and multiple pending scheduled tasks perform data migration operations according to a predetermined schedule and rules, thereby achieving efficient and accurate migration of the data to be migrated from the source end to the target end.

[0042] In summary, in response to the target database being in an initialization state, it is determined whether the target instance of the target solution has completed initialization; wherein the target solution is used to migrate the data to be migrated in the target database; in response to the target instance not completing initialization, the solution configuration information of the target solution is read; according to the solution configuration information, a queue of pending tasks of the target solution is generated to complete the initialization of the target instance; wherein the task queue includes a plurality of pending scheduled tasks; the target instance is used to process the plurality of pending scheduled tasks to realize the data migration of the data to be migrated; thus, by checking the initialization state of the target instance and in the case that the target instance has not completed initialization, In this case, a pending task queue of the target solution is generated according to the solution configuration information of the target solution, so that the target instance is in a correct and ready state before data migration, and the data migration process is decomposed into multiple pending scheduled tasks. The pending scheduled tasks will be processed step by step according to the order in the pending task queue. Therefore, even if the interval between two data migrations is long, it will not cause the problem of excessive data volume involved in the migration. At the same time, the data migration of the data to be migrated is completed automatically and orderly. This automated processing method based on task queue not only improves the efficiency and accuracy of data migration, but also reduces the possibility of human intervention and errors.

[0043] In order to clearly illustrate how the target instance is used in the above embodiment to process multiple scheduled tasks to be processed, the present disclosure proposes another data migration method.

[0044] Figure 2 It is a flowchart of a data migration method shown in the second embodiment of the present disclosure.

[0045] like Figure 2 As shown, the data migration method includes the following steps:

[0046] Step 201 , in response to a target database being in an initialization state, determining whether a target instance running a target solution has completed initialization.

[0047] The target solution is used to migrate the data to be migrated in the target database.

[0048] Step 202: In response to the target instance not completing initialization, read the solution configuration information of the target solution.

[0049] Step 203: Generate a to-be-processed task queue of the target solution according to the solution configuration information to complete the initialization of the target instance.

[0050] The task queue includes multiple scheduled tasks to be processed.

[0051] Step 204 , in response to not receiving the solution control instruction, determining a current process node from at least one process node of the target solution.

[0052] Among them, the solution control instruction is used to terminate or pause the current process node.

[0053] In order to improve the controllability of data migration, as a possible implementation method, before executing each process node of the target solution, it is determined whether a solution control instruction of the user is received, wherein the solution control instruction is used to terminate or pause the current process node.

[0054] As an example, when no solution control instruction is received, a current process node to be executed is determined from at least one process node of the target solution, and then, a pending scheduled task corresponding to the current process node is executed.

[0055] As another example, when a solution control instruction is received, the execution status of the current process node and the execution status of the target solution are updated, and the current process node is paused or terminated.

[0056] Step 205, obtaining the pending scheduled task corresponding to the current process node from the multiple pending scheduled tasks.

[0057] In order to make the data migration process proceed according to the predetermined process sequence and logic and avoid confusion and conflict between tasks, as an example, each process node corresponds to at least one pending scheduled task, and from multiple pending scheduled tasks, the pending scheduled tasks that match the current process node are identified and extracted.

[0058] Step 206 , determining whether the current process node in the target solution is a process node for batch submission transactions.

[0059] It should be understood that each process node in the target solution has a corresponding flag bit, and the value of the flag bit is used to indicate whether the corresponding process node is a process node for batch submission transactions.

[0060] As an example, the value of the flag bit of the current process node in the target solution is used to determine whether the current process node in the target solution is a process node for batch submission transactions. For example, the value of the flag bit is "true", indicating that the current process node is a process node for batch submission transactions, and the value of the flag bit is "false", indicating that the current process node is a process node for non-batch submission transactions.

[0061] Step 207 , in response to the current process node being a process node for submitting a batch transaction, using the target instance, at least one batch process is performed on the data to be migrated indicated by the scheduled task to be processed corresponding to the current process node.

[0062] As an example, when the current process node is a process node for submitting transactions in batches, the target instance is used to divide the data to be migrated indicated by the pending scheduled tasks corresponding to the current process node into at least one batch for processing.

[0063] Wherein, one of the batch processes in at least one batch process comprises the following steps:

[0064] 1. Determine whether a control instruction for the batch processing scheme is received; i is a positive integer;

[0065] 2. If not, then obtaining the data to be migrated processed in one of the batches from the data to be migrated;

[0066] 3. Use the target instance to batch process the data to be migrated in one of the batches.

[0067] That is to say, in order to further improve the controllability of the data migration process, for the first batch processing, it is determined whether the solution control instruction is received. If the solution control instruction is not received, the first batch of data to be migrated is determined from the data to be migrated, and the target instance is used to batch process the first batch of data to be migrated; at the end of the first batch processing, it is determined whether the solution control instruction is received. If the solution control instruction is not received, the second batch of data to be migrated is obtained from the data to be migrated; the target instance is used to batch process the second batch of data to be migrated; and so on, until the last batch processing of at least one batch processing is completed; for each batch processing, if the solution control instruction is received, the data migration is suspended or terminated, and the execution status of the current process node to which the batch belongs and the execution status of the target solution are updated.

[0068] In order to facilitate the monitoring of the execution progress of data migration, as an example, in response to the completion of the last batch processing in at least one batch processing, it is determined whether the current process node is the last process node in the target solution; when the current process node is the last process node, the execution status of the current process node and the execution status of the target solution are updated; if the current process node is not the last process node, the execution status of the current process node is updated; in addition, each time a solution control instruction is received, the execution status of the process node corresponding to the received solution control instruction and the execution status of the target solution are updated.

[0069] As another example, in response to the current process node being a process node for non-batch submission transactions, the processing logic of the pending scheduled tasks corresponding to the current process node is obtained, and the target instance is used to process the data to be migrated indicated by the pending scheduled tasks corresponding to the current process node according to the processing logic.

[0070] That is to say, when the current process node is a process node for non-batch submission transactions, the pending scheduled tasks corresponding to the current process node have pre-defined processing logic. Therefore, the target instance is used to accurately and efficiently process the data to be migrated indicated by the pending scheduled tasks corresponding to the current process node according to the defined processing logic. This processing process ensures that data migration is both in line with business logic and can be carried out efficiently in non-batch mode.

[0071] It should be noted that the execution process of steps 201 to 203 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0072] In summary, in response to not receiving a solution control instruction, the current process node is determined from at least one process node of the target solution; the pending scheduled tasks corresponding to the current process node are obtained from multiple pending scheduled tasks; it is determined whether the current process node in the target solution is a process node for batch submission transactions; in response to the current process node being a process node for batch submission transactions, the target instance is used to perform at least one batch processing on the data to be migrated indicated by the pending scheduled tasks corresponding to the current process node. Therefore, before executing each process node of the target solution, it is determined whether the user's solution control instructions are received, thereby improving the controllability of data migration. Furthermore, when the solution control instructions are not received, the pending scheduled tasks corresponding to the current process node are accurately obtained from multiple pending scheduled tasks, thereby achieving the pertinence and accuracy of data migration and avoiding unnecessary waste of resources. Next, judging whether the current process node is a process node for batch submission transactions helps to adopt corresponding processing strategies according to different types of process nodes, thereby improving data migration efficiency. Finally, when the current process node is a process node for batch submission transactions, the target instance is used to perform at least one batch processing, thereby achieving batch and high efficiency of data migration, enabling large amounts of data to be processed quickly and accurately, while improving the reliability and efficiency of data migration, and enhancing the controllability and flexibility of the entire migration process.

[0073] In order to clearly explain how to generate a to-be-processed task queue of a target solution according to solution configuration information in the above embodiment, the present disclosure proposes another data migration method.

[0074] Figure 3 It is a flowchart of a data migration method shown in the third embodiment of the present disclosure.

[0075] like Figure 3 As shown, the data migration method includes the following steps:

[0076] Step 301 , in response to the target database being in an initialization state, determining whether the target instance running the target solution has completed initialization.

[0077] The target solution is used to migrate the data to be migrated in the target database.

[0078] Step 302: In response to the target instance not completing initialization, read the solution configuration information of the target solution.

[0079] Step 303: In response to obtaining the database lock of the target database, it is determined whether the creation of multiple scheduled tasks to be processed is completed.

[0080] In order to ensure the consistency and integrity of the data and prevent data conflicts or inconsistencies during task creation and processing, as an example, the database lock of the target database is obtained. When the database lock of the target database is obtained, it is determined whether multiple pending scheduled tasks have been created, that is, when the target database is locked, it is determined whether the scheduled tasks have been created.

[0081] Step 304 , in response to the fact that multiple pending scheduled tasks have not been created, the pending scheduled tasks of multiple process nodes of the target solution are sequentially created according to the solution configuration information, and written into the pending task queue.

[0082] Among them, each process node corresponds to at least one scheduled task to be processed.

[0083] In order to reduce human intervention and errors, as a possible implementation method, when multiple pending scheduled tasks are not completely created, a pending task queue with multiple pending scheduled tasks is automatically generated according to the solution configuration information.

[0084] As an example, according to the solution configuration information, the pending scheduled tasks of multiple process nodes of the target solution are created in sequence. When the pending scheduled tasks of all process nodes are created, the pending scheduled tasks of each process node are written into the pending task queue in sequence.

[0085] As another example, according to the solution configuration information, the pending scheduled tasks of each process node of the target solution are created in sequence, and when the pending scheduled tasks of each process node are created, the pending scheduled tasks are written into the pending task queue.

[0086] Step 305 , using the target instance to process the multiple scheduled tasks to be processed, so as to achieve data migration of the data to be migrated.

[0087] It should be noted that the execution process of steps 301 to 302 and step 305 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0088] In summary, in response to obtaining the database lock of the target database, it is determined whether multiple pending scheduled tasks have been created. Through the database lock mechanism, conflicts or data inconsistencies caused by multiple migration tasks operating the same data resource at the same time are avoided. At the same time, judging the status of pending scheduled tasks in a locked state can timely discover and handle problems in the task creation process, ensure the effective implementation of data migration, and improve the reliability and stability of data migration. Furthermore, in the case that multiple pending scheduled tasks have not been created, the pending scheduled tasks are automatically created according to the solution configuration information, reducing human intervention and errors, and writing the pending scheduled tasks into the pending task queue realizes the orderly execution of tasks, avoids resource conflicts and repeated processing, and improves processing efficiency.

[0089] In order to achieve orderly execution of tasks during data migration and avoid resource conflicts and repeated processing, as an example, according to the solution configuration information, the pending scheduled tasks of each process node of the target solution are created in sequence. When the pending scheduled tasks of each process node are created, the pending scheduled tasks are written into the pending task queue. Then, the target instance is used to process multiple pending scheduled tasks to achieve data migration of the data to be migrated. Figure 4 The embodiments are described in detail.

[0090] Figure 4 It is a flowchart of a data migration method shown in the fourth embodiment of the present disclosure.

[0091] like Figure 4 As shown, the data migration method includes the following steps:

[0092] Step 401 , in response to the target database being in an initialization state, determining whether the target instance running the target solution has completed initialization.

[0093] The target solution is used to migrate the data to be migrated in the target database.

[0094] Step 402: In response to the target instance not completing initialization, read the solution configuration information of the target solution.

[0095] Step 403: In response to obtaining the database lock of the target database, it is determined whether the creation of multiple pending scheduled tasks is completed.

[0096] Step 404 , in response to the fact that multiple pending scheduled tasks have not been created, a pending scheduled task for the current process node is created according to the solution configuration information.

[0097] In order to ensure that data is migrated according to user needs, as a possible implementation method, when multiple pending scheduled tasks have not been created, a pending scheduled task for the current process node is created based on the solution configuration information.

[0098] Step 405 , in response to the completion of the creation of the pending scheduled task of the current process node, the pending scheduled task of the current process node is written into the pending task queue.

[0099] In order to avoid conflicts between tasks, further, when the pending scheduled tasks of the current process node are created, the pending scheduled tasks of the current process node are written into the pending task queue.

[0100] Step 406, determining whether the current process node is the last process node in the target solution.

[0101] In order to avoid unnecessary operations and reduce resource waste, as an example, it is determined whether the current process node is the last process node in the target solution, wherein the target solution includes m process nodes.

[0102] Step 407: If yes, then the writing process of the pending task queue ends.

[0103] As an example, when the current process node is the last process node in the target solution, that is, the pending timings of all process nodes of the target solution have been created and written into the pending task queue, the writing process of the pending task queue is ended.

[0104] Step 408, if not, create a pending scheduled task for the next process node of the current process node according to the solution configuration information, so that when the pending scheduled task for the next process node is created, the pending scheduled task for the next process node is written into the pending task queue.

[0105] As another example, when the current process node is not the last process node in the target plan, continue to create the pending scheduled tasks of the next process node of the current process node according to the plan configuration information, and when the pending scheduled tasks of the next process node are created, write the pending scheduled tasks of the next process node into the pending task queue.

[0106] In order to avoid occupying database resources for a long time and reduce the waiting time of other threads or processes, as an example, when multiple pending scheduled tasks are written into the pending task queue, the database lock is released, or when multiple pending scheduled tasks are created, the database lock is released.

[0107] Step 409 , using the target instance to process the multiple scheduled tasks to be processed, so as to achieve data migration of the data to be migrated.

[0108] It should be noted that the execution process of steps 401 to 403 and step 409 can be implemented in any manner in the embodiments of the present disclosure, and the embodiments of the present disclosure do not limit this and will not be described in detail.

[0109] In summary, by creating a pending scheduled task for the current process node according to the scheme configuration information when multiple pending scheduled tasks have not been completed, the orderliness and accuracy of task creation are achieved, and task omission or repeated creation is avoided. After the pending scheduled task of the current process node is created, the pending scheduled task is written into the pending task queue, which achieves orderly queuing and efficient scheduling of tasks. At the same time, the queue mechanism realizes the processing of pending scheduled tasks in a predetermined order and priority, reducing the waiting time and conflict between pending scheduled tasks. Then, by judging whether the current process node is the last process node in the target scheme, the system can accurately control the writing process of the pending task queue. If the current process node is the last node, the writing of the pending task queue is terminated, avoiding unnecessary resource occupation and waste; if the current process node is not the last process node, the pending scheduled task of the next process node is continued to be created according to the scheme configuration information, and written into the pending task queue, ensuring that the entire process can be carried out in a predetermined order and logic, and improving the integrity of the pending task queue.

[0110] Based on any embodiment of the present disclosure, the data migration method of the embodiment of the present disclosure can also be implemented based on the following steps:

[0111] 1. When starting, a cyclic timing task will be initialized to implement the initialization action of the instance. The initialization flow chart is as follows Figure 5 As shown, the specific steps include:

[0112] 1) Get the initialization status of REDIS (target database). If the current instance (target instance) is not initialized, start the subsequent initialization action, otherwise exit directly;

[0113] 2) Read the configuration information of the solution process in the database (solution configuration information);

[0114] 3) Loop configuration information and create a scheduled task for the solution process;

[0115] 4) After all process configurations are created, the initialization status is updated to completed, that is, the status of the target database is updated from the initialization status to the completed status;

[0116] 5) If the front-end adjusts the solution process configuration information, the initialization status on REDIS is cleared, that is, the status of the target database is updated from the completed status to the initialized status;

[0117] 2. Loop configuration information, create the core logic of the program process timing task is only responsible for generating the task queue of the process involved in the current program, and the generated task queue is consumed and processed by other background tasks.

[0118] Among them, the processing flow chart of loop configuration information and creation of program flow timing task is as follows Figure 6 As shown, the specific steps include:

[0119] 1) Try to acquire the lock. If the lock is acquired successfully, continue with the following actions, otherwise exit;

[0120] 2) After acquiring the lock, determine whether a task has been created. If it has been created, exit directly after unlocking;

[0121] 3) If it has not been created, create a pending task queue corresponding to the solution according to the configuration information of the current solution process;

[0122] 4) After the creation is completed, the lock is released and the processing logic is exited;

[0123] 3. After consumers receive the task to be consumed, they follow Figure 7 The process shown performs each step as follows:

[0124] 1) Before executing the solution step logic, first determine whether the current process node is suspended or terminated. If so, update the log status and exit the logic directly; otherwise, continue with the following logic;

[0125] 2) If the current process node is a process node for non-batch submission transactions, start the transaction before starting execution, and then execute the steps one by one in sequence until the last step is completed, update the log status, and end the solution processing logic;

[0126] 3) If the current process node is a process node for batch transaction submission, when the step is obtained, first determine whether it is a step for starting a batch transaction. If so, determine whether the process is suspended or terminated. If not, start the transaction, and then execute the logic of the steps that require batch submission. After completion, submit the batch transaction. After the transaction is submitted, determine whether there are any unprocessed batches. If there are, continue to process the next batch.

[0127] Corresponding to the data migration method provided in the above-mentioned embodiment, the present disclosure also provides a data migration device. Since the data migration device provided in the embodiment of the present disclosure corresponds to the data migration method provided in the above-mentioned embodiment, the implementation method of the data migration method is also applicable to the data migration device provided in the embodiment of the present disclosure, and will not be described in detail in the embodiment of the present disclosure.

[0128] Figure 8It is a structural diagram of a data migration device shown in the fifth embodiment of the present disclosure.

[0129] like Figure 8 As shown, the data migration device 800 includes: a judgment module 810, a reading module 820, a generation module 830 and a processing module 840.

[0130] Among them, the judgment module 810 is used to judge whether the target instance running the target solution has completed initialization in response to the target database being in an initialization state; wherein the target solution is used to migrate the data to be migrated in the target database; the reading module 820 is used to read the solution configuration information of the target solution in response to the target instance not completing initialization; the generation module 830 is used to generate a queue of pending tasks of the target solution according to the solution configuration information to complete the initialization of the target instance; wherein the task queue includes multiple pending scheduled tasks; the processing module 840 is used to use the target instance to process multiple pending scheduled tasks to realize data migration of the data to be migrated.

[0131] As a possible implementation method of the embodiment of the present disclosure, the processing module 840 is used to determine the current process node from at least one process node of the target solution in response to not receiving the solution control instruction; wherein the solution control instruction is used to terminate or pause the current process node; obtain the pending scheduled task corresponding to the current process node from multiple pending scheduled tasks; determine whether the current process node in the target solution is a process node for batch submission transactions; in response to the current process node being a process node for batch submission transactions, use the target instance to perform at least one batch processing on the data to be migrated indicated by the pending scheduled task corresponding to the current process node.

[0132] As a possible implementation method of the embodiment of the present disclosure, one of the batch processing in at least one batch processing includes: determining whether a scheme control instruction of one of the batch processing is received; if not, obtaining the data to be migrated of one of the batch processing from the data to be migrated; and using the target instance to batch process the data to be migrated of one of the batch processing.

[0133] As a possible implementation method of the embodiment of the present disclosure, the processing module 840 is also used to determine whether the current process node is the last process node in the target solution in response to the completion of the last batch processing in at least one batch processing; in response to the current process node being the last process node, updating the execution status of the current process node and the execution status of the target solution; in response to the current process node being not the last process node, updating the execution status of the current process node.

[0134] As a possible implementation method of the embodiment of the present disclosure, the processing module 840 is also used to obtain the processing logic of the pending scheduled task corresponding to the current process node in response to the current process node being a process node for non-batch submission transactions; and use the target instance to process the data to be migrated indicated by the pending scheduled task corresponding to the current process node according to the processing logic.

[0135] As a possible implementation method of the embodiment of the present disclosure, the generation module 830 is used to determine whether multiple pending scheduled tasks have been created in response to obtaining the database lock of the target database; in response to multiple pending scheduled tasks not being created, create the pending scheduled tasks of multiple process nodes of the target solution in sequence according to the solution configuration information, and write them into the pending task queue; wherein each process node corresponds to at least one pending scheduled task.

[0136] As a possible implementation method of the embodiment of the present disclosure, the target plan includes m process nodes, and the generation module 830 is used to create a pending scheduled task for the current process node in response to the incomplete creation of multiple pending scheduled tasks according to the plan configuration information; in response to the completion of the creation of the pending scheduled task of the current process node, write the pending scheduled task of the current process node into the pending task queue; determine whether the current process node is the last process node in the target plan; if so, end the writing process of the pending task queue; if not, create a pending scheduled task for the next process node of the current process node according to the plan configuration information, so that when the creation of the pending scheduled task of the next process node is completed, the pending scheduled task of the next process node is written into the pending task queue.

[0137] As a possible implementation of the embodiment of the present disclosure, the data migration device 800 further includes: a release module.

[0138] The release module is used to release the database lock in response to multiple pending scheduled tasks being written into the pending task queue; or to release the database lock in response to multiple pending scheduled tasks being created.

[0139] As a possible implementation of the embodiment of the present disclosure, the data migration device 800 further includes: an update module.

[0140] The updating module is used to update the state of the target database from the initialization state to the completion state; in response to the update of the solution configuration information, the state of the target database is updated from the completion state to the initialization state.

[0141] The data migration device of the embodiment of the present disclosure determines whether the target instance of the target scheme is initialized in response to the target database being in the initialization state; wherein the target scheme is used to migrate the data to be migrated in the target database; in response to the target instance not being initialized, the scheme configuration information of the target scheme is read; according to the scheme configuration information, a queue of pending tasks of the target scheme is generated to complete the initialization of the target instance; wherein the task queue includes multiple scheduled tasks to be processed; the target instance is used to process the multiple scheduled tasks to be processed to achieve data migration of the data to be migrated; thus, by checking the initialization state of the target instance and generating the queue of pending tasks of the target scheme according to the scheme configuration information of the target scheme when the target instance is not initialized, the target instance is put in a correct and ready state before data migration, and the data migration process is decomposed into multiple scheduled tasks to be processed, and the scheduled tasks to be processed are processed step by step according to the order in the queue of pending tasks, so that even if the interval between two data migrations is long, it will not cause the problem of excessive amount of data involved in the migration, and at the same time, the data migration of the data to be migrated is automatically and orderly completed, which not only improves the efficiency and accuracy of data migration, but also reduces the possibility of human intervention and errors.

[0142] In an exemplary embodiment, an electronic device is also provided.

[0143] Among them, electronic equipment includes:

[0144] processor;

[0145] a memory for storing processor-executable instructions;

[0146] The processor is configured to execute instructions to implement the data migration method proposed in any of the aforementioned embodiments.

[0147] As an example, Fig. 9 is a schematic diagram of the structure of an electronic device 900 shown in an exemplary embodiment of the present disclosure, such as Fig. 9 As shown, the electronic device 900 may further include:

[0148] The memory 910 and the processor 920, a bus 930 connecting different components (including the memory 910 and the processor 920), the memory 910 stores a computer program, and when the processor 920 executes the program, the data migration method described in the embodiment of the present disclosure is implemented.

[0149] Bus 930 represents one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processor or a local bus using any of a variety of bus architectures. For example, these architectures include but are not limited to Industry Standard Architecture (ISA) bus, Micro Channel Architecture (MAC) bus, Enhanced ISA bus, Video Electronics Standards Association (VESA) local bus and Peripheral Component Interconnect (PCI) bus.

[0150] The electronic device 900 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by the electronic device 900, including volatile and non-volatile media, removable and non-removable media.

[0151] The memory 910 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 940 and / or cache memory 950. The server 900 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, the storage system 960 may be used to read and write non-removable, non-volatile magnetic media ( Fig. 9 not shown, usually called a "hard drive"). Although Fig. 9 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk"), and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, a DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to the bus 930 via one or more data medium interfaces. The memory 910 may include at least one program product having a set (e.g., at least one) of program modules that are configured to perform the functions of the various embodiments of the present disclosure.

[0152] A program / utility 980 having a set (at least one) of program modules 970 may be stored, for example, in the memory 910, such program modules 970 including but not limited to an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment. The program modules 970 generally perform the functions and / or methods of the embodiments described in the present disclosure.

[0153] The electronic device 900 may also communicate with one or more external devices 990 (e.g., keyboard, pointing device, display 991, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 900, and / or communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., network card, modem, etc.). Such communication may be performed through an input / output (I / O) interface 992. In addition, the electronic device 900 may also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN) and / or public network, such as the Internet) through a network adapter 993. As shown, the network adapter 993 communicates with other modules of the electronic device 900 through a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 900, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0154] The processor 920 executes various functional applications and data processing by running the programs stored in the memory 910 .

[0155] It should be noted that the implementation process and technical principles of the electronic device of this embodiment refer to the aforementioned explanation of the data migration method of the embodiment of the present disclosure, and will not be repeated here.

[0156] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, and the above instructions can be executed by a processor of an electronic device to complete the data migration method proposed in any of the above embodiments. Optionally, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0157] In an exemplary embodiment, a computer program product is further provided, including a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the data migration method proposed in any of the above embodiments.

[0158] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The description and examples are to be considered exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0159] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. A data migration method, characterized in that: include: In response to the target database being in an initialization state, determining whether the target instance running the target solution has completed initialization; wherein the target solution is used to migrate the data to be migrated in the target database; In response to the target instance not completing initialization, reading solution configuration information of the target solution; Generate a pending task queue of the target solution according to the solution configuration information to complete the initialization of the target instance; wherein the task queue includes a plurality of pending scheduled tasks; The target instance is used to process the multiple scheduled tasks to be processed to achieve data migration of the data to be migrated.

2. The method according to claim 1, characterized in that: The adopting the target instance to process the multiple scheduled tasks to be processed includes: In response to not receiving a solution control instruction, determining a current process node from at least one process node of the target solution; wherein the solution control instruction is used to terminate or pause the current process node; From the multiple scheduled tasks to be processed, obtain the scheduled task to be processed corresponding to the current process node; Determine whether the current process node in the target solution is a process node for batch submission of transactions; In response to the current process node being a process node for batch submission of transactions, the target instance is used to perform at least one batch process on the to-be-migrated data indicated by the to-be-processed scheduled task corresponding to the current process node.

3. The method according to claim 2, characterized in that One of the batch processes in the at least one batch process comprises: Determining whether one of the batch processing scheme control instructions is received; If not, obtaining the data to be migrated processed in one of the batches from the data to be migrated; The target instance is used to batch process the data to be migrated in one of the batches.

4. The method according to claim 2, characterized in that: The method further comprises: In response to the last batch processing of the at least one batch processing being completed, determining whether the current process node is the last process node in the target solution; In response to the current process node being the last process node, updating the execution status of the current process node and the execution status of the target solution; In response to the current process node being not the last process node, updating the execution state of the current process node.

5. The method according to claim 2, characterized in that: The adopting the target instance to process the plurality of scheduled tasks to be processed further includes: In response to the current process node being a process node for non-batch submission transactions, obtaining the processing logic of the to-be-processed scheduled task corresponding to the current process node; The target instance is used to process the data to be migrated indicated by the scheduled task to be processed corresponding to the current process node according to the processing logic.

6. The method according to claim 1, characterized in that The step of generating a queue of pending tasks of the target solution according to the solution configuration information includes: In response to acquiring the database lock of the target database, determining whether the plurality of pending scheduled tasks have been created; In response to the incomplete creation of the multiple pending scheduled tasks, the pending scheduled tasks of the multiple process nodes of the target solution are created in sequence according to the solution configuration information, and written into the pending task queue; wherein each of the process nodes corresponds to at least one pending scheduled task.

7. The method according to claim 6, characterized in that The target solution includes m process nodes, In response to the multiple pending scheduled tasks not being created, sequentially creating the pending scheduled tasks of the multiple process nodes of the target solution according to the solution configuration information, and writing them into the pending task queue, including: In response to the multiple pending scheduled tasks not being created, creating a pending scheduled task for the current process node according to the solution configuration information; In response to the completion of the creation of the pending scheduled task of the current process node, writing the pending scheduled task of the current process node into the pending task queue; Determine whether the current process node is the last process node in the target solution; If yes, then the writing process of the pending task queue is terminated; If not, create a pending scheduled task for the next process node of the current process node according to the scheme configuration information, so that when the pending scheduled task for the next process node is created, the pending scheduled task for the next process node is written into the pending task queue.

8. The method according to claim 6, characterized in that The method further comprises: In response to the plurality of pending scheduled tasks being written into the pending task queue, releasing the database lock; or, In response to the completion of creation of the plurality of pending scheduled tasks, the database lock is released.

9. The method according to claim 6, characterized in that After writing the pending scheduled tasks into the pending task queue according to the task execution times of the plurality of pending scheduled tasks, the method further includes: Updating the state of the target database from an initialization state to a completion state; In response to the scheme configuration information being updated, the state of the target database is updated from a completed state to an initialized state.

10. A data migration device, characterized in that: include: A judgment module, configured to judge whether the target instance running the target solution has completed initialization in response to the target database being in an initialization state; wherein the target solution is used to migrate the data to be migrated in the target database; A reading module, configured to read the solution configuration information of the target solution in response to the target instance not completing initialization; A generation module, used to generate a pending task queue of the target solution according to the solution configuration information to complete the initialization of the target instance; wherein the task queue includes a plurality of pending scheduled tasks; A processing module is used to process the multiple scheduled tasks to be processed using the target instance to achieve data migration of the data to be migrated.

Images