Data migration processing method and device, equipment and medium

Abstract

The invention provides a data migration processing method and device, equipment and a medium. In the scheme, the system comprises a control device and a data center device. And the control equipment sends a data locking message to the plurality of data center equipment storing the to-be-migrated data based on the data identifier of the to-be-migrated data input by the user, and after the plurality of data center equipment returns a data locking completion message, the control equipment verifies the data in the intermediate data tables in all the data center equipment. And after verification is completed, the control device sends a data format conversion message to each data center device, and the multiple data center devices execute data format conversion operation. And when the control equipment receives format conversion success messages returned by the plurality of data center equipment, data migration messages are sent to each data center equipment, so that the data migration process is completed. According to the data migration processing method provided by the invention, automation of the data migration process is realized through cooperation of the control equipment and the data center equipment.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method, device, equipment and medium for processing data migration. Background Art

[0002] With the rapid development of computer technology, various business systems have undergone a certain degree of upgrade and transformation. Among them, in the process of cross-domain account consolidation, the migration of relevant user data from the old system to the new system is crucial.

[0003] In the specific implementation scheme, when a cross-domain account consolidation operation needs to be performed, the deployed cross-domain account consolidation system first performs the migration operation of user data between different types of databases. Specifically, before and after the deployed cross-domain account consolidation system is upgraded, the user data related to the cross-domain account consolidation business is stored in different ORACLE databases and DRDS databases. When executing the cross-domain account consolidation business, the deployed cross-domain account consolidation system first extracts the relevant user data from different ORACLE databases, performs data conversion operations on the obtained user data according to the format of the DRDS database to store data, and transmits the obtained user data to the DRDS database to finally obtain complete user data.

[0004] However, when executing the user data migration process, if an error occurs in any link or the number of migrated users exceeds a certain threshold, the above cross-domain account consolidation system will stop the migration process and return an error message. The subsequent process needs to be triggered manually, and it is impossible to achieve the complete migration of user data in an automated manner, which is inefficient. Summary of the invention

[0005] The present application provides a data migration processing method, device, equipment and medium, which are used to solve the problem in the prior art that data migration operations rely too much on manual labor and have low efficiency.

[0006] In a first aspect, the present application provides a data migration processing method, which is applied to a control device, and the method includes:

[0007] In response to a user operation, obtaining a data identifier of the data to be migrated input by the user;

[0008] When it is determined according to the data identifier that the corresponding data to be migrated is not stored in the DRDS database, a data lock message is sent to multiple data center devices storing the data to be migrated according to a preset message sending rule, wherein the message sending rule is set to send a message to each data center device in the order of user, account, and customer;

[0009] After receiving the data locking completion messages returned by the multiple data center devices, checking whether the data stored in the intermediate data table of each data center device is repeated;

[0010] After all the data in the intermediate data table of each data center device has been verified, a data table verification message is sent to each data center device respectively, wherein the data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to a preset verification rule;

[0011] After receiving the data table verification pass message returned by the multiple data center devices, sending a data format conversion message to each data center device, wherein the data format conversion message includes a converted target format;

[0012] After receiving format conversion success messages returned by all the multiple data center devices, a data migration message is sent to each data center device, where the data migration message is used to instruct the data in the intermediate data table to be migrated to the DRDS database.

[0013] In a possible design of the first aspect, sending a data lock message to a plurality of data center devices storing the data to be migrated according to a preset message sending rule includes:

[0014] The data locking message is sent to the plurality of data center devices in sequence in the form of a message queue in the order of users, accounts, and customers, wherein the data locking message instructs each data center device to lock the storage space of the data to be migrated in the database.

[0015] In a possible design of the first aspect, the method further includes:

[0016] If a data locking failure message returned by any data center device is received, a first fallback message is sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

[0017] In a possible design of the first aspect, the method further includes:

[0018] If a data table verification failure message returned by any data center device is received, a second fallback message and a database unlocking message are respectively sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

[0019] In a possible design of the first aspect, the method further includes:

[0020] If a format conversion failure message returned by any data center device is received, a data conversion failure notification message is pushed to the user through a graphical user interface, and the data conversion failure notification message includes an identifier of the data center device where the conversion failed.

[0021] In a second aspect, the present application provides a data migration processing method, which is applied to a data center device, wherein an ORACLE database and a DRDS database are provided in the data center device, and the method comprises:

[0022] receiving a data locking message sent by a control device, wherein the data locking message includes a data identifier of the data to be migrated;

[0023] According to the data identifier of the data to be migrated, the data to be migrated is stored in an intermediate data table, and a storage space in the ORACLE database storing the data to be migrated is locked;

[0024] After the locking is completed, a data locking completion message is returned to the control device;

[0025] Receiving a data table verification message sent by the control device, wherein the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule;

[0026] Verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes;

[0027] receiving a data format conversion message sent by the control device, wherein the data format conversion message includes a target data format after conversion;

[0028] According to the data format conversion message, converting the format of the data to be migrated stored in the intermediate data table into the target data format;

[0029] After the format conversion of the data to be migrated is completed, sending a format conversion success message to the control device;

[0030] receiving a data migration message sent by the control device, wherein the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database;

[0031] The data to be migrated in the intermediate data table after the format conversion is stored in the DRDS database according to the data migration message.

[0032] In a possible design of the second aspect, the method further includes:

[0033] If locking the storage space storing the data to be migrated in the ORACLE database fails, sending a data locking failure message to the control device;

[0034] A first back-off message sent by the control device is received, and a back-off process is performed according to the first back-off message.

[0035] In a possible design of the second aspect, the method further includes:

[0036] If the verification of the intermediate data table fails, sending a data table verification failure message to the control device;

[0037] Receiving a second rollback message and a database unlocking message sent by the control device;

[0038] Rollback processing is performed according to the second rollback message, and after the rollback is completed, the storage space storing the data to be migrated in the ORACLE database is unlocked.

[0039] In a possible design of the second aspect, the method further includes:

[0040] If the format conversion of the data to be migrated in the intermediate data table fails, a format conversion failure message is sent to the control device, and the conversion time of each data and the failure reason of the data that failed to be converted are recorded in the intermediate data table.

[0041] In a third aspect, the present application provides a data migration processing device, including:

[0042] A receiving module, configured to obtain, in response to a user operation, a data identifier of the data to be migrated input by the user;

[0043] A sending module, configured to send a data lock message to a plurality of data center devices storing the data to be migrated according to a preset message sending rule, when it is determined according to the data identifier that the corresponding data to be migrated is not stored in the DRDS database, wherein the message sending rule is configured to send a message to each data center device in the order of user, account, and customer;

[0044] A processing module, configured to verify whether the data stored in the intermediate data table of each data center device is repeated after receiving the data locking completion message returned by the plurality of data center devices;

[0045] The sending module is also used to send a data table verification message to each data center device after all the data in the intermediate data table of each data center device has been verified, and the data table verification message is used to command the data center device to verify the data in the intermediate data table according to the preset verification rules;

[0046] The sending module is further used to send a data format conversion message to each data center device after receiving the data table verification pass message returned by the multiple data center devices, wherein the data format conversion message includes a converted target format;

[0047] The sending module is also used to send a data migration message to each data center device after receiving format conversion success messages returned by all the multiple data center devices, and the data migration message is used to instruct the migration of data in the intermediate data table to the DRDS database.

[0048] In a fourth aspect, the present application provides a data migration processing device, including:

[0049] A receiving module, configured to receive a data locking message sent by a control device, wherein the data locking message includes a data identifier of the data to be migrated;

[0050] A processing module, configured to store the data to be migrated in an intermediate data table according to the data identifier of the data to be migrated, and lock the storage space storing the data to be migrated in the ORACLE database;

[0051] A sending module, used for returning a data locking completion message to the control device after the locking is completed;

[0052] The receiving module is further used to receive a data table verification message sent by the control device, wherein the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule;

[0053] The processing module is also used to verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes;

[0054] The receiving module is further used to receive a data format conversion message sent by the control device, wherein the data format conversion message includes a target data format after conversion;

[0055] The processing module is further used to convert the format of the data to be migrated stored in the intermediate data table into the target data format according to the data format conversion message;

[0056] The sending module is further used to send a format conversion success message to the control device after the format conversion of the data to be migrated is completed;

[0057] The receiving module is also used to receive a data migration message sent by the control device, wherein the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database;

[0058] The processing module is further configured to store the data to be migrated in the intermediate data table after the format conversion into the DRDS database according to the data migration message.

[0059] In a fifth aspect, the present application provides an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0060] The memory stores computer-executable instructions;

[0061] The processor executes the computer-executable instructions stored in the memory to implement the data migration processing method as described in any one of the first to second aspects.

[0062] In a sixth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer execution instructions, and when the computer execution instructions are executed by a processor, the processing method for data migration as described in any one of the first to second aspects is implemented.

[0063] In a seventh aspect, the present application provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, it is used to implement the data migration processing method described in any one of the first to second aspects.

[0064] The present application provides a processing method, device, equipment and medium for data migration, and the processing method for data migration is applied to control equipment and data center equipment. After the control equipment determines that the corresponding data to be migrated is not stored in the DRDS database based on the identifier of the data to be migrated input by the user, it starts to interact with multiple data center equipment. During the whole process of interaction, the control equipment sends data lock messages, data table verification messages, data format conversion messages and data migration messages to multiple data center equipment at different stages according to the message sending rules, and multiple data center equipment performs corresponding operations after receiving the messages, and finally realizes the migration of format-converted data from the ORACLE database to the DRDS database. During the whole process of data migration, the control equipment and multiple data center equipment interact frequently without manual intervention, and finally successfully realize the migration of all data. By applying the above-mentioned data migration processing method, the user only needs to input the data to be queried in the control device, and the subsequent processes are all completed autonomously by the control device and the data center equipment, realizing the automation of the whole process of data migration and improving the efficiency of data migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0066] Figure 1A schematic diagram of an application scenario of the data migration processing method provided in this application;

[0067] Figure 2 A flowchart of a first embodiment of a method for processing data migration provided by the present application;

[0068] Figure 3 An illustration of the data migration process provided for this application;

[0069] Figure 4 A flowchart of a second embodiment of the data migration processing method provided by the present application;

[0070] Figure 5 A flowchart of a third embodiment of a data migration processing method is provided for this application;

[0071] Figure 6 A flowchart of a fourth embodiment of the data migration processing method provided by the present application;

[0072] Figure 7 A schematic diagram of the structure of a first embodiment of a data migration processing device provided by the present application;

[0073] Figure 8 A schematic diagram of the structure of a second embodiment of a data migration processing device provided by the present application;

[0074] Fig. 9 A schematic diagram of the structure of an electronic device for processing data migration provided in this application.

[0075] The above drawings have shown clear embodiments of the present application, which will be described in more detail later. These drawings and text descriptions are not intended to limit the scope of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. DETAILED DESCRIPTION

[0076] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0077] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0078] Figure 1 This is a schematic diagram of an application scenario of the data migration processing method provided in this application. Figure 1 As shown, the application scenario of the solution provided by the present application includes a server 100 and a server 101. The server 100 is used to deploy a control device 1001, and the server 101 is used to deploy a data center device 1011.

[0079] The server 100 is mainly used to deploy the control device 1001. The control device 1001 is mainly used for user data migration in the cross-domain joint account business. When a user wants to view the details of his account, he can directly enter key information in the graphical user interface provided by the control device 1001, and the control device 1001 can return detailed information that meets the requirements. Figure 1 Only one server 100 is shown in the figure, but it should be understood that there may be two or more servers 100. The control device 1001 in the server 100 is connected to the data center device 1011 in the server 101 through a communication network.

[0080] The server 101 is mainly used to deploy the data center device 1011. The data center device 1011 is provided with a database for storing user service data. The data center device 1011 is mainly used to receive the service data query message sent by the control device 1001, and execute the process of transmitting the data that meets the requirements from the data center device 1011 to the control device 1001. Figure 1 Only one server 101 is shown in FIG. 1 , but it should be understood that there may be two or more servers 101. At the same time, it should be understood that although Figure 1 Only one data center device 1011 is shown, but there may be two or more data center devices 1011.

[0081] Server 100 and server 101 are both high-performance computers with functions such as computing, storing and processing data.

[0082] When the control device 1001 deployed in the server 100 receives the cross-domain account-joint business data query request, it interacts with the data center device 1011 deployed in the server 101. Based on the data identifier in the cross-domain account-joint business data query request, the control device 1001 searches for business data that meets the requirements in the data center device 1011 deployed in the server 101. After locating all business data that meets the requirements, the data center device 1011 transmits them to the control device 1001 deployed in the server 100, and the control device 1001 presents all business data that meets the requirements to the user in the form of a visual interface, thereby completing the entire process of the cross-domain account-joint business.

[0083] However, with the rapid development of computer technology, the cross-domain joint account business system has undergone a certain degree of upgrade and transformation. Before the cross-domain joint account system was upgraded, that is, in the 1.0 system stage, user data was uniformly stored in the ORACLE database according to different business types. After the cross-domain joint account system was upgraded, that is, in the 2.0 system stage, user data was uniformly stored in the newly deployed DRDS database. Due to the cross-domain joint account business, all user data in the 2.0 system will be restricted from being migrated back to the 1.0 system. After the cross-domain joint account system is upgraded, only the scenario of user data in the 1.0 system being migrated to the 2.0 system exists. Therefore, when processing cross-domain joint account business, how to ensure a successful one-time cutover based on the same account and group relationship, and not cutover in batches, and the problem of executing data migration from the ORACLE database to the DRDS database needs to be solved urgently.

[0084] In view of this situation, the commonly used processing method is to limit the number of migrated users when processing cross-domain joint account services. When the number of migrated users does not exceed the set threshold, the control device 1001 sends a data migration instruction to the data center device 1011 deployed in the server 101. After the data center device 1011 receives the data migration instruction, it executes the operation of migrating data from the ORACLE database to the DRDS database, thereby completing the process of user data migration.

[0085] However, the above processing scheme has the following problems: during the process of executing user data migration in the cross-domain joint account system, if the number of users handling cross-domain joint account business exceeds the set threshold, the cross-domain joint account system will return an error message, prompting that the number of current operating users exceeds the set threshold, which is inconsistent with the business rules, and the current cross-domain joint account business cannot be completed, and it needs to be restarted manually. At the same time, during the actual operation of the cross-domain joint account system, if an error occurs in a certain execution link, the user data migration process will stop immediately, and the subsequent process will also need to be triggered manually. Therefore, the above processing scheme relies too much on manual methods, fails to form full process automation of cross-domain joint account business, and has low execution efficiency.

[0086] In view of the above problems, the inventors found in the process of studying the problem of low execution efficiency of data migration operation in the process of cross-domain joint account business due to excessive reliance on manual methods that once the system executing the cross-domain joint account business detects that the number of migrated users exceeds the specified threshold or an error occurs in a certain link, the user data migration operation will be stopped immediately, and the data migration operation needs to be restarted manually later. The cross-domain joint account business system lacks automated processing functions and relies too much on manual processing, which is easy to cause user complaints and low execution efficiency. Based on this, the inventors consider whether the control device 1001 can adopt a microservice architecture based on a message queue, in which the message queue is a middleware component for storing messages. The message queue can ensure the reliability and sequence of messages, and can realize persistent storage and batch processing of messages. The microservice architecture is an architectural model that improves the scalability, reliability and flexibility of the system by splitting a large system into multiple small, autonomous services. The services in the microservice architecture need to be decoupled to achieve efficient service collaboration. By using message queues, services can communicate asynchronously to achieve decoupling. In the process of executing the cross-domain joint account business, the data query requests of multiple users are isolated from each other and do not interfere with each other. The control device 1001 and the multi-data center device 1011 cooperate with each other. The control device 1001 is responsible for sending messages about user data query requests, and the data center device 1011 performs corresponding operations based on the received messages. When an error occurs in a certain link, the control device 1001 can instruct the data center device 1011 to automatically perform a rollback operation, thereby completing the automation of the entire cross-domain account consolidation business process.

[0087] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described below in conjunction with the accompanying drawings.

[0088] Figure 2 The flowchart of the first embodiment of the data migration processing method provided by this application is as follows. Figure 2 As shown, the process of the data migration processing method may include:

[0089] S201: In response to the user's operation, obtain the data identifier of the data to be migrated input by the user. In this step, the control device responds to the user's operation, that is, after the user inputs the key information of the data request to be queried through the graphical user interface provided by the control device, the control device is triggered. Among them, the key information of the data request to be queried may include the user's mobile phone number, the user's ID number, the time range to be queried, and the business type to be queried.

[0090] After the control device is triggered, the basis for executing the subsequent user data migration operation is the data identifier of the data to be migrated input by the user, which can be the user's mobile phone number, user ID number, etc. contained in the key information. Among them, the data identifier can determine the identity of the user, is unique, and can help the control device determine whether it is necessary to perform the user data migration operation subsequently.

[0091] For example, user A who handles cross-domain joint account business inputs a data request for the data to be queried through the graphical interface provided by the control device, where the key information contained in the data request includes the mobile phone number 131XXXX3421 that user A wants to query, and the time range to be queried is from January 1, 2023 to May 31, 2023. Based on the information entered by user A, the control device obtains the mobile phone number 131XXXX3421, the data identifier of the data to be migrated entered by user A, and provides a basis for judging whether the subsequent user data needs to be migrated.

[0092] S202: When it is determined based on the data identifier that the corresponding data to be migrated is not stored in the DRDS database, a data lock message is sent to multiple data center devices storing the data to be migrated according to a preset message sending rule. The message sending rule is set to send messages to each data center device in the order of users, accounts, and customers.

[0093] This step mainly involves a process in which the control device determines whether the user data needs to be relocated according to the data identifier.

[0094] Based on the data identifier of the data to be migrated input by the user obtained by the control device in step S201, the first operation that the control device needs to perform is to determine whether the data corresponding to the user identifier is stored in the ORACLE database or the DRDS database deployed in the data center device, thereby triggering the master control service to call the storage process to retrieve the migration data. The specific reason for performing this operation is: before the system that performs the cross-domain joint account business is upgraded, all business data related to the user are stored in the ORACLE database according to the business category. After the cross-domain joint account system is upgraded, all business data related to the user are stored in the newly deployed DRDS database and will no longer be stored in the ORACLE database. Therefore, when the control device is processing the cross-domain joint account business, in order to ensure the accuracy of the query data, it is necessary to first determine the specific location of the user data to be queried.

[0095] The control device performs page verification, account period verification, and time verification based on the obtained user ID, and determines that the storage location of the corresponding data is the ORACLE database, not the DRDS database. Next, the control device bus needs to confirm whether there is a repeated migration in the current state, such as two numbers in the same group triggering migration, that is, migrating at the same time. If not, the control device will officially start the first step of the user data migration operation, that is, sending data lock messages to multiple data center devices storing the data to be migrated. Among them, the control device sends data lock messages to multiple data center devices according to the preset message sending rules, that is, the message queue MQ. MQ realizes the decoupling between producers and consumers by storing messages in the queue. The order of sending data lock messages is user, account, and customer, following the three-household model, which is a model commonly used in the construction and operation support system in the telecommunications industry. The data lock message sent includes the locking method for the ORACLE database table in each data center device. Among them, the lock is mainly used to ensure database integrity and consistency in a multi-user environment. If multiple users manipulate data in the same database at the same time, data inconsistency will occur, such as lost updates, dirty reads, non-repeatable reads, and phantom reads. Therefore, the way to handle concurrent access by multiple users is to add locks.

[0096] When the control device obtains the specific location of the data storage corresponding to the user identifier, it must first send a locking message to the data center device where the data is located, and notify the data center device to perform a locking operation on the database involved in the form of a message queue to avoid data inconsistency and other problems.

[0097] For example, based on user A involved in step S201, after the control device determines that the corresponding data to be migrated is not stored in the DRDS database according to the data identifier mobile phone number 131XXXX3421, it sends a data lock message to data center device B and data center device C in the order of user, customer, and account in the form of a message queue.

[0098] S203: Receive a data locking message sent by the control device, where the data locking message includes a data identifier of the data to be migrated.

[0099] In this step, after the control device sends a data lock message to the data center device based on step S202, the data center device receives the data lock message sent by the control device. Among them, the data lock message includes the data identifier of the data to be migrated, which can provide a basis for the subsequent data center device to determine the target ORACLE database. At the same time, each data center device needs to build its own relocation details table, which is used to record all operations performed by the data center device and the specific process of data flow. For example, the relocation details table TF_F_VIP_TRANS_DETAIL that records the customer VIP migration data details, the relocation details tables TF_F_ACCT_TRANS_DETAIL and TF_F_USER_TRANS_DETAIL that record the account center migration data details, and the relocation details table TF_F_USER_TRANS_DETAIL that records the user center migration data details. In addition, each data center device also needs to build a set of FH tables to store relocation history data.

[0100] For example, based on step S202, after the control device completes sending the data locking message to the data center device B and the data center device C, the data center device B and the data center device C receive the data locking message respectively.

[0101] S204: According to the data identifier of the data to be migrated, the data to be migrated is stored in the intermediate data table, and the storage space storing the data to be migrated in the ORACLE database is locked.

[0102] This step mainly involves how the data center equipment locks the target storage space based on the data identifier.

[0103] Based on step S203, after each data center device receives the data locking message sent by the control device, it constructs an intermediate data table, namely the DETAIL table, ensures that the structure is consistent with DRDS, and adds the migration flow (index, joint primary key) and the insertion time INSERT_TIME. At the same time, according to the data identifier of the data to be migrated contained in the data locking message, the storage space for storing the data to be migrated in the target ORACLE database is determined respectively, and a locking operation is performed on it. After the data is locked, the lock time of TF_F_USER_TRANS_DETAIL is updated. Among them, the intermediate data table is used to temporarily store the data to be migrated. The purpose of performing a locking operation on the storage space of the data to be migrated is to prevent other thread operations from destroying the accuracy of this part of the data. Among them, the locking method is determined according to the storage space of the data to be migrated, such as a global lock, a table-level lock, a row-level lock, etc.

[0104] For example, according to the data identifier mobile phone number 131XXXX3421, the storage space for the data to be migrated is the entire ORACLE database in data center device B and a table in the ORACLE database in data center device C. Next, data center device B and data center device C respectively perform global lock and table-level lock operations according to different storage spaces to lock the data to be migrated. At the same time, data center device B and data center device C each temporarily stores the data to be migrated in the intermediate data table DETAIL table they have constructed.

[0105] S205: After the locking is completed, a data locking completion message is returned to the control device.

[0106] In this step, after each data center device has completed the locking operation on the data storage space to be migrated in step S204, each data center device returns a message of data locking completion to the control device in the form of a message queue.

[0107] For example, based on step S204, both data center device B and data center device C have completed the locking operation on the data to be migrated in the ORACLE database, and data center device B and data center device C notify the control device in a message queue that the locking operation has been completed.

[0108] S206: After receiving the data locking completion messages returned by the multiple data center devices, check whether the data stored in the intermediate data table of each data center device is duplicated.

[0109] This step mainly involves the process of the control device performing verification after receiving a message that the locking operation is successfully completed.

[0110] After the control device receives the data lock message returned by multiple data center devices, the control device bus needs to perform rule verification on the data in the intermediate data tables set in the multiple data center devices. The purpose of the rule verification is to check whether there is duplication in the data stored in the intermediate data table, thereby ensuring the uniqueness of the data.

[0111] For example, the control device bus checks whether there is any duplication in the data to be migrated stored in the DETAIL table set by the data center device B and the data center device C. If no duplicate data is found after the rule verification is completed, the control device proceeds to the next step. If duplicate data exists, the control device instructs the data center device to delete the duplicate data.

[0112] S207: After all the data in the intermediate data table of each data center device has been verified, a data table verification message is sent to each data center device respectively, where the data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to a preset verification rule.

[0113] This step mainly involves a process in which the control device sends a data table check message to multiple data center devices.

[0114] Based on step S206, after all the data in the intermediate data tables of each data center device have been verified, the control device sends a data table verification message to each data center device in the form of a message queue. The data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to the preset verification rules, that is, to instruct each data center device to perform a rule verification on the data in the intermediate data table stored in itself again to check whether there are data omissions, data migration errors, data duplications, etc., to ensure the accuracy of the final migrated data, and the subsequent control device bus waits for each data center device to return the verification results.

[0115] S208: Receive a data table verification message sent by the control device, where the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule.

[0116] In this step, multiple data center devices receive the data table verification message sent by the control device, and perform the following data verification operation according to the data verification table message. The data verification table is used to indicate that the data stored in the intermediate data table is verified according to the preset verification rules. For example, the data verification rules include data type, data length, data range, data consistency, data integrity, etc.

[0117] S209: Verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes.

[0118] In this step, multiple data center devices verify the intermediate data table according to the verification rules to check whether the data stored in the intermediate data table has omissions, duplications, errors, etc. After the verification is completed, multiple data center devices return a message that the data table verification has passed to the control device through the verification result message queue.

[0119] For example, after the data to be migrated stored in the DETAIL table set by data center device B and data center device C passes the rule verification and the above problems do not exist, then data center device B and data center device C need to feed back the message that the verification data has passed to the control device, thereby completing the rule verification process of each center.

[0120] S210: After receiving data table verification pass messages returned by multiple data center devices, a data format conversion message is sent to each data center device, where the data format conversion message includes a converted target format.

[0121] In this step, after the control device receives the data table verification message returned by multiple data center devices, it then sends a data format conversion message to each data center device, that is, triggers the customer, account, user, letter control and other data center devices to perform data format conversion through the message trigger notification service, and notifies the flow of each center. The data format conversion message is carried in the format message, which is consistent with the locking message, and is mainly used to indicate how to perform format conversion on the data stored in the intermediate data table in multiple data center devices. Among them, the data format conversion message includes the converted target format, which is consistent with the format of the data stored in the DRDS database.

[0122] S211: Receive a data format conversion message sent by a control device, where the data format conversion message includes a target data format after conversion.

[0123] In this step, multiple data center devices receive the data format conversion message sent by the control device, and through parsing, the converted target data format included in the format conversion message can be obtained. Next, multiple data center devices will instruct their respective stored intermediate data tables to perform corresponding data format conversion operations. Among them, the target data format can be type conversion, character conversion, date conversion, etc.

[0124] S212: According to the data format conversion message, the format of the data to be migrated stored in the intermediate data table is converted into the target data format.

[0125] In this step, multiple data center devices perform format conversion operations on the format of the data to be migrated stored in the intermediate data table according to the data format conversion message according to the target data format. The order of data format conversion is to convert one by one according to users and accounts. At the same time, multiple data center devices add a conversion storage process. When the data stored in the intermediate data table is converted, the data stored in the intermediate data table is updated to ensure that the data format stored in the intermediate data table conforms to the target data format after the data format conversion.

[0126] S213: After the format conversion of the data to be migrated is completed, a format conversion success message is sent to the control device.

[0127] In this step, after the format conversion of the migrated data is completed according to the data format conversion message sent by the control device, multiple data center devices send a message of successful format conversion to the control device in the form of a message queue. Each data center device performs the format conversion synchronously, but the speed may be different. Therefore, when a data center device completes the data format conversion, it can directly send a message of successful format conversion to the control device.

[0128] S214: After receiving format conversion success messages returned by all the data center devices, a data migration message is sent to each data center device, where the data migration message is used to instruct to migrate the data in the intermediate data table to the DRDS database.

[0129] In this step, based on step S213, the control device needs to receive messages from multiple data center devices indicating that the format conversion is successful before the next data migration operation can be performed. The control device bus sends a data migration message to each data center device in a parallel message format to notify each data center device to start migration. The data migration message is used to indicate that all data in the intermediate data table in the ORACLE database is migrated to the DRDS database.

[0130] S215: Receive a data migration message sent by the control device, where the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database.

[0131] In this step, multiple data center devices receive data migration messages sent by the control device. The data migration message is used to indicate that the data to be migrated after the conversion format in the intermediate data table is migrated to the DRDS database. Among them, data migration is to transfer data from one computing environment to another computing environment. Before the actual transmission, the data needs to be selected, prepared, extracted and converted, and these operations have been completed in the previous steps. The input parameters in the data migration message are migration flow, etc., and the data center device returns the success or failure status in real time.

[0132] S216: storing the data to be migrated in the converted format in the intermediate data table into the DRDS database according to the data migration message.

[0133] In this step, based on step S215, after receiving the data migration message sent by the control device, the multiple data center devices store the data to be migrated after the format conversion in the intermediate data table into the DRDS database according to the instruction of the data migration message.

[0134] The process of each data center device executing data migration is recorded in detail in the migration details table. The order of data migration is carried out one by one according to users and accounts. The data of the migration results updates the migration field status and time of the intermediate data table DETAIL. Tools for performing data migration between different databases include Dbmate, Apache NiFi, Ladder, TiDB, etc. Although there are many different database migration tools available, the specific application depends on the type of programming language, cost and specific needs. If there is a data migration failure, the migration process will not trigger the subsequent process, and will not trigger the rollback operation. It will wait for manual intervention, that is, after going online, it will determine whether to automatically roll back the operation based on the manual processing situation and the number of errors.

[0135] When all the equipment in each data center in the same batch is successfully relocated, the control device notifies the billing center to relocate, and the billing center processes asynchronously, and internally guarantees that the processing is successful and returns the result. That is, the billing center guarantees positive success, there are internal errors, and the positive repair guarantees success. After the billing center successfully processes, the control device triggers subsequent routing, unlocking and other processes through the message queue. At the same time, the control device notifies the order center to relocate, and the order center processes asynchronously, and internally guarantees that the processing is successful and returns the result, that is, the order center guarantees positive success, there are internal errors, and the positive repair guarantees success. After the order center successfully processes, it notifies the control device to trigger the subsequent process.

[0136] The data migration processing method provided in this embodiment mainly involves how to migrate the data to be migrated from the ORACLE database to the DRDS database through the control device. Among them, the control device maintains connection and communication with multiple data center devices in real time, so as to ensure the timeliness of cross-domain joint account business processing. At the same time, the control device and multiple data center devices establish message connections with each other in the form of message queues, which can improve the time utilization rate during business processing and shorten the data format conversion time. The control device controls the overall situation, and multiple data center devices perform different operations according to the message instructions, add fallback services, complete the entire process of data migration, reduce manual intervention, and thus realize the automation of the entire process of cross-domain joint account business processing.

[0137] Figure 3 This is a diagram illustrating the data migration process provided by this application. Figure 3 As shown, it includes a data center device and a control device. The control device frequently interacts with the data center device in the form of a message queue via a bus. The specific execution process can refer to steps S201 to S216, so it will not be repeated here.

[0138] As an optional implementation manner, the data migration processing method provided in this embodiment, based on any of the above embodiments, sends a data lock message to multiple data center devices storing data to be migrated according to a preset message sending rule, and the data migration processing method further includes the following steps:

[0139] The data locking message is sent to multiple data center devices in sequence in the form of a message queue in the order of users, accounts, and customers. The data locking message instructs each data center device to lock the storage space of the data to be migrated in the database.

[0140] In this step, the order of the data locking message sent by the control device and the key contents included are mainly involved. When the control device determines that the data to be queried is not stored in the DRDS database, it will perform the first step of the data migration operation and perform a locking operation on the storage space of the data to be migrated.

[0141] The control device sends data lock messages in the order of user, account, and customer, following the three-household model, which is a commonly used model in the construction and operation support system in the telecommunications industry. The data lock message is sent in the form of message queues, including ActiveMQ, RabbitMQ, ZeroMQ, Kafka, RocketMQ, etc. The specific message queue mode depends on actual needs. The data lock message instructs each data center device to lock the storage space of the data to be migrated in the database. The purpose of locking is to ensure the integrity and accuracy of the data to be migrated, and to prevent data loss and interference from other threads during the data migration process.

[0142] The data migration processing method provided in this embodiment mainly involves the order of the data lock message sent by the control device and the key content included. Among them, the sending order of the data lock message follows the three-household model, which can ensure the logic of message sending. The sending mode of the data lock message adopts a message queue, which can avoid the dependency between systems and reduce the coupling between systems. The data lock message indicates that the storage space is locked, which can ensure the non-interference of the subsequent data migration process and ensure the integrity and accuracy of the data to be migrated.

[0143] Figure 4 This is a flow chart of the second embodiment of the data migration processing method provided by this application. Figure 4 As shown, the process of the data migration processing method may include:

[0144] S401: In response to a user operation, obtaining a data identifier of the data to be migrated input by the user.

[0145] S402: When it is determined based on the data identifier that the corresponding data to be migrated is not stored in the DRDS database, a data lock message is sent to multiple data center devices storing the data to be migrated according to a preset message sending rule. The message sending rule is set to send messages to each data center device in the order of users, accounts, and customers.

[0146] S403: Receive a data locking message sent by the control device, where the data locking message includes a data identifier of the data to be migrated.

[0147] S404: According to the data identifier of the data to be migrated, the data to be migrated is stored in the intermediate data table, and the storage space storing the data to be migrated in the ORACLE database is locked.

[0148] The specific implementation of steps S401 to S404 may refer to steps S201 to S204, and thus will not be described in detail here.

[0149] S405: If the storage space storing the data to be migrated in the ORACLE database fails to be locked, a data locking failure message is sent to the control device.

[0150] In this step, based on the data lock message sent by the control device, multiple data center devices perform the lock operation according to the message instructions. If a data center device fails to lock the storage space storing the data to be migrated in the ORACLE database, the service is terminated and a data lock failure message is sent to the control device, that is, each center notifies the bus rollback result. After receiving the failure message, the control device decides the subsequent operation to be performed.

[0151] Among them, the reasons why the data center equipment fails to lock the storage space storing the data to be migrated in the ORACLE database may be network delay, ORACLE database downtime, system failure, etc.

[0152] S406: If a data locking failure message returned by any data center device is received, a first fallback message is sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

[0153] In this step, based on step S405, after receiving the data lock failure message returned by any data center device, the control device needs to trigger a message to notify other devices in multiple data center devices except the data center device to send a first fallback message, that is, there is no need to notify the failed data center device. At the same time, note that the order of sending the first fallback message is customer, account, and user.

[0154] Among them, the data center device that sends the data lock failure message can actively perform a rollback operation after detecting the data lock failure. This function is built-in to the ORACLE database. Other data center devices that do not send the data lock failure message need to wait for the first rollback message sent by the control device. The purpose of performing the rollback operation is to restore the data to the state of the previous specific time point to ensure the accuracy of the data.

[0155] S407: Receive a first back-off message sent by the control device, and perform a back-off process according to the first back-off message.

[0156] In this step, based on step S406, the data center device that has not sent a data lock failure message performs a rollback operation according to the first rollback message after receiving the first rollback message sent by the control device, that is, executes statements to delete or modify data, commits or rolls back transactions, and checks whether the data returns to its original state. If not, repeat the execution until the data returns to its original state.

[0157] The data migration processing method provided in this embodiment mainly deals with the problem of how to continue the operation after the data center device fails to lock the data. When a data center device fails to lock the data, it automatically executes the rollback operation and notifies the control device at the same time. The control device then notifies other data center devices to also execute the synchronous rollback operation. The rollback operation can undo the executed transaction and restore the data to the state before the transaction started, thereby avoiding data loss and damage and ensuring the integrity, consistency and accuracy of the data to be subsequently migrated.

[0158] Figure 5 This is a flow chart of the third embodiment of the data migration processing method provided by this application. Figure 5 As shown, the process of the data migration processing method may include:

[0159] S501: After all data in the intermediate data table of each data center device has been verified, a data table verification message is sent to each data center device respectively, where the data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to preset verification rules.

[0160] S502: Receive a data table verification message sent by a control device, where the data table verification message is used to instruct to verify data stored in the intermediate data table according to a preset verification rule.

[0161] The specific implementation of step S501 to step S502 may refer to step S207 to step S208, so it will not be described here in detail.

[0162] S503: If the verification of the intermediate data table fails, a data table verification failure message is sent to the control device.

[0163] In this step, based on the data table verification message sent by the control device, multiple data center devices perform data table verification operations according to the message instructions. If a data center device fails to verify the intermediate data table, it needs to send a data table verification failure message to the control device. After receiving the data table verification failure message, the control device decides the subsequent operation to be performed.

[0164] The data table verification operation is completed independently by each data center device. The specific verification process can be obtained in the data table verification message. Common data table verification contents include format verification, range verification, consistency verification, logic verification, uniqueness verification, etc. When performing data verification, it is necessary to select the appropriate verification method according to specific business needs and data characteristics, and combine relevant programming languages ​​and related tools to implement it.

[0165] S504: If a data table verification failure message returned by any data center device is received, a second fallback message and a database unlocking message are respectively sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

[0166] In this step, if the control device receives a message indicating that the data table verification has failed from any data center device, it sends a second fallback message and a database unlock message to other data center devices except the data center device in the plurality of data center devices in a message queue manner.

[0167] Among them, after detecting that the data table verification fails, the data center device that sends the data table verification failure message sends the data table verification failure message to the control device while autonomously performing the rollback operation and unlocking operation. The purpose of this operation is to allow the control device to notify other data center devices to synchronously perform the rollback and unlocking operations. After a data table verification fails, it means that there is a problem with the data and the next data format conversion operation cannot be performed, otherwise it will cause data errors.

[0168] S505: Receive a second rollback message and a database unlock message sent by the control device.

[0169] In this step, a plurality of data center devices that have not sent a data table verification failure message to the control device receive a second rollback message and a database locking message sent by the control device, and then perform corresponding rollback operations and locking operations.

[0170] S506: Perform rollback processing according to the second rollback message, and unlock the storage space storing the data to be migrated in the ORACLE database after the rollback is completed.

[0171] In this step, based on the second rollback message and database unlocking message received in step S505, the data center device first performs the rollback operation in the order of customer, credit control, and user, and returns the business logic to the locked operation state of the intermediate data table. After the rollback operation is completed, the data center device unlocks the storage space in the ORACLE database storing the data to be migrated. The execution order of the unlocking operation is customer, account, and user, and the specific unlocking method needs to be executed according to different locking methods.

[0172] The data migration processing method provided in this embodiment mainly deals with the problem of how to continue the operation after the data table verification of the data center equipment fails. When the data table verification of a data center equipment fails, the message of the data table verification failure is fed back to the control device. This process can ensure that the control device understands the situation of the data center equipment in real time and ensures the transparency of the data migration process. After receiving the message of the data table verification failure, the control device sends a second fallback message and unlocking instruction to other devices, which can ensure that all ORACLE databases fall back to the state before locking, and re-perform a new round of locking and data table verification process, thereby ensuring the integrity and accuracy of the data before data migration. Among them, the data table verification operation is an important link to ensure data quality and reliability. Through different data table verification methods, data errors can be effectively detected and corrected, and the accuracy and integrity of the data can be improved, thereby ensuring the validity and credibility of the data to be migrated in the end, and providing a reliable basis for business processing.

[0173] Figure 6 This is a flow chart of the fourth embodiment of the data migration processing method provided by this application. Figure 6 As shown, the process of the data migration processing method may include:

[0174] S601: After receiving data table verification pass messages returned by multiple data center devices, a data format conversion message is sent to each data center device, where the data format conversion message includes a converted target format.

[0175] S602: Receive a data format conversion message sent by a control device, where the data format conversion message includes a target data format after conversion.

[0176] S603: According to the data format conversion message, the format of the data to be migrated stored in the intermediate data table is converted into the target data format.

[0177] The specific implementation of steps S601 to S603 may refer to steps S210 to S212, so they will not be described in detail here.

[0178] S604: If the format conversion of the data to be migrated in the intermediate data table fails, a format conversion failure message is sent to the control device, and the conversion time of each data and the failure reason of the data that failed to be converted are recorded in the intermediate data table.

[0179] In this step, based on the data format conversion message sent by the control device, multiple data center devices perform data format conversion operations on the data to be migrated in the intermediate data table according to the message instructions. If a data center device fails to perform a data format conversion operation, the data center device needs to send a format conversion failure message to the control device, and will not automatically trigger a rollback, waiting for the next operation instruction. At the same time, the data center device records the conversion time of each data and the failure reason of the data that failed to be converted in the intermediate data table.

[0180] Among them, the reasons for the failure of data format conversion include incompatible data structures and data types, differences in database versions, problems with conversion tools, and database security issues. Subsequent personalized solutions need to be made based on the specific reasons for the failure recorded.

[0181] S605: If a format conversion failure message returned by any data center device is received, a data conversion failure notification message is pushed to the user through the graphical user interface, and the data conversion failure notification message includes an identifier of the data center device where the conversion failed.

[0182] In this step, based on step S604, after receiving the format conversion failure message returned by any data center device, the control device recommends the data conversion failure notification message to the user through the graphical user interface provided by the control device, does not trigger the subsequent process, does not trigger the rollback operation, and waits for manual intervention.

[0183] Among them, after the user receives the data conversion failure notification message, manual intervention is performed based on the identifier of the data center device where the conversion failed carried in the conversion failure notification message. After going online, the manual processing situation and the number of errors are observed to decide whether to automatically roll back the operation.

[0184] Table 1 shows the overall situation of the center / topic relocation with a large-screen monitoring detail diagram. As shown in Table 1, the entire process of data migration can be clearly presented in Table 1. The salesperson can use this menu to relocate users who exceed the relocation limit of 15 households (from the ORACLE database to the DRDS database). Among them, this menu can be relocated by number, and the page can query the overall situation of the center / topic relocation, that is, success or failure. Click to view details and add a refresh button to refresh the pop-up page data. Each center / topic sheet page can query the failed users and the reasons for failure. At the same time, each center / topic can repeat some operations for the failed users after manual intervention. If the manual intervention fails, the overall rollback can be performed without the need to roll back by center / topic.

[0185] Among them, the basic process of adding and unlocking at the source end includes receiving the adding and unlocking messages of the control device bus, parsing the data to lock or unlock (delete the main table and move the act library table account and user to the standby table), and feedback the bus locking and unlocking results. The basic process of data verification includes receiving the data verification message of the bus, parsing the data to determine the user and account in-transit orders, and recording the log for menu query, and feedback the bus verification results. The basic process of data conversion and rollback includes receiving the conversion or conversion rollback message of the bus, parsing the data for data conversion (the granularity requires specific users, accounts and specific tables), generating the data to be cutover by the core table data according to the user and account, and putting it into the DRDS table of the old architecture, and doing a good job of error processing (a certain table or a certain user fails, and other users and tables continue) and recording. The rollback operation does not process the DRDS table data at present, and will be optimized according to specific needs later, and feedback the result message of bus conversion and conversion rollback.

[0186] The basic process of core table data migration includes receiving the core table migration message of the control device, parsing the data and migrating the successfully converted data (the granularity requires specific users, accounts and specific tables), migrating to the 2.0 database, and handling the error (a certain table or a certain user fails, other users and tables continue) and recording, and feedback the bus migration result. The basic process of core table data migration rollback includes receiving the core table rollback message of the bus, parsing the data and deleting the successfully migrated data. And handle the error (a certain table or a certain user fails, other users and tables continue) and record, and feedback the bus rollback result. The basic process of non-core table data migration is the same as the core table data migration process, except that the message of this consumption is sent to the billing, and the return message is temporarily not added and is not in the control process of the control device bus. The return bus message is changed to send a deletion message to the deletion application. The basic process of peer unlocking includes receiving the peer unlocking message of the control device bus, parsing the data to unlock the account (update the account remove_tag in the DRDS library data to the normal state), and feedback the bus peer unlocking result. The basic deletion process includes receiving deletion messages, parsing data to perform data deletion actions, and updating the status after successful deletion. The asynchronous migration program development includes the need to monitor accounts and complete the query and migration of cross-domain tables in the accounting center. The development of repair tools and cutover history query development needs to cover every step from data conversion to deletion, including returning the migration results to the bus. It is also necessary to timely analyze common errors based on the online situation and evaluate whether tool development and program optimization can be carried out.

[0187] Table 1 Overall situation of center / topic relocation Large screen full monitoring details

[0188]

[0189] The data migration processing method provided in this embodiment mainly deals with the problem of how to continue the operation after the data format conversion of the data center device fails. When a data center device fails to perform data format conversion on the data in the migration data table, the data center device needs to feed back the message of data conversion failure to the control device through the message queue to ensure that the control device can follow up the progress of data format conversion of each data center device in time, and automatically perform the recording operation of the time and reason of data format conversion failure in the intermediate data table, which can provide a basis for subsequent technical personnel to understand the cause of failure and determine whether to perform automatic rollback operation.

[0190] Figure 7 This is a schematic diagram of the structure of the first embodiment of the data migration processing device provided by the present application. Figure 7 As shown, the data migration processing device 700 includes:

[0191] The receiving module 701 is used to obtain the data identifier of the data to be migrated input by the user in response to the user's operation;

[0192] The sending module 702 is used to send a data lock message to multiple data center devices storing the data to be migrated according to a preset message sending rule when it is determined that the corresponding data to be migrated is not stored in the DRDS database according to the data identifier, and the message sending rule is set to send a message to each data center device in the order of user, account, and customer;

[0193] Processing module 703, for verifying whether the data stored in the intermediate data table of each data center device is duplicated after receiving the data locking completion message returned by the multiple data center devices;

[0194] The sending module 702 is also used to send a data table verification message to each data center device after all the data in the intermediate data table of each data center device have been verified, and the data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to the preset verification rules;

[0195] The sending module 702 is further configured to send a data format conversion message to each data center device after receiving a data table verification pass message returned by multiple data center devices, wherein the data format conversion message includes a converted target format;

[0196] The sending module 702 is further used to send a data migration message to each data center device after receiving format conversion success messages returned by all multiple data center devices. The data migration message is used to instruct to migrate the data in the intermediate data table to the DRDS database.

[0197] Optionally, the sending module 702 is further used to send data locking messages to multiple data center devices in sequence in the form of message queues according to the order of users, accounts, and customers, and the data locking messages instruct each data center device to lock the storage space of the data to be migrated in the database.

[0198] Optionally, the sending module 702 is further configured to send a first fallback message to other devices among the multiple data center devices except the data center device according to a message sending rule if a data lock failure message returned by any data center device is received.

[0199] Optionally, the sending module 702 is further used to send a second fallback message and a database unlock message to other devices in the multiple data center devices except the data center device according to the message sending rules if a data table verification failure message returned by any data center device is received.

[0200] Optionally, if the sending module 702 receives a format conversion failure message returned by any data center device, it pushes a data conversion failure notification message to the user through the graphical user interface, and the data conversion failure notification message includes the identifier of the data center device where the conversion failed.

[0201] The data migration processing device provided in the above embodiments can be used to execute the data migration processing method in any of the above method embodiments, and its implementation principle and technical effect are similar. The data migration processing device follows the basic principle that the same account and the same group relationship must be successfully cut over at one time and cannot be cut over in batches, which ensures the integrity of the account arrears credit control and group inter-call free data. At the same time, the data migration processing device uses the control device as the overall control, breaks through the limit on the number of migrated users, improves the efficiency of cross-domain joint account operations, and realizes the automation of cross-domain joint account business data migration operations.

[0202] Figure 8 This is a schematic diagram of the structure of the second embodiment of the data migration processing device provided by this application. Figure 8 As shown, the data migration processing device 800 includes:

[0203] A receiving module 801 is used to receive a data locking message sent by a control device, where the data locking message includes a data identifier of the data to be migrated;

[0204] Processing module 802, for storing the data to be migrated in an intermediate data table according to the data identifier of the data to be migrated, and locking the storage space storing the data to be migrated in the ORACLE database;

[0205] The sending module 803 is used to return a data locking completion message to the control device after the locking is completed;

[0206] The receiving module 801 is further used to receive a data table verification message sent by the control device, where the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule;

[0207] The processing module 802 is also used to verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes;

[0208] The receiving module 801 is further used to receive a data format conversion message sent by the control device, where the data format conversion message includes a target data format after conversion;

[0209] The processing module 802 is also used to convert the format of the data to be migrated stored in the intermediate data table into the target data format according to the data format conversion message;

[0210] The sending module 803 is further configured to send a format conversion success message to the control device after the format conversion of the data to be migrated is completed;

[0211] The receiving module 801 is also used to receive a data migration message sent by the control device, where the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database;

[0212] The processing module 802 is further configured to store the data to be migrated in the converted format in the intermediate data table into the DRDS database according to the data migration message.

[0213] Optionally, the sending module 803 is further used for:

[0214] If locking of the storage space storing the data to be migrated in the ORACLE database fails, a data locking failure message is sent to the control device.

[0215] Optionally, the receiving module 801 is further used for:

[0216] A first back-off message sent by the control device is received, and a back-off process is performed according to the first back-off message.

[0217] Optionally, the sending module 803 is further used for:

[0218] If the verification of the intermediate data table fails, a data table verification failure message is sent to the control device.

[0219] Optionally, the receiving module 801 is further used for:

[0220] A second rollback message and a database unlocking message sent by the control device are received.

[0221] Optionally, the processing module 802 is further configured to:

[0222] The rollback process is performed according to the second rollback message, and after the rollback is completed, the storage space storing the data to be migrated in the ORACLE database is unlocked.

[0223] Optionally, the sending module 803 is further used for:

[0224] If the format conversion of the data to be migrated in the intermediate data table fails, a format conversion failure message is sent to the control device, and the conversion time of each data and the failure reason of the data that failed to be converted are recorded in the intermediate data table.

[0225] The data migration processing device provided in this embodiment can be used to execute the data migration processing method in any of the aforementioned method embodiments. Its implementation principle and technical effects are similar and will not be described in detail here.

[0226] Fig. 9This is a schematic diagram of the structure of the electronic device for processing data migration provided by this application. Fig. 9 As shown, the electronic device may specifically include a receiver 900, a transmitter 901, a processor 902, and a memory 903. The receiver 900 and the transmitter 901 are used to implement data transmission between the electronic device and the application client, the memory 903 stores computer execution instructions; the processor 902 executes the computer execution instructions stored in the memory 903 to implement the data migration processing method in the above embodiment.

[0227] This embodiment provides a computer-readable storage medium, in which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, they are used to implement the data migration processing method in the above embodiment.

[0228] An embodiment of the present application also provides a computer program product, including a computer program, which, when executed by a processor, implements the data migration processing method provided in any one of the above embodiments.

[0229] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily required by the present application.

[0230] It should be further noted that, although the various steps in the flowchart are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless there is a clear description in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowchart may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these sub-steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0231] It should be understood that the above-mentioned device embodiments are only illustrative, and the device of the present application can also be implemented in other ways. For example, the division of units / modules in the above-mentioned embodiments is only a logical function division, and there may be other division methods in actual implementation. For example, multiple units, modules or components can be combined, or can be integrated into another system, or some features can be ignored or not executed.

[0232] In addition, unless otherwise specified, each functional unit / module in each embodiment of the present application may be integrated into one unit / module, each unit / module may exist physically separately, or two or more units / modules may be integrated together. The above-mentioned integrated unit / module may be implemented in the form of hardware or in the form of a software program module.

[0233] If the integrated unit / module is implemented in the form of hardware, the hardware may be a digital circuit, an analog circuit, etc. The physical implementation of the hardware structure includes but is not limited to transistors, memristors, etc. If not specifically stated, the processor may be any appropriate hardware processor, such as a CPU, a GPU, an FPGA, a DSP, an ASIC, etc. If not specifically stated, the storage unit may be any appropriate magnetic storage medium or magneto-optical storage medium, such as a resistive random access memory RRAM (Resistive Random Access Memory), a dynamic random access memory DRAM (Dynamic Random Access Memory), a static random access memory SRAM (Static Random-Access Memory), an enhanced dynamic random access memory EDRAM (Enhanced Dynamic Random Access Memory), a high-bandwidth memory HBM (High-Bandwidth Memory), a hybrid memory cube HMC (Hybrid Memory Cube), etc.

[0234] If the integrated unit / module is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or all or part of the technical solution, can be embodied in the form of a software product, which is stored in a memory and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned memory includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, disk or optical disk and other media that can store program codes.

[0235] In the above embodiments, the description of each embodiment has its own emphasis. For the part not described in detail in a certain embodiment, please refer to the relevant description of other embodiments. The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0236] Those skilled in the art will readily appreciate other embodiments of the present application after considering the specification and practicing the invention disclosed herein. The present application is intended to cover any modification, use or adaptation of the present application, which follows the general principles of the present application and includes common knowledge or customary techniques in the art that are not disclosed in the present application. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present application are indicated by the following claims.

[0237] It should be understood that the present application is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present application is limited only by the appended claims.

Claims

1. A data migration processing method, characterized in that: Applied to a control device, the method comprises: In response to a user operation, obtaining a data identifier of the data to be migrated input by the user; When it is determined according to the data identifier that the corresponding data to be migrated is not stored in the DRDS database, a data lock message is sent to multiple data center devices storing the data to be migrated according to a preset message sending rule, wherein the message sending rule is set to send a message to each data center device in the order of user, account, and customer; After receiving the data locking completion messages returned by the multiple data center devices, checking whether the data stored in the intermediate data table of each data center device is repeated; After all the data in the intermediate data table of each data center device has been verified, a data table verification message is sent to each data center device respectively, wherein the data table verification message is used to instruct the data center device to verify the data in the intermediate data table according to a preset verification rule; After receiving the data table verification pass message returned by the multiple data center devices, sending a data format conversion message to each data center device, wherein the data format conversion message includes a converted target format; After receiving format conversion success messages returned by all the multiple data center devices, a data migration message is sent to each data center device, where the data migration message is used to instruct the data in the intermediate data table to be migrated to the DRDS database.

2. The method according to claim 1, characterized in that: The sending of a data locking message to a plurality of data center devices storing the data to be migrated according to a preset message sending rule includes: The data locking message is sent to the plurality of data center devices in sequence in the form of a message queue in the order of users, accounts, and customers, wherein the data locking message instructs each data center device to lock the storage space of the data to be migrated in the database.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: If a data locking failure message returned by any data center device is received, a first fallback message is sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

4. The method according to claim 1 or 2, characterized in that: The method further comprises: If a data table verification failure message returned by any data center device is received, a second fallback message and a database unlocking message are respectively sent to other devices among the multiple data center devices except the data center device according to the message sending rule.

5. The method according to claim 1 or 2, characterized in that: The method further comprises: If a format conversion failure message returned by any data center device is received, a data conversion failure notification message is pushed to the user through a graphical user interface, and the data conversion failure notification message includes an identifier of the data center device where the conversion failed.

6. A method for processing data migration, characterized in that: Applied to a data center device, wherein the data center device is provided with an ORACLE database and a DRDS database, the method comprises: receiving a data locking message sent by a control device, wherein the data locking message includes a data identifier of the data to be migrated; According to the data identifier of the data to be migrated, the data to be migrated is stored in an intermediate data table, and a storage space in the ORACLE database storing the data to be migrated is locked; After the locking is completed, a data locking completion message is returned to the control device; Receiving a data table verification message sent by the control device, wherein the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule; Verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes; receiving a data format conversion message sent by the control device, wherein the data format conversion message includes a target data format after conversion; According to the data format conversion message, converting the format of the data to be migrated stored in the intermediate data table into the target data format; After the format conversion of the data to be migrated is completed, sending a format conversion success message to the control device; receiving a data migration message sent by the control device, wherein the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database; The data to be migrated in the intermediate data table after the format conversion is stored in the DRDS database according to the data migration message.

7. The method according to claim 6, characterized in that The method further comprises: If locking the storage space storing the data to be migrated in the ORACLE database fails, sending a data locking failure message to the control device; A first back-off message sent by the control device is received, and a back-off process is performed according to the first back-off message.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: If the verification of the intermediate data table fails, sending a data table verification failure message to the control device; Receiving a second rollback message and a database unlocking message sent by the control device; Rollback processing is performed according to the second rollback message, and after the rollback is completed, the storage space storing the data to be migrated in the ORACLE database is unlocked.

9. The method according to claim 6 or 7, characterized in that: The method further comprises: If the format conversion of the data to be migrated in the intermediate data table fails, a format conversion failure message is sent to the control device, and the conversion time of each data and the failure reason of the data that failed to be converted are recorded in the intermediate data table.

10. A data migration processing device, characterized in that: include: A receiving module, configured to obtain, in response to a user operation, a data identifier of the data to be migrated input by the user; A sending module, configured to send a data lock message to a plurality of data center devices storing the data to be migrated according to a preset message sending rule, when it is determined according to the data identifier that the corresponding data to be migrated is not stored in the DRDS database, wherein the message sending rule is configured to send a message to each data center device in the order of user, account, and customer; A processing module, configured to verify whether the data stored in the intermediate data table of each data center device is repeated after receiving the data locking completion message returned by the plurality of data center devices; The sending module is also used to send a data table verification message to each data center device after all the data in the intermediate data table of each data center device has been verified, and the data table verification message is used to command the data center device to verify the data in the intermediate data table according to the preset verification rules; The sending module is further used to send a data format conversion message to each data center device after receiving the data table verification pass message returned by the multiple data center devices, wherein the data format conversion message includes a converted target format; The sending module is also used to send a data migration message to each data center device after receiving format conversion success messages returned by all the multiple data center devices, and the data migration message is used to instruct the migration of data in the intermediate data table to the DRDS database.

11. A data migration processing device, characterized in that: include: A receiving module, configured to receive a data locking message sent by a control device, wherein the data locking message includes a data identifier of the data to be migrated; A processing module, configured to store the data to be migrated in an intermediate data table according to the data identifier of the data to be migrated, and lock the storage space storing the data to be migrated in the ORACLE database; A sending module, used for returning a data locking completion message to the control device after the locking is completed; The receiving module is further used to receive a data table verification message sent by the control device, wherein the data table verification message is used to instruct to verify the data stored in the intermediate data table according to a preset verification rule; The processing module is also used to verify the intermediate data table according to the verification rule, and return a data table verification pass message to the control device after the verification passes; The receiving module is further used to receive a data format conversion message sent by the control device, wherein the data format conversion message includes a target data format after conversion; The processing module is further used to convert the format of the data to be migrated stored in the intermediate data table into the target data format according to the data format conversion message; The sending module is further used to send a format conversion success message to the control device after the format conversion of the data to be migrated is completed; The receiving module is also used to receive a data migration message sent by the control device, wherein the data migration message is used to instruct to migrate the data to be migrated after the format conversion in the intermediate data table to the DRDS database; The processing module is further configured to store the data to be migrated in the intermediate data table after the format conversion into the DRDS database according to the data migration message.

12. An electronic device, characterized in that: include: A processor, and a memory communicatively connected to the processor; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory to implement the method according to any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, which are used to implement the method according to any one of claims 1 to 9 when executed by a processor.

Images