Data migration method, device, computer equipment and storage medium

Abstract

The present application relates to a data migration method, device, computer equipment, storage medium and computer program product, which can be applied to various systems, such as enterprise management systems, ERP systems, production and manufacturing management systems, supply chain systems, personnel systems, financial systems and tax systems. The above method includes: obtaining a migration request initiated by the target system, determining the data to be migrated in the system to be migrated corresponding to the target system according to the migration request; calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice; when the data detection passes, determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice, and the target thread pool is configured with multiple threads; extracting the data to be migrated from the system to be migrated in parallel by calling the migration microservice, and migrating the data to be migrated to the target system. The use of this method can improve the efficiency of data migration.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data migration method, apparatus, computer equipment, storage medium and computer program product. Background Art

[0002] As the enterprise system is upgraded or the database cannot meet business needs, it is sometimes necessary to migrate huge amounts of data from the system to be migrated to another system, which involves the issue of data migration.

[0003] The existing data migration method is mainly for enterprise personnel to manually export the data of the system to be migrated in the form of electronic spreadsheets, organize the data in the form required by the new system, and then manually import it into the new system. However, in the face of huge amounts of data, the manual export / import method is very inefficient. Summary of the invention

[0004] Based on this, it is necessary to provide a data migration method, apparatus, computer device, computer-readable storage medium and computer program product that can improve data migration efficiency in response to the above technical problems.

[0005] In a first aspect, the present application provides a data migration method. The method comprises:

[0006] Acquire a migration request initiated by the target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request;

[0007] Calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice;

[0008] When the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads;

[0009] The migration microservice is called to run multiple threads in the target thread pool, to extract the data to be migrated from the system to be migrated in parallel, and to migrate the data to be migrated to the target system.

[0010] In some embodiments, after obtaining the migration request initiated by the target system, the method further includes:

[0011] Extracting credential information carried in the migration request that matches the corresponding system to be migrated;

[0012] The validity of the credential information is verified, and if the validity is passed, a to-be-migrated list including multiple sets of to-be-migrated data in the to-be-migrated system is obtained, and the list is returned to the target system, so that the target system can determine the to-be-migrated data in the to-be-migrated list.

[0013] In some embodiments, after performing data detection on the data to be migrated by the migration microservice, the method further includes:

[0014] When it is detected that abnormal data exists in the data to be migrated, the migration microservice calls a first system interface connected to the target system to feed back abnormal information containing the abnormal data to the target system.

[0015] In some embodiments, after determining the data to be migrated in the to-be-migrated system corresponding to the target system according to the migration request, the method further includes:

[0016] Dividing the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration order;

[0017] The sub-data to be migrated are arranged according to the migration order to obtain a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0018] In some embodiments, the step of calling the migration microservice to run multiple threads in the target thread pool, extracting the data to be migrated from the system to be migrated in parallel, and migrating the data to be migrated to the target system includes:

[0019] Determining the sub-data to be migrated currently to be migrated in the migration queue;

[0020] By running the multiple threads in the target thread pool in parallel through the migration microservice, the sub-data to be migrated currently to be migrated is extracted and cached into the process of the migration microservice;

[0021] Migrating the cached sub-data to be migrated to the target system by invoking the second system interface connected to the target system through the migration microservice;

[0022] The next sub-data to be migrated in the migration queue is used as the sub-data to be migrated currently to be migrated, and the process returns to the step of running multiple threads in the target thread pool in parallel through the migration microservice until all sub-data to be migrated in the migration queue are migrated.

[0023] In some embodiments, the method of determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice includes at least one of the following:

[0024] By calling the migration microservice, the migration microservice sequentially queries the usage status of each thread pool, and determines the first thread pool currently in an idle state obtained by the query as the target thread pool corresponding to the target system;

[0025] By calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently in an idle state to determine it as a target thread pool corresponding to the target system;

[0026] By calling the migration microservice, the migration microservice queries the usage status of each thread in each thread pool, and determines the target thread pool corresponding to the target system according to the number of threads required to be occupied by the data to be migrated.

[0027] In a second aspect, the present application also provides a data migration device. The device comprises:

[0028] An acquisition module, configured to acquire a migration request initiated by a target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request;

[0029] A detection module, used to call a migration microservice and perform data detection on the data to be migrated through the migration microservice;

[0030] An allocation module, configured to determine, when data detection passes, a target thread pool corresponding to the target system from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads;

[0031] The migration module is used to extract the data to be migrated from the system to be migrated in parallel by calling the migration microservice to run multiple threads in the target thread pool, and migrate the data to be migrated to the target system.

[0032] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0033] Acquire a migration request initiated by the target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request;

[0034] Calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice;

[0035] When the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads;

[0036] The migration microservice is called to run multiple threads in the target thread pool, to extract the data to be migrated from the system to be migrated in parallel, and to migrate the data to be migrated to the target system.

[0037] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0038] Acquire a migration request initiated by the target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request;

[0039] Calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice;

[0040] When the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads;

[0041] The migration microservice is called to run multiple threads in the target thread pool, to extract the data to be migrated from the system to be migrated in parallel, and to migrate the data to be migrated to the target system.

[0042] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0043] Acquire a migration request initiated by the target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request;

[0044] Calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice;

[0045] When the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads;

[0046] The migration microservice is called to run multiple threads in the target thread pool, to extract the data to be migrated from the system to be migrated in parallel, and to migrate the data to be migrated to the target system.

[0047] The above-mentioned data migration method, apparatus, computer equipment, storage medium and computer program product obtain a migration request initiated by a target system and determine the data to be migrated, first call the migration microservice to perform data detection on the data to be migrated, without manually checking whether the data is abnormal, avoiding the repetitive operation of re-exporting the data to be migrated after the abnormality is found, and improving the migration efficiency and success rate; when the data detection passes, the target thread pool is determined from multiple thread pools by calling the migration microservice, and the data to be migrated is extracted and migrated in parallel by running multiple threads in the target thread pool, which significantly improves the migration efficiency of the data to be migrated and greatly improves the migration speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 An application environment diagram of a data migration method in an embodiment;

[0049] Figure 2 A schematic diagram of a data migration method in one embodiment;

[0050] Figure 3 A flowchart of steps for determining a migration order in one embodiment;

[0051] Figure 4 is a flowchart of steps of performing data migration according to a migration sequence in one embodiment;

[0052] Figure 5 A schematic diagram of a use scenario of a data migration method in an embodiment;

[0053] Figure 6 A system logic diagram of a data migration method in one embodiment;

[0054] Figure 7 is a structural block diagram of a data migration device in one embodiment;

[0055] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0057] There are two main existing data migration methods. The first is to manually export data to a spreadsheet, then organize the data into an import template according to the requirements of the new system, and then manually import the import template into the new system. This method is extremely inefficient when processing large amounts of data, with a large workload, cumbersome operations, and prone to errors. It has high requirements for the order and dependency of the imported data, and cannot fully check the imported data in advance. The second method is to download the installation package (or plug-in) and use the plug-in to migrate the data. However, this method has strict requirements on the version of the installation package (or plug-in), the operating system of the system, and the system version, and has poor scalability.

[0058] The data migration method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown. Among them, the target system 102 and the system to be migrated 106 communicate with the server 104 respectively. In some embodiments, the target system 102 can be deployed on the server 104, and the user can log in to the target system 102 through the platform provided by the server 104, thereby realizing the communication connection between the target system 102 and the server 104. The server 104 can communicate by calling the system interface of the system to be migrated 106. The server 104 obtains the migration request initiated by the target system 102, and obtains the data to be migrated from the system to be migrated 106; the server 104 performs data detection on the data to be migrated through a separately deployed migration microservice, and when the data detection passes, the data to be migrated is migrated from the system to be migrated 106 to the target system 102 by calling the migration microservice. Among them, the server 104 can be implemented by an independent server or a server cluster composed of multiple servers. Among them, the target system 102 and the system to be migrated 106 both include but are not limited to one or more of an enterprise management system, an ERP system, a production and manufacturing management system, a supply chain system, a personnel system, a financial system, a tax system, and an audit system.

[0059] In some embodiments, Figure 2 As shown, a data migration method is provided, which is applied to Figure 1 The server in the example is used to illustrate the following steps:

[0060] Step 202: Acquire a migration request initiated by the target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request.

[0061] The data to be migrated is data to be migrated from the system to be migrated to the target system. In some embodiments, the data to be migrated is, for example, account set data. The account set data refers to the general term for all data files storing the basic information of the enterprise, business data such as business documents, and financial data such as accounting vouchers. The account set data is usually stored in a database associated with the enterprise business system.

[0062] Accordingly, the target system is the system into which the data to be migrated will be migrated, and the system to be migrated is the system from which the data to be migrated will be migrated. For the target system, a set of data to be migrated can be established, and the data to be migrated is migrated from the system to be migrated. Among them, the target system and the system to be migrated can be different business systems, or they can be two systems of the same business system with different system versions, such as a system to be migrated with a lower system version and a target system with a higher system version.

[0063] Specifically, the server obtains a migration request initiated by the target system, for example, the migration request may be initiated automatically by the target system at a fixed time, or the migration request may be initiated by a user's operation in the target system. The server determines the data to be migrated this time from multiple sets of data to be migrated in the system to be migrated based on the request parameters carried in the migration request. The request parameters are ID information of the data to be migrated, and the ID information is used to indicate which set of data to be migrated.

[0064] In some embodiments, the target system can be logged in by the user entering an account and password in a platform, and the platform is deployed in the above-mentioned server, so that the server establishes a data connection with the target system. The user can enter the account and password of the system to be migrated in the user interface provided by the target system, so that the server can determine the system to be migrated for data migration, and obtain the data to be migrated indicated by the request parameters from the system to be migrated according to the request parameters in the migration request by calling the system interface of the system to be migrated.

[0065] Step 204: call the migration microservice, and perform data detection on the data to be migrated through the migration microservice.

[0066] Among them, a microservice is a business function that is deployed and operated independently, and implements the corresponding business function by providing an API (Application Programming Interface) for external system calls. In the embodiment of the present application, a migration microservice that can be called is provided, and the migration microservice is used to provide a data migration function.

[0067] Since the amount of data to be migrated is large, direct migration may fail due to abnormal data. Therefore, before data migration, the server calls the migration microservice to detect whether there is abnormal data to ensure the success rate and accuracy of data migration. Abnormal data includes, for example, erroneous data caused by system operation errors or old data that is incompatible with product versions.

[0068] Specifically, the server calls the migration microservice, and the migration microservice performs data detection on the data to be migrated based on the preset data inspection rules. Exemplarily, the server calls the migration microservice to perform the data detection function by calling the API for the data detection function provided by the migration microservice. The data inspection rules include, for example, one or more of whether the name of the data to be migrated complies with the specification, whether the data value is within the normal range, whether the data item is legal, and whether it is erroneous data caused by system operation errors.

[0069] In some embodiments, when the server detects that there is abnormal data in the data to be migrated by calling the migration microservice, the server calls the first system interface connected to the target system through the migration microservice to feed back abnormal information including the abnormal data to the target system.

[0070] Specifically, the migration microservice performs data detection on the data to be migrated based on the preset data inspection rules. When abnormal data is detected in the data to be migrated, the migration microservice calls the system interface connected to the target system (in order to avoid confusion with other system interfaces, it is referred to as the first system interface here), and feeds back the abnormal information containing the abnormal data to the target system through the first system interface. After receiving the abnormal information, the target system visually displays it to the user through the interface, so that the user can modify or delete the abnormal data. Among them, the abnormal information includes one or more of the data value of the abnormal data, the data item of the abnormal data, and the description of the abnormal situation.

[0071] In the above embodiment, by performing data detection on the data to be migrated and timely filtering out abnormal data, the accuracy of the data to be migrated is ensured, thereby ensuring the success rate of data migration.

[0072] Step 206, when the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling the migration microservice, and multiple threads are configured in the target thread pool.

[0073] Specifically, if the data detection passes (i.e., no abnormal data is detected in the data to be migrated), the server calls the migration microservice, and the migration microservice allocates a data pool for data migration for the migration request initiated by the target system based on the preset allocation rules. For example, the migration microservice creates three thread pools by default when it is started, and each thread pool is configured with 20 threads by default.

[0074] Among them, since the migration microservice is a separately deployed microservice, it can be called by multiple servers at the same time to simultaneously process data migration operations between multiple target systems and the corresponding systems to be migrated. The data migration operation includes the following set of actions: reading the data to be migrated from the system to be migrated, assembling it into a data packet, and calling the system interface of the target system to save the data to be migrated to the target system. Therefore, the migration microservice allocates corresponding thread pools to perform data migration for each migration request initiated by the target system. Among them, each thread pool is configured with multiple threads to perform data migration operations concurrently. For example, both system A and system B initiate migration requests, and the server calls the migration microservice. The migration microservice allocates a thread pool to system A and a thread pool to system B respectively, wherein the thread pools allocated to system A and system B respectively can be the same thread pool, and the data migration of system A and system B respectively occupies different threads; or, the thread pools allocated to system A and system B respectively can also be different thread pools, and the data migration of system A and system B respectively is independent of each other and does not interfere with each other.

[0075] The allocation rules preset in the migration microservice include, but are not limited to, one or more of round-robin allocation, random allocation, and allocation according to thread occupancy, etc. Exemplarily, the migration microservice can read the allocation rules preset in a separately set configuration platform, and allocate the thread pool based on the read allocation rules.

[0076] In some embodiments, the method of determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice includes: by calling the migration microservice, the migration microservice sequentially queries the usage status of each thread pool, and determines the first thread pool currently in an idle state obtained by the query as the target thread pool corresponding to the target system. In an embodiment of the present application, when a thread in a thread pool is executing a data migration operation, the thread pool is in an occupied state; when all threads in the thread pool do not perform any operation, the thread pool is in an idle state. Specifically, the server calls the migration microservice, and the migration microservice polls all thread pools, that is, sequentially queries the usage status of each thread pool, and uses the thread pool that is first queried as an idle state as the target thread pool for processing the migration request of the target system.

[0077] In other embodiments, the method of determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice includes: by calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools currently in an idle state as the target thread pool corresponding to the target system. Specifically, the server calls the migration microservice, and the migration microservice first determines the thread pools currently in an idle state, and randomly selects a thread pool from these thread pools in an idle state as the target thread pool for processing the migration request of the target system.

[0078] The data migration operation between a target system and the corresponding system to be migrated is executed by a thread pool through polling allocation or random allocation. The data migration operations corresponding to the migration requests initiated by different target systems are independent of each other. One thread pool is only used to process one data migration operation, thereby ensuring the data integrity and security of data migration.

[0079] In other embodiments, the method of determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice includes: calling the migration microservice, the migration microservice querying the usage status of each thread in each thread pool, and determining the target thread pool corresponding to the target system according to the number of threads required for the data to be migrated.

[0080] Specifically, the server calls the migration microservice, and the migration microservice queries the usage status of each thread in each thread pool, calculates the number of threads required for the data to be migrated, and comprehensively determines the target thread pool according to the number of threads and the number of threads in the idle state in the thread pool. In order to reasonably allocate thread resources, the migration microservice gives priority to the thread pool in the occupied state. If the number of threads in the idle state in a thread pool is not less than the number of threads required for the data to be migrated, the thread pool is used as the target thread pool for processing the migration request of the target system. If the number of threads in the idle state in the thread pool is less than the number of threads required for the data to be migrated, the migration microservice selects the next thread pool in the occupied state and continues the above judgment. When the number of threads in the idle state in the thread pool in the occupied state cannot meet the number of threads required for the data to be migrated, the migration microservice selects the thread pool in the idle state as the target thread pool for processing the migration request of the target system. This can greatly improve the thread utilization rate, avoid resource consumption, and increase the number of data migrations that can be processed in parallel at the same time, thereby improving the efficiency of data migration.

[0081] Step 208: extract the data to be migrated from the system to be migrated in parallel by calling the migration microservice to run multiple threads in the target thread pool, and migrate the data to be migrated to the target system.

[0082] Specifically, the server calls the migration microservice, for example, the server calls the data migration API interface provided by the migration microservice, and the migration microservice allocates and determines the target thread pool, and then runs multiple threads in the target thread pool in parallel. Each thread extracts the data to be migrated from the system to be migrated in parallel and caches it in the process of the migration microservice, and then migrates the data to be migrated to the target system.

[0083] Exemplarily, the migration microservice runs multiple threads in the target thread pool, concurrently reads the data to be migrated from the system to be migrated and assembles them into data packets, and passes the data to be migrated assembled into data packets to the target system by calling the system interface connected to the target system, thereby completing the data migration.

[0084] In some embodiments, since the amount of data to be migrated often reaches hundreds of thousands or even more, during the data migration process, the server uses a high-performance cache component to cache the data to be migrated in the process of the migration microservice, which greatly shortens the time spent on reading the data to be migrated, and other objects can directly read the cache when referencing, which improves efficiency and performance, reduces the number of interface requests, and reduces the processing pressure of the migration microservice. Furthermore, in order to avoid the situation where the migration microservice consumes too much memory and reduces the system availability, in some embodiments, the server uses JDBC (Java DataBaseConnectivity) streaming technology to smoothly pull the data to be migrated in the system to be migrated, ensuring that the memory load of the migration microservice is always stable in a safe range.

[0085] In the above data migration method, a migration request initiated by the target system is obtained and the data to be migrated is determined. The migration microservice is first called to perform data detection on the data to be migrated. By performing a comprehensive check before data migration, the migration failure caused by abnormal data is reduced, and there is no need to manually check whether the data is abnormal, avoiding the repetitive operation of re-exporting the data to be migrated after the abnormality is found, thereby improving the migration efficiency and success rate; when the data detection passes, the target thread pool is determined from multiple thread pools by calling the migration microservice, and the data to be migrated is extracted and migrated in parallel by running multiple threads in the target thread pool, which significantly improves the migration efficiency of the data to be migrated and greatly improves the migration speed. At the same time, the one-click migration of the account set data is realized by calling the migration microservice. Since the migration microservice is deployed separately and runs in a calling manner, the migration microservice itself will not modify and store any original data to be migrated, ensuring the integrity, legality, and security of the original data to be migrated; and the accuracy of the data to be migrated is ensured by detecting the data to be migrated through the migration microservice.

[0086] In some embodiments, the target system can display the progress of data migration to the user in real time through the account creation interface, and prompt the user that the migration is successful after the data migration is completed, so as to achieve good interaction with the user.

[0087] In some embodiments, in order to maximize the data transmission speed and to avoid frequently establishing TCP (Transmission Control Protocol) connections with the target system, the server in the embodiment of the present application uses HTTP (HyperText Transfer Protocol) long connection technology, so that the transmission of the entire data to be migrated is completed in one connection, thereby improving the efficiency of data migration.

[0088] In some embodiments, after obtaining the migration request initiated by the target system, the above method also includes: extracting the credential information carried in the migration request that matches the corresponding system to be migrated; verifying the legitimacy of the credential information, and if the legitimacy is passed, obtaining a list to be migrated including multiple sets of candidate data to be migrated in the system to be migrated, and returning it to the target system, so that the target system can determine the data to be migrated in the list to be migrated.

[0089] The migration request initiated by the target system also includes credential information that matches the corresponding system to be migrated. The server verifies whether it can establish a connection with the system to be migrated according to the migration request through the credential information. The credential information includes but is not limited to one or more login credentials such as the account number, password, passcode, and key of the system to be migrated.

[0090] Specifically, the server extracts the credential information carried in the migration request and verifies the legitimacy of the credential information. For example, the server verifies whether the credential information carried in the migration request is correct, or whether it is consistent with the credential information of the system to be migrated that is pre-stored, and so on, thereby determining the legitimacy of the credential information. If the legitimacy is passed, the server establishes a connection with the system to be migrated, for example, logs in to the system to be migrated using the credential information, and obtains multiple sets of candidate data to be migrated in the system to be migrated, and forms a list to be migrated and returns it to the target system, so that the target system can determine a set of candidate data to be migrated in the list to be migrated as the target data to be migrated, that is, the data to be migrated that is finally determined.

[0091] Exemplarily, after receiving the list of data to be migrated returned by the server, the target system displays it to the user through an interface. The user selects a set of data to be migrated from the list of data to be migrated displayed on the interface. The target system obtains the identification information (e.g., data ID) of the selected data to be migrated and returns it to the server. The server thereby determines that the current data to be migrated is the target data to be migrated.

[0092] In the above embodiment, by verifying the legitimacy of the credential information carried in the migration request, the data security in the system to be migrated is guaranteed, and the security of data migration is improved.

[0093] In some embodiments, Figure 3 As shown, after determining the data to be migrated in the system to be migrated corresponding to the target system according to the migration request, the method further includes:

[0094] Step S302: divide the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration order.

[0095] Among them, data categories can be divided according to the importance of the data, such as basic data categories, important categories, secondary categories, and other categories. Data categories can also be divided according to the attributes of the data, such as enterprise information categories, document business categories, and voucher categories. The data categories are pre-set with a migration order. Usually, the account set data of the basic data category will be upgraded first. Specifically, the server calls the migration microservice, and the migration microservice divides the data to be migrated into different data categories to obtain the sub-data to be migrated belonging to various data categories.

[0096] Step S304: Arrange the sub-data to be migrated according to the migration order and obtain a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0097] Specifically, the migration microservice arranges each set of sub-data to be migrated according to the migration order corresponding to various data categories to obtain a migration queue, for example, the basic data category, important category, secondary category and other categories in order of priority of the migration order. The migration microservice then prioritizes the data migration operations of the sub-data to be migrated in the basic data category. When all the sub-data to be migrated in the basic data category have completed data migration, the migration microservice continues to process the data migration operations of the sub-data to be migrated in the important category... and so on. For each data category of sub-data to be migrated, the migration microservice executes the migration operation in parallel by running multiple threads in the thread pool.

[0098] In the above embodiment, since the amount of data to be migrated is huge, the data to be migrated is divided into sub-data to be migrated of various data categories by setting data categories with different priority levels, and then the sub-data to be migrated of each data category is processed accordingly, and the data to be migrated with higher importance (or higher priority level) is migrated first, thereby improving the migration efficiency of the data to be migrated.

[0099] It is easy to understand that, based on the classification of data categories for the data to be migrated, the various types of data to be migrated obtained by the classification can have a reference relationship between them. For example, taking the data to be migrated as document data, the data common to various documents can be divided into basic data (such as document fields, document time information, etc.), and the data of the document itself can be divided into business document data (such as specific data values ​​corresponding to document fields, and non-common fields, etc.). Thus, the business document data can form complete document data by referencing the basic data. To this end, in some embodiments, the data to be migrated includes basic data and multiple business document data, and at least part of the data in each of the business document data references the basic data.

[0100] Correspondingly, the extracting the data to be migrated from the system to be migrated in parallel and migrating the data to be migrated to the target system includes: extracting basic data from the data to be migrated in the system to be migrated in parallel, and caching the basic data in the process of the migration microservice; extracting each business document data from the data to be migrated in parallel respectively, and combining each business document data with the basic data cached in the process of the migration microservice respectively, and sending the combined data to the target system.

[0101] Specifically, the migration microservice first extracts the basic data in the data to be migrated from the system to be migrated in parallel by running multiple threads in the target thread pool, and caches the basic data in the process of the migration microservice. After the migration of the basic data is completed, the migration microservice then extracts each business document data in the data to be migrated in parallel. For each business document data, after extraction, the migration microservice can combine the business document data with the cached basic data, and then send the combined data to the target system.

[0102] For example, if the data to be migrated is multiple documents, each document contains information such as document fields, document time, and specific values ​​corresponding to the document fields. For example, if the document fields common to each document are product fields, supplier fields, etc., these document fields can be used as basic data. After the migration microservice extracts and caches the common document fields as basic data, it extracts other specific data of each document, and then combines the common document fields with other specific data to form a complete document and migrate it to the target system.

[0103] In some embodiments, since the target system has different document format requirements from the system to be migrated, the migration microservice can also combine the basic data and business document data according to the requirements of the target system, and migrate the combined data that meets the requirements of the target system to the target system, thereby improving the efficiency of migration.

[0104] As a result, the migration microservice no longer needs to migrate duplicate or shared data multiple times, greatly shortening the time spent on data migration. At the same time, by caching basic data, other data can directly read the basic data in the cache when referencing it, without having to extract it again from the system to be migrated, reducing the number of requests to the system interface of the system to be migrated and alleviating the pressure on the server.

[0105] In some embodiments, Figure 4 As shown, the method of calling the migration microservice to run multiple threads in the target thread pool, extracting the data to be migrated from the system to be migrated in parallel, and migrating the data to be migrated to the target system includes:

[0106] Step S402: determining the sub-data to be migrated currently to be migrated in the migration queue.

[0107] Specifically, the server calls the migration microservice, and the migration microservice selects the data category with the first migration order in the migration queue, and uses the sub-data to be migrated of the data category as the sub-data to be migrated currently.

[0108] Step S404: Run multiple threads in the target thread pool in parallel through the migration microservice, extract the sub-data to be migrated currently to be migrated, and cache them into the process of the migration microservice.

[0109] Specifically, the server calls the migration microservice, runs multiple threads in the target thread pool in parallel through the migration microservice, extracts the sub-data to be migrated from the system to be migrated, and caches it in the process of the migration microservice.

[0110] Step S406: The cached sub-data to be migrated are migrated to the target system by invoking the second system interface connected to the target system through the migration microservice.

[0111] Specifically, the server calls the migration microservice, and the migration microservice encapsulates the cached sub-data to be migrated into a data packet. Then, the server calls the second system interface connected to the target system to save the encapsulated sub-data to be migrated to the target system, thereby completing the data migration of the sub-data to be migrated of this data category.

[0112] It should be noted that the above terms "first" and "second" are used in this application to describe different system interfaces, but these system interfaces should not be limited by these terms. These terms are only used to distinguish one system interface from another system interface. For example, the first system interface may be referred to as the second system interface, and similarly, the second system interface may be referred to as the first system interface. The first system interface and the second system interface may be different system interfaces, and in other embodiments, the first system interface and the second system interface may also be the same system interface, which is not limited in the embodiments of the present application.

[0113] Step S408, taking the next sub-data to be migrated in the migration queue as the sub-data to be migrated currently to be migrated, and returning to the step of running multiple threads in the target thread pool in parallel through the migration microservice to continue execution until all sub-data to be migrated in the migration queue complete data migration.

[0114] Specifically, after completing the data migration of the sub-data to be migrated of a data category, the migration microservice takes the sub-data to be migrated that is next in the migration order in the migration queue as the sub-data to be migrated currently, and returns to step S404 to continue execution until all the sub-data to be migrated in the migration queue have completed data migration.

[0115] In the above embodiment, since the amount of data to be migrated is huge, the data to be migrated is divided into sub-data to be migrated of various data categories by setting data categories with different priority levels, and then the sub-data to be migrated of each data category is processed accordingly, and the data to be migrated with higher importance (or higher priority level) is migrated first, thereby improving the migration efficiency of the data to be migrated.

[0116] In order to better understand the inventive concept of the present application, a specific example is given below to illustrate. Taking the account set data to be migrated as an example, in a specific scenario, Figure 5 As shown, the user opens the account creation interface in the target system and selects the data range and corresponding product type of the account set data to be migrated. The account and password of the system to be migrated are entered in the account creation interface so that the backend server can verify its legitimacy through the workbench. If the server verifies that any of the account or password is wrong, the verification result is returned to the target system, and the target system prompts the user that it is illegal. If the legitimacy verification passes, the server calls the system interface of the system to be migrated, obtains one or more sets of account set data from the system to be migrated, and forms a list to be migrated and returns it to the target system for the target system to display and select for the user. After the user selects the account set data to be migrated in the account creation interface, the server calls the migration microservice to extract the account set data from the database of the system to be migrated, and performs data detection on the account set data. If abnormal data is detected, the migration microservice returns abnormal information to the target system for the user to modify or delete. If the data detection passes, the migration microservice can further organize the data, for example, divide the account set data into different data categories, and perform data migration operations in sequence according to the migration order corresponding to different data categories. During the data migration process, the target system can display the progress of data migration in real time through the account creation interface, and display a prompt message indicating that the migration is complete after all data migration is completed. Therefore, users only need to log in to the system to be migrated in the target system to achieve one-click migration of the account data in the system to be migrated, which greatly simplifies the operation difficulty.

[0117] Let's take a specific example and combine it with Figure 5 And as Figure 6As shown in the figure, the user enters the account and password of the system to be migrated on the target system account creation page to log in, and the backend server calls the API interface provided by the workbench to check whether the user is legitimate. If it is legitimate, the backend server calls the API interface provided by the workbench to obtain the account set data of the system to be migrated, forms a list to be migrated and displays it on the account creation page. The user selects the account set data to be upgraded and clicks the "Confirm" button to start the migration. The server calls the API interface of the migration service and sends a migration request for the account set data of the system to be migrated. The migration service will process a single migration request in the following steps: The migration service establishes a connection with the database of the system to be migrated based on the account set ID in the request parameter, and checks the data integrity of the account set data to be migrated according to the preset data inspection rules. When abnormal data that does not meet the rules is detected, the migration service returns the abnormal information to the target system through the API interface. After the target system encapsulates the returned information, it presents it to the user through a user-friendly interface. After the user modifies the abnormal data, repeat the previous steps to continue migrating in the target system.

[0118] When the data detection passes, illustratively, the target system obtains account creation parameters, which include one or more of the account set name, account set date, and currency. The server calls the migration service API interface, and the migration service returns the corresponding account creation parameters so that the target system completes the account creation, and performs data migration by calling the migration service API. The migration service will assign a thread pool to each migration request. The thread pool allocation rules can be the polling allocation, random allocation, and allocation rules allocated according to thread occupancy in the aforementioned embodiment, and also support the extension of other rules. Exemplarily, the migration service will read the default allocation rules preset in the Apollo configuration platform when it starts. After the thread pool is allocated, the data migration phase is entered. In this phase, the account set data is divided into different data categories, and the appropriate migration order is arranged (which can be determined according to needs). Under normal circumstances, the data of the basic information category will be migrated first. For each category of data, the migration microservice will concurrently execute the migration action, which includes reading data, assembling JSON data packets, and calling the target system API interface to save data to the target system. Therefore, through concurrent execution, the migration speed is greatly improved.

[0119] In the process of migrating data, in order to increase the data transmission speed as much as possible, optimization is performed in the embodiments of the present application. First, in order to avoid frequently establishing TCP connections with the target system, HTTP long connection technology is used to complete the transmission of the entire account data in one connection. Secondly, because a large amount of account data is migrated, it is necessary to read the basic information data that is migrated first. Therefore, when migrating the basic information data, the Caffeine high-performance cache component is used to cache the data in the process of the migration service, which greatly shortens the time spent on reading data; thirdly, in order to avoid excessive memory consumption of the migration service, which leads to reduced system availability, JDBC streaming technology is used to smoothly pull the system data to be migrated, ensuring that the memory load of the service is always stable in a safe range.

[0120] During the entire migration process, users can see the real-time progress in the target system account creation interface. When the data of the entire account set is migrated, the migration service will return a message of successful migration to the target system. After the target system encapsulates the data, it will be presented to the user through a friendly interface. When an exception occurs during the migration process, the migration microservice will also prompt the target system with the exception information. Therefore, users only need to log in to the system to be migrated in the target system to achieve one-click migration of the account set data in the system to be migrated, which greatly simplifies the difficulty of operation and provides a good user experience.

[0121] It should be understood that, although the various steps in the flowcharts involved in the above-mentioned embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps does not have a strict order restriction, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-mentioned embodiments can include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0122] Based on the same inventive concept, the embodiment of the present application also provides a data migration device for implementing the data migration method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more data migration device embodiments provided below can refer to the limitations on the data migration method above, and will not be repeated here.

[0123] In some embodiments, Figure 7As shown, a data migration device 700 is provided, comprising: an acquisition module 701, a detection module 702, an allocation module 703 and a migration module 704, wherein:

[0124] The acquisition module 701 is used to acquire a migration request initiated by a target system, and determine the data to be migrated in the system to be migrated corresponding to the target system according to the migration request.

[0125] The detection module 702 is used to call the migration microservice and perform data detection on the data to be migrated through the migration microservice.

[0126] The allocation module 703 is used to determine a target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice when the data detection passes, and the target thread pool is configured with multiple threads.

[0127] The migration module 704 is used to call the migration microservice to run multiple threads in the target thread pool, extract the data to be migrated from the system to be migrated in parallel, and migrate the data to be migrated to the target system.

[0128] In some embodiments, the above-mentioned device also includes a verification module, which is used to extract the credential information carried in the migration request and matching the corresponding system to be migrated; verify the legitimacy of the credential information, and if the legitimacy is passed, obtain a list to be migrated including multiple sets of candidate data to be migrated in the system to be migrated, and return it to the target system so that the target system can determine the data to be migrated in the list to be migrated.

[0129] In some embodiments, the detection module is further used to, when abnormal data is detected in the data to be migrated, call a first system interface connected to the target system through the migration microservice to feed back abnormal information containing the abnormal data to the target system.

[0130] In some embodiments, the above-mentioned device also includes an arrangement module, which is used to divide the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration orders; the sub-data to be migrated are arranged according to the migration order and a migration queue is obtained, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0131] In some embodiments, the migration module is also used to determine the sub-data to be migrated currently in the migration queue; run multiple threads in the target thread pool in parallel through the migration microservice, extract the sub-data to be migrated currently and cache them in the process of the migration microservice; call the second system interface connected to the target system through the migration microservice to migrate the cached sub-data to be migrated to the target system; use the next sub-data to be migrated in the migration queue as the sub-data to be migrated currently, and return to the step of running multiple threads in the target thread pool in parallel through the migration microservice to continue execution until all sub-data to be migrated in the migration queue are migrated.

[0132] In some embodiments, the method of determining the target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice includes at least one of the following: by calling the migration microservice, the migration microservice queries the usage status of each thread pool in turn, and determines the first thread pool that is currently idle as the target thread pool corresponding to the target system; by calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently idle as the target thread pool corresponding to the target system; by calling the migration microservice, the migration microservice queries the usage status of each thread in each thread pool, and determines the target thread pool corresponding to the target system according to the number of threads required for the data to be migrated.

[0133] Each module in the above data migration device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0134] In some embodiments, a computer device is provided. The computer device may be a server, and its internal structure diagram may be as follows: Figure 8As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, referred to as I / O) and a communication interface. The processor, the memory and the input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data to be migrated. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, a data migration method is implemented.

[0135] Those skilled in the art will understand that Figure 8 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0136] In some embodiments, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the following steps when executing the computer program: obtaining a migration request initiated by a target system, and determining the data to be migrated in the system to be migrated corresponding to the target system according to the migration request; calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice; when the data detection passes, determining a target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice, wherein multiple threads are configured in the target thread pool; extracting the data to be migrated from the system to be migrated in parallel by calling the migration microservice, and migrating the data to be migrated to the target system.

[0137] In some embodiments, when the processor executes the computer program, the following steps are also implemented: extracting the credential information carried in the migration request that matches the corresponding system to be migrated; verifying the legitimacy of the credential information, and if the legitimacy is passed, obtaining a list to be migrated that includes multiple sets of candidate data to be migrated in the system to be migrated, and returning it to the target system so that the target system can determine the data to be migrated in the list to be migrated.

[0138] In some embodiments, when the processor executes the computer program, the following steps are also implemented: when abnormal data is detected in the data to be migrated, the migration microservice calls a first system interface connected to the target system to feed back abnormal information containing the abnormal data to the target system.

[0139] In some embodiments, when the processor executes the computer program, the following steps are also implemented: dividing the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration orders; arranging the sub-data to be migrated according to the migration order and obtaining a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0140] In some embodiments, when the processor executes the computer program, the following steps are also implemented: determining the sub-data to be migrated currently to be migrated in the migration queue; running multiple threads in the target thread pool in parallel through the migration microservice, extracting the sub-data to be migrated currently to be migrated and caching them into the process of the migration microservice; calling a second system interface connected to the target system through the migration microservice to migrate the cached sub-data to be migrated to the target system; taking the next sub-data to be migrated in the migration queue as the sub-data to be migrated currently to be migrated, and returning to the step of running multiple threads in the target thread pool in parallel through the migration microservice to continue executing until all sub-data to be migrated in the migration queue are migrated.

[0141] In some embodiments, the processor further implements the following steps when executing the computer program: by calling the migration microservice, the migration microservice queries the usage status of each thread pool in turn, and determines the first thread pool that is currently idle as the target thread pool corresponding to the target system; by calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently idle as the target thread pool corresponding to the target system; by calling the migration microservice, the migration microservice queries the usage status of each thread in each thread pool, and determines the target thread pool corresponding to the target system according to the number of threads required for the data to be migrated.

[0142] In some embodiments, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the following steps are implemented: obtaining a migration request initiated by a target system, and determining the data to be migrated in the system to be migrated corresponding to the target system according to the migration request; calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice; when the data detection passes, determining a target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice, wherein multiple threads are configured in the target thread pool; extracting the data to be migrated from the system to be migrated in parallel by calling the migration microservice, and migrating the data to be migrated to the target system.

[0143] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: extracting the credential information carried in the migration request that matches the corresponding system to be migrated; verifying the legitimacy of the credential information, and if the legitimacy is passed, obtaining a list to be migrated including multiple sets of candidate data to be migrated in the system to be migrated, and returning it to the target system so that the target system can determine the data to be migrated in the list to be migrated.

[0144] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: when abnormal data is detected in the data to be migrated, the migration microservice calls a first system interface connected to the target system, and abnormal information containing the abnormal data is fed back to the target system.

[0145] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: dividing the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration orders; arranging the sub-data to be migrated according to the migration order and obtaining a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0146] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: determining the sub-data to be migrated currently in the migration queue; running multiple threads in the target thread pool in parallel through the migration microservice, extracting the sub-data to be migrated currently and caching them in the process of the migration microservice; calling a second system interface connected to the target system through the migration microservice to migrate the cached sub-data to be migrated to the target system; taking the next sub-data to be migrated in the migration queue as the sub-data to be migrated currently, and returning to the step of running multiple threads in the target thread pool in parallel through the migration microservice to continue execution until all sub-data to be migrated in the migration queue are migrated.

[0147] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: by calling the migration microservice, the migration microservice queries the usage status of each thread pool in turn, and determines the first thread pool that is currently idle as the target thread pool corresponding to the target system; or, by calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently idle as the target thread pool corresponding to the target system; or, by calling the migration microservice, the migration microservice queries the usage status of each thread in each thread pool, and determines the target thread pool corresponding to the target system according to the number of threads required for the data to be migrated.

[0148] In some embodiments, a computer program product is provided, including a computer program, which implements the following steps when executed by a processor: obtaining a migration request initiated by a target system, and determining the data to be migrated in the system to be migrated corresponding to the target system according to the migration request; calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice; when the data detection passes, determining a target thread pool corresponding to the target system from multiple thread pools by calling the migration microservice, wherein multiple threads are configured in the target thread pool; extracting the data to be migrated from the system to be migrated in parallel by calling the migration microservice, and migrating the data to be migrated to the target system.

[0149] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: extracting the credential information carried in the migration request that matches the corresponding system to be migrated; verifying the legitimacy of the credential information, and if the legitimacy is passed, obtaining a list to be migrated including multiple sets of candidate data to be migrated in the system to be migrated, and returning it to the target system so that the target system can determine the data to be migrated in the list to be migrated.

[0150] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: when abnormal data is detected in the data to be migrated, the migration microservice calls a first system interface connected to the target system, and abnormal information containing the abnormal data is fed back to the target system.

[0151] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: dividing the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration orders; arranging the sub-data to be migrated according to the migration order and obtaining a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue.

[0152] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: determining the sub-data to be migrated currently in the migration queue; running multiple threads in the target thread pool in parallel through the migration microservice, extracting the sub-data to be migrated currently and caching them in the process of the migration microservice; calling a second system interface connected to the target system through the migration microservice to migrate the cached sub-data to be migrated to the target system; taking the next sub-data to be migrated in the migration queue as the sub-data to be migrated currently, and returning to the step of running multiple threads in the target thread pool in parallel through the migration microservice to continue execution until all sub-data to be migrated in the migration queue are migrated.

[0153] In some embodiments, when the computer program is executed by the processor, the following steps are also implemented: by calling the migration microservice, the migration microservice queries the usage status of each thread pool in turn, and determines the first thread pool that is currently idle as the target thread pool corresponding to the target system; or, by calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently idle as the target thread pool corresponding to the target system; or, by calling the migration microservice, the migration microservice queries the usage status of each thread in each thread pool, and determines the target thread pool corresponding to the target system according to the number of threads required for the data to be migrated.

[0154] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0155] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0156] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data migration method, characterized in that: The method comprises: Obtaining a migration request initiated by the target system, extracting credential information carried in the migration request that matches the system to be migrated, verifying the legitimacy of the credential information, and establishing a connection with the system to be migrated if the legitimacy is passed, and determining the data to be migrated in the system to be migrated; wherein the data to be migrated includes account set data; Calling a migration microservice, and performing data detection on the data to be migrated through the migration microservice; When the data detection passes, a target thread pool corresponding to the target system is determined from multiple thread pools by calling a migration microservice, wherein the target thread pool is configured with multiple threads; Dividing the data to be migrated into different data categories to obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration order; Arrange the sub-data to be migrated according to the migration order and obtain a migration queue, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue; For each data category of sub-data to be migrated, the migration microservice is called to run multiple threads in the target thread pool, the sub-data to be migrated are extracted in parallel from the system to be migrated, and the sub-data to be migrated are migrated to the target system; wherein, in the case where the sub-data to be migrated of each data category include basic data and business document data, and at least part of the data in each business document data references the basic data, when performing data migration, the migration microservice extracts the basic data in parallel, and caches the basic data in the process of the migration microservice through a cache component, and for each business document data, the basic data in the cache is directly read when the basic data needs to be referenced, so as to combine the business document data with the basic data to form complete document data and send it to the target system.

2. The method according to claim 1, characterized in that The determining the data to be migrated in the system to be migrated comprises: A to-be-migrated list including multiple sets of candidate to-be-migrated data in the to-be-migrated system is obtained, and the list is returned to the target system, so that the target system can determine the to-be-migrated data in the to-be-migrated list.

3. The method according to claim 1, characterized in that After the migration microservice performs data detection on the data to be migrated, the method further includes: When it is detected that abnormal data exists in the data to be migrated, the migration microservice calls a first system interface connected to the target system to feed back abnormal information containing the abnormal data to the target system.

4. The method according to claim 1, characterized in that: For each data category of the sub-data to be migrated, the sub-data to be migrated are extracted from the system to be migrated in parallel by calling the migration microservice to run multiple threads in the target thread pool, and the sub-data to be migrated are migrated to the target system, including: Determining the sub-data to be migrated currently to be migrated in the migration queue; By running the multiple threads in the target thread pool in parallel through the migration microservice, the sub-data to be migrated currently to be migrated is extracted and cached into the process of the migration microservice; Migrating the cached sub-data to be migrated to the target system by invoking the second system interface connected to the target system through the migration microservice; The next sub-data to be migrated in the migration queue is used as the sub-data to be migrated currently to be migrated, and the process returns to the step of running multiple threads in the target thread pool in parallel through the migration microservice until all sub-data to be migrated in the migration queue are migrated.

5. The method according to claim 1, characterized in that The method of determining the target thread pool corresponding to the target system from the multiple thread pools by calling the migration microservice includes at least one of the following: By calling the migration microservice, the migration microservice sequentially queries the usage status of each thread pool, and determines the first thread pool currently in an idle state obtained by the query as the target thread pool corresponding to the target system; By calling the migration microservice, the migration microservice randomly selects a thread pool from all thread pools that are currently in an idle state to determine it as a target thread pool corresponding to the target system; By calling the migration microservice, the migration microservice queries the occupancy status of threads in each thread pool, and determines the target thread pool corresponding to the target system according to the occupancy status and the number of threads required to be occupied by the data to be migrated.

6. A data migration device, characterized in that: The device comprises: An acquisition module, used for acquiring a migration request initiated by a target system; A verification module, used to extract the credential information matching the system to be migrated carried in the migration request, verify the legitimacy of the credential information, establish a connection with the system to be migrated if the legitimacy is passed, and determine the data to be migrated in the system to be migrated; wherein the data to be migrated includes account set data; A detection module, used to call a migration microservice and perform data detection on the data to be migrated through the migration microservice; An allocation module, configured to determine a target thread pool corresponding to the target system from multiple thread pools by calling a migration microservice when the data detection passes, wherein the target thread pool is configured with multiple threads; An arrangement module, used to divide the data to be migrated into different data categories, and obtain sub-data to be migrated of each data category; wherein different data categories correspond to their own migration orders; the sub-data to be migrated are arranged according to the migration order and a migration queue is obtained, so that the migration microservice can migrate according to the migration order corresponding to each sub-data to be migrated in the migration queue; A migration module is used for extracting the sub-data to be migrated from the system to be migrated in parallel and migrating the sub-data to be migrated to the target system by calling the migration microservice to run multiple threads in the target thread pool for the sub-data to be migrated of each data category; wherein, in the case where the sub-data to be migrated of each data category include basic data and business document data, and at least part of the data in each business document data references the basic data, when performing data migration, the migration microservice extracts the basic data in parallel and caches the basic data in the process of the migration microservice through a cache component; for each business document data, the basic data in the cache is directly read when the basic data needs to be referenced, so as to combine the business document data with the basic data to form complete document data and send it to the target system.

7. The device according to claim 6, characterized in that The detection module is further configured to, when abnormal data is detected in the data to be migrated, call a first system interface connected to the target system through the migration microservice to feed back abnormal information containing the abnormal data to the target system.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 5 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 5 are implemented.

Images