Data migration method and device, equipment and medium

Abstract

The embodiment of the invention discloses a data migration method, device and equipment and a medium, and is suitable for the field of financial science and technology, the method comprises the following steps: determining a migration time point of a source end database, and generating an incremental data block according to an operation log and the migration time point of the source end database; wherein the source end database comprises historical data blocks of a plurality of application services, the operation of the application services is realized through the plurality of historical data blocks, and the incremental data block records the data change condition of the source end database after the migration time point; a service dependency graph of the source end database is obtained, and the service dependency graph represents the service calling relation between the multiple application services and the incidence relation between historical data blocks of the application services; according to the service dependency graph, determining a migration sequence of each historical data block of each application service; and migrating the historical data blocks and the incremental data blocks to the target database according to the migration sequence. According to the technical scheme provided by the invention, the data migration efficiency can be improved.

Description

Technical Field

[0001] This invention relates to the field of data transmission, and more particularly to a data migration method, apparatus, device, and medium. Background Technology

[0002] In the banking sector, data migration is a crucial step in supporting core system upgrades, ensuring business continuity, and meeting financial regulatory compliance requirements. With business development and technological evolution, database migration has become routine. Reasons for data migration include system upgrades and data center downsizing or consolidation, and it holds irreplaceable strategic value for maintaining financial stability and business development.

[0003] However, bank data migration often results in a chaotic and inefficient process due to the massive volume of data. More importantly, data that has not been properly organized and is directly migrated to the target database must be processed again before it can be used, which is a huge waste of time and computing resources. Summary of the Invention

[0004] This invention provides a data migration method, apparatus, device, and medium. The method of this invention can determine the migration order of each data, thereby improving the efficiency of data migration. At the same time, it reduces the difficulty of organizing each data after it arrives at the target database and saves resources.

[0005] This invention provides a data migration method, including:

[0006] Determine the migration time point of the source database, and generate incremental data blocks based on the operation logs of the source database and the migration time point; wherein, the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record the data changes of the source database after the migration time point;

[0007] Obtain the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application service;

[0008] Based on the service dependency graph, determine the migration order of each historical data block of each application service;

[0009] According to the migration order, each historical data block and incremental data block is migrated to the target database.

[0010] Secondly, embodiments of the present invention provide a data migration apparatus, comprising:

[0011] The generation module is used to determine the migration time point of the source database and generate incremental data blocks based on the operation logs of the source database and the migration time point. The source database includes historical data blocks of multiple application services, and the operation of the application services is implemented through multiple historical data blocks. The incremental data blocks record the data changes of the source database after the migration time point.

[0012] The acquisition module is used to acquire the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application service;

[0013] The sequence determination module is used to determine the migration order of each historical data block of each application service based on the service dependency graph.

[0014] The migration module is used to migrate each historical data block and incremental data block to the target database according to the migration order.

[0015] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising:

[0016] At least one processor; and,

[0017] A memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data migration method described in any one of the embodiments of the present invention.

[0019] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing computer instructions that, when executed by a processor, implement the data migration method described in any one of the embodiments of the present invention.

[0020] This invention provides a data migration method, apparatus, device, and medium. The method includes: determining a migration time point for a source database; generating incremental data blocks based on the operation logs of the source database and the migration time point; wherein the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record data changes in the source database after the migration time point; obtaining a service dependency graph of the source database, wherein the service dependency graph represents the service call relationships between the multiple application services and the association relationships between the historical data blocks of each application service; determining the migration order of each historical data block of each application service based on the service dependency graph; and migrating each historical data block and incremental data block to a target database according to the migration order. Specifically, the service dependency graph includes the call relationships between application services and the association relationships between their historical data blocks. Determining the migration order of each historical data block of each application service based on the association relationships ensures that interrelated data is migrated to the target database in a centralized and orderly manner, improving data migration efficiency while reducing the difficulty of data processing and resource consumption after migration. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart of a data migration method provided in Embodiment 1 of the present invention;

[0023] Figure 2 This is a flowchart of a data migration method provided in Embodiment 2 of the present invention;

[0024] Figure 3 This is a schematic diagram of the structure of a data migration device provided in Embodiment 3 of the present invention;

[0025] Figure 4 This is a schematic diagram of the structure of an electronic device provided in Embodiment 4 of the present invention. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0027] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0028] It should be noted that the collection, storage, use, processing, transmission, provision, and disclosure of user personal information involved in the technical solution disclosed herein all comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0029] Example 1

[0030] Figure 1 This is a flowchart illustrating a data migration method according to Embodiment 1 of the present invention. This method is specifically applicable to the fintech field, particularly for the rapid and orderly migration of data content from bank databases. The method can be executed using a data migration device, which can be composed of software and / or hardware and configured in a computer or server.

[0031] like Figure 1 As shown, it includes:

[0032] Step 110: Determine the migration time point of the source database, and generate incremental data blocks based on the operation logs of the source database and the migration time point; wherein, the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record the data changes of the source database after the migration time point.

[0033] The source database is the database that provides the data to be migrated. The migration time point refers to a specific moment determined in the source database, serving as a reference point for data migration and marking the start of the migration task. The operation log records the data operations performed by users on the source database, including the operation object, operation time, and operation content. Incremental data blocks record data changes in the source database after the migration time point. It should be noted that from the migration time point onwards, historical data blocks in the source database will no longer be modified. However, due to continuous business operation, users may still perform data operations on the source database. Therefore, data changes after the migration time point need to be temporarily stored in incremental data blocks. After the migration task is completed, the data will be updated in the target database based on the incremental data blocks. Application services refer to business services that rely on historical data blocks to run. Historical data blocks are data units stored in the source database to support the operation of application services, corresponding to specific data entity files, such as data tables, program code, and other data files. Operations on historical data blocks are essentially operations on the data entity files within them, such as adding, deleting, querying, and modifying data tables to complete the tasks involved in the application service.

[0034] Optionally, generating incremental data blocks based on the operation logs of the source database and the migration time point includes:

[0035] Obtain data operation records after the migration time point from the operation log and store the data operation records in an empty data block; when the amount of data in the empty data block reaches a preset threshold, the empty data block is determined as an incremental data block.

[0036] Specifically, by acquiring post-migration operation records and storing them in batches to generate incremental data blocks, the incremental data block size can be increased, ensuring that data is not lost and business operations are not interrupted during the data migration process, thereby improving data integrity.

[0037] Specifically, the operation identifier of the data operation can be determined through the operation log, and the operation identifier and operation content can be associated and stored in an empty data block, wherein the operation identifier represents the operation object of the data operation.

[0038] Specifically, by associating storage operation identifiers and content, the traceability and accuracy of data can be ensured.

[0039] Furthermore, after migrating the incremental data blocks and the historical data blocks of the application services to the target database, the process further includes:

[0040] Based on the operation identifier of the data operation in the incremental data block, the corresponding operation object is determined in the historical data block, and the operation object is updated according to the operation content of the data operation.

[0041] It should be noted that by associating operation identifiers and operation content within incremental data blocks, the target database can accurately locate the data objects that need to be updated based on the operation identifiers, then retrieve historical data blocks, and complete the data update of the data objects according to the corresponding operation content. This method effectively ensures the continuity and integrity of the data migration process.

[0042] Step 120: Obtain the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application services.

[0043] The service dependency graph represents the service call relationships between multiple application services, as well as the association relationships between the historical data blocks corresponding to each application service. When executing a specific data task, multiple application services typically need to work together; the service call relationship is the collaborative logic of these services, such as an application service requesting or accessing the functions of other application services during its operation. The association relationships between historical data blocks represent the references or access relationships between the data entity files corresponding to different historical data blocks during service execution. For example, when an application service executes a task, it may need to sequentially call data entities such as table A, file B, and program C.

[0044] Step 130: Determine the migration order of each historical data block of each application service according to the service dependency graph.

[0045] The migration order characterizes the order in which data is sent and the priority of data block migration. For example, the migration order may include serial sending, parallel sending, and combined packet sending.

[0046] For example, a service needs to simultaneously access a weight record data block, a height record data block, and a length record data block. These three data blocks are closely related and require joint queries. Therefore, the migration order of these three data blocks can be determined as either sending them together in a package or sending them in parallel.

[0047] For example, if the operation of historical data block A depends on historical data block B, the migration order can be determined as serial transmission: historical data block B is sent first, and then historical data block A is sent.

[0048] Step 140: According to the migration order, migrate each historical data block and incremental data block to the target database.

[0049] Specifically, historical data blocks can be sent to the target database in parallel or serially based on the migration order to complete the data migration process. It should be noted that because the data blocks are sent in the migration order, the target database will configure its receiving process accordingly; therefore, the difficulty and resource consumption of data processing after migration can be reduced.

[0050] This invention provides a data migration method, comprising: determining a migration time point for a source database; generating incremental data blocks based on the operation logs of the source database and the migration time point; wherein the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record data changes in the source database after the migration time point; obtaining a service dependency graph of the source database, wherein the service dependency graph represents the service call relationships between the multiple application services and the association relationships between the historical data blocks of each application service; determining the migration order of each historical data block of each application service based on the service dependency graph; and migrating each historical data block and incremental data block to a target database according to the migration order. Specifically, the service dependency graph includes the call relationships between application services and the association relationships between their historical data blocks. Determining the migration order of each historical data block of each application service based on the association relationships ensures that interrelated data is migrated to the target database in a centralized and orderly manner, improving data migration efficiency while reducing the difficulty of data processing and resource consumption after migration.

[0051] Example 2

[0052] Figure 2 This is a flowchart of a data migration method provided in Embodiment 2 of the present invention. Based on the above embodiments, this method further defines the method for generating a service dependency graph, such as... Figure 2 As shown, it includes:

[0053] Step 210: Determine the migration time point of the source database, and generate incremental data blocks based on the operation logs of the source database and the migration time point; wherein, the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record the data changes of the source database after the migration time point.

[0054] Step 220: Obtain the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application services.

[0055] Optional methods for generating the service dependency graph include:

[0056] Based on the monitoring logs during the application service operation, determine the running order of the data files corresponding to each historical data block; based on the running order, determine the running dependencies between each historical data block; based on the historical data blocks and running dependencies of each application service, generate a service subgraph for each application service; based on the service subgraphs of each application service, generate a service dependency graph.

[0057] Runtime dependencies refer to the dependency logic between historical data blocks within a single application service. This includes the order in which the data files corresponding to the data blocks are called during service execution, as well as the reference and hierarchical relationships between the data files corresponding to the data blocks.

[0058] For example, table A in data block A may contain a key field that references table B in data block B. This reference relationship constitutes a dependency of A on B.

[0059] For example, a hierarchical relationship is a logical hierarchy or subordinate structure that exists between data blocks. For instance, table A in data block A might be used to store basic information about the parent entity, while table B in data block B might be used to store detailed information about the child entity. Tables A and B form a parent-child hierarchy (table A is the higher level, and table B is the lower level).

[0060] Specifically, by analyzing the runtime sequence of application services through monitoring logs, the actual call order of data files in each historical data block is determined, thereby deriving the runtime dependencies between data blocks. Based on this, a service subgraph is constructed for each application service, using historical data blocks as nodes and runtime dependencies as edges. Finally, all service subgraphs are integrated to form a global service dependency graph. Furthermore, the generated service dependency graph accurately illustrates the execution order and relationships between historical data blocks. Calling the service dependency graph during the migration task can accurately determine the migration order and improve migration efficiency.

[0061] Furthermore, generating a service dependency graph based on the service subgraphs of each application service includes:

[0062] Obtain the service call order of each application service during runtime; and the service subgraph of each application service, and generate a service dependency graph.

[0063] Specifically, when performing specific business operations, multiple application services often need to work together. By analyzing the interface connection relationships or call sequence information between these application services, the service call order at runtime can be determined, thereby constructing a service dependency graph. This method can accurately identify the call relationships between application services, allowing data blocks corresponding to each application service to be sent sequentially according to these relationships during database migration, effectively improving migration efficiency.

[0064] Step 230: For any application service, according to the running dependencies of each historical data block of the application service in the service dependency graph and the preset data volume threshold of the data packet, the historical data blocks of the application service are sequentially packaged to obtain multiple data packets.

[0065] Step 240: Determine the migration order of data packets for each application service based on the service dependency relationships of each application service in the service dependency graph.

[0066] Among them, runtime dependencies are the dependency logic between historical data blocks of a single application service, representing the order in which historical data blocks are called during the execution of the application service. Service dependencies are the call and dependency logic between different application services, representing other application services that the normal operation of an application service depends on, such as constraints on the execution order of application services at runtime. The preset data volume threshold of the data packet is used to limit its storage capacity, ensuring that at least one data block can be contained in the same data packet to achieve the overall transmission of the data block.

[0067] Specifically, the service dependency graph is composed of service subgraphs corresponding to each application service. By analyzing the connections between historical data blocks within a service subgraph, the runtime dependencies of each historical data block within that application service can be determined. By analyzing the connections between different service subgraphs, the service dependencies between application services can be determined. Based on this, historical data blocks of the same application service can be packaged into multiple data packets according to runtime dependencies, and the sending order of each historical data packet can be determined. At the same time, the overall execution order between application services can be clarified according to service dependencies, thereby organizing the data migration process in an orderly manner to optimize the migration order and improve efficiency.

[0068] Step 250: According to the migration order, migrate each historical data block and incremental data block to the target database.

[0069] For example, this data migration method can also be applied to migration scenarios across hardware architectures. Before performing data migration, the system automatically identifies the hardware differences between the source and target databases, constructs a virtualization compatibility layer to shield the underlying heterogeneous environment, and ensures that the virtual machine can run normally on the target side without modification. The specific process includes: First, the engine scans the hardware configuration of both sides and generates a difference comparison report; then, the instruction translation layer is activated on the target side to translate the source instructions in real time and synchronize the execution status. At the same time, applicable device drivers or emulation modules are dynamically loaded according to the difference comparison report, so that the virtual machine disk, network, and other devices maintain compatible access in the target environment. Instruction translation and driver adaptation work together to ensure seamless migration and stable operation of virtual machines between heterogeneous hardware. To support performance consistency, the system also pre-allocates cloud resources, such as computing power, memory, and bandwidth, on the target side through a resource reservation agent to match the performance of the source side.

[0070] For example, the method for generating incremental data blocks can be implemented as follows:

[0071] Incremental data capture is implemented using a dynamic hierarchical structure based on a layered merged storage architecture. All write operations to the source database are uniformly converted into write operations to incremental data blocks. The layered storage architecture acts as a dynamic node system for capturing incremental data during the migration process. First, all storage nodes that have been updated or generated after the migration time point are scanned. Then, the corresponding data files are merged and scanned, and the latest data content is retained to generate incremental data blocks. An incremental data block refers to the data content that has been modified or added since the last complete or incremental synchronization. Each incremental data block logically corresponds to the latest state of at least one piece of data. This method relies entirely on the change records maintained internally by the storage structure for the capture process, enabling efficient scanning of ordered data and naturally obtaining the latest value of each piece of data. It does not require comparing the complete data of the source and target ends during migration execution, thus significantly improving the efficiency and performance of incremental synchronization. The migration engine processes data using a two-stage strategy. Stage one prioritizes stable data (historical data blocks), that is, scanning and migrating all data nodes that were frozen or persisted before the current migration time point. The data content of these nodes is stable and unchanged, and can be migrated safely and efficiently. Phase 2 is responsible for capturing new changes (incremental data blocks) during the migration: Simultaneously or after Phase 1, new write operations generated at the source end are continuously recorded in the log; the system reads all data written since the start of the migration and appends these new changes as new data blocks to the current synchronization batch.

[0072] Furthermore, data verification can be performed after the migration is complete. A small amount of real traffic is copied from the source production environment as canary traffic. Traffic replication rules are customized according to server roles, such as filtering by user hash and request path, to ensure that canary traffic is accurately routed to the target. Real-time comparison verification compares the processing results of the same canary traffic at the source and target ends in real time, including output response, database changes, message output, and file changes. The verification content should cover functional correctness, data consistency, and performance indicators to ensure that the target end's behavior meets expectations. Gradual traffic switching: After the low-proportion canary traffic verification is stable, the canary proportion is gradually increased. Consistency verification and performance monitoring must be performed again after each round of increase. If anomalies are found, the increase or rollback of the traffic proportion should be paused, and timely investigation and repair should be carried out. Final business switching: After the canary traffic proportion reaches the preset threshold and the system behavior continues to be stable, it is confirmed that the target end has full capacity to take over. All business requests originally directed to the source end are switched to the target end through the global traffic scheduling mechanism. After that, the source end device can be converted to read-only backup or enter a waiting-to-retirement state.

[0073] Furthermore, data file packaging operations can be performed by training a grouping model. For small file groups with strong dependencies and frequent simultaneous access, the model recommends packaging these small file groups into a single large chunk for transmission to reduce network connection overhead. When an extremely large monolithic file is identified, it is recommended to split it into multiple smaller chunks for parallel transmission to improve throughput. For cold data with low access frequency, a low-priority scheduling and high compression ratio strategy can be adopted to optimize overall transmission efficiency. Data blocks with strong startup dependencies are aggregated into logical migration units to ensure they are migrated or rolled back as a whole data packet. The model also supports real-time network awareness and dynamic routing selection. By combining real-time network probe data and historical performance information, the optimal transmission protocol and path are dynamically selected for different types of data packets: high-reliability, low-latency links are prioritized for latency-sensitive small files; high-bandwidth paths are prioritized for large, non-sensitive data blocks. In cross-regional transmission scenarios, optimized public network acceleration links or dedicated lines can be automatically selected. At the scheduling level, the model uses a dependency-aware scheduling mechanism to ensure that parent data blocks migrate before child data blocks based on the service dependency graph, and optimizes critical paths based on predicted latency. Leveraging resource awareness, the model combines predicted resource consumption with real-time system load to rationally allocate migration tasks, avoid overload of CPU, memory, network, or storage I / O, and prioritize scheduling data blocks on non-bottleneck resources, continuously improving migration efficiency and stability.

[0074] The data migration method of this invention can accurately determine the migration order between historical data blocks of each application service through a service dependency graph. Data migration can be carried out through the migration order, which can improve the data migration efficiency and reduce the difficulty of data processing and resource consumption after data migration.

[0075] Example 3

[0076] Figure 3 This is a schematic diagram of a data migration device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device includes:

[0077] The generation module 310 is used to determine the migration time point of the source database and generate incremental data blocks based on the operation log of the source database and the migration time point; wherein, the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record the data changes of the source database after the migration time point;

[0078] The acquisition module 320 is used to acquire the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application service;

[0079] The sequence determination module 330 is used to determine the migration order of each historical data block of each application service according to the service dependency graph;

[0080] The migration module 340 is used to migrate each historical data block and incremental data block to the target database according to the migration order.

[0081] This invention provides a data migration apparatus that: determines the migration time point of a source database; generates incremental data blocks based on the operation logs of the source database and the migration time point; wherein the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record data changes in the source database after the migration time point; obtains a service dependency graph of the source database, wherein the service dependency graph represents the service call relationships between the multiple application services and the association relationships between the historical data blocks of each application service; determines the migration order of each historical data block of each application service based on the service dependency graph; and migrates each historical data block and incremental data block to a target database according to the migration order. Specifically, the service dependency graph includes the call relationships between application services and the association relationships between their historical data blocks. Determining the migration order of each historical data block of each application service based on the association relationships ensures that interrelated data is migrated to the target database in a centralized and orderly manner, improving data migration efficiency while reducing the difficulty of data processing and resource consumption after migration.

[0082] Optionally, the generation module 310 includes:

[0083] Storage unit, used to obtain data operation records after the migration time point in the operation log, and store the data operation records in an empty data block;

[0084] The determining unit is used to determine the empty data block as an incremental data block when the amount of data in the empty data block reaches a preset threshold.

[0085] Specifically, the storage unit is used to determine the operation identifier of the data operation, and associate the operation identifier with the operation content in an empty data block for storage, wherein the operation identifier represents the operation object of the data operation.

[0086] Optionally, the apparatus further includes a graph generation module, comprising:

[0087] The extraction unit is used to determine the execution order of the data files corresponding to each historical data block based on the monitoring logs during the operation of the application service.

[0088] The relationship determination unit is used to determine the operational dependencies between historical data blocks based on the operational sequence.

[0089] The generation unit is used to generate service subgraphs for each application service based on the historical data blocks and runtime dependencies of each application service.

[0090] The combination unit is used to generate a service dependency graph based on the service subgraphs of each application service.

[0091] Optionally, the combination unit is specifically used to obtain the service call order of each application service during runtime; and the service subgraph of each application service, to generate a service dependency graph.

[0092] Optionally, the sequence determination module 330 includes:

[0093] The packaging unit is used to sequentially package the historical data blocks of any application service according to the service dependency relationship of each historical data block of the application service in the service dependency relationship diagram and the preset data volume threshold of the data packets, to obtain multiple data packets.

[0094] The sequence determination unit is used to determine the migration order of data packets of each application service based on the service dependency relationship of each application service in the service dependency relationship graph.

[0095] Optionally, the device further includes an update unit, specifically configured to determine the corresponding operation object in the historical data block based on the operation identifier of the data operation in the incremental data block, and update the operation object according to the operation content of the data operation.

[0096] The data migration apparatus provided in the embodiments of the present invention can execute the data migration method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of executing the method.

[0097] Example 4

[0098] Figure 4 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0099] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory 12 or a random access memory 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the read-only memory 12 or loaded from storage unit 18 into the random access memory 13. The random access memory 13 may also store various programs and data required for the operation of the electronic device 10. The processor 11, read-only memory 12, and random access memory 13 are interconnected via a bus 14. An input / output interface 15 is also connected to the bus 14.

[0100] Multiple components in electronic device 10 are connected to input / output interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of monitors, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0101] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, central processing units, graphics processing units, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, digital signal processors, and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as data migration methods.

[0102] In some embodiments, the data migration method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via read-only memory 12 and / or communication unit 19. When the computer program is loaded into random access memory 13 and executed by processor 11, one or more steps of the data migration method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform the data migration method by any other suitable means (e.g., by means of firmware).

[0103] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays, application-specific integrated circuits (ASICs), application-specific standard products (ASICs), systems-on-a-chip (SoCs), complex programmable logic devices, computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0104] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0105] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory, read-only memory, erasable programmable read-only memory (EP memory or flash memory), optical fibers, portable compact disk read-only memory, optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0106] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a cathode ray tube or liquid crystal display, a monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0107] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0108] A computing system can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product within the cloud computing service system. This addresses the shortcomings of traditional physical hosts and cloud host services, such as high management difficulty and weak business scalability.

[0109] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0110] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A data migration method, characterized in that, include: Determine the migration time point of the source database, and generate incremental data blocks based on the operation logs of the source database and the migration time point; wherein, the source database includes historical data blocks of multiple application services, the operation of the application services is implemented through multiple historical data blocks, and the incremental data blocks record the data changes of the source database after the migration time point; Obtain the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application service; Based on the service dependency graph, determine the migration order of each historical data block of each application service; According to the migration order, each historical data block and incremental data block is migrated to the target database.

2. The method according to claim 1, characterized in that, The step of generating incremental data blocks based on the operation logs of the source database and the migration time point includes: Obtain the data operation records after the migration time point in the operation log, and store the data operation records in an empty data block; When the amount of data in the empty data block reaches a preset threshold, the empty data block is identified as an incremental data block.

3. The method according to claim 2, characterized in that, The step of storing the data operation record in an empty data block includes: A data operation identifier is determined, and the operation identifier and operation content are associated and stored in an empty data block, wherein the operation identifier represents the operation object of the data operation.

4. The method according to claim 1, characterized in that, The method for generating the service dependency graph includes: Based on the monitoring logs during the application service operation, determine the execution order of the data files corresponding to each historical data block; Based on the described execution order, the execution dependencies between each historical data block are determined. Based on the historical data blocks and runtime dependencies of each application service, a service subgraph is generated for each application service. Generate a service dependency graph based on the service subgraphs of each application service.

5. The method according to claim 4, characterized in that, The step of generating a service dependency graph based on the service subgraphs of each application service includes: Obtain the service call order of each application service during runtime; and the service subgraph of each application service, and generate a service dependency graph.

6. The method according to claim 1, characterized in that, Determining the migration order of historical data blocks for each application service based on the service dependency graph includes: For any application service, based on the runtime dependencies of each historical data block of the application service in the service dependency graph and the preset data volume threshold of the data packet, the historical data blocks of the application service are sequentially packaged to obtain multiple data packets; Based on the service dependencies of each application service in the service dependency graph, the migration order of data packets for each application service is determined.

7. The method according to claim 3, characterized in that, After migrating the incremental data blocks and each of the historical data blocks of the application service to the target database, the process further includes: Based on the operation identifier of the data operation in the incremental data block, the corresponding operation object is determined in the historical data block, and the operation object is updated according to the operation content of the data operation.

8. A data migration device, characterized in that, include: The generation module is used to determine the migration time point of the source database and generate incremental data blocks based on the operation logs of the source database and the migration time point. The source database includes historical data blocks of multiple application services, and the operation of the application services is implemented through multiple historical data blocks. The incremental data blocks record the data changes of the source database after the migration time point. The acquisition module is used to acquire the service dependency graph of the source database, wherein the service dependency graph represents the service call relationship between the multiple application services and the association relationship between each historical data block of the application service; The sequence determination module is used to determine the migration order of each historical data block of each application service based on the service dependency graph. The migration module is used to migrate each historical data block and incremental data block to the target database according to the migration order.

9. An electronic device, characterized in that, The electronic device includes: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the data migration method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the data migration method of any one of claims 1-7.

Images