Data migration method and device based on adaptive heterogeneous environment and medium

Abstract

The invention discloses a data migration method and device based on a self-adaptive heterogeneous environment and a medium. The method comprises the steps that a project plan is obtained, an information set directory is determined according to the project plan, and a data list is determined according to the information set directory; performing data acquisition on the source database and the target database according to the data list to obtain a source field and a target field, and determining a mapping relationship between the source field and the target field; and performing data cleaning and standardization processing on the source data, and executing a migration task on the processed source data through a preset migration executor according to the mapping relation so as to write the source data into the target database. The method comprises the following steps: gradually determining an information set directory and a data list according to a project plan, and accurately positioning data; a field mapping relation is determined through collection; cleaning the standardized source data; finally, the migration executor is used for writing according to the mapping relation, the process is clear, and the accuracy and efficiency of data migration can be guaranteed.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of data processing, and particularly relates to a data migration method, device and medium based on an adaptive heterogeneous environment. BACKGROUND

[0002] At present, enterprise evaluation systems generally adopt a distributed heterogeneous database migration mode based on DataX. This mode provides a stable and efficient offline synchronization solution between heterogeneous data sources. However, this mode has the key problem of insufficient data migration and business adaptability.

[0003] Specifically, the overall data migration only focuses on the physical transfer of data and does not deeply consider the business-level solution, making it difficult to meet the diversified needs of different business scenarios. At the same time, the system lacks precise recognition ability of business semantics and cannot effectively carry out conversion operations between fields, resulting in that the data is difficult to be directly applied to the new business environment after migration. In addition, this mode cannot realize standard verification and cleaning processing of data, so that the data quality is uneven, and then the accuracy and reliability of the evaluation results are affected.

[0004] These problems not only reduce the efficiency of data migration, but also restrict the effective operation of the enterprise evaluation system. SUMMARY

[0005] In order to solve the above problems, the present application provides a data migration method based on an adaptive heterogeneous environment, comprising: obtaining a project plan, determining an information set directory according to the project plan, and determining a data list according to the information set directory; collecting data from a source database and a target database according to the data list to obtain source fields and target fields, and determining a mapping relationship between the source fields and the target fields; performing data cleaning and standardization processing on the source data, and executing a migration task on the processed source data according to the mapping relationship through a pre-set migration executor, so as to write the source data into the target database.

[0006] In one example, the data list is determined according to the information set directory, specifically comprising: generating a structured data list according to the information set directory, the migration list including a source table name, a target object, a field list, and a default condition; determining a pre-set filtering logic, the filtering logic including an organizational level, a personnel category, and a time window, so as to circumscribe a data range according to the filtering logic, thereby avoiding the migration of irrelevant historical data according to the data range.

[0007] In one example, the mapping relationship between the source field and the target field is determined, specifically including: connecting the interface between the source database and the target database, and extracting table structure information through the connected interface; performing field matching based on a pre-configured semantic dictionary, and supporting user manual review, correction or locking of the result of field matching; performing code value comparison on the result of field matching to determine a field mapping table, and using a fuzzy matching algorithm to identify the mapping table to obtain the mapping relationship.

[0008] In one example, the source data is subjected to data cleaning and standardization processing, specifically including: sequentially performing field type conversion, format forced verification, mandatory field integrity check, and value range verification on the source data to find out abnormal data; recording the abnormal data to an exception queue and recording error causes.

[0009] In one example, the processed source data is subjected to a migration task according to the mapping relationship through a pre-set migration executor, specifically including: the migration executor includes an API or a database connection pool, a write program is determined through the migration executor, the write program includes a batch submission strategy, concurrency control, and binary data processing; a write rate is determined according to the system load of the target database, so that the source data is written into the target database according to the write program and the write rate.

[0010] In one example, the method further includes: recording the task state of the migration task in real time through Redis, and monitoring the real-time progress of the migration task through the front end to determine whether the migration task is normally executed; if the task is interrupted or fails, a pre-set repair process is automatically triggered, and the task is resumed from the breakpoint; if the task is successful, the total number of records and the hash value of the key field are checked, and a task report is generated.

[0011] In one example, the method further includes: defining an information set directory according to the project plan, determining a migration range according to the information set directory, and generating a data list according to the information set directory; the data list supports filtering logic, data is circumscribed according to the filtering logic, and the filtering logic is dynamically adjusted through organization level, personnel category or time window parameter.

[0012] In one example, the method further includes: after the migration task is completed, a multi-dimensional data verification is performed on the source database and the target database through a pre-set verifier to determine the total number of records and sample comparison; a structured verification report is generated according to the total number of records and sample comparison, and the verification report is used for project acceptance and audit.

[0013] In another aspect, the present application also provides a data migration device based on adaptive heterogeneous environment, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the data migration device based on adaptive heterogeneous environment to perform the method as described in any one of the above examples.

[0014] In another aspect, the present application also provides a non-volatile computer storage medium storing computer executable instructions configured to perform the method as described in any one of the above examples.

[0015] The present application determines information set catalog according to project plan, generates data list containing multiple key information, and can also dynamically adjust data range by preset filtering logic according to parameters such as organizational level, accurately circumscribe data range, avoid irrelevant historical data migration, reduce manual grooming complexity, and improve migration accuracy. Table structure information is extracted through connection of source and target database interfaces, fields are automatically matched based on semantic dictionary, manual review and correction locking are supported, code value comparison is also performed, fuzzy matching algorithm is used to determine mapping relationship, automation and accuracy are taken into account, and manual configuration workload is reduced. Abnormal data is found out and recorded to abnormal queue and error reason through a variety of operations such as field type conversion and format verification on source data, and dirty data is intercepted from the source to ensure data quality. The migration executor supports multiple writing modes and has strategies such as batch submission, can dynamically adjust writing rate according to target database load, and ensures stable and efficient large-scale data migration. Task status is recorded in real time through Redis, progress can be monitored in real time on the front end, repair process is automatically triggered and breakpoint resume is performed when task is interrupted or fails, and report is verified and generated when task is successful. After migration is completed, multi-dimensional data verification is performed, structured verification report is generated for project acceptance and audit, and migration result is ensured to be reliable. BRIEF DESCRIPTION OF DRAWINGS

[0016] The accompanying drawings, which are included to provide a further understanding of the present application and constitute a part of this application, illustrate certain illustrative embodiments of the present application and together with the description serve to explain the present application. In the drawings: Figure 1 A flowchart of a data migration method based on adaptive heterogeneous environment in an embodiment of the present application; Figure 2 A schematic diagram of a data migration system based on adaptive heterogeneous environment in an embodiment of the present application; Figure 3 A schematic diagram of a data migration device based on adaptive heterogeneous environment in an embodiment of the present application. DETAILED DESCRIPTION

[0017] For the purpose, technical solutions and advantages of the present application, the following will be combined with the specific embodiments of the present application and the corresponding drawings to clearly and completely describe the technical solutions of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative labor are within the scope of protection of the present application.

[0018] The technical solutions provided by the embodiments of the present application are described in detail below in combination with the drawings.

[0019] As Figure 1 shown, in order to solve the above problems, the data migration method based on adaptive heterogeneous environment provided by the embodiments of the present application is applied in a data migration system based on adaptive heterogeneous environment, as Figure 2 shown, the system includes a business configuration and interaction layer, an intelligent conversion and verification engine layer, an execution scheduling and adaptation layer, a monitoring and operation guarantee layer. The business configuration and interaction layer includes a migration range configuration module and a mapping rule management module. The intelligent conversion and verification engine layer includes a metadata comparison and field mapping engine, a data standardization and cleaning module, and a consistency verifier. The execution scheduling and adaptation layer includes a migration executor and a plug-in data source adapter. The monitoring and operation guarantee layer includes an exception recovery and resource scheduling module. The system adopts hierarchical design, and each layer has clear division of labor and collaborative work, and completes data standard unification, data cleaning and migration work and verification from top to bottom.

[0020] The method comprises: S101, obtaining a project plan, determining an information set directory according to the project plan, and determining a data list according to the information set directory.

[0021] In one embodiment, the migration range configuration module allows the implementation personnel or business user to predefine the "information set directory" according to the project online plan, and then generate a structured data migration list. The list covers the source table name, target object, field list, and default conditions, such as "employee status = in service" or "date of employment ≥ 2020-01-01". Users can further adjust the filtering logic to accurately define the data range by organization level, personnel category, or time window, avoiding irrelevant historical data from being migrated. This module tightly binds the migration task with the business module, reducing the complexity of manually sorting out the data migration range and improving the accuracy and efficiency of the migration range definition. Compared with the traditional ETL tool that requires manual writing of SQL filtering statements, this module realizes data selection driven by business language, reducing the problem of data redundancy or loss caused by range misjudgment, laying a solid foundation for subsequent data mapping and verification work. In addition, this module supports multi-project parallel configuration, improving the reusability in the implementation process.

[0022] In one embodiment, the mapping rule management module can display the field correspondence automatically generated by the intelligent mapping engine, and support user manual review, correction or locking operation on the matching results. The module has built-in code value comparison management function, allowing users to maintain the mapping table of enumerated fields, such as the gender value "1 / 2" in the source system corresponding to "M / F" in the target system. All mapping rules are stored in a standardized JSON format, supporting version management and cross-project reuse, ensuring the auditability and traceability of the mapping logic. Through the structured rule storage method and version management mechanism, this module effectively solves the mapping error problem caused by inconsistent field naming and ambiguous semantics between different systems. Its rule centralized management mode significantly reduces the repeated configuration cost, especially suitable for the scene of unified cloud migration of group multi-subsidiaries, further improving the consistency of data migration and the standardization level of implementation.

[0023] S102, collecting data from the source database and the target database according to the data list to obtain source fields and target fields, and determining the mapping relationship between the source fields and the target fields.

[0024] In one embodiment, the metadata comparison and field mapping engine automatically extracts table structure information of both the source database and the target system through the metadata interface connecting the source database and the target system, and performs field matching work according to the pre-configured HR field business semantic dictionary. Fuzzy matching algorithms such as edit distance algorithm and synonym expansion technology are used to identify potential field correspondence, and secondary screening and filtering are performed in combination with the compatibility of field types, and finally high-confidence field mapping suggestions are output. Due to the deep integration of HR business context, the matching accuracy of this engine is significantly higher than that of general NLP models or simple regular matching schemes. With the help of this engine, non-technical personnel can also quickly complete the field alignment task across systems.

[0025] S103, data cleaning and standardization processing is performed on the source data, and a pre-set migration executor performs a migration task on the processed source data according to the mapping relationship, so as to write the source data into the target database.

[0026] In one embodiment, before the data is written into the target system, the data standardization and cleaning module will perform a series of standardization and cleaning operations. Specifically, the following aspects are covered: automatic execution of field type conversion, format forced verification, checking the integrity of mandatory fields, and verifying the field value range. Automatic execution of field type conversion, such as converting VARCHAR type to DATE type, format forced verification, such as mobile phone number must be 11 digits, and email must comply with RFC standard format. For records that do not meet the data standards of the target platform, the system will isolate them to the exception queue and record the error reasons in detail. In this process, the main process is not affected, and the effective data can continue to migrate. This module can intercept dirty data from the source, avoid API call failures or business function abnormalities caused by format errors, and truly realize migration compliance, effectively reducing the rework cost, especially suitable for old system migration scenarios with uneven data quality.

[0027] In one embodiment, after the migration task is completed, the consistency verifier performs multi-dimensional data consistency verification work. Specifically, the total number of records of the source system and the target system is compared first, and then sampling comparison is performed. In this way, data verification is changed from manual sampling to systematic operation, which can accurately identify content-level differences and greatly reduce the risk of data problems after the system goes online. In addition, the generated verification report can provide objective evidence for project acceptance, effectively accelerating the customer's signature confirmation process.

[0028] In one embodiment, the migration executor supports two data writing modes, one is a secure call through the official OpenAPI, and the other is a direct writing to the target database through a database connection pool. The executor is built-in with batch submission strategy, concurrency control mechanism and binary data processing capability, and can dynamically adjust the data writing rate according to the load of the target system. The API and DB dual-channel design adopted by it takes into account the security and performance of data migration. At the same time, the batch submission and flow control mechanism can effectively avoid memory overflow or lock table problems caused by a single transaction, ensuring that large-scale data migration work is stable and efficient.

[0029] In one embodiment, the plug-in data source adapter adopts a plug-in-based architecture design, and each adapter independently encapsulates the connection, query and paging logic for a specific system. When a new data source needs to be added, only the unified interface such as fetch_metadata(), read_batch() needs to be implemented to complete the adaptation work without modifying the core engine code. Facing the diversified old system environment of enterprises, there is no need to develop complete migration logic for each source system, and this design is the key technical foundation for realizing "one tool for multiple sources".

[0030] In one embodiment, the module focuses on ensuring the robustness of the migration task and optimizing resource use. It uses Redis to record the task status in real time, thereby supporting real-time monitoring of the migration progress by the front end. When network interruption or queue abnormalities cause task failure, the system will automatically trigger the repair process, such as calling fix_celery_error mechanism to try to recover the task and continue execution from the breakpoint, avoiding full re-run of the task. Through the combination of breakpoint resume mechanism and automatic triggering mechanism during business low peak period, the module ensures the reliability of data migration while significantly improving the overall implementation efficiency.

[0031] In one embodiment, the migration process is divided into migration preparation and mapping construction process, data execution and verification closed loop process. In the migration preparation and mapping construction process, first, the scope of the business module is circled according to the plan, and the HR information set directory to be migrated is confirmed to clearly define the migration boundary; then a structured data list is generated, which includes the source table name, field list and business filtering conditions such as "only active employees"; then the source and target metadata are collected, the source system database structure is scanned and the target system metadata interface is connected, and the standard model definitions of both parties are obtained; then the field mapping relationship is intelligently matched, based on the pre-set HR business semantic dictionary and code value rule library, the semantic level correspondence between the source field and the target field is automatically established; finally, manual review is performed and the mapping rule is locked, providing users with the opportunity to adjust the mapping result, and after confirmation, it is persisted as an auditable and reusable configuration file. In the data execution and verification closed loop process, first, data cleaning and standardization is performed, the source data is format checked, such as date format YYYY-MM-DD, type conversion and value range filtering are performed, and abnormal records are isolated; then API or DB batch writing is called, supporting batch submission through OpenAPI safe writing or database direct connection, and compatible with binary data such as photos; at the same time, Redis is used to store task status, real-time monitoring of task status progress, follow-up of migration progress and current batch information; if the task is interrupted abnormally, the failure point can be automatically detected and the execution can be continued from the breakpoint, realizing automatic breakpoint resume when interrupted, avoiding re-running; after migration is completed, the verification report is automatically compared and generated, the record total number and key field hash value are checked, and the structured report is generated for UAT acceptance and audit.

[0032] As shown in Figure 3 The embodiments of the present application also provide a data migration device based on an adaptive heterogeneous environment, comprising: at least one processor; and a memory in communication connection with the at least one processor; wherein The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable a data migration device based on an adaptive heterogeneous environment to perform the method described in any one of the above embodiments.

[0033] The embodiments of the present application also provide a non-volatile computer storage medium, which stores computer executable instructions, and the computer executable instructions are configured to perform the method described in any one of the above embodiments.

[0034] In the 1990s, it was possible to distinguish whether an improvement in a technology was a hardware improvement (e.g., an improvement in the circuit structure of a diode, transistor, switch, etc.) or a software improvement (an improvement in a method flow). However, as technology has advanced, many improvements in method flows today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain a corresponding hardware circuit structure by programming an improved method flow into a hardware circuit. Therefore, it cannot be said that an improvement in a method flow cannot be implemented using a hardware entity module. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it himself, without having to ask a chip manufacturer to design and manufacture a special integrated circuit chip. Moreover, instead of manually fabricating an integrated circuit chip, this programming is now mostly implemented using "logic compiler" software, which is similar to a software compiler used when developing a program, and the original code before compilation must also be written in a specific programming language, which is called a hardware description language (HDL), and there are many types of HDL, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should also be aware that it is very easy to obtain a hardware circuit that implements a logical method flow by simply logically programming the method flow in one of the above-mentioned hardware description languages and programming it into an integrated circuit.

[0035] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the microprocessor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to implementing the controller in pure computer readable program code, it is also possible to implement the controller in the form of logic gates, switches, application specific integrated circuits, programmable logic controllers and embedded microcontrollers, etc. to achieve the same function by logically programming the method steps. Therefore, such a controller can be considered as a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can even be considered as both a software module implementing a method and a structure within a hardware component.

[0036] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0037] For the sake of convenience, the above apparatuses are described in functional form with various units described separately for the sake of description. Of course, the functions of each unit can be implemented in the same or multiple software and / or hardware when implementing the present specification.

[0038] Each of the embodiments in the present application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. Each embodiment mainly describes the differences from other embodiments. In particular, the device and medium embodiments are described more simply because they are basically similar to the method embodiments, and the relevant parts can be referred to the part of the method embodiments.

[0039] The device and medium provided by the embodiments of the present application are one-to-one corresponding, and therefore the device and medium also have similar beneficial technical effects to the corresponding method. Since the beneficial technical effects of the method have been described in detail above, the beneficial technical effects of the device and medium will not be described here again.

[0040] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. In addition, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROMs, optical storage, etc.) containing computer-usable program code. The present application is described with reference to flowcharts and / or block diagrams of the method, device (system), and computer program product according to the embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device produce a device implemented in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0041] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including instruction apparatus, which implements the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0042] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide a process for implementing the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that performs the functions specified in one or more flows and / or blocks.

[0043] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memories.

[0044] Memory can include non-persistent memory, such as volatile random access memory (RAM) and / or non-volatile memory, such as read only memory (ROM), electrically erasable read only memory (EEPROM), flash memory, or other memory technologies, which can be either volatile or non-volatile. Memory is an example of computer readable media.

[0045] Computer readable media includes permanent and non-permanent, removable and non-removable media, which can be implemented by any method or technology for storage of information such as computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), electrically erasable programmable read only memory (EEPROM), flash memory or other memory technologies, compact disc read only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to computing devices. According to the definition herein, computer readable media does not include transitory media, such as modulated data signals and carrier waves.

[0046] It should also be noted that the terms "comprising", "containing", or any other variant thereof, are intended to encompass non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without further limitation, an element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0047] The above only illustrates the embodiments of the present application and is not intended to limit the present application. The present application can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. within the spirit and principle of the present application shall be included in the scope of claims of the present application.

Claims

1. A method for data migration based on adaptive heterogeneous environment, characterized in that, include: Obtain the project plan, determine the information set catalog based on the project plan, and determine the data list based on the information set catalog; Data is collected from the source database and the target database according to the data list to obtain source fields and target fields, and to determine the mapping relationship between the source fields and the target fields; The source data is cleaned and standardized. A pre-set migration executor performs a migration task on the processed source data according to the mapping relationship, so as to write the source data into the target database.

2. The method of claim 1, wherein, The data list is determined based on the information set catalog, specifically including: A structured data list is generated based on the information set catalog, and the migration list includes source table name, target object, field list, and default conditions; A pre-defined filtering logic is established, which includes organizational hierarchy, personnel category, and time window, to define the data range based on the filtering logic, thereby avoiding the migration of irrelevant historical data based on the data range.

3. The method of claim 1, wherein, Determining the mapping relationship between the source field and the target field specifically includes: Connect the interface between the source database and the target database, and extract table structure information through the connected interface; Field matching is performed based on a pre-configured semantic dictionary, and users can manually review, correct, or lock the field matching results. The code values ​​of the field matching results are compared to determine the field mapping table, and the fuzzy matching algorithm is used to identify the mapping table to obtain the mapping relationship.

4. The method of claim 1, wherein, The source data undergoes data cleaning and standardization processing, specifically including: The source data is sequentially subjected to field type conversion, format enforcement validation, required field integrity check, and value range validation to identify abnormal data. The abnormal data is recorded in the error queue, along with the reason for the error.

5. The method of claim 1, wherein, The migration executor, pre-configured according to the mapping relationship, performs a migration task on the processed source data, specifically including: The migration executor includes an API or database connection pool, and the write program is determined through the migration executor. The write program includes a batch commit strategy, concurrency control, and binary data processing. The write rate is determined based on the system load of the target database, and the source data is written to the target database according to the write procedure and the write rate.

6. The method of claim 1, wherein, The method further includes: The migration task status is recorded in real time by Redis, and the progress of the migration task is monitored in real time by the front end to determine whether the migration task is executed normally. If the task is interrupted or fails, a pre-set repair process will be automatically triggered, and the transmission will resume from the point of interruption. If the task is successful, the total number of records and the hash value of the key field are verified, and a task report is generated.

7. The method of claim 1, wherein, The method further includes: The project plan defines an information set catalog to determine the migration scope and generate a data list based on the information set catalog. The data list supports filtering logic to define data based on the filtering logic, and the filtering logic can be dynamically adjusted by organizational level, personnel category, or time window parameters.

8. The method of claim 1, wherein, The method further includes: After the migration task is completed, performing multi-dimensional data verification on the source database and the target database by a pre-set verifier to determine record total number and sample comparison; Generating a structured verification report according to the record total number and the sample comparison, so as to use the verification report for project acceptance and auditing.

9. A data migration device based on adaptive heterogeneous environment, characterized by, Comprise: at least one processor; and, a memory connected to the at least one processor in communication; wherein, the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the adaptive heterogeneous environment-based data migration device to perform the method of any one of claims 1-8.

10. A non-transitory computer storage medium storing computer-executable instructions that, when executed, cause a computer to perform: The computer executable instructions are configured to perform the method of any one of claims 1-8.

Images