Data transfer method applied to cross-database and related equipment

Abstract

The embodiment of the invention belongs to the technical field of data processing, and relates to a cross-database data transfer method and related device.The method comprises the steps that a data transfer request sent by a user terminal is received, and the data transfer request comprises a source database identifier and a target database identifier; obtaining source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; performing type comparison operation on the source database data and the target database data to obtain a type comparison result; constructing a data type mapping table according to a type comparison result; performing data conversion operation on the source database data according to the data type mapping table and a preset data conversion rule to obtain converted database data; and importing the converted database data into a target database corresponding to the target database identifier. The efficiency and accuracy of data migration can be effectively improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, is applicable to the financial field, and in particular, to a method for transferring data across databases and related equipment. Background Art

[0002] With the development of big data technology, data transfer is usually required to achieve business connectivity. Based on this, data transfer methods have emerged.

[0003] There is an existing data transfer method, which is to save the dynamic data to be inserted into a database file, and import it in batches through database backup or through a loading tool provided by the database.

[0004] However, the applicant has found that the conventional data transfer method generally has shortcomings, specifically, it requires additional disk space, and the input / output operation when saving files is relatively time-consuming. It can be seen that the conventional data transfer method has the problem of low data transfer efficiency. Summary of the invention

[0005] The purpose of the embodiments of the present application is to provide a data transfer method and related equipment applied to cross-databases, so as to solve the problem of low data transfer efficiency in traditional data transfer methods.

[0006] In order to solve the above technical problems, the present application embodiment provides a data transfer method applied to cross-databases, which adopts the following technical solutions:

[0007] Receiving a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier;

[0008] Acquire source database data corresponding to the source database identifier and target database data corresponding to the target database identifier;

[0009] Performing a type comparison operation on the source database data and the target database data to obtain a type comparison result;

[0010] Constructing a data type mapping table according to the type comparison result;

[0011] Performing data conversion operation on the source database data according to the data type mapping table and preset data conversion rules to obtain converted database data;

[0012] The converted database data is imported into a target database corresponding to the target database identifier.

[0013] Furthermore, the step of importing the converted database data into a target database corresponding to the target database identifier specifically includes the following steps:

[0014] Processing the converted database data in batches to obtain batch database data;

[0015] The batch database data is imported into the target database according to multi-threading technology.

[0016] Furthermore, the step of importing the batch database data into the target database according to the multi-threading technology specifically includes the following steps:

[0017] Calculate the hash values ​​of the batch database data and the source database data imported in each batch respectively, and confirm whether the batch database data imported in the current batch and the source database data are consistent according to the hash values;

[0018] If the batch database data imported in the current batch is consistent with the source database data, marking the batch database data imported in the current batch as normal data;

[0019] If the batch database data imported in the current batch is inconsistent with the source database data, the batch database data imported in the current batch is marked as abnormal data, and the abnormal data is re-imported.

[0020] Furthermore, the step of importing the converted database data into a target database corresponding to the target database identifier specifically includes the following steps:

[0021] The converted database data is imported into a target database corresponding to the target database identifier according to an incremental import method.

[0022] Furthermore, the step of importing the converted database data into a target database corresponding to the target database identifier specifically includes the following steps:

[0023] During the data import process, when a fault event occurs, the breakpoint information of the currently imported conversion database data is recorded;

[0024] The currently imported conversion database data is restored and imported from the breakpoint information according to the fault tolerance mechanism and the retry strategy.

[0025] Furthermore, the step of importing the converted database data into a target database corresponding to the target database identifier specifically includes the following steps:

[0026] Confirm whether the current converted database data to be imported is critical business data;

[0027] If the currently converted database data is key business data, then after the step of importing the converted database data into the target database corresponding to the target database identifier, the following step is also included:

[0028] A key data check operation is performed on the imported current conversion database data according to a data verification mechanism.

[0029] In order to solve the above technical problems, the embodiment of the present application further provides a data transfer device applied to cross-databases, which adopts the following technical solution:

[0030] A request acquisition module, configured to receive a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier;

[0031] A data acquisition module, used to acquire source database data corresponding to the source database identifier and target database data corresponding to the target database identifier;

[0032] A type comparison module, used to perform a type comparison operation on the source database data and the target database data to obtain a type comparison result;

[0033] A mapping table construction module, used to construct a data type mapping table according to the type comparison result;

[0034] A data conversion module, used to perform data conversion operations on the source database data according to the data type mapping table and preset data conversion rules to obtain converted database data;

[0035] A data import module is used to import the converted database data into a target database corresponding to the target database identifier.

[0036] Furthermore, the data import module includes:

[0037] A batch processing submodule, used for processing the converted database data in batches to obtain batch database data;

[0038] The data import submodule is used to import the batch database data into the target database according to the multi-threading technology.

[0039] In order to solve the above technical problems, the embodiment of the present application further provides a computer device, which adopts the following technical solution:

[0040] The system comprises a memory and a processor, wherein the memory stores computer-readable instructions, and the processor implements the steps of the above-mentioned method for data transfer across databases when executing the computer-readable instructions.

[0041] In order to solve the above technical problems, the embodiment of the present application further provides a computer-readable storage medium, which adopts the following technical solution:

[0042] The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the above-mentioned method for data transfer across databases are implemented.

[0043] The present application provides a data transfer method applied to cross-databases, including: receiving a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier; obtaining source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; performing a type comparison operation on the source database data and the target database data to obtain a type comparison result; constructing a data type mapping table according to the type comparison result; performing a data conversion operation on the source database data according to the data type mapping table and a preset data conversion rule to obtain conversion database data; and importing the conversion database data into a target database corresponding to the target database identifier. Compared with the prior art, the present application forms a complete and efficient data migration process through clear request reception, accurate data acquisition, detailed type comparison, reasonable mapping table construction, accurate data conversion and efficient import operation. Each link is closely connected and supports each other, effectively improving the efficiency and accuracy of data migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the scheme in the present application, a brief introduction is given below to the drawings required for use in the description of the embodiments of the present application. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0045] Figure 1 is an exemplary system architecture diagram to which the present application may be applied;

[0046] Figure 2 is a flowchart of an implementation of a data transfer method applied to a cross-database provided in an embodiment of the present application;

[0047] Figure 3 is a schematic diagram of the structure of a data transfer device applied to a cross-database provided in an embodiment of the present application;

[0048] Figure 4 It is a structural diagram of an embodiment of a computer device according to the present application. DETAILED DESCRIPTION

[0049] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by technicians in the technical field of the present application; the terms used in the specification of the application herein are only for the purpose of describing specific embodiments and are not intended to limit the present application; the terms "including" and "having" and any variations thereof in the specification and claims of the present application and the above-mentioned drawings are intended to cover non-exclusive inclusions. The terms "first", "second", etc. in the specification and claims of the present application or the above-mentioned drawings are used to distinguish different objects, not to describe a specific order.

[0050] Reference to "embodiments" herein means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described herein may be combined with other embodiments.

[0051] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the accompanying drawings.

[0052] like Figure 1 As shown, the system architecture 100 may include a terminal device 101, a network 102 and a server 103. The terminal device 101 may be a laptop 1011, a tablet computer 1012 or a mobile phone 1013. The network 102 is used to provide a medium for a communication link between the terminal device 101 and the server 103. The network 102 may include various connection types, such as wired, wireless communication links or optical fiber cables, etc.

[0053] The user can use the terminal device 101 to interact with the server 103 through the network 102 to receive or send messages, etc. Various communication client applications can be installed on the terminal device 101, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0054] The terminal device 101 can be any electronic device with a display screen and supporting web browsing. In addition to the laptop computer 1011, tablet computer 1012 or mobile phone 1013, the terminal device 101 can also be an e-book reader, an MP3 player (Moving Picture Experts Group Audio Layer III), an MP4 (Moving Picture Experts Group Audio Layer IV), a laptop computer, a desktop computer, etc.

[0055] The server 103 may be a server that provides various services, such as a background server that provides support for a web page displayed on the terminal device 101 .

[0056] It should be noted that the cross-database data transfer method provided in the embodiment of the present application is generally executed by a server / terminal device, and accordingly, the cross-database data transfer device is generally set in the server / terminal device.

[0057] It should be understood that Figure 1 The number of terminal devices, networks and servers in the embodiment is only for illustration. Any number of terminal devices, networks and servers may be provided according to implementation requirements.

[0058] Continue to refer Figure 2 , shows a flow chart of an embodiment of a data transfer method applied to a cross-database according to the present application. The data transfer method applied to a cross-database includes: step S201, step S202, step S203, step S204, step S205 and step S206.

[0059] In step S201, a data transfer request sent by a user terminal is received, wherein the data transfer request includes a source database identifier and a target database identifier.

[0060] In an embodiment of the present application, a request for data migration can be submitted through a terminal, and this request is initiated through a user interface or an API interface, wherein the user terminal refers to a terminal device used to execute the image processing method for preventing document abuse provided by the present application, and the user terminal can be a mobile terminal such as a mobile phone, a smart phone, a laptop computer, a digital broadcast receiver, a PDA (personal digital assistant), a PAD (tablet computer), a PMP (portable multimedia player), a navigation device, etc., as well as a fixed terminal such as a digital TV, a desktop computer, etc. It should be understood that the examples of user terminals here are only for the convenience of understanding and are not used to limit the present application.

[0061] In the embodiment of the present application, the request includes two key information: a source database identifier and a target database identifier. The source database identifier is used to identify the source of the original data, while the target database identifier indicates the destination of the data migration.

[0062] In step S202, source database data corresponding to the source database identifier and target database data corresponding to the target database identifier are acquired.

[0063] In the embodiment of the present application, the source database data is the specific content that the user wants the system to transfer data. Specifically, the source database data can be "transaction data or payment data or business data or purchase data" related to a financial institution (such as a bank, etc.).

[0064] In the embodiment of the present application, the system connects to the corresponding source database according to the source database identifier and extracts data therefrom.

[0065] In the embodiment of the present application, the system may also obtain some metadata or structural information of the target database, which is helpful for the subsequent data comparison and conversion steps. However, this step may not directly involve obtaining data in the target database, because data migration is usually a one-way flow from the source database to the target database.

[0066] In step S203, a type comparison operation is performed on the source database data and the target database data to obtain a type comparison result.

[0067] In the embodiment of the present application, in order to ensure that data will not cause errors or data loss due to type mismatch during the data migration process, the system compares the data types of the fields in the source database and the target database.

[0068] In the embodiment of the present application, the type comparison result will indicate which fields require special attention because their types are different between the two databases.

[0069] In step S204, a data type mapping table is constructed according to the type comparison result.

[0070] In the embodiment of the present application, based on the result of the type comparison, the system will create a data type mapping table. This table will list each field in the source database and the data type they should correspond to in the target database.

[0071] In the embodiment of the present application, the mapping table is an important reference for the data conversion step because it guides how to correctly convert the data in the source database into a format that can be understood and accepted by the target database.

[0072] In step S205, a data conversion operation is performed on the source database data according to the data type mapping table and preset data conversion rules to obtain converted database data.

[0073] In the embodiment of the present application, the system converts the data in the source database according to the data type mapping table and the preset data conversion rules. These rules may include data format adjustment, data cleaning, data conversion, etc.

[0074] In the embodiment of the present application, the converted data is called converted database data, which is ready to be imported into the target database.

[0075] In step S206, the converted database data is imported into a target database corresponding to the target database identifier.

[0076] In the embodiment of the present application, the last step is to import the converted data into the target database, which involves writing the data into the corresponding tables and fields of the target database.

[0077] In the embodiment of the present application, the import process may also need to consider factors such as transaction processing, data consistency and error handling to ensure the integrity and reliability of data migration.

[0078] In some optional implementations of the embodiments of the present application, the SQL statements for data import can be optimized, and the SQL statements can be precompiled to reduce the overhead of SQL parsing and compilation. At the same time, according to the characteristics of the target database, the insert statements are optimized, such as using multi-value insert, batch insert, etc., to reduce the number of interactions with the database and improve the insertion efficiency.

[0079] In an embodiment of the present application, a method for data transfer applied to cross-databases is provided, including: receiving a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier; obtaining source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; performing a type comparison operation on the source database data and the target database data to obtain a type comparison result; constructing a data type mapping table according to the type comparison result; performing a data conversion operation on the source database data according to the data type mapping table and a preset data conversion rule to obtain conversion database data; and importing the conversion database data into the target database corresponding to the target database identifier. Compared with the prior art, the present application forms a complete and efficient data migration process through clear request reception, accurate data acquisition, detailed type comparison, reasonable mapping table construction, accurate data conversion and efficient import operation. Each link is closely connected and supports each other, effectively improving the efficiency and accuracy of data migration.

[0080] In some optional implementations of the embodiments of the present application, the step of importing the converted database data into the target database corresponding to the target database identifier specifically includes the following steps:

[0081] Processing the converted database data in batches to obtain batch database data;

[0082] Import database data in batches into the target database based on multi-threading technology.

[0083] In the embodiment of the present application, since a large-scale data set may contain millions or even billions of records, direct processing may cause performance problems or memory overflow. Therefore, dividing the converted data into smaller batches for processing is an effective solution. Batch processing can be based on the amount of data (such as 1,000 records per batch) or time intervals (such as one batch per hour).

[0084] In the embodiment of the present application, multithreading technology allows multiple threads to run simultaneously, and each thread can execute tasks independently. In the database data import scenario, multithreading technology can significantly improve the speed and efficiency of data import.

[0085] In the embodiment of the present application, batch processing of the converted database data and importing the batched data into the target database according to the multi-threading technology is a complex and important process. Through reasonable batch processing strategies, multi-threaded import technology and performance optimization measures, the speed and efficiency of data import can be significantly improved while ensuring the integrity and consistency of the data.

[0086] In some optional implementations of the embodiments of the present application, the step of importing the batch database data into the target database according to the multi-threading technology specifically includes the following steps:

[0087] Calculate the hash values ​​of the batch database data and the source database data imported in each batch respectively, and confirm whether the batch database data imported in the current batch and the source database data are consistent according to the hash values;

[0088] If the batch database data imported in the current batch is consistent with the source database data, the batch database data imported in the current batch is marked as normal data;

[0089] If the batch database data imported in the current batch is inconsistent with the source database data, the batch database data imported in the current batch is marked as abnormal data, and the abnormal data is re-imported.

[0090] In the embodiment of the present application, it is crucial to ensure the integrity and accuracy of data during database migration or data synchronization. To achieve this goal, the consistency of the data can be verified by calculating the hash value of the data.

[0091] In the embodiment of the present application, the hash function is an algorithm that maps data of any length to a hash value of a fixed length, and the hash function includes MD5, SHA-1, SHA-256, etc.

[0092] In the embodiment of the present application, for each batch of imported batch database data, a selected hash function is used to calculate its hash value. At the same time, a hash value calculation is also performed on the corresponding data in the source database. The calculation of the hash value can be based on the overall content of the data, or based on a specific field or field combination of the data.

[0093] In the embodiment of the present application, the calculated hash value of the batch database data is compared with the hash value of the source database data. If the two hash values ​​are exactly the same, it can be considered that the batch database data imported in the current batch is consistent with the source database data in content.

[0094] In the embodiment of the present application, calculating the hash value and comparing its consistency to verify the accuracy of the data during the migration or synchronization process is an effective method. This method can ensure the integrity and accuracy of the data, while providing a mechanism for processing and re-importing abnormal data.

[0095] In some optional implementations of the embodiments of the present application, the step of importing the converted database data into the target database corresponding to the target database identifier specifically includes the following steps:

[0096] The converted database data is imported into a target database corresponding to the target database identifier according to the incremental import method.

[0097] In the embodiment of the present application, during the import process, an incremental import strategy is adopted to identify newly added, modified, and deleted data by comparing the timestamps or version numbers of the source data and the target data, and only import the changed data to avoid the overhead of full import. Incremental import can significantly reduce the amount of imported data and increase the import speed.

[0098] In some optional implementations of the embodiments of the present application, the step of importing the converted database data into the target database corresponding to the target database identifier specifically includes the following steps:

[0099] During the data import process, when a fault event occurs, the breakpoint information of the currently imported conversion database data is recorded;

[0100] According to the fault tolerance mechanism and retry strategy, the currently imported conversion database data is restored and imported from the breakpoint information.

[0101] In the embodiments of the present application, a fault-tolerant mechanism and a retry strategy are designed for abnormal situations during the import process, such as network interruption, database failure, etc. By recording the import progress and breakpoint information, the import can be resumed from the breakpoint when an abnormality occurs, avoiding data loss and repeated import. At the same time, the data that failed to be imported is recorded and alarmed to facilitate subsequent manual processing.

[0102] In some optional implementations of the embodiments of the present application, the step of importing the converted database data into the target database corresponding to the target database identifier specifically includes the following steps:

[0103] Confirm whether the current converted database data to be imported is critical business data;

[0104] If the current converted database data is key business data, after the step of importing the converted database data into the target database corresponding to the target database identifier, the following steps are also included:

[0105] According to the data verification mechanism, key data checking operations are performed on the current converted database data after import.

[0106] In an embodiment of the present application, data verification is introduced for the import of key business data. After the import is completed, a sample check is performed on the key data, and the correctness of the imported data is verified by comparing it with the source data.

[0107] In some optional implementations of the embodiments of the present application, an audit mechanism may also be introduced to audit the import process and record information such as the time, operator, and data volume of the import operation to ensure the traceability and accountability of the import process.

[0108] The embodiments of the present application can acquire and process relevant data based on artificial intelligence technology. Among them, artificial intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results.

[0109] AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, robotics technology, biometrics technology, speech processing technology, natural language processing technology, and machine learning / deep learning.

[0110] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through computer-readable instructions, and the computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, the aforementioned storage medium can be a non-volatile storage medium such as a disk, an optical disk, a read-only memory (ROM), or a random access memory (RAM).

[0111] It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

[0112] Further references Figure 3 , as a response to the above Figure 2 The present application provides an embodiment of a data transfer device for use across databases. Figure 2 Corresponding to the method embodiment shown, the device can be specifically applied to various electronic devices.

[0113] like Figure 3 As shown, the data migration device 200 applied to a cross-database in the embodiment of the present application includes:

[0114] The request acquisition module 210 is used to receive a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier;

[0115] The data acquisition module 220 is used to acquire source database data corresponding to the source database identifier and target database data corresponding to the target database identifier;

[0116] The type comparison module 230 is used to perform a type comparison operation on the source database data and the target database data to obtain a type comparison result;

[0117] A mapping table construction module 240 is used to construct a data type mapping table according to the type comparison result;

[0118] The data conversion module 250 is used to perform data conversion operations on the source database data according to the data type mapping table and the preset data conversion rules to obtain converted database data;

[0119] The data import module 260 is used to import the converted database data into a target database corresponding to the target database identifier.

[0120] In an embodiment of the present application, a data transfer device 200 for cross-database is provided, including: a request acquisition module 210, for receiving a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier; a data acquisition module 220, for acquiring source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; a type comparison module 230, for performing a type comparison operation on the source database data and the target database data to obtain a type comparison result; a mapping table construction module 240, for constructing a data type mapping table according to the type comparison result; a data conversion module 250, for performing a data conversion operation on the source database data according to the data type mapping table and a preset data conversion rule to obtain converted database data; a data import module 260, for importing the converted database data into the target database corresponding to the target database identifier. Compared with the prior art, the present application forms a complete and efficient data migration process through clear request reception, accurate data acquisition, detailed type comparison, reasonable mapping table construction, accurate data conversion and efficient import operation. Each link is closely connected and supports each other, effectively improving the efficiency and accuracy of data migration.

[0121] In some optional implementations of the embodiments of the present application, the data import module includes:

[0122] A batch processing submodule is used to process the converted database data in batches to obtain batch database data;

[0123] The data import submodule is used to import batches of database data into the target database based on multi-threading technology.

[0124] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of a computer device according to an embodiment of the present application.

[0125] The computer device 300 includes a memory 310, a processor 320, and a network interface 330 that are interconnected and communicated through a system bus. It should be noted that the figure only shows a computer device 300 having components 310-330, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0126] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0127] The memory 310 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 310 can be an internal storage unit of the computer device 300, such as a hard disk or memory of the computer device 300. In other embodiments, the memory 310 can also be an external storage device of the computer device 300, such as a plug-in hard disk equipped on the computer device 300, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card, etc. Of course, the memory 310 can also include both the internal storage unit of the computer device 300 and its external storage device. In the embodiment of the present application, the memory 310 is generally used to store the operating system and various application software installed on the computer device 300, such as computer-readable instructions for a cross-database data transfer method, etc. In addition, the memory 310 can also be used to temporarily store various types of data that have been output or are to be output.

[0128] The processor 320 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 320 is generally used to control the overall operation of the computer device 300. In the embodiment of the present application, the processor 320 is used to run the computer-readable instructions stored in the memory 310 or process data, such as running the computer-readable instructions applied to the cross-database data transfer method.

[0129] The network interface 330 may include a wireless network interface or a wired network interface. The network interface 330 is generally used to establish a communication connection between the computer device 300 and other electronic devices.

[0130] The computer equipment provided by this application forms a complete and efficient data migration process through clear request reception, accurate data acquisition, detailed type comparison, reasonable mapping table construction, accurate data conversion and efficient import operation. Each link is closely connected and supports each other, effectively improving the efficiency and accuracy of data migration.

[0131] The present application also provides another embodiment, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores computer-readable instructions, and the computer-readable instructions can be executed by at least one processor to enable the at least one processor to perform the steps of the data transfer method applied to cross-databases as described above.

[0132] The computer-readable storage medium provided by the present application forms a complete and efficient data migration process through clear request reception, accurate data acquisition, detailed type comparison, reasonable mapping table construction, accurate data conversion and efficient import operation. Each link is closely connected and supports each other, effectively improving the efficiency and accuracy of data migration.

[0133] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0134] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A data transfer method applied to cross-databases, characterized in that: The steps include: Receiving a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier; Acquire source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; Performing a type comparison operation on the source database data and the target database data to obtain a type comparison result; Constructing a data type mapping table according to the type comparison result; Performing data conversion operation on the source database data according to the data type mapping table and preset data conversion rules to obtain converted database data; The converted database data is imported into a target database corresponding to the target database identifier.

2. The method for data transfer across databases according to claim 1, characterized in that: The step of importing the converted database data into a target database corresponding to the target database identifier specifically comprises the following steps: Processing the converted database data in batches to obtain batch database data; The batch database data is imported into the target database according to multi-threading technology.

3. The method for data transfer across databases according to claim 2, characterized in that: The step of importing the batch database data into the target database according to the multi-threading technology specifically includes the following steps: Calculate the hash values ​​of the batch database data and the source database data imported in each batch respectively, and confirm whether the batch database data imported in the current batch and the source database data are consistent according to the hash values; If the batch database data imported in the current batch is consistent with the source database data, marking the batch database data imported in the current batch as normal data; If the batch database data imported in the current batch is inconsistent with the source database data, the batch database data imported in the current batch is marked as abnormal data, and the abnormal data is re-imported.

4. The method for data transfer across databases according to claim 1, characterized in that: The step of importing the converted database data into a target database corresponding to the target database identifier specifically comprises the following steps: The converted database data is imported into a target database corresponding to the target database identifier according to an incremental import method.

5. The method for data transfer across databases according to claim 1, characterized in that: The step of importing the converted database data into a target database corresponding to the target database identifier specifically comprises the following steps: During the data import process, when a fault event occurs, the breakpoint information of the currently imported conversion database data is recorded; The currently imported conversion database data is restored and imported from the breakpoint information according to the fault tolerance mechanism and the retry strategy.

6. The method for data transfer across databases according to claim 1, characterized in that: The step of importing the converted database data into a target database corresponding to the target database identifier specifically comprises the following steps: Confirm whether the current converted database data to be imported is critical business data; If the currently converted database data is key business data, then after the step of importing the converted database data into the target database corresponding to the target database identifier, the following step is also included: A key data check operation is performed on the imported current conversion database data according to a data verification mechanism.

7. A data transfer device applied to cross-databases, characterized in that: include: A request acquisition module, configured to receive a data transfer request sent by a user terminal, wherein the data transfer request includes a source database identifier and a target database identifier; A data acquisition module, used to acquire source database data corresponding to the source database identifier and target database data corresponding to the target database identifier; A type comparison module, used to perform a type comparison operation on the source database data and the target database data to obtain a type comparison result; A mapping table construction module, used to construct a data type mapping table according to the type comparison result; A data conversion module, used to perform data conversion operations on the source database data according to the data type mapping table and preset data conversion rules to obtain converted database data; A data import module is used to import the converted database data into a target database corresponding to the target database identifier.

8. The device for data transfer across databases according to claim 7, characterized in that: The data import module includes: A batch processing submodule, used for processing the converted database data in batches to obtain batch database data; The data import submodule is used to import the batch database data into the target database according to the multi-threading technology.

9. A computer device comprising a memory and a processor, characterized in that: The memory stores computer-readable instructions, and when the processor executes the computer-readable instructions, the steps of the method for data transfer across databases according to any one of claims 1 to 6 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-readable instructions, and when the computer-readable instructions are executed by a processor, the steps of the method for data transfer across databases according to any one of claims 1 to 6 are implemented.

Images