Data migration method based on message-oriented middleware and related equipment

Abstract

The invention relates to the technical field of data processing, in particular to a data migration method based on message-oriented middleware and related equipment, and the method comprises the steps: extracting to-be-migrated data from a source database, and writing the to-be-migrated data into first-stage message-oriented middleware of a message-oriented middleware cluster; carrying out streaming computation processing on the to-be-migrated data and then transmitting the to-be-migrated data to the secondary message middleware; judging whether a data asynchronous landing requirement exists or not according to the next-level connection of the secondary message middleware; when the secondary message middleware exists, transmitting the data to be migrated to an intermediate database connected with the secondary message middleware, and the data falling into a target database in an ETL mode; and when the to-be-migrated data does not exist, performing streaming computation processing on the to-be-migrated data, transmitting the to-be-migrated data to subsequent cascaded message middleware, and the to-be-migrated data falling into a target database. According to the method, the middleware is constructed to serve as an intermediate bridge of data migration, so that the data falls from the source database to the target database, and the method has the advantages of high real-time performance, low time delay and high transmission rate.

Description

technical field [0001] The invention relates to the technical field of data processing, in particular to a message middleware-based data migration method and related equipment. Background technique [0002] Data migration is a typical method of data storage technology, which is used to transfer data stored on one storage system to another storage system. For example, based on the replacement requirements of the hardware of the data storage device, data is migrated from the source database where the old device is located to the target database where the new device is located. In general use, it is also commonly used to transfer data that is less frequently used or even not used from the source database to the target database, for example, using a high-speed, high-capacity off-line storage device as the next-level device of the disk device, Then, the commonly used data in the disk is automatically migrated to secondary mass storage devices such as tape libraries (referred to ...

AI Technical Summary

Problems solved by technology

[0005] 1) The cost of time consumption caused by the delay problem increases, and the timeliness of the data cannot be guaranteed. When the data in the source database changes during migration, the change may not correspond to the target database;

[0006] 2) It is impossible to achieve high-speed landing of data, that is, the speed of data migration is relatively slow

Images