Method for migrating DB2 / 400 data to heterogeneous database

Abstract

The invention relates to a DB2 / 400 data migration heterogeneous database method and system, and the method comprises the steps: obtaining first real-time migration data, compressing the first real-time migration data part, and generating second real-time migration data; connecting the AS / 400 end with an open platform end; performing decompression and field data type analysis on second real-time migration data based on a Head part in the first real-time migration data to obtain field type information; performing data splitting on the second real-time migration data according to the field type information to obtain split data partitions; and constructing a pre-transcoding rule and calling the open platform end to transcode the split data partitions to obtain third real-time migration data, and migrating the third real-time migration data to the heterogeneous database. The technical problem that in the prior art, applicability and migration timeliness are poor is solved.

Description

Technical Field

[0001] The present invention relates to the technical field related to data migration processing, and in particular to a DB2 / 400 data migration heterogeneous database method. Background Art

[0002] The database of IBM AS / 400 mainframe is DB2 / 400. Different from DB2 UDB used in open platform environment, DB2 / 400 used in simplified Chinese environment natively adopts EBCDIC CCSID 935 or 1388 to support Chinese (used by O-type field of physical file PF), CCSID 836 or 37 to support English half-width characters (used by A-type field of PF), CCSID (coded characterset identifiers) is similar to Windows code page; for numeric fields, there are P type (compressed decimal) and S type (uncompressed decimal); other common field type information includes Z (timestamp) type, B (binary) type, etc. In data management, in order to improve the efficiency of data sharing and exchange, or to conduct project testing, it is necessary to migrate the data in the database.

[0003] There are currently two main ways to migrate DB2 / 400 database data. One is to export and download database files through plug-ins and then upload them to an open platform heterogeneous database. The other is to use migration tools from third-party products.

[0004] However, the inventors of this application found that the above technology has at least the following technical problems in the process of implementing the technical solution of the invention in the embodiment of this application:

[0005] The prior art has technical problems such as poor applicability and migration timeliness due to functional limitations such as mismatch of special field storage types and export files occupying hard disk space. Summary of the invention

[0006] The embodiment of the present application provides a DB2 / 400 data migration heterogeneous database method, which solves the technical problems in the prior art that the applicability and migration timeliness are poor due to functional limitations such as special field storage type mismatch and export files occupying hard disk space. By acquiring the data to be migrated in real time, when the data volume of the data to be migrated reaches a preset value, the data to be migrated is compressed, and the data field type information of the data to be migrated is further analyzed. The corresponding data is processed according to the split result matching the predetermined transcoding rules, which solves the problem of field type information mismatch, compresses the data to be migrated, and solves the problem of migration timeliness, achieving the technical effect of matching the storage data field type information and good migration timeliness.

[0007] To achieve the above objectives, the technical solutions adopted by the present invention include:

[0008] A method for DB2 / 400 data migration of heterogeneous databases, comprising:

[0009] Acquire first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records;

[0010] Compressing the Body part of the first real-time migration data to generate second real-time migration data;

[0011] Connecting the AS / 400 terminal to the open platform terminal;

[0012] decompressing the second live migration data and analyzing field data types based on the Head part in the first live migration data to obtain field type information;

[0013] Splitting the second live migration data according to the field type information, wherein a Head part is sent as a first block of data to be transmitted, and a Body part is split into multiple data blocks for transmission, to obtain split data partitions;

[0014] The open platform end constructs a pre-transcoding rule based on the default transcoding rule of the Head part field type;

[0015] Calling the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data;

[0016] The third real-time migration data is migrated to the heterogeneous database through the open platform.

[0017] Further, the pre-transcoding rules include garbled code replacement rules;

[0018] By performing field transcoding on the second real-time migration data, a garbled field set is obtained, wherein the garbled field set is a field that cannot be converted according to a transcoding rule during field transcoding;

[0019] Performing character replacement on the garbled code field set based on the garbled code replacement rule to obtain a replacement field set;

[0020] A record log is generated according to the replacement field set.

[0021] Furthermore, before obtaining the first real-time migration data, the method further includes:

[0022] Setting a preset quantization coefficient, wherein the preset quantization coefficient is a data compression critical value;

[0023] Obtaining a quantization coefficient by performing data quantization analysis on the first real-time migration data;

[0024] When the quantization coefficient is greater than a preset quantization coefficient, a compression instruction is obtained, and the compression instruction is used for compressing the real-time migration data.

[0025] Furthermore, the second real-time migration data is obtained by connecting to a client on the AS / 400 side, and can support multiple client connections on the AS / 400 side in a multi-process manner, allowing multiple PF table data on the AS / 400 side to be migrated simultaneously.

[0026] Furthermore, before migrating the third real-time migration data to the heterogeneous database through the open platform, the method further includes:

[0027] Performing CRC32 verification on the third real-time migration data to obtain a verification result;

[0028] If the verification result is that the verification fails, a return instruction is obtained;

[0029] If the verification result is that the verification passes, a migration instruction is obtained;

[0030] The third real-time migration data is processed according to the return instruction and the migration instruction.

[0031] Furthermore, the record log can locate the location of the problematic data;

[0032] The problem data location includes the relative record number RRN, the field name, and the offset of the error in the field.

[0033] Furthermore, the method further comprises:

[0034] According to the compression instruction, obtaining the compression algorithm of the AS / 400 end;

[0035] According to the compression algorithm, a decompression algorithm is obtained, wherein the compression algorithm corresponds to the decompression algorithm;

[0036] The decompression algorithm is used as the decompression algorithm of the open platform end to perform decompression processing.

[0037] The present invention also relates to a system for DB2 / 400 data migration of heterogeneous databases, comprising:

[0038] An acquisition module is used to acquire first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records;

[0039] A first generating module, configured to compress a Body portion of the first real-time migration data to generate second real-time migration data;

[0040] A connection module, used for connecting the AS / 400 terminal to the open platform terminal;

[0041] a splitting module, configured to decompress the second real-time migration data and analyze the field data type based on the Head part of the first real-time migration data to obtain field type information; and split the second real-time migration data according to the field type information, wherein the Head part is sent as the first block of data to be transmitted, and the Body part is split into multiple data blocks for transmission to obtain split data partitions;

[0042] A rule building module, wherein the open platform end builds a pre-transcoding rule based on the default transcoding rule of some field types of Head;

[0043] A second generating module is used to call the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data;

[0044] A migration module is used to migrate the third real-time migration data to the heterogeneous database through the open platform end.

[0045] The present invention also relates to a computer-readable storage medium, on which a computer program is stored, and the computer program implements the above method when executed by a processor.

[0046] The present invention also relates to an electronic device, comprising a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor is used to execute the computer program stored in the memory to execute the above method.

[0047] The beneficial effects of the present invention are:

[0048] The present invention provides a method for DB2 / 400 data migration in heterogeneous databases, which solves the technical problems in the prior art that the applicability and migration timeliness are poor due to functional limitations such as mismatch of special field storage types and export files occupying hard disk space. By acquiring the data to be migrated in real time, when the quantization coefficient of the data to be migrated reaches a preset value, the Body part of the data to be migrated is compressed, and the data field type information of the data to be migrated is further analyzed. The corresponding data is processed according to the split result matching the predetermined transcoding rules, which solves the problem of mismatch of field type information, compresses the data to be migrated, and solves the problem of migration timeliness, achieving the technical effect of matching the field type information of the stored data and good migration timeliness; and no error is reported when encountering garbled characters, and logs can be recorded to locate the problem data; no additional hard disk space is occupied, and the network transmission efficiency is improved by compressing the data and adjusting the socket parameters according to the BDP, thereby reducing the transmission time. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 The present invention is a flow chart of a method for DB2 / 400 data migration to heterogeneous databases.

[0050] Figure 2 The present invention is a schematic diagram of application command execution in a DB2 / 400 data migration heterogeneous database.

[0051] Figure 3 The present invention is a schematic diagram of the system structure of DB2 / 400 data migration heterogeneous database. DETAILED DESCRIPTION

[0052] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0053] The first aspect of the present invention relates to a process of steps as follows Figure 1 The method for DB2 / 400 data migration to heterogeneous databases shown includes:

[0054] S000: Setting a preset quantization coefficient, wherein the preset quantization coefficient is a data compression critical value;

[0055] Obtaining a quantization coefficient by performing data quantization analysis on the first real-time migration data;

[0056] When the quantization coefficient is greater than a preset quantization coefficient, a compression instruction is obtained, wherein the compression instruction is used for compressing the real-time migration data;

[0057] S100: Obtaining first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records;

[0058] Specifically, the heterogeneous database described in the embodiment of the present application refers to a database that receives migration data due to work requirements, and examples include: MySQL, Oracle, MS SQL Server and other types of databases; the first real-time migration data is the metadata of the AS / 400-side PF that needs to be migrated to the heterogeneous database. The data to be migrated is read on the AS / 400 side. During the data reading process, avoid using configuration files / tables to register the metadata of the DB2 / 400 database table, which is easy to cause the problem of mismatch after modification. There is no restriction on other download tools. An exemplary download method is: obtain the metadata of the PF to be transmitted in real time through APIQUSLFLD(), and select "FLDL0100" as the format name; read any specified PF through record-level file reading APIs such as _Ropen() and _Rreadd(); specify the transmission PF and related options according to the command line parameters, and calculate the bandwidth-delay product (Bandwidth-Delay Product, BDP). The command line specifies the BDP parameters, and the program adjusts the SO_RCVBUF parameter of the TCP window accordingly to make full use of the bandwidth.

[0059] Further, exemplary: Linux is a free and open source UNIX-like operating system, which can process mismatches in field type information of data separately, and download the first real-time migration data to the Linux end for processing after processing on the AS / 400 end, wherein, Figure 2 As shown in the figure, data migration is performed on the Linux platform as the server.

[0060] S200: compressing the Body part of the first real-time migration data to generate second real-time migration data.

[0061] Specifically, in the embodiment of the present application, since the traditional data migration is to generate an export file on the local hard disk of the host by AS / 400PF, occupying the hard disk space of the host. All files are transferred by FTP method, and the transmission efficiency is not high and there is no integrity verification means, so the first real-time migration data is subjected to data quantization analysis to obtain the quantization coefficient that characterizes the data volume of the first real-time migration data; further, the preset quantization coefficient is a data volume threshold that characterizes the timeliness of data migration; the quantization coefficient is compared with the preset quantization coefficient, if the quantization coefficient is less than or equal to the preset quantization coefficient, it is not necessary to compress the first real-time migration data and it will not affect the timeliness of data migration, then the first real-time migration data can be directly updated to the second real-time migration data and transmitted to the Linux end, and the uncompressed command is recorded as command 2: cmd 0x02; if the quantization coefficient is greater than the preset quantization coefficient, it indicates that the data volume of the first real-time migration data will affect the timeliness of data migration, then it is necessary to generate a compression instruction to facilitate the control of the compression / decompression function module in the AS / 400 end to compress the second real-time migration data.

[0062] When the compression / decompression function module receives the compression instruction, it calls the preset compression algorithm to compress the first real-time migration data to obtain the second real-time migration data. The preset compression algorithm is a customizable compression algorithm with high flexibility, which ensures that it is consistent with the compression / decompression algorithm in the server program of the later Linux end.

[0063] The C language structure of the compressed data is exemplarily recorded as command 4: cmd 0x04:

[0064] Table 1

[0065] C language structure members illustrate unsigned char flag[2]; ^F, ^S = 0x06, 0x13 char ver; Version:ver3=0x03 char cmd; 0x04:fCompData unsigned int seq; data4 The record sequence number of the first record unsigned int rrn; data4 RRN of the first record unsigned int crc32; crc32 before compression unsigned int len; The length of all the following information CMD4_DATA_T data4; Command 4 Data

[0066] Table 2

[0067] C language structure members illustrate unsigned int rec_num; Number of records before compression unsigned int datalena; Compressed size unsigned int datalenb; Size before compression char data[1]; varchar: compressed data

[0068] S300: Connecting the AS / 400 terminal to the open platform terminal;

[0069] Furthermore, step S300 also includes:

[0070] S310: The second real-time migration data is obtained by connecting to the client of the AS / 400 end, and can support multiple client connections of the AS / 400 end through a multi-process mode, allowing multiple PF table data of the AS / 400 end to be migrated simultaneously.

[0071] Specifically, in the embodiment of the present application, the open platform end refers to a platform that assists in data transfer processing, such as, for example, a Linux end; after the first real-time migration data is processed in the AS / 400 end to obtain the second real-time migration data, the client program of the AS / 400 end is called to interact with the server program of the Linux end, and the file sending protocol for the interaction between the client program of the AS / 400 end and the server program of the Linux end can be customized: For example:

[0072] Table 3 File sending protocol, request C language structure

[0073]

[0074] Table 4 File sending protocol, response C language structure

[0075]

[0076]

[0077] Table 5 PF file field information structure

[0078] C language structure members illustrate unsigned short fldno; Field number unsigned char fldname

[10] ; Field Name char eos; 0x00,NUL,end of string unsigned char VLFI; Variable Length Field Ind. unsigned char fldtype; Field type information unsigned short fldlen; Field length unsigned short flddec; Decimal Places unsigned short fldpos; Field location int Field_Data_CCSID; Field data CCSID

[0079] The connection status refers to information that characterizes the interactive configuration and data transmission status of the client program on the AS / 400 side and the server program on the Linux side. If the configuration is completed, the Linux side is set to a file-waiting-to-receive state, and the second real-time migration data is transmitted to the open platform through the client program on the AS / 400 side based on the file transfer protocol.

[0080] Furthermore, the second real-time migration data is obtained through an AS / 400 client connection, and can support multiple AS / 400 client connections through a multi-process mode, allowing multiple PF table data of AS / 400 to be migrated simultaneously. This is because it is difficult to compress the PF table of a very large database alone. Therefore, when the first real-time migration data meets the preset quantization coefficient, it is necessary to determine whether the compression threshold of the split transmission is met, which represents the data volume threshold that requires the compressed data to be split and compressed. When the compression threshold of the split transmission is met, the first real-time migration data needs to be split through the split instruction issued by the preprocessor on the AS / 400 side to obtain the multiple data transmission partitions representing multiple sharded small PFs, and then compressed and transmitted, which can improve transmission efficiency and real-time performance. Exemplary: a preprocessor can be used to split it into multiple sharded small PFs of 300,000 to 400,000 records and then transmitted in parallel.

[0081] The transmission command is recorded as command 1: cmd 0x01, and the C language structure is as follows:

[0082] Table 6

[0083] C language structure members illustrate char lib

[10] ; PF library char file

[10] ; PF Name char OrgnPF

[10] ; Original PF name (before split) char MySQLtab

[64] ; Corresponding MySQL table name char KeyFldsSeq

[34] ; Key field sequence number, separated by commas, such as: 3,6,2 unsigned int maxrrn; Maximum RRN unsigned int recnum; Number of records unsigned short recsize; Record size unsigned short fldnum; Number of fields unsigned int bufsize; Buffer size before compression pf_ffd_T ffds[MAX_PF_FLDS]; File field structure

[0084] S400: decompressing the second live migration data and analyzing the field data type based on the Head part in the first live migration data to obtain field type information;

[0085] S500: splitting the second live migration data according to the field type information, wherein the Head part is sent as the first block of data to be transmitted, and the Body part is split into multiple data blocks for transmission, thereby obtaining split data partitions;

[0086] Specifically, when the connection status shows that the open platform end: hereinafter referred to as the Linux end has completely downloaded the second real-time migration data, in order to process the second real-time migration data, the open platform end calls the same compression / decompression algorithm as the AS / 400 end to decompress the second real-time migration data; the field type information refers to the analysis of the mismatched field data type of the second real-time migration data after the Linux end has completed the decompression of the second real-time migration data of the AS / 400 end PF, and obtains data information with different field type information identifiers; further, the split data partition is to partition the second real-time migration data according to the field type information to obtain multiple groups of partition data, wherein a group of partitions corresponds to a group of data field type information. Exemplary mismatched field type information is: actually storing O-type Chinese in an A-type field, for early data that does not comply with the current IBM transcoding rules, for the case where multiple A / O / P / S / Z and other type small fields containing structures are stored in an A / O-type large field, and for mismatched scenarios such as garbled characters in the field. By extracting the mismatched field type information, it is convenient for subsequent steps to process them separately, thereby solving the problem of mismatched field type information.

[0087] S600: The open platform constructs a pre-transcoding rule based on the default transcoding rule of some field types of Head;

[0088] S700: Calling the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data;

[0089] S800: Migrating the third real-time migration data to the heterogeneous database through the open platform.

[0090] Specifically, the pre-transcoding rule is a transcoding processing rule preset by the open platform end, i.e., the Linux end, for different types of mismatched data field type information. For example: for a large A / O type field containing multiple A / O / P / S / Z type small fields and a type O Chinese character is actually stored in an A type field, the small fields can be transcoded one by one according to the structure configuration; for the garbled characters in the field, they can be converted into half-width question marks, full-width question marks and other replacement characters according to the predetermined rules; the upload transcoding process can record logs, save the bytes of the EBCDIC encoding before the conversion for future reference and other preset processing rules. Further, the third real-time migration data is the migration data obtained by transcoding the mismatched data field type information according to the pre-transcoding rule. At this time, the migration data does not have garbled characters and mismatched field type information, and the data has higher accuracy and applicability after migration. Further, the third real-time migration data is transmitted to the heterogeneous database through the open platform end, i.e., the Linux end, to complete the data migration, and the data garbled characters and mismatched field type information are processed by the open platform end, thereby ensuring the accuracy of data migration.

[0091] Furthermore, the method step S600 also includes:

[0092] S610: Constructing a garbled code replacement rule, where the garbled code replacement rule is a sub-rule of the pre-transcoding rule;

[0093] S620: Obtain a garbled field set by performing field transcoding on the second real-time migration data, wherein the garbled field set is a field that cannot be converted according to a transcoding rule during field transcoding;

[0094] S630: Perform character replacement on the garbled code field set based on the garbled code replacement rule to obtain a replacement field set;

[0095] S640: Generate a record log according to the replacement field set.

[0096] Furthermore, step S640 also includes S641: the record log can locate the location of the problem data, wherein the location of the problem data includes the relative record number RRN, the field name, and the offset of the error in the field.

[0097] Specifically, the garbled code replacement rule refers to the preset rule of the pre-transcoding rule for transcoding the garbled code in the field; the garbled code field set refers to the field garbled code data set extracted from the field type information, exemplarily: early data that does not comply with the current IBM transcoding rules, multiple A / O / P / S / Z and other type small fields containing structures are stored in a large A / O type field, and the official or third-party tools are used to download / export data errors or garbled codes, and the field data type does not match, such as the O-type Chinese is actually stored in the A-type field, resulting in the official or third-party tools downloading / exporting data as garbled codes and other garbled codes; further, the replacement field set refers to the replacement field set preset by the garbled code replacement rule and matching the garbled code field set, and the garbled code field set is replaced by the replacement field set, exemplarily: the garbled codes in the above examples are converted into half-width question marks, full-width question marks and other replacement characters. Furthermore, the record log records the bytes of EBCDIC encoding before and after the conversion, which is convenient for reference. For example, the record log can locate the problem data position represented by the relative record number RRN, the field name, and the offset of the error in the field, and then the location of the problem can be accurately traced back based on the record log, which is convenient for adjustment.

[0098] The C language structure of the record data command is exemplified as follows:

[0099] Table 7

[0100] C language structure members illustrate unsigned int seq; Record sequence number unsigned int rrn; RRN char data[1]; varchar: data before compression

[0101] Table 8

[0102]

[0103]

[0104] Further, before migrating the third real-time migration data to the heterogeneous database through the open platform, the method further includes step S900:

[0105] S910: Inputting the third real-time migration data into a data verification unit to perform CRC32 verification to obtain a verification result;

[0106] S920: If the verification result is that the verification fails, a return instruction is obtained;

[0107] S930: If the verification result is that the verification passes, a migration instruction is obtained;

[0108] S940: Process the third real-time migration data according to the return instruction and the migration instruction.

[0109] Specifically, in order to ensure the integrity of data transmission, before transmission, that is, when the second real-time migration data is processed and the third real-time migration data is obtained on the Linux side, command 3: cmd 0x03 is issued. Through command 3, the integrity and accuracy of the data are verified before the third real-time migration data is migrated to the heterogeneous database. The CRC32 is the most commonly used error check code in the field of data communication, and the length of its information field and check field can be arbitrarily selected; the verification result is the result after the third real-time migration data is input into the data verification unit for CRC32 verification. When the length of the information field and the check field do not match, the integrity is insufficient, and the Linux side returns the third real-time migration data to the AS / 400 side through the return instruction to supplement the data until the CRC32 verification requirement is met before transmission; when the length of the information field and the check field matches, the verification result is verification passed, and the third real-time migration data is transmitted to the heterogeneous database through the migration instruction to complete the data migration. By verifying the integrity and accuracy of the migration data, the technical effect of ensuring the accuracy of data migration is achieved.

[0110] The C language structure of the verification command is exemplified as follows:

[0111] Table 9

[0112]

[0113]

[0114] Furthermore, the method comprises step S1000:

[0115] S1010: Obtaining a compression algorithm of the AS / 400 terminal according to the compression instruction;

[0116] S1020: Obtain a decompression algorithm according to the compression algorithm, wherein the compression algorithm corresponds to the decompression algorithm;

[0117] S1030: Using the decompression algorithm as the decompression algorithm of the open platform to perform decompression processing.

[0118] Specifically, the migration dual-ends of the migrated data refer to the dual-ends for transferring and receiving data, which are recorded as the first migration end and the second migration end. For example, the first migration end is the local AS / 400 end; the second migration end is the open platform end for data transfer processing, that is, the Linux end. After receiving the compression instruction, the first migration end matches the compression algorithm to compress the data, and further sets the decompression algorithm corresponding to the compression algorithm to the decompression algorithm of the open platform end to avoid the situation where different versions are incompatible and cause garbled characters.

[0119] Another aspect of the present invention also relates to a system for DB2 / 400 data migration of heterogeneous databases, the structure of which is as follows: Figure 2 As shown, including:

[0120] An acquisition module is used to acquire first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records;

[0121] A first generating module, configured to compress a Body portion of the first real-time migration data to generate second real-time migration data;

[0122] A connection module, used for connecting the AS / 400 terminal to the open platform terminal;

[0123] a splitting module, configured to decompress the second real-time migration data and analyze the field data type based on the Head part of the first real-time migration data to obtain field type information; and split the second real-time migration data according to the field type information, wherein the Head part is sent as the first block of data to be transmitted, and the Body part is split into multiple data blocks for transmission to obtain split data partitions;

[0124] A rule building module, wherein the open platform end builds a pre-transcoding rule based on the default transcoding rule of some field types of Head;

[0125] A second generating module is used to call the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data;

[0126] A migration module is used to migrate the third real-time migration data to the heterogeneous database through the open platform end.

[0127] By using this system, the above-mentioned operation processing method can be executed and the corresponding technical effects can be achieved.

[0128] An embodiment of the present invention also provides a computer-readable storage medium capable of implementing all steps of the DB2 / 400 data migration heterogeneous database method in the above embodiment, on which a computer program is stored, and when the computer program is executed by a processor, all steps of the DB2 / 400 data migration heterogeneous database method in the above embodiment are implemented.

[0129] An embodiment of the present invention also provides an electronic device for executing the above method. As an implementation device of the method, the electronic device has at least a processor and a memory, and in particular, the memory stores data and related computer programs required for executing the method, and the processor calls the data and programs in the memory to execute all steps of the implementation method and obtains the corresponding technical effect.

[0130] Preferably, the electronic device may include a bus architecture, the bus may include any number of interconnected buses and bridges, and the bus will include various circuits linked together by one or more processors and memories. The bus may also link together various other circuits such as peripherals, voltage regulators, and power management circuits, which are all well known in the art and are therefore not further described herein. The bus interface provides an interface between the bus and the receiver and transmitter. The receiver and transmitter may be the same component, i.e., a transceiver, providing a unit for communicating with various other systems over a transmission medium. The processor is responsible for managing the bus and general processing, while the memory may be used to store data used by the processor when performing operations.

[0131] In addition, the electronic device may further include components such as a communication module, an input unit, an audio processor, a display, and a power supply. The processor (or controller, operation control) used may include a microprocessor or other processor device and / or a logic device, which receives input and controls the operation of various components of the electronic device; the memory may be one or more of a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory, or other suitable devices, which may store the above-mentioned related data information, and may also store a program for executing related information, and the processor may execute the program stored in the memory to realize information storage or processing, etc.; the input unit is used to provide input to the processor, for example, it may be a key or a touch input device; the power supply is used to provide power to the electronic device; the display is used to display display objects such as images and text, for example, it may be an LCD display. The communication module is a transmitter / receiver that sends and receives signals via an antenna. The communication module (transmitter / receiver) is coupled to the processor to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal. Based on different communication technologies, multiple communication modules may be provided in the same electronic device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module (transmitter / receiver) is also coupled to a speaker and a microphone via an audio processor to provide an audio output via the speaker and receive an audio input from the microphone, thereby realizing a common telecommunication function. The audio processor may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor is also coupled to a central processor, so that recording can be performed on the machine through the microphone, and the sound stored on the machine can be played through the speaker.

[0132] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0133] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A system that specifies the functions of a box or multiple boxes.

[0134] These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction system, which is implemented in the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0135] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps of the functions specified in a box or multiple boxes. Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they know the basic creative concept. Therefore, the attached claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0136] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for DB2 / 400 data migration of heterogeneous databases, characterized in that: include: Acquire first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records; Compressing the Body part of the first real-time migration data to generate second real-time migration data; Connecting the AS / 400 terminal to the open platform terminal; decompressing the second live migration data and analyzing field data types based on the Head part in the first live migration data to obtain field type information; Splitting the second live migration data according to the field type information, wherein a Head part is sent as a first block of data to be transmitted, and a Body part is split into multiple data blocks for transmission, to obtain split data partitions; The open platform end constructs a pre-transcoding rule based on the default transcoding rule of the Head part field type; Calling the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data; The third real-time migration data is migrated to the heterogeneous database through the open platform.

2. The method according to claim 1, characterized in that The pre-transcoding rules include garbled code replacement rules; By performing field transcoding on the second real-time migration data, a garbled field set is obtained, wherein the garbled field set is a field that cannot be converted according to a transcoding rule during field transcoding; Performing character replacement on the garbled code field set based on the garbled code replacement rule to obtain a replacement field set; A record log is generated according to the replacement field set.

3. The method according to claim 1, characterized in that Before obtaining the first real-time migration data, the method further includes: Setting a preset quantization coefficient, wherein the preset quantization coefficient is a data compression critical value; Obtaining a quantization coefficient by performing data quantization analysis on the first real-time migration data; When the quantization coefficient is greater than a preset quantization coefficient, a compression instruction is obtained, and the compression instruction is used for compressing the real-time migration data.

4. The method according to claim 1, characterized in that The second real-time migration data is obtained by connecting to a client on the AS / 400 side, and can support multiple client connections on the AS / 400 side in a multi-process manner, allowing multiple PF table data on the AS / 400 side to be migrated simultaneously.

5. The method according to claim 1, characterized in that Before migrating the third real-time migration data to the heterogeneous database through the open platform, the method further includes: Performing CRC32 verification on the third real-time migration data to obtain a verification result; If the verification result is that the verification fails, a return instruction is obtained; If the verification result is that the verification passes, a migration instruction is obtained; The third real-time migration data is processed according to the return instruction and the migration instruction.

6. The method according to claim 2, characterized in that The record log can locate the location of the problematic data; The problem data location includes the relative record number RRN, the field name, and the offset of the error in the field.

7. The method according to claim 1 or 3, characterized in that: The method further comprises: According to the compression instruction, obtaining the compression algorithm of the AS / 400 end; According to the compression algorithm, a decompression algorithm is obtained, wherein the compression algorithm corresponds to the decompression algorithm; The decompression algorithm is used as the decompression algorithm of the open platform end to perform decompression processing.

8. A DB2 / 400 data migration system for heterogeneous databases, characterized in that: The system comprises: An acquisition module is used to acquire first real-time migration data, wherein the first real-time migration data is data of a DB2 / 400 database in an AS / 400 terminal, and the first real-time migration data includes a Head part and a Body part, wherein the Head part is metadata including key information required for subsequent processing, and the Body part is detailed data including specific data records; A first generating module, configured to compress a Body portion of the first real-time migration data to generate second real-time migration data; A connection module, used for connecting the AS / 400 terminal to the open platform terminal; a splitting module, configured to decompress the second real-time migration data and analyze the field data type based on the Head part of the first real-time migration data to obtain field type information; and split the second real-time migration data according to the field type information, wherein the Head part is sent as the first block of data to be transmitted, and the Body part is split into multiple data blocks for transmission to obtain split data partitions; A rule building module, wherein the open platform end builds a pre-transcoding rule based on the default transcoding rule of some field types of Head; A second generating module is used to call the open platform end to transcode the split data partition according to the pre-transcoding rule to obtain third real-time migration data; A migration module is used to migrate the third real-time migration data to the heterogeneous database through the open platform end.

9. A computer-readable storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: The processor is configured to execute a computer program stored in a memory to perform the method according to any one of claims 1 to 7.

Images