A greenplum database migration method and device

Abstract

The application relates to the technical field of computer coding software, and particularly provides a Greenplum database migration method, wherein a Greenplum plug-in is uploaded in a Yunxi migration platform, database connection information is configured, a data transform plug-in is matched, data transmission between the Greenplum database is realized, and data information is encrypted and then decrypted for use. Compared with the prior art, the application has more application scenarios, is more widely used, and is more perfect in adaptation. Further market demands are met. According to the syntax rules and characteristics of different databases, specific syntax trees and mapping schemes are provided, automatic recognition is completely realized by using a program, and automatic matching conversion is carried out according to the syntax. The latest version of the syntax of each database is matched, so that the accuracy and timeliness of the tool are guaranteed.

Description

Technical Field

[0001] This invention relates to the field of computer coding software technology, and specifically provides a Greenplum database migration method and apparatus. Background Technology

[0002] With Greenplum's open-source release in October 2015, distributed databases gained further development and attention, achieving widespread application and market share similar to Inspur Cloud's ZNBase. This led to the development of the Greenplum plugin. It's a loosely coupled, highly cohesive pluggable plugin used for data conversion between different databases and the Greenplum database, including data transfer, schema conversion, migration verification, etc. It's written in C / C++ and compatible with various operating systems including Linux, Mac, and Windows. It's small in size and consumes little memory, using batch loading APIs for data transfer. Key features: ①Low coupling ②Pluggable ③Small size ④Low memory consumption.

[0003] The existing Inspur Cloud Stream migration platform supports data migration and exchange between various databases, but it lacks support for Greenplum database. Summary of the Invention

[0004] This invention addresses the shortcomings of the prior art by providing a highly practical method for migrating Greenplum databases.

[0005] A further technical objective of this invention is to provide a reasonably designed, safe, and applicable Greenplum database migration device.

[0006] The technical solution adopted by this invention to solve its technical problem is:

[0007] A method for migrating a Greenplum database involves uploading the Greenplum plugin to the Yunxi migration platform, configuring the database connection information, and using the datatransform data conversion plugin to achieve data transfer between the Greenplum database and the database. The data is encrypted and then decrypted before use.

[0008] Furthermore, upload the Greenplum plugin to the Yunxi migration platform. The platform will temporarily store the component package on the server, add a runtime server, and select the option to install the Greenplum plugin along with the cloud message load balancer to the target server.

[0009] Furthermore, the task configuration database connection information is created, connecting to the target or source database, and the configuration information is saved as a file on the server for later use.

[0010] Furthermore, under the aforementioned task, an information flow is created before the database is established, and then a configuration file is sent out. This file will be parsed and used by the Greenplum plugin to start the task and wait for the data migration to complete.

[0011] Furthermore, the Greenplum plugin reads the collected SQL statement set or object set, uses the JudgeType() function to determine the necessary flags, and transmits them along with the set to the datatransform component.

[0012] Furthermore, the datatransform uses pre-designed syntax tree rules to map and transform the corresponding statement sets layer by layer, and then passes the transformed dataset to the Write component of the corresponding database for database migration.

[0013] Furthermore, the data information undergoes unified desensitization and encryption processing. The connection configuration information is encapsulated through the linkedInfo() function and passed to the encryption function encryptUtil() for encryption.

[0014] Furthermore, the encrypted information is configured and entered into the connection pool for use by connection threads, and decrypted using decryptUtil() when needed.

[0015] A Greenplum database migration apparatus includes: at least one memory and at least one processor;

[0016] The at least one memory is used to store a machine-readable program;

[0017] The at least one processor is configured to invoke the machine-readable program to execute a Greenplum database migration method.

[0018] Compared with existing technologies, the Greenplum database migration method and apparatus of the present invention have the following outstanding advantages:

[0019] This invention features a richer source database, adding Greenplum to work with mainstream enterprise databases for data migration and syntax conversion. It offers more application scenarios, broader uses, and more comprehensive compatibility, further meeting market demands.

[0020] Depending on the syntax rules and characteristics of different databases, there are specific syntax trees and mapping schemes. The program automatically identifies and matches the syntax.

[0021] The syntax is matched to the latest version of each database, ensuring the accuracy and timeliness of the tool. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Appendix Figure 1 This is a flowchart illustrating the use of Greenplum plugins in a Greenplum database migration method.

[0024] Appendix Figure 2 This is a data migration flowchart in a Greenplum database migration method.

[0025] Appendix Figure 3 This is a schematic diagram of the data encryption and decryption process in a Greenplum database migration method. Detailed Implementation

[0026] To enable those skilled in the art to better understand the present invention, the present invention will be further described in detail below with reference to specific embodiments. Obviously, the described embodiments are merely some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] The following is a preferred embodiment:

[0028] This embodiment describes a Greenplum database migration method. The method involves uploading a Greenplum plugin to the Yunxi migration platform, configuring the database connection information, and using the datatransform plugin to achieve data transmission between the Greenplum database and the database. The data is then encrypted and decrypted before use.

[0029] The specific steps are as follows:

[0030] like Figure 1 As shown, upload the Greenplum plugin to the Inspur Cloud Creek migration platform. The platform will temporarily store the component package on the server. Add a runtime server and check the box to install the Greenplum plugin along with the cloud message load balancer to the target server.

[0031] Upload the Greenplum plugin to the Inspur Cloud Stream migration platform. The platform will temporarily store the component package on the server. Add a runtime server and check the box to install the Greenplum plugin along with the cloud message load balancer to the target server.

[0032] The task creates a job to establish the information flow before the database is set up, and then distributes the configuration file. This file will be parsed and used by the Greenplum plugin. Start the task and wait for the data migration to complete.

[0033] like Figure 2 As shown, the Greenplum plugin reads the collected SQL statement set or object set, uses the JudgeType() function to determine the required flags, and transmits them along with the set to the datatransform component.

[0034] DataTransform uses pre-designed syntax tree rules to map and transform corresponding statement sets such as select, update, and delete layer by layer.

[0035] The transformed dataset is passed to the Write component of the corresponding database for database migration.

[0036] like Figure 3 As shown, the information undergoes unified de-identification and encryption processing. Configuration information such as connection information is encapsulated through the linkedInfo() function and passed to the encryption function encryptUtil() for encryption.

[0037] After the information is encrypted, it is entered into the connection pool through configuration and made available for use by connection threads.

[0038] Use decryptUtil() to decrypt the code when you need to use it.

[0039] Based on the above method, a Greenplum database migration device in this embodiment includes: at least one memory and at least one processor;

[0040] The at least one memory is used to store a machine-readable program;

[0041] The at least one processor is configured to invoke the machine-readable program to execute a Greenplum database migration method.

[0042] The specific embodiments described above are merely specific examples of the present invention. The patent protection scope of the present invention includes, but is not limited to, the specific embodiments described above. Any Greenplum database migration method and apparatus that conforms to the present invention, and any appropriate changes or substitutions made by those skilled in the art, should fall within the patent protection scope of the present invention.

[0043] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A Greenplum database migration method, characterized in that, Upload the Greenplum plugin to the Yunxi migration platform, configure the database connection information, and use the datatransform data transformation plugin to achieve data transmission between the Greenplum database. The data information is encrypted and then decrypted before use. Upload the Greenplum plugin to the Yunxi Migration Platform. The platform will temporarily store the component package on the server, add a runtime server, and select the option to install the Greenplum plugin along with the cloud message load balancer to the target server. Create a task, configure database connection information, connect to the target or source database, and save the configuration information. The information will be saved to the server as a file for later use. The task creates an information flow before the database is established, and then the configuration file is sent out. The Greenplum plugin will parse and use the file, start the task, and wait for the data migration to complete. The Greenplum plugin reads the collected SQL statement set or object set, uses the JudgeType() function to determine the required flags, and transmits them along with the set to the datatransform component. The datatransform uses pre-designed syntax tree rules to map and transform the corresponding statement set layer by layer, and then passes the transformed dataset to the Write component of the corresponding database to connect to the database for migration. Data information undergoes unified desensitization and encryption processing. Connection configuration information is encapsulated through the linkedInfo() function and passed to the encryption function encryptUtil() for encryption. After the information is encrypted, it is entered into the connection pool through configuration for use by connection threads. When it is about to be used, it is decrypted using decryptUtil().

2. A Greenplum database migration device, characterized in that, include: At least one memory and at least one processor; The at least one memory is used to store a machine-readable program; The at least one processor is configured to invoke the machine-readable program to execute the method of claim 1.

Images