ETL software automatic adaptation method and device based on credential environment, equipment and medium

Abstract

The invention belongs to the technical field of ETL, and provides an ETL software automatic adaptation method and device based on a credential environment, equipment and a medium, and the method comprises the following steps: receiving adaptation resources, and executing an adaptation scheduling script; calling the first adaptive execution script, and checking a server environment in which the ETL software runs; calling a second adaptive execution script, decompressing the ETL installation medium, and installing the ETL software dependency package; calling a third adaptive execution script, and modifying a system configuration file according to a preset ETL deployment configuration file; calling a fourth adaptive execution script, and respectively generating a data warehouse connection encryption file and an ETL database connection encryption file according to a preset ETL deployment configuration file; calling a fifth adaptation execution script, verifying the ETL process, and completing automatic adaptation; according to the method, all the adaptation instructions are integrated in the adaptation resources, a unified automatic adaptation process is realized through a one-key operation mode, the error probability is reduced, the working difficulty is reduced, and the working efficiency is improved.

Description

Technical Field

[0001] The present invention belongs to the field of ETL technology, and in particular relates to an automatic adaptation method, device, equipment and medium for ETL software based on a trusted computing environment. Background Art

[0002] ETL is the abbreviation of three English words: Extract, Transform, and Load. It is the process of extracting (Extract), cleaning, transforming (Transform), and then loading (Load) the data from the business system into the data warehouse. It is also called data warehouse technology.

[0003] In order to realize the full localization of information systems as soon as possible and maintain a high level of service, the software trust creation replacement work has been launched. The operating system and database that the ETL software relies on will be replaced by the domestic technology stack in the future, so the adaptation of ETL software based on the trust creation environment is an indispensable step.

[0004] However, in the prior art, the adaptation of ETL software requires manual operation by technicians, and automatic adaptation across platforms and batch servers cannot be achieved. Since different technicians perform different operations during the adaptation process, this increases the probability of errors in the adaptation process. At the same time, when the system environment changes, manual adaptation is required again, which also greatly increases the difficulty and workload of technicians. Summary of the invention

[0005] In response to the problems in the background technology, the present invention proposes an automatic adaptation method for ETL software based on the trusted innovation environment.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention proposes an automatic adaptation method for ETL software based on a trusted innovation environment, comprising:

[0008] Receive adaptation resources and execute adaptation scheduling scripts; wherein the adaptation resources include adaptation scripts, preset external parameter files and ETL installation media, the adaptation scripts include adaptation scheduling scripts and adaptation execution scripts, the preset external parameter files include ETL deployment configuration files and adaptation execution script serial number files, and the adaptation execution script serial number files are used to record the serial number of the next adaptation execution script;

[0009] Call the first adaptation execution script to check the server environment where the ETL software runs;

[0010] Call the second adapter execution script, decompress the ETL installation media, and install the ETL software dependency package;

[0011] Call the third adapter execution script to modify the system configuration file according to the preset ETL deployment configuration file;

[0012] Calling the fourth adapter execution script to generate a data warehouse connection encryption file and an ETL database connection encryption file respectively according to a preset ETL deployment configuration file;

[0013] Call the fifth adaptation execution script to verify the ETL process and complete the automatic adaptation.

[0014] Further, the calling of the first adaptation execution script to check the server environment in which the ETL software runs includes:

[0015] Check the mlocate command installation;

[0016] Read the user password file and check the creation of users etluser, etlftp, and etlcheck;

[0017] Read the user group file and check whether the group etladmin is created;

[0018] Check parameter configuration;

[0019] Check the port connectivity between servers.

[0020] Furthermore, the ETL software dependency packages include: Perl software package and DBD-Pg database driver;

[0021] The installation of ETL software dependency packages includes:

[0022] Traverse the array storing the Perl software package name and the array storing the installation method respectively to obtain the software package name and installation method data of the Perl software package;

[0023] Install the Perl software package according to the installation method data of the Perl software package;

[0024] Create a Makefile in the DBD-Pg installation package directory, compile and link the source code, and install the DBD-Pg database driver.

[0025] Further, the system configuration files include: user environment configuration files, ETL common variable configuration files and FTP configuration files;

[0026] Call the third adapter execution script and modify the system configuration file according to the preset ETL deployment configuration file, including:

[0027] Read the preset ETL deployment configuration file, obtain the data warehouse installation directory, ETL database name and ETL directory respectively, and write the data warehouse installation directory, ETL database name and ETL directory into the user environment configuration file respectively;

[0028] Read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse IP address and data warehouse port respectively, and write the data warehouse name, data warehouse IP address and data warehouse port respectively into the ETL common variable configuration file;

[0029] Read the preset ETL deployment configuration file, obtain the data warehouse user name, concatenate the data warehouse user name and the data warehouse IP, generate a data warehouse connection file name, and write the data warehouse connection file name into the ETL public variable configuration file;

[0030] Read the preset ETL deployment configuration file, obtain the ETL server IP, and write the ETL server IP into the ETL public variable configuration file;

[0031] Read the preset ETL deployment configuration file, obtain the ETL main directory, generate the ETL source data receiving directory by string concatenation, and write the ETL source data receiving directory into the FTP configuration file.

[0032] Furthermore, before reading the preset ETL deployment configuration file, obtaining the data warehouse installation directory, and writing the data warehouse installation directory into the user environment configuration file, it also includes:

[0033] Back up the user environment configuration file.

[0034] Further, the fourth adaptation execution script is called to generate a data warehouse connection encryption file and an ETL database connection encryption file according to a preset ETL deployment configuration file, including:

[0035] Read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse user name and data warehouse password respectively, and generate the data warehouse connection encryption file;

[0036] Read the preset ETL deployment configuration file, obtain the ETL database name, ETL database user name and ETL database password, and generate an ETL database connection encryption file.

[0037] Further, the calling of the fifth adaptation execution script, verifying the ETL process, and completing the automatic adaptation includes:

[0038] Start the ETL process and gpfdist process;

[0039] Verify the ETL process and complete automatic adaptation.

[0040] In a second aspect, the present invention proposes an automatic adaptation device for ETL software based on a trusted innovation environment, comprising:

[0041] A preparation unit, configured to receive an adaptation resource and execute an adaptation scheduling script; wherein the adaptation resource includes an adaptation script, a preset external parameter file, and an ETL installation medium; the adaptation script includes an adaptation scheduling script and an adaptation execution script; the preset external parameter file includes an ETL deployment configuration file and an adaptation execution script sequence number file; the adaptation execution script sequence number file is used to record the sequence number of the next adaptation execution script;

[0042] A checking unit, used to call the first adaptation execution script to check the server environment in which the ETL software runs;

[0043] An installation unit is used to call the second adaptation execution script, decompress the ETL installation medium, and install the ETL software dependency package;

[0044] A configuration unit, used to call a third adaptation execution script and modify a system configuration file according to a preset ETL deployment configuration file;

[0045] A generating unit, used to call a fourth adaptation execution script, and generate a data warehouse connection encryption file and an ETL database connection encryption file respectively according to a preset ETL deployment configuration file;

[0046] The startup unit is used to call the fifth adaptation execution script, verify the ETL process, and complete the automatic adaptation.

[0047] In the third aspect, the present invention proposes a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the above-mentioned ETL software automatic adaptation method based on the trusted innovation environment can be implemented.

[0048] In a fourth aspect, the present invention proposes a device comprising a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and wherein the processor executes the above-mentioned ETL software automatic adaptation method based on the trusted innovation environment when running the computer instructions.

[0049] Beneficial effects of the present invention:

[0050] The method of the present invention integrates all adaptation instructions into the adaptation resources, realizes a unified automated adaptation process through a one-key operation mode, reduces the error probability, reduces the work difficulty, and improves the work efficiency. At the same time, the adaptation operation process is saved in the execution script in the form of a template, which makes it easier for technical personnel to maintain and upgrade the operation process.

[0051] Other features and advantages of the present invention will be described in the following description, and partly become obvious from the description, or be understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures pointed out in the description and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0053] Figure 1 The flowchart of the ETL software automatic adaptation method based on the trust creation environment of the present invention is shown;

[0054] Figure 2 The structure diagram of the preset external parameter file of the present invention is shown;

[0055] Figure 3 The structure diagram of the array storing the Perl software package name and the array storing the installation mode of the present invention is shown;

[0056] Figure 4 A flow chart showing a method for modifying a system configuration file of the present invention is shown;

[0057] Figure 5 A schematic diagram showing the successful start of the ETL process of the present invention;

[0058] Figure 6 Shown is a system framework diagram of the present invention;

[0059] Figure 7 A schematic diagram of the device structure of the present invention is shown. DETAILED DESCRIPTION

[0060] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0061] Refer to the instruction manual Figure 1 As shown in the figure, the automatic adaptation method of ETL software based on the trusted innovation environment specifically includes the following steps:

[0062] S1. Receive adaptation resources and execute adaptation scheduling script;

[0063] In step S1, the adaptation resources include an adaptation script, a preset external parameter file and an ETL installation medium. The adaptation script includes an adaptation scheduling script and an adaptation execution script. The preset external parameter file includes an ETL deployment configuration file etldeploy.conf and an adaptation execution script serial number file etltag.dat.

[0064] Among them, the adaptation scheduling script is used to coordinate the execution order of the entire adaptation process, call each adaptation execution script, and ensure that the adaptation steps are carried out in a predetermined order; the adaptation execution script contains specific adaptation operation instructions, such as checking the server environment where the ETL software runs, installing dependency packages, modifying configuration files, etc.; the preset ETL deployment configuration file stores various configuration parameters required in the adaptation process, such as ETL database name, data warehouse name, IP address, etc.;

[0065] The adaptation execution script sequence number file is used to record the sequence number of the next adaptation execution script, and its default value is 1, that is, execution starts from the first adaptation execution script.

[0066] Exemplarily, the system first checks the sequence number in etltag.dat and determines which adaptation execution script to execute based on the sequence number;

[0067] If the current value of etltag.dat is 1, the system will execute the first adaptation execution script;

[0068] After the first adaptation execution script is executed, the value of etltag.dat is updated to 2, indicating that the next script to be executed is the second adaptation execution script;

[0069] The system will repeat the above steps until all the adaptation execution scripts are executed in order;

[0070] S2, calling the first adaptation execution script to check whether the server environment where the ETL software runs meets the adaptation requirements;

[0071] A comprehensive inspection of the server environment where the ETL software runs is the basis for ensuring that the subsequent adaptation process can proceed smoothly.

[0072] Specifically, the first adaptation execution script is called to perform the following checks:

[0073] S21. Check the installation of mlocate command;

[0074] mlocate is a tool for quickly finding files. It improves search efficiency by indexing the database. ETL software relies on this command to find specific files or directories. Therefore, checking whether the mlocate command is installed can ensure the normal operation of the ETL software.

[0075] S22. Read the user password file and check the creation of users etluser, etlftp, and etlcheck.

[0076] S23, read the user group file and check the creation of the group etladmin;

[0077] S24, check parameter configuration;

[0078] S25, checking the port connectivity between servers;

[0079] In a specific embodiment, refer to the attached specification Figure 2 As shown, it is a structure diagram of the preset external parameter file of the present invention, which includes parameters such as ETL installation directory, database information and data warehouse information. By running the command `yumlist installed|grep mlocate`, check whether the operating system mlocate command is installed;

[0080] Read the / etc / shadow file and check whether users etluser, etlftp, and etlcheck are created;

[0081] Read the / etc / group file and check whether the group etladmin has been created.

[0082] Read the preset external parameter file etldeploy.conf and check whether the parameter configuration is correct.

[0083] Run the telnet${ip}${port} command to check whether the ports between servers are connected.

[0084] If all the above checks pass, the external parameter file etltag.dat is updated and the recorded adaptation execution script number is changed to the next script number to be executed.

[0085] S3, calling the second adapter execution script, decompressing the ETL installation media, and installing the ETL software dependency package;

[0086] Specifically, install the ETL software dependency package, which includes the Perl software package and the DBD-Pg database driver, including the following steps:

[0087] S31, traversing the array storing the Perl software package name and the array storing the installation method respectively, and obtaining the software package name and installation method data of the Perl software package;

[0088] By traversing the two arrays, we can understand which Perl packages need to be installed and how to install each package (such as through yum or rpm), ensuring that the subsequent installation process can proceed correctly.

[0089] S32, installing the Perl software package according to the installation method data of the Perl software package;

[0090] S33. Enter the DBD-Pg installation package directory, create a Makefile file, compile and link the source code, and install the DBD-Pg database driver.

[0091] Installing the DBD-Pg database driver enables the ETL software to interact smoothly with the database to implement operations such as data extraction, conversion, and loading.

[0092] In a specific embodiment, refer to the attached specification Figure 3 As shown, Figure 3 The structure diagram of an array storing the name of a Perl software package and an array storing the installation method of the present invention calls the second adaptation execution script to decompress the ETL installation medium ETL.tar to the / opt / app / directory;

[0093] Install ETL software dependency packages, call ToInstall() function, traverse array @name_perl storing Perl package names and array @method_perl storing installation methods, obtain package names and installation method data, and install them one by one. When the installation method parameter method_perl = yum, run the command `yum-y install${name_perl[i]}`; when the installation method parameter method_perl = rpm, run the command `rpm-ivh${name_perl[i]}.rpm`;

[0094] To install the DBD-Pg database driver, first enter the DBD-Pg installation package directory, run the `perlMakefile.PL -l` command to create the Makefile file, then run the `make` command to compile and link the source code, and finally run the `make install` command to install it.

[0095] After all the above installations are completed, update the value of the external parameter file etltag.dat to the sequence number of the next adaptation execution script.

[0096] S4, calling the third adaptation execution script, and modifying the system configuration file according to the preset ETL deployment configuration file;

[0097] Specifically, modify the system configuration files, which include: user environment configuration files, ETL common variable configuration files, and FTP configuration files, including the following steps:

[0098] S41, modify the user environment configuration file;

[0099] By modifying the user environment configuration file, you can configure the environment variables required by the ETL software, such as the data warehouse installation directory (GPHOME), ETL database name (AUTO_DSN), ETL directory (AUTO_HOME), etc.

[0100] S411, read the preset ETL deployment configuration file, obtain the data warehouse installation directory, ETL database name and ETL directory respectively, and write the data warehouse installation directory, ETL database name and ETL directory into the user environment configuration file respectively;

[0101] S42, modify the ETL common variable configuration file;

[0102] Through the ETL common variable configuration file, you can configure various parameters required for the ETL software to connect to the data warehouse, such as the data warehouse name, IP address, etc., to ensure that the ETL software can correctly connect to the data warehouse and perform data extraction, transformation, and loading (ETL) operations.

[0103] S421, read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse IP and data warehouse port respectively, and write the data warehouse name, data warehouse IP and data warehouse port respectively into the ETL common variable configuration file;

[0104] S422, read the preset ETL deployment configuration file, obtain the data warehouse user name, splice the data warehouse user name and the data warehouse IP, generate the data warehouse connection file name, and write the data warehouse connection file name into the ETL public variable configuration file;

[0105] S423, read the preset ETL deployment configuration file, obtain the ETL server IP, and write the ETL server IP into the ETL public variable configuration file;

[0106] S43, modify the FTP configuration file;

[0107] By modifying the FTP configuration file, you can configure parameters such as the ETL source data receiving directory to ensure that FTP can correctly receive and transmit data files to the ETL software.

[0108] S431. Read the preset ETL deployment configuration file, obtain the ETL main directory, generate the ETL source data receiving directory by string concatenation, and write the ETL source data receiving directory into the FTP configuration file.

[0109] In a specific embodiment, refer to the attached specification Figure 4 As shown, Figure 4 The flowchart of the method for modifying the system configuration file of the present invention is to call the third adaptation execution script, first modify the user environment configuration file .bashrc: run the `locate greenplum_path.sh` command to obtain the absolute path of the shell file, that is, the data warehouse installation directory, call the ExchangeFile() function, and write the value into the GPHOME variable of the user environment configuration file .bashrc;

[0110] Read the preset external parameter file etldeploy.conf, get the REPO_SERVICE_NAME parameter value, that is, the ETL database name, call the ExchangeFile() function, and write the value to the AUTO_DSN variable of the user environment configuration file .bashrc;

[0111] Get the ETL_HOME parameter value, that is, the ETL directory, call the ExchangeFile() function, and write the value to the AUTO_HOME variable in the user environment configuration file .bashrc;

[0112] Then modify the ETL public variable configuration file PubVar.pm: Get the BATCH_DATABASE parameter value, that is, the data warehouse name, call the ExchangeFile() function, and write the value to the EDW_GP_DB_Name variable in the ETL public variable configuration file PubVar.pm; Get the BATCH_IP parameter value, that is, the data warehouse IP, call the ExchangeFile() function, and write the value to the EDW_GP_Svr variable in the ETL public variable configuration file PubVar.pm; Get the BATCH_PORT parameter value, that is, the data warehouse port, call the ExchangeFile() function, and write the value to the EDW_GP_Po rt variable; get the BATCH_USERNAME parameter value, and generate the data warehouse connection file name after splicing, $gp_login_file = "DS_DB_GP_Pdt".${BATCH_IP}."_".${BATCH_USERNAME}, call the ExchangeFile() function, and write the $gp_login_file value to the EncrypedGPLogOnFile variable of the ETL public variable configuration file PubVar.pm; get the ETL_IP parameter value, that is, the ETL server IP, call the ExchangeFile() function, and write the value to the EDW_GP_FileHost variable of the ETL public variable configuration file PubVar.pm;

[0113] Finally, modify the FTP configuration file vsftpd.conf: get the ETL_HOME parameter value, generate the ETL source data receiving directory after splicing, $ftp_path = ${ETL_HOME}." / ETL / DATA / hostfile", call the ExchangeFile() function, and write the $ftp_path value into the local_root variable of the FTP configuration file vsftpd.conf;

[0114] After all the above configurations are completed, update the etltag.dat value to the next adaptation execution script number.

[0115] It should be noted that the user environment configuration file .bashrc is an important file of the operating system. Therefore, before reading the preset external parameter file etldeploy.conf, obtaining the data warehouse installation directory, and writing the data warehouse installation directory into the user environment configuration file, you also need to back up the user environment configuration file.

[0116] S5, calling the fourth adaptation execution script, and generating a data warehouse connection encryption file and an ETL database connection encryption file respectively according to a preset ETL deployment configuration file;

[0117] Specifically, the steps include:

[0118] S51, read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse user name and data warehouse password respectively, and generate a data warehouse connection encryption file;

[0119] S52 , reading a preset ETL deployment configuration file, obtaining an ETL database name, an ETL database user name, and an ETL database password, respectively, and generating an ETL database connection encryption file.

[0120] In a specific embodiment, the fourth adaptation execution script is called to read the preset external parameter file etldeploy.conf, obtain the BATCH_DATABASE (data warehouse name), BATCH_USERNAME (data warehouse user name) and BATCH_PASS parameter values ​​(data warehouse password), call the CreateLogonFile() function, generate a data warehouse connection encrypted file, and save the file in the ${AUTO_HOME} / DATA / DB_Connection / directory;

[0121] Get the parameter values ​​of REPO_SERVICE_NAME (ETL database name), REPO_USERNAME (ETL database user name), and REPO_PASS (ETL database password), call the CreateLogonFile() function, generate the ETL database connection encryption file, and save the file in the ${AUTO_HOME} / etc / DataSet / directory.

[0122] After all the above files are generated, update the value of the external parameter file etltag.dat to the sequence number of the next adaptation execution script;

[0123] S6. Call the fifth adaptation execution script to verify the ETL process and complete the automatic adaptation;

[0124] Specifically, the steps include:

[0125] S61. Start the ETL process and gpfdist process;

[0126] S62. Verify the ETL process and complete automatic adaptation.

[0127] In a specific embodiment, refer to the attached specification Figure 5 As shown, Figure 5This is a schematic diagram of a successful start of an ETL process of the present invention. The fifth adaptation execution script is called, and the runuser-letluser-c'startetl' command is run to start the ETL process; runuser-l etluser-c'startgpfdist' is run to start the gpfdist process; and `etlstatus` is run to verify whether the ETL process is successfully started. The entire adaptation process ends.

[0128] The present invention integrates all adaptation instructions into the adaptation resources and realizes a unified automatic adaptation process through one-key operation, thereby reducing the error probability, reducing work difficulty, and improving work efficiency. At the same time, the adaptation operation process is saved in the execution script in the form of a template, which makes it easier for technical personnel to maintain and upgrade the operation process.

[0129] Based on the same inventive concept, refer to the attached specification Figure 6 As shown, the present invention also discloses an ETL software automatic adaptation device based on a trust creation environment, comprising:

[0130] The preparation unit 601 is used to receive the uploaded adaptation resources and execute the adaptation scheduling script; wherein the adaptation resources include the adaptation script, the preset external parameter file and the ETL installation medium, the adaptation script includes the adaptation scheduling script and the adaptation execution script, the preset external parameter file includes the ETL deployment configuration file and the adaptation execution script sequence number file, and the adaptation execution script sequence number file is used to record the sequence number of the next adaptation execution script;

[0131] The checking unit 602 is used to call the first adaptation execution script to check the server environment where the ETL software runs;

[0132] The installation unit 603 is used to call the second adaptation execution script, decompress the ETL installation medium, and install the ETL software dependency package;

[0133] The configuration unit 604 is used to call the third adaptation execution script to modify the system configuration file according to the preset ETL deployment configuration file;

[0134] The generating unit 605 is used to call the fourth adaptation execution script to generate the data warehouse connection encryption file and the ETL database connection encryption file according to the preset ETL deployment configuration file;

[0135] The starting unit 606 is used to call the fifth adaptation execution script, verify the ETL process, and complete the automatic adaptation.

[0136] The specific working process of the above units can be found in the corresponding method implementation examples, which will not be repeated here.

[0137] Based on the same inventive concept, the present invention also discloses a computer-readable storage medium on which computer instructions are stored. When the computer instructions are executed, the above-mentioned ETL software automatic adaptation method based on the trusted innovation environment can be implemented.

[0138] Based on the same inventive concept, refer to the attached specification Figure 7 As shown, the present invention also discloses a device, including a memory and a processor, wherein the memory stores computer instructions that can be run on the processor, and is characterized in that the processor executes the above-mentioned ETL software automatic adaptation method based on the trusted innovation environment when running the computer instructions.

[0139] It should be noted that, in this article, relational terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0140] Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent substitutions for some of the technical features therein; and these modifications or substitutions do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. The ETL software automatic adaptation method based on the trust creation environment is characterized by: include: Receive adaptation resources and execute adaptation scheduling scripts; wherein the adaptation resources include adaptation scripts, preset external parameter files and ETL installation media; the adaptation scripts include adaptation scheduling scripts and adaptation execution scripts, the preset external parameter files include ETL deployment configuration files and adaptation execution script serial number files, and the adaptation execution script serial number files are used to record the serial number of the next adaptation execution script; Call the first adaptation execution script to check the server environment where the ETL software runs; Call the second adapter execution script, decompress the ETL installation media, and install the ETL software dependency package; Call the third adapter execution script to modify the system configuration file according to the preset ETL deployment configuration file; Calling the fourth adapter execution script to generate a data warehouse connection encryption file and an ETL database connection encryption file respectively according to a preset ETL deployment configuration file; Call the fifth adaptation execution script to verify the ETL process and complete the automatic adaptation.

2. The ETL software automatic adaptation method based on the trust creation environment according to claim 1 is characterized in that: The calling of the first adaptation execution script to check the server environment in which the ETL software runs includes: Check the mlocate command installation; Read the user password file and check the creation of users etluser, etlftp, and etlcheck; Read the user group file and check whether the group etladmin is created; Check parameter configuration; Check the port connectivity between servers.

3. The ETL software automatic adaptation method based on the trust creation environment according to claim 1 is characterized in that: The ETL software dependency packages include: Perl software package and DBD-Pg database driver; The installation of ETL software dependency packages includes: The array storing the Perl software package name and the array storing the installation method are traversed respectively to obtain the software package name and installation method data of the Perl software package; Install the Perl software package according to the installation method data of the Perl software package; Create a Makefile file in the DBD-Pg installation package directory, compile and link the source code, and install the DBD-Pg database driver.

4. The ETL software automatic adaptation method based on the trust creation environment according to claim 1 is characterized in that: The system configuration files include: user environment configuration files, ETL common variable configuration files and FTP configuration files; Call the third adapter execution script and modify the system configuration file according to the preset ETL deployment configuration file, including: Read the preset ETL deployment configuration file, obtain the data warehouse installation directory, ETL database name and ETL directory respectively, and write the data warehouse installation directory, ETL database name and ETL directory into the user environment configuration file respectively; Read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse IP address and data warehouse port respectively, and write the data warehouse name, data warehouse IP address and data warehouse port respectively into the ETL common variable configuration file; Read the preset ETL deployment configuration file, obtain the data warehouse user name, concatenate the data warehouse user name and the data warehouse IP, generate a data warehouse connection file name, and write the data warehouse connection file name into the ETL public variable configuration file; Read the preset ETL deployment configuration file, obtain the ETL server IP, and write the ETL server IP into the ETL public variable configuration file; Read the preset ETL deployment configuration file, obtain the ETL main directory, generate the ETL source data receiving directory by string concatenation, and write the ETL source data receiving directory into the FTP configuration file.

5. The automatic adaptation method for ETL software based on the trust creation environment according to claim 4 is characterized in that: Before reading the preset ETL deployment configuration file, obtaining the data warehouse installation directory, and writing the data warehouse installation directory into the user environment configuration file, the method further includes: Back up the user environment configuration file.

6. The ETL software automatic adaptation method based on the trust creation environment according to claim 1 is characterized in that: The fourth adaptation execution script is called to generate the data warehouse connection encryption file and the ETL database connection encryption file according to the preset ETL deployment configuration file, including: Read the preset ETL deployment configuration file, obtain the data warehouse name, data warehouse user name and data warehouse password respectively, and generate the data warehouse connection encryption file; Read the preset ETL deployment configuration file, obtain the ETL database name, ETL database user name and ETL database password respectively, and generate the ETL database connection encryption file.

7. The ETL software automatic adaptation method based on the trust creation environment according to claim 1 is characterized in that: The calling of the fifth adaptation execution script, verifying the ETL process, and completing the automatic adaptation includes: Start the ETL process and gpfdist process; Verify the ETL process and complete automatic adaptation.

8. The ETL software automatic adaptation device based on the trust creation environment is characterized by: include: A preparation unit, configured to receive an adaptation resource and execute an adaptation scheduling script; wherein the adaptation resource includes an adaptation script, a preset external parameter file, and an ETL installation medium; the adaptation script includes an adaptation scheduling script and an adaptation execution script; the preset external parameter file includes an ETL deployment configuration file and an adaptation execution script sequence number file; the adaptation execution script sequence number file is used to record the sequence number of the next adaptation execution script; A checking unit, used to call the first adaptation execution script to check the server environment in which the ETL software runs; An installation unit is used to call the second adaptation execution script, decompress the ETL installation medium, and install the ETL software dependency package; A configuration unit, used to call a third adaptation execution script and modify a system configuration file according to a preset ETL deployment configuration file; A generating unit, used to call a fourth adaptation execution script, and generate a data warehouse connection encryption file and an ETL database connection encryption file respectively according to a preset ETL deployment configuration file; The startup unit is used to call the fifth adaptation execution script, verify the ETL process, and complete the automatic adaptation.

9. A computer-readable storage medium having computer instructions stored thereon, characterized in that: When the computer instructions are executed, the automatic adaptation method of ETL software based on the trusted innovation environment as described in any one of claims 1 to 7 can be implemented.

10. A device comprising a memory and a processor, wherein the memory stores computer instructions that can be executed on the processor, characterized in that: When the processor runs the computer instructions, it executes the ETL software automatic adaptation method based on the trusted innovation environment as described in any one of claims 1 to 7.

Images