Data Migration Method, Device, Equipment and Storage Medium

Abstract

This application relates to the field of artificial intelligence technology, and discloses a data migration method, device, equipment and storage medium. The method includes: obtaining a data migration request, determining input parameter configuration, storage configuration and JSON format conversion configuration according to the data migration request and a preset data migration configuration library; performing message encryption on the input parameter configuration by using a preset message encryption function to obtain a message encryption result; sending a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and obtaining a return result sent by the unified data service platform according to the target Post request; performing format conversion on the return result according to the JSON format conversion configuration to obtain target data; and performing storage processing on the target data according to the storage configuration. Thus, for different data migrations, there is no need to code separately, and there is no need for developers to be deeply involved, which improves the low efficiency of data migration.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a data migration method, apparatus, device, and storage medium. Background Art

[0002] With the continuous growth and expansion of the business of the data middle platform, the demand for data migration with business systems is also increasing. And data migration needs to be accessed or output by calling APIs (interfaces). How to call APIs quickly, efficiently, and securely becomes particularly important. Calling an API can send a Post request through coding in Java (an object-oriented programming language) or Python (a computer programming language) to connect to a database to obtain data, and then write SQL (Structured Query Language) statements to insert the obtained data into the target table. Coding separately for different data migrations results in low data migration efficiency and difficult subsequent maintenance. Moreover, for those who do not understand Java or Python coding, there are certain difficulties in learning programming syntax, creating a project, and configuring the scheduling method in the project, resulting in the need for developers to be deeply involved in data migration, increasing the development cost. Summary of the Invention

[0003] The main purpose of this application is to provide a data migration method, apparatus, device, and storage medium, aiming to solve the technical problems in the existing data migration method that coding separately for different data migrations results in low data migration efficiency, difficult subsequent maintenance, the need for developers to be deeply involved, and increased development cost.

[0004] To achieve the above invention purpose, this application proposes a data migration method. The method is applicable to the Kettle tool, and the method includes:

[0005] Obtain a data migration request, and determine the input parameter configuration, storage configuration, and JSON format conversion configuration according to the data migration request and a preset data migration configuration library;

[0006] Use a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result;

[0007] Send a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result, and obtain the return result sent by the unified data service platform according to the target Post request;

[0008] Convert the format of the return result according to the JSON format conversion configuration to obtain target data;

[0009] Perform storage processing on the target data according to the storage configuration.

[0010] Further, the step of encrypting the input parameter configuration by using a preset message encryption function to obtain a message encryption result includes:

[0011] Using the message encryption function to encrypt the interface address, interface key, interface identifier, and interface name in the input parameter configuration to obtain a Java interface signature, a timestamp, and a globally unique identifier;

[0012] Taking the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result.

[0013] Further, before the step of using the unified data service platform with the Oneservice platform and encrypting the input parameter configuration by using a preset message encryption function to obtain a message encryption result, the following steps are also included:

[0014] Obtaining the SHA256 encryption function code;

[0015] Adding the SHA256 encryption function code to a preset JavaScript component to obtain the message encryption function.

[0016] Further, the step of sending a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result includes:

[0017] Using the RestClient component to generate a Post request with the message encryption result, the interface identifier, and the interface address in the input parameter configuration as the target Post request, and sending the target Post request to the unified data service platform.

[0018] Further, before the step of obtaining the data migration request, the following steps are also included:

[0019] Obtaining input parameter data through a constant control;

[0020] Obtaining JSON format conversion data through a JSON input control;

[0021] Obtaining warehousing data through a table output control;

[0022] Generating associated data according to the input parameter data, the JSON format conversion data, and the warehousing data;

[0023] Storing the associated data in the data migration configuration library.

[0024] Further, the step of obtaining input parameter data through a constant control includes:

[0025] Obtain a query statement through a table input control, execute the query statement, and obtain an interface parameter specification set from a preset input parameter table;

[0026] Display the interface parameter specification set in a configuration interface;

[0027] Obtain a constant control call request in the configuration interface;

[0028] Respond to the constant control call request and display the constant control;

[0029] Obtain the input parameter data through the constant control.

[0030] Furthermore, the step of performing warehousing processing on the target data according to the warehousing configuration includes:

[0031] Write the target data into a target database corresponding to the database link address in the warehousing configuration according to the detailed warehousing data in the warehousing configuration;

[0032] Query from the target database according to the result query data in the warehousing configuration to obtain a query result;

[0033] Display the query result in a table.

[0034] This application also proposes a data migration device, and the device includes:

[0035] A request acquisition module, configured to acquire a data migration request, and determine an input parameter configuration, a warehousing configuration, and a JSON format conversion configuration according to the data migration request and a preset data migration configuration library;

[0036] A message encryption result determination module, configured to encrypt the input parameter configuration with a preset message encryption function to obtain a message encryption result;

[0037] A return result determination module, configured to send a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and acquire a return result sent by the unified data service platform according to the target Post request;

[0038] A target data determination module, configured to perform format conversion on the return result according to the JSON format conversion configuration to obtain target data;

[0039] A warehousing processing module, configured to perform warehousing processing on the target data according to the warehousing configuration.

[0040] The present application also provides a computer device, including a memory and a processor. The memory stores a computer program, and when the processor executes the computer program, the steps of any one of the above methods are implemented.

[0041] The present application also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of any one of the above methods are implemented.

[0042] The data migration method, device, equipment and storage medium of the present application. The method obtains a data migration request, and determines input parameter configuration, storage configuration and JSON format conversion configuration according to the data migration request and a preset data migration configuration library; uses a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result; sends a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and obtains a return result sent by the unified data service platform according to the target Post request; performs format conversion on the return result according to the JSON format conversion configuration to obtain target data; performs storage processing on the target data according to the storage configuration. By performing data migration based on a data migration configuration library, there is no need to code separately for different data migrations, and there is no need for developers to be deeply involved, which improves the efficiency of data migration and reduces the development cost; by encrypting the input parameter configuration to generate a Post request, the security of data migration is improved, making the present application applicable to application scenarios with high requirements for data security. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is a schematic flowchart of the data migration method according to an embodiment of the present application;

[0044] Figure 2 It is a schematic block diagram of the structure of the data migration device according to an embodiment of the present application;

[0045] Figure 3 It is a schematic block diagram of the structure of the computer device according to an embodiment of the present application.

[0046] The implementation, functional features and advantages of the purpose of the present application will be further described in conjunction with the embodiments with reference to the drawings. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0047] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application, and are not used to limit the present application.

[0048] Refer to Figure 1, in an embodiment of the present application, a data migration method is provided. The method is applicable to the Kettle tool, and the method includes:

[0049] S1: Obtain a data migration request, and determine input parameter configuration, storage configuration, and JSON format conversion configuration according to the data migration request and a preset data migration configuration library;

[0050] S2: Use a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result;

[0051] S3: Send a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result, and obtain a return result sent by the unified data service platform according to the target Post request;

[0052] S4: Perform format conversion on the return result according to the JSON format conversion configuration to obtain target data;

[0053] S5: Perform storage processing on the target data according to the storage configuration.

[0054] In this embodiment, data migration is performed based on a data migration configuration library. For different data migrations, there is no need to code separately and no need for developers to be deeply involved, which improves the efficiency of data migration and reduces the development cost; by encrypting the input parameter configuration to generate a Post request, the security of data migration is improved, making the present application applicable to application scenarios with high requirements for data security.

[0055] For S1, the data migration request input by the user can be obtained in the interface of the Kettle tool, or the data migration request sent by a third-party application can be obtained.

[0056] The Kettle tool, a commonly used ETL (Extract-Transform-Load) tool, is responsible for receiving the extracted data, performing transformation, and then loading it to the destination.

[0057] The data migration request is a request to migrate the data in the data source registered in the unified data service platform.

[0058] Optionally, according to the data migration configuration identifier carried in the data migration request, obtain input parameter data, storage data, and JSON format conversion data from the data migration configuration library, use the obtained input parameter data as the input parameter configuration, use the obtained storage data as the storage configuration, and use the obtained JSON format conversion data as the JSON format conversion configuration.

[0059] The data migration configuration library includes: a data migration configuration identifier and associated data. The associated data includes: input parameter data, data to be stored in the database, and JSON format conversion data. The data migration configuration identifier can be a configuration name, a configuration ID, or other unique identifier for an associated data item.

[0060] The input parameter configuration includes: an interface address, an interface key, an interface identifier, and an interface name. The interface address is the URL address (network address) of the interface. The interface key is the appkey. The interface identifier is the appid. The interface name is the appname. The interface address, interface key, and interface identifier are generated by the unified data service platform. The interface name is pre-set in the unified data service platform. The interface identifier can be an interface ID, which is used to uniquely identify an interface. The interface key is the key required to call the interface.

[0061] The database storage configuration includes: a database connection address, a target table name, and the owner of the table (the schema of the database). The database connection address is the connection address of the database into which the data is to be migrated. The target table name is the name of the data table into which the data is to be migrated. The owner of the table is the owner of the data table into which the data is to be migrated.

[0062] Optionally, the database storage configuration further includes: a trimmed table configuration and a specified database field configuration. The trimmed table configuration has only one value, and the value range of the trimmed table configuration includes yes or no; if the trimmed table configuration is yes, first clear the data in the data table into which the data is to be migrated, and then insert the data into the cleared data table; if the trimmed table configuration is no, directly insert the data into the data table into which the data is to be migrated. The specified database field configuration has only one value, and the value range of the specified database field configuration includes yes or no; if the specified database field configuration is yes, the database storage configuration further includes: a table field configuration and a stream field configuration.

[0063] The JSON format conversion configuration includes: a field acquisition source configuration. The field acquisition source configuration is the configuration data of the field that needs to be format-converted.

[0064] The unified data service platform is used for registering various data sources and then providing interface call services to applications. By calling the interfaces of the unified data service platform, an application can obtain the data in the registered data sources.

[0065] Optionally, the unified data service platform adopts the Oneservice platform. Oneservice refers to the unified data service, that is, the data middle platform provides unified data access and data query services.

[0066] For S2, use a preset message encryption function to encrypt the input parameter configuration, and use the encrypted data as the message encryption result, thereby obtaining the encrypted input parameters.

[0067] For S3, generate a Post request according to the input parameter configuration and the message encryption result, use the generated Post request as the target Post request, and send the target Post request to the unified data service platform.

[0068] Post request, a request for transmitting data to the unified data service platform.

[0069] The unified data service platform calls the data source to obtain data according to the target Post request, and uses the obtained data as the return result.

[0070] Among them, the parameters in the Body field (request body) of the target Post request are all passed from the input parameter configuration and the message encryption result. The Headers (request headers) of the target Post request include the interface identifier in the input parameter configuration and the message encryption result.

[0071] For S4, according to the JSON format conversion configuration, convert the return result to the specified format in the JSON format conversion configuration in JSON format, and use the converted data as the target data.

[0072] For S5, according to the storage configuration, store the target data into the target database corresponding to the storage configuration. Thus, the data in the data source corresponding to the input parameter configuration is migrated to the target database.

[0073] In one embodiment, the step of using a preset message encryption function to perform message encryption on the input parameter configuration to obtain a message encryption result includes:

[0074] S21: Use the message encryption function to perform message encryption on the interface address, interface key, interface identifier, and interface name in the input parameter configuration to obtain a Java interface signature, a timestamp, and a globally unique identifier;

[0075] S22: Use the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result.

[0076] This embodiment realizes message encryption of the interface address, interface key, interface identifier, and interface name in the input parameter configuration, thereby improving the security of interface calls and making this application suitable for application scenarios with high requirements for data security.

[0077] For S21, the message encryption function is adopted to encrypt the interface address, interface key, interface identifier, and interface name in the input parameter configuration to obtain a message digest, where the message digest includes: Java interface signature, timestamp, and globally unique identifier.

[0078] For S22, the Java interface signature, the timestamp, and the globally unique identifier are used as the message encryption result, thereby obtaining the encryption result of the input parameter configuration.

[0079] In one embodiment, the above unified data service platform adopts the Oneservice platform. Before the step of encrypting the input parameter configuration with the preset message encryption function to obtain the message encryption result, it further includes:

[0080] S021: Obtain the SHA256 encryption function code;

[0081] S022: Add the SHA256 encryption function code to the preset JavaScript component to obtain the message encryption function.

[0082] This embodiment realizes adding custom SHA256 encryption function code to the JavaScript component based on the Kettle tool, providing a basis for subsequent SHA256 encryption of the input parameter configuration using the message encryption function; since SHA256 is currently the most popular secure encryption algorithm, it will generate a 256-bit hash value (i.e., message digest) for messages of any length, and this algorithm is difficult to crack and has higher security, thus further improving the security of interface calls.

[0083] For S021, obtain the SHA256 encryption function code input by the user.

[0084] The hash value length used by the SHA256 encryption function is 256 bits. This is an abstract class, and the only implementation of this abstract class is SHA256Managed.

[0085] The SHA256 encryption function code is the JavaScript code that implements the SHA256 encryption function.

[0086] For S022, add the SHA256 encryption function code to the JavaScript component of the Kettle tool, and use the added SHA256 encryption function code as the message encryption function.

[0087] JavaScript (abbreviation: "JS") is a lightweight, interpreted or just-in-time compiled programming language with function priority.

[0088] It is understandable that when other platforms are adopted by the unified data service platform, other encryption algorithms can also be used for the message encryption function.

[0089] In one embodiment, the step of sending a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result includes:

[0090] S31: Using the RestClient component, generate a Post request with the message encryption result, the interface identifier, and the interface address in the input parameter configuration as the target Post request, and send the target Post request to the unified data service platform.

[0091] In this embodiment, by using the RestClient component of the Kettle tool, a Post request is generated with the message encryption result, the interface identifier, and the interface address in the input parameter configuration, thereby realizing the transmission of the encrypted result of the input parameter configuration to the unified data service platform, improving the security of data migration, and making this application suitable for application scenarios with high requirements for data security.

[0092] For S31, using the RestClient component of the Kettle tool, the message encryption result, the interface identifier, and the interface address in the input parameter configuration are used as parameters of the Post request, the generated Post request is used as the target Post request, and the target Post request is sent to the unified data service platform. Thus, the encrypted result of the input parameter configuration is transmitted to the unified data service platform.

[0093] The RestClient component is used to simulate sending requests.

[0094] In one embodiment, before the step of obtaining the data migration request, the following steps are further included:

[0095] S11: Obtain input parameter data through a constant control;

[0096] S12: Obtain JSON format conversion data through a JSON input control;

[0097] S13: Obtain warehousing data through a table output control;

[0098] S14: Generate associated data according to the input parameter data, the JSON format conversion data, and the warehousing data;

[0099] S15: Store the associated data in the data migration configuration library.

[0100] In this embodiment, data migration configuration is achieved through the controls of the Kettle tool in a graphical interface. Configuration personnel do not need to understand Java or Python coding, which reduces the professionalism of configuration operations. As a result, data migration does not require in-depth participation of developers, reducing development costs.

[0101] For S11, through the constant control of the Kettle tool, the input parameter data entered by the user is obtained.

[0102] The input parameter data includes: interface address, interface key, interface identifier, and interface name.

[0103] For S12, through the JSON input control of the Kettle tool, the JSON format conversion data entered by the user is obtained.

[0104] The JSON format conversion data includes: field acquisition source configuration.

[0105] For S13, through the table output control of the Kettle tool, the data to be stored in the database entered by the user is obtained.

[0106] The data to be stored in the database includes: database connection address, target table name, and table owner.

[0107] For S14, based on the input parameter data, the JSON format conversion data, and the data to be stored in the database, associated data is generated. It can be understood that when generating the associated data, a data migration configuration identifier will be automatically generated, and the data migration configuration identifier will be carried as the data of the associated data.

[0108] For S15, the associated data is stored in the data migration configuration library, thus providing a basis for subsequent response to data migration requests.

[0109] In one embodiment, the step of obtaining the input parameter data through the constant control includes:

[0110] S151: Through the table input control, obtain a query statement, execute the query statement, and obtain an interface input parameter specification set from a preset input parameter table;

[0111] S152: Display the interface input parameter specification set on the configuration interface;

[0112] S153: In the configuration interface, obtain a constant control call request;

[0113] S154: Respond to the constant control call request and display the constant control;

[0114] S155: Through the constant control, obtain the input parameter data.

[0115] In this embodiment, the interface input parameter specification set is obtained from the input parameter table through the table input control, and the input parameter data is obtained based on the interface input parameter specification set, further reducing the professionalism of the configuration operation.

[0116] For S151, through the table input control of the Kettle tool, the query statement configured by the user is obtained, the query statement is executed, and the interface input parameter specification set is obtained from the preset input parameter table, so as to query the input parameter specification of the interface to be called.

[0117] The interface input parameter specification set includes: specification identifier and interface input parameter specification. The specification identifier can be a unique identifier such as a specification name, specification ID, etc. for an interface input parameter specification. The interface input parameter specification includes: interface name and interface input parameters. Interface input parameters are the parameters that need to be provided when calling the interface. For example, the interface input parameters include: interface address, interface key, interface identifier, and interface name.

[0118] For S152, the interface input parameter specification set is displayed on the interface, and the displayed interface is used as the configuration interface.

[0119] For S153, in the configuration interface, the constant control call request input by the user is obtained, where the constant control call request carries the specification identifier.

[0120] The constant control call request is a request to call the constant control.

[0121] For S154, in response to the constant control call request, the constant control is displayed, and the displayed constant control contains the interface input parameter specification corresponding to the specification identifier carried by the constant control call request.

[0122] For S155, through the displayed constant control, the input parameter data input by the user is obtained.

[0123] In one embodiment, the step of performing the warehousing process on the target data according to the warehousing configuration includes:

[0124] S51: Write the target data into the target database corresponding to the database link address in the warehousing configuration according to the detailed warehousing data in the warehousing configuration;

[0125] S52: Query from the target database according to the result query data in the warehousing configuration to obtain a query result;

[0126] S53: Display the query result in a table.

[0127] In this embodiment, the target data is processed for storage through storage configuration, and then the query result data is queried and presented in a table, reducing the professionalism of storage inspection and eliminating the need for developers to participate in storage inspection.

[0128] For S51, the detailed storage data includes: the target table name and the owner of the table.

[0129] Optionally, the detailed storage data further includes: cropped table configuration and specified database field configuration.

[0130] According to the detailed storage data in the storage configuration, the target data is written into the target database corresponding to the database link address in the storage configuration, thereby realizing the migration of the data in the data source corresponding to the input parameter configuration to the target database.

[0131] For S52, the query result data includes: the target table name and the owner of the table.

[0132] Optionally, the query result data further includes: specified database field configuration.

[0133] For S53, the query result is presented in a table on the interface. Through the table on the interface, users can quickly check whether the stored data is incorrect.

[0134] Referring to Figure 2 , this application also proposes a data migration device, which includes:

[0135] A request acquisition module 100, configured to acquire a data migration request, and determine an input parameter configuration, a storage configuration, and a JSON format conversion configuration according to the data migration request and a preset data migration configuration library;

[0136] A message encryption result determination module 200, configured to encrypt the input parameter configuration using a preset message encryption function to obtain a message encryption result;

[0137] A return result determination module 300, configured to send a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and acquire the return result sent by the unified data service platform according to the target Post request;

[0138] A target data determination module 400, configured to perform format conversion on the return result according to the JSON format conversion configuration to obtain target data;

[0139] A storage processing module 500, configured to perform storage processing on the target data according to the storage configuration.

[0140] In this embodiment, data migration is performed based on the data migration configuration library. For different data migrations, there is no need for separate coding and in-depth participation of developers, which improves the efficiency of data migration and reduces the development cost. By encrypting the input parameter configuration to generate a Post request, the security of data migration is improved, making this application suitable for application scenarios with high requirements for data security.

[0141] In one embodiment, the above-mentioned message encryption result determination module 200 includes: a message encryption sub-module;

[0142] The message encryption sub-module is used to encrypt the interface address, interface key, interface identifier, and interface name in the input parameter configuration by using the message encryption function to obtain a Java interface signature, a timestamp, and a globally unique identifier, and use the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result.

[0143] In one embodiment, the above-mentioned device further includes: a message encryption function generation module;

[0144] The message encryption function generation module is used to obtain the SHA256 encryption function code and add the SHA256 encryption function code to a preset JavaScript component to obtain the message encryption function.

[0145] In one embodiment, the above-mentioned return result determination module 300 includes: a Post request generation and sending sub-module;

[0146] The Post request generation and sending sub-module is used to generate a Post request by using the RestClient component with the message encryption result, the interface identifier, and the interface address in the input parameter configuration as the target Post request, and send the target Post request to the unified data service platform.

[0147] In one embodiment, the above-mentioned device further includes: a configuration module;

[0148] The configuration module is used to obtain input parameter data through a constant control, obtain JSON format conversion data through a JSON input control, obtain warehousing data through a table output control, generate associated data according to the input parameter data, the JSON format conversion data, and the warehousing data, and store the associated data in the data migration configuration library.

[0149] In one embodiment, the above-mentioned configuration module includes: an interface input parameter specification set acquisition sub-module and an input parameter data acquisition sub-module;

[0150] The interface input parameter specification set acquisition sub-module is used to obtain a query statement through a table input control, execute the query statement, obtain an interface input parameter specification set from a preset input parameter table, and display the interface input parameter specification set in a configuration interface;

[0151] The input parameter data acquisition sub-module is used to obtain a constant control call request in the configuration interface, respond to the constant control call request, display the constant control, and obtain the input parameter data through the constant control.

[0152] In one embodiment, the above-mentioned warehousing processing module 500 includes: a data writing sub-module and a query result display sub-module;

[0153] The data writing sub-module is used to write the target data into a target database corresponding to the database link address in the warehousing configuration according to the detailed warehousing data in the warehousing configuration;

[0154] The query result display sub-module is used to query from the target database according to the result query data in the warehousing configuration to obtain a query result, and display the query result in a table.

[0155] Refer to Figure 3 , in the embodiment of the present application, a computer device is further provided. The computer device may be a server, and its internal structure may be as Figure 3 shown. The computer device includes a processor, a memory, a network interface, and a database connected through a system bus. Among them, the processor of the computer design is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program, and a database. The memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store data such as data migration methods. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, it realizes a data migration method. The data migration method includes: obtaining a data migration request, and determining an input parameter configuration, a warehousing configuration, and a JSON format conversion configuration according to the data migration request and a preset data migration configuration library; using a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result; sending a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and obtaining a return result sent by the unified data service platform according to the target Post request; performing format conversion on the return result according to the JSON format conversion configuration to obtain target data; performing warehousing processing on the target data according to the warehousing configuration.

[0156] In this embodiment, data migration is performed based on a data migration configuration library. For different data migrations, there is no need for separate coding and in-depth participation of developers, which improves the efficiency of data migration and reduces the development cost. By encrypting the input parameter configuration to generate a Post request, the security of data migration is improved, making this application suitable for application scenarios with high requirements for data security.

[0157] In one embodiment, the step of encrypting the input parameter configuration with a preset message encryption function to obtain a message encryption result includes:

[0158] Using the message encryption function to encrypt the interface address, interface key, interface identifier, and interface name in the input parameter configuration to obtain a Java interface signature, a timestamp, and a globally unique identifier;

[0159] Taking the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result.

[0160] In one embodiment, before the step of encrypting the input parameter configuration with a preset message encryption function to obtain a message encryption result, the above unified data service platform adopts the Oneservice platform, and further includes:

[0161] Obtaining the SHA256 encryption function code;

[0162] Adding the SHA256 encryption function code to a preset JavaScript component to obtain the message encryption function.

[0163] In one embodiment, the step of sending a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result includes:

[0164] Using the RestClient component to generate a Post request with the message encryption result, the interface identifier, and the interface address in the input parameter configuration as the target Post request, and sending the target Post request to the unified data service platform.

[0165] In one embodiment, before the step of obtaining a data migration request, it further includes:

[0166] Obtaining input parameter data through a constant control;

[0167] Obtaining JSON format conversion data through a JSON input control;

[0168] Obtaining warehousing data through a table output control;

[0169] Generate associated data based on the input parameter data, the JSON format conversion data, and the data to be stored in the database;

[0170] Store the associated data in the data migration configuration library.

[0171] In one embodiment, the step of obtaining input parameter data through a constant control includes:

[0172] Obtain a query statement through a table input control, execute the query statement, and obtain an interface input parameter specification set from a preset input parameter table;

[0173] Display the interface input parameter specification set in a configuration interface;

[0174] Obtain a constant control call request in the configuration interface;

[0175] Respond to the constant control call request and display the constant control;

[0176] Obtain the input parameter data through the constant control.

[0177] In one embodiment, the step of performing database storage processing on the target data according to the database storage configuration includes:

[0178] Write the target data into a target database corresponding to the database link address in the database storage configuration according to the detailed database storage data in the database storage configuration;

[0179] Query the target database according to the result query data in the database storage configuration to obtain a query result;

[0180] Display the query result in a table.

[0181] One embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, a data migration method is implemented, including the steps of: obtaining a data migration request, and determining an input parameter configuration, a database storage configuration, and a JSON format conversion configuration according to the data migration request and a preset data migration configuration library; using a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result; sending a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and obtaining a return result sent by the unified data service platform according to the target Post request; performing format conversion on the return result according to the JSON format conversion configuration to obtain target data; performing database storage processing on the target data according to the database storage configuration.

[0182] The above-mentioned data migration method, by performing data migration based on the data migration configuration library, does not require separate coding for different data migrations and does not require in-depth participation of developers, which improves the efficiency of data migration and reduces the development cost; by encrypting the input parameter configuration and then generating a Post request, it improves the security of data migration and makes this application suitable for application scenarios with high requirements for data security.

[0183] In one embodiment, the step of encrypting the input parameter configuration with a preset message encryption function to obtain a message encryption result includes:

[0184] Using the message encryption function to encrypt the interface address, interface key, interface identifier, and interface name in the input parameter configuration to obtain a Java interface signature, a timestamp, and a globally unique identifier;

[0185] Taking the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result.

[0186] In one embodiment, before the step of encrypting the input parameter configuration with a preset message encryption function to obtain a message encryption result, the above-mentioned unified data service platform uses the Oneservice platform, and further includes:

[0187] Obtaining the SHA256 encryption function code;

[0188] Adding the SHA256 encryption function code to a preset JavaScript component to obtain the message encryption function.

[0189] In one embodiment, the step of sending a target Post request to the unified data service platform according to the input parameter configuration and the message encryption result includes:

[0190] Using the RestClient component to generate a Post request with the message encryption result and the interface identifier and interface address in the input parameter configuration as the target Post request, and sending the target Post request to the unified data service platform.

[0191] In one embodiment, before the step of obtaining a data migration request, it further includes:

[0192] Obtaining input parameter data through a constant control;

[0193] Obtaining JSON format conversion data through a JSON input control;

[0194] Obtaining warehousing data through a table output control;

[0195] Generate associated data based on the input parameter data, the JSON format conversion data, and the data to be stored in the database;

[0196] Store the associated data in the data migration configuration database.

[0197] In one embodiment, the step of obtaining input parameter data through a constant control includes:

[0198] Obtain a query statement through a table input control, execute the query statement, and obtain an interface input parameter specification set from a preset input parameter table;

[0199] Display the interface input parameter specification set in a configuration interface;

[0200] Obtain a constant control call request in the configuration interface;

[0201] Respond to the constant control call request and display the constant control;

[0202] Obtain the input parameter data through the constant control.

[0203] In one embodiment, the step of performing database storage processing on the target data according to the database storage configuration includes:

[0204] Write the target data into a target database corresponding to the database link address in the database storage configuration according to the detailed database storage data in the database storage configuration;

[0205] Query from the target database according to the result query data in the database storage configuration to obtain a query result;

[0206] Display the query result in a table.

[0207] Those of ordinary skill in the art can understand that all or part of the processes in the above-described method embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-described method embodiments. Among them, any reference to a memory, storage, database, or other medium provided in this application and used in the embodiments can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0208] It should be noted that in this article, the terms "include", "comprise", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, apparatus, article, or method that includes a series of elements includes not only those elements but also other elements not expressly listed, or elements that are inherent to such process, apparatus, article, or method. Without further limitation, an element defined by the phrase "including one..." does not exclude the existence of additional identical elements in the process, apparatus, article, or method that includes such element.

[0209] The above are only the preferred embodiments of this application, and do not limit the patent scope of this application accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of this application, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of this application.

Claims

1. A data migration method, characterized in that, The method is applicable to the Kettle tool, and the method includes: Obtain a data migration request, and determine the input parameter configuration, the storage configuration, and the JSON format conversion configuration according to the data migration request and a preset data migration configuration library; Use a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result; Send a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and obtain a return result sent by the unified data service platform according to the target Post request; Convert the format of the return result according to the JSON format conversion configuration to obtain target data; Perform storage processing on the target data according to the storage configuration; The step of using a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result includes: Use the message encryption function to encrypt the interface address, the interface key, the interface identifier, and the interface name in the input parameter configuration to obtain a Java interface signature, a timestamp, and a globally unique identifier; Use the Java interface signature, the timestamp, and the globally unique identifier as the message encryption result; The unified data service platform uses the Oneservice platform. Before the step of using a preset message encryption function to encrypt the input parameter configuration to obtain a message encryption result, it further includes: Obtain the SHA256 encryption function code; Add the SHA256 encryption function code to a preset JavaScript component to obtain the message encryption function; The step of sending a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result includes: Use the RestClient component to generate a Post request from the message encryption result, the interface identifier, and the interface address in the input parameter configuration as the target Post request, and send the target Post request to the unified data service platform; Before the step of obtaining the data migration request, it further includes: Obtain input parameter data through a constant control; Obtain JSON format conversion data through a JSON input control; Obtain storage data through a table output control; Generate associated data according to the input parameter data, the JSON format conversion data, and the storage data; Store the associated data in the data migration configuration library; The step of obtaining input parameter data through a constant control includes: Obtain a query statement through a table input control, execute the query statement, and obtain an interface input parameter specification set from a preset input parameter table; Display the interface input parameter specification set in a configuration interface; In the configuration interface, obtain a constant control call request; Respond to the constant control call request and display the constant control; Obtain the input parameter data through the constant control.

2. The data migration method according to claim 1, wherein The step of performing storage processing on the target data according to the storage configuration includes: Write the target data into the target database corresponding to the database link address in the warehousing configuration according to the detailed warehousing data in the warehousing configuration; Query the target database according to the result query data in the warehousing configuration to obtain a query result; Present the query result in a table.

3. A data migration device, characterized in that, For implementing the data migration method according to any one of claims 1 to 2, the device includes: A request acquisition module, configured to acquire a data migration request, and determine an input parameter configuration, a warehousing configuration, and a JSON format conversion configuration according to the data migration request and a preset data migration configuration library; A message encryption result determination module, configured to encrypt the input parameter configuration using a preset message encryption function to obtain a message encryption result; A return result determination module, configured to send a target Post request to a unified data service platform according to the input parameter configuration and the message encryption result, and acquire a return result sent by the unified data service platform according to the target Post request; A target data determination module, configured to perform format conversion on the return result according to the JSON format conversion configuration to obtain target data; A warehousing processing module, configured to perform warehousing processing on the target data according to the warehousing configuration.

4. A computer device, comprising a memory and a processor, the memory storing a computer program, characterized in that, When the processor executes the computer program, the steps of the method according to any one of claims 1 to 2 are implemented.

5. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 2 are implemented.

Images