Data migration method and device, model training method and device, equipment, medium and product

Abstract

The invention provides a data migration method and device, a model training method and device, equipment, a medium and a product. The data migration method comprises the steps of determining a first script of first data in a first database and an identifier of a second database; generating a prompt word of the first data according to the first script; generating a second script of the first data in the second database according to the identifier of the second database and the cue word of the first data; and migrating the first data from the first database to the second database according to the second script. According to the embodiment of the invention, the complexity of data migration can be reduced.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a data migration method, a model training method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] Data migration refers to the process of migrating data from one database to another. The specific codes of different types of databases vary greatly, and the data migration operation in related technologies is highly complex. Summary of the invention

[0003] The present disclosure provides a data migration method, a model training method and device, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] In a first aspect, the present disclosure provides a data migration method, the data migration method comprising:

[0005] Determine a first script of the first data in the first database and an identifier of the second database; generate a prompt word for the first data according to the first script; generate a second script of the first data in the second database according to the identifier of the second database and the prompt word for the first data; and migrate the first data from the first database to the second database according to the second script.

[0006] In a second aspect, the present disclosure provides a model training method, the model training method comprising:

[0007] The third script of the sample data in the sample database is identified by the first model to obtain a prompt word of the sample data; the sample database includes multiple types of databases; the identifier of the sample database and the prompt word of the sample data are script generated by the first model to obtain a fourth script of the sample data; the first model is trained according to the third script and the fourth script; wherein the first model is used to implement the above-mentioned data migration method.

[0008] In a third aspect, the present disclosure provides a data migration device, the data migration device comprising:

[0009] A determination module, used to determine a first script of the first data in the first database and an identifier of the second database;

[0010] A prompt word generating module, used for generating a prompt word of the first data according to the first script;

[0011] A script generation module, used for generating a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data;

[0012] A migration module is used to migrate the first data from the first database to the second database according to the second script.

[0013] In a fourth aspect, the present disclosure provides a model training device, the model training device comprising:

[0014] A sample script recognition module, used to perform script recognition on a third script of the sample data in a sample database by using a first model to obtain a prompt word of the sample data; the sample database includes multiple types of databases;

[0015] A sample script generation module, used for generating a script based on the identifier of the sample database and the prompt word of the sample data by using the first model to obtain a fourth script of the sample data;

[0016] A training module is used to train the first model according to the third script and the fourth script; wherein the first model is used to implement the above-mentioned data migration device.

[0017] In a fifth aspect, the present disclosure provides an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores one or more computer programs executable by the at least one processor, and one or more of the computer programs are executed by the at least one processor so that the at least one processor can execute the above-mentioned data migration method, or execute the above-mentioned model training method.

[0018] In a sixth aspect, the present disclosure provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program implements the above-mentioned data migration method or the above-mentioned model training method when executed by a processor.

[0019] In the seventh aspect, the present disclosure provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned data migration method, or executes the above-mentioned model training method.

[0020] According to the data migration method provided by the embodiment of the present disclosure, a prompt word for the data can be generated according to a script of the data in the first database, thereby eliminating the need to learn the specific code of the database and directly generating a second script for the data in the second database according to the prompt word and the identifier of the second database; and then migrating the data from the first database to the second database according to the second script, thereby reducing the learning cost of the second database, reducing the complexity of data migration, and improving the efficiency of data migration.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The accompanying drawings are used to provide a further understanding of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the present disclosure and do not constitute a limitation of the present disclosure. The above and other features and advantages will become more apparent to those skilled in the art by describing detailed example embodiments with reference to the accompanying drawings, in which:

[0023] Figure 1 A schematic diagram of implementing data migration using a migration tool of a target database provided in the related art;

[0024] Figure 2 A schematic diagram of using metadata to achieve data migration provided in the related art;

[0025] Figure 3 A flowchart of a data migration method provided by an embodiment of the present disclosure;

[0026] Figure 4 A schematic diagram of a training process of a data migration method provided in an embodiment of the present disclosure;

[0027] Figure 5 A schematic diagram of a data migration process provided by an embodiment of the present disclosure;

[0028] Figure 6 A schematic diagram of a model training method provided in an embodiment of the present disclosure;

[0029] Figure 7 A block diagram of a data migration device provided by an embodiment of the present disclosure;

[0030] Figure 8 A block diagram of a model training device provided in an embodiment of the present disclosure;

[0031] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0032] In order to enable those skilled in the art to better understand the technical solutions of the present disclosure, the exemplary embodiments of the present disclosure are described below in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, the description of well-known functions and structures is omitted in the following description.

[0033] In the absence of conflict, the various embodiments of the present disclosure and the various features therein may be combined with each other.

[0034] As used herein, the term "and / or" includes any and all combinations of one or more of the associated listed items.

[0035] The terms used herein are only used to describe specific embodiments and are not intended to limit the present disclosure. As used herein, the singular forms "a" and "the" are also intended to include plural forms, unless the context clearly indicates otherwise. It will also be understood that when the terms "including" and / or "made of" are used in this specification, the presence of the features, wholes, steps, operations, elements and / or components is specified, but the presence or addition of one or more other features, wholes, steps, operations, elements, components and / or groups thereof is not excluded. "Connected" or "connected" and similar words are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect.

[0036] Unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as commonly understood by those of ordinary skill in the art. It will also be understood that terms such as those defined in commonly used dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant art and the present disclosure, and will not be interpreted as having an idealized or overly formal meaning unless explicitly defined as such herein.

[0037] Since the specific codes of different types of databases differ greatly, when the source database and the target database are of different types, the operation of migrating data from the source database to the target database in the related art is also relatively complicated.

[0038] The processing method of the related technology includes using the migration tool of the target database to migrate the data of the source database. Figure 1 A schematic diagram of a method of implementing data migration using a migration tool of a target database provided in the related art. Figure 1In the example, the database A migration tool is used to migrate the data of the source database to the target database A, and the database B migration tool is used to migrate the data of the source database to the target database B. However, this processing method relies on the database migration tool provided by the target database manufacturer, and it is necessary to learn the database migration tool provided by the target database manufacturer, which has a high learning cost.

[0039] The processing method of the related technology also includes extracting metadata of the source database and then using a data conversion tool to convert the metadata into a data table of the target database. Figure 2 A schematic diagram of using metadata to achieve data migration provided in the related art. Figure 2 In this method, metadata extraction tools are used to extract metadata from the source database, and then data conversion tools are used to convert metadata into data tables of target database A, and data conversion tools are used to convert metadata into data tables of target database B. However, in this processing method, both metadata extraction tools and data conversion tools need to be rigorously designed in advance and need to be able to work normally in various complex situations, which is highly complex.

[0040] In order to solve the problem of high complexity of data migration operations, the embodiments of the present disclosure provide a data migration method, which can generate a prompt word for the data according to a script of the data in a source database, thereby eliminating the need to learn the specific code of the database and directly generating a script for the data in a target database according to the prompt word and an identifier of the target database; and then migrating the data from the source database to the target database according to the script, thereby reducing the learning cost of the target database, reducing the complexity of data migration, and improving the efficiency of data migration.

[0041] The data migration method according to the embodiment of the present disclosure can be performed by an electronic device such as a terminal device or a server, and the terminal device can be a user equipment (User Equipment, UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory.

[0042] Figure 3 A flowchart of a data migration method provided by an embodiment of the present disclosure. Figure 3 , the method comprising:

[0043] Step 301: Determine a first script of first data in a first database and an identifier of a second database.

[0044] In some embodiments, the database where the first data to be migrated is located may be referred to as the first database, and the database as the migration target may be referred to as the second database. The scripts in the first database may be scripts for multiple scenarios, for example, they may be database table creation scripts, database synchronization scripts, SQL (Structured Query Language) query scripts, SQL update scripts, etc. The present disclosure does not limit the specific scenarios of the scripts in the first database. The present disclosure takes the first script of the first database as a database table creation script as an example for explanation.

[0045] In some embodiments, the identifier of the second database is used to characterize the type of the second database, for example, it can be any one of the postgresql database, the DAMO database, the Renmin University of China Jincang database and the MySQL database. The identifier of the second database can also be the identifier of other databases not listed, and the present disclosure does not impose any restrictions on this.

[0046] Step 302: Generate prompt words for first data according to the first script.

[0047] In some embodiments, the prompt words are used to represent information of the data table structure of the first database, and the prompt words corresponding to the same data table structure in different databases are the same. There can be multiple prompt words for the first data, which are represented in the form of a prompt word set.

[0048] In some embodiments, the prompt is an artificial intelligence (AI) prompt, which is a method of using natural language to guide or inspire an AI model to complete a specific task. Prompts are a new paradigm that has emerged in the field of natural language processing (NLP) in recent years, also known as "pretrain-prompt-predict".

[0049] In some embodiments, when the prompt word is represented in the form of a prompt word set, the prompt word set may be an ordered set of prompt words, and the ordered set of prompt words includes a plurality of prompt words arranged in a preset order.

[0050] In some embodiments, the data table structure includes a table name, field names, field types, field attributes, and a primary key, wherein the field attributes include field length, and the field attributes may also include other information, which are not listed here one by one.

[0051] For example, the first database is a MySQL database, and the first script of the first database is as follows:

[0052] create table `sample_table`(

[0053] `id`bigint not null auto_increment,

[0054] `code`varchar(64)not null,

[0055] `name`varchar(128),

[0056] primary key (`id`) )

[0058] The data table structure of the first database is as follows:

[0059] The data table sample_table has three fields:

[0060] Field id, auto-increment field;

[0061] Field code, string type, length 64 bytes, non-empty;

[0062] Field name, string type, length 128 bytes;

[0063] The field id is the primary key.

[0064] The first script of the first database is a database DDL (Data Definition Language) table creation script. The prompt words generated according to the first script of the first database are as follows:

[0065] Create a table, sample_table, with the following fields:

[0066] Self-increment primary key, id;

[0067] String type, code, length 64, not empty;

[0068] String type, name, length 128;

[0069] The table creation is complete.

[0070] The prompt words generated by the first script of the first database have nothing to do with the type of the first database. No matter whether the first database is a postgresql database, a DAMO database, a Renmin University of China Golden Warehouse database, or a MySQL database, or whether the first database is other databases not listed, the prompt words corresponding to the same data table structure are the same.

[0071] Step 303: Generate a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data.

[0072] In some embodiments, the identifier of the second database is, for example, a postgresql database, and the prompt words are expressed as follows:

[0073] Create a table, sample_table, with the following fields:

[0074] Self-increment primary key, id;

[0075] String type, code, length 64, not empty;

[0076] String type, name, length 128;

[0077] The table creation is complete.

[0078] According to the identifier of the second database and the prompt word of the first data, the second script of the generated first data in the second database is expressed as follows:

[0079] create table "sample_table"(

[0080] "id"bigserial not null,

[0081] "code"varchar(64)not null,

[0082] "name"varchar(128),

[0083] primary key("id") );

[0085] It should be understood that those skilled in the art may set the identifier of the second database according to actual conditions, and the present disclosure does not limit this.

[0086] Step 304: Migrate the first data from the first database to the second database according to the second script.

[0087] In some embodiments, after the second script is obtained, the second script may be stored in the second database, completing the migration process of the first data from the first database to the second database.

[0088] The data migration method provided by the embodiment of the present disclosure can generate a prompt word for the data according to a script of the data in the first database, thereby eliminating the need to learn the specific code of the database and directly generating a second script for the data in the second database according to the prompt word and the identifier of the second database; and then migrating the data from the first database to the second database according to the second script, thereby reducing the learning cost of the second database, reducing the complexity of data migration, and improving the efficiency of data migration.

[0089] The data migration method according to the embodiment of the present disclosure is described below.

[0090] As described above, the first script of the first data in the first database and the identifier of the second database may be determined in step 301 , and then the prompt word of the first data may be generated according to the first script in step 302 .

[0091] In some embodiments, step 302 includes:

[0092] Extracting a first parameter value corresponding to a preset character string from the first script;

[0093] Obtaining a prompt word template corresponding to the preset character string, wherein the prompt word templates corresponding to the preset character strings in different databases are the same;

[0094] Generate a prompt word for the first data according to the first parameter value and the prompt word template.

[0095] For example, the preset character string includes a character string used to represent the data table structure, and the data table structure includes the table name, the field names, field types, field attributes, and primary keys that make up the data table, etc. The contents of the same type of scripts in different databases vary greatly, but the preset character string used to represent the data table structure is the same, and the corresponding prompt word template is also the same.

[0096] In the example, for the table creation script in the script type, it is represented as follows in the postgresql database:

[0097] create table "training_table"(

[0098] "trainKey"bigserial not null,

[0099] "trainCode"varchar(32)not null,

[0100] "trainName"varchar(64),

[0101] primary key("trainKey"))

[0102] In the DAMO database, it is represented as:

[0103] create table "training_table"(

[0104] "trainKey"bigint identity,

[0105] "trainCode"varchar(32)not null,

[0106] "trainName"varchar(64),

[0107] primary key("trainKey"))

[0108] In the Renmin University of China Jincang database, it is represented as:

[0109] create table "training_table"(

[0110] "trainKey"bigserial not null,

[0111] "trainCode"varchar(32)not null,

[0112] "trainName"varchar(64),

[0113] primary key("trainKey"))

[0114] In the MySQL database it is represented as:

[0115] create table `training_table`(

[0116] `trainKey`bigint not null auto_increment,

[0117] `trainCode`varchar(32)not null,

[0118] `trainName`varchar(64),

[0119] primary key(`trainKey`)

[0120] In the example, the preset string "create table" in different databases is used to indicate that the script is a table creation script, and the corresponding prompt word templates are:

[0121] Create a table, training_table, with the following fields:

[0122] {Output prompt words for embedded subtasks}

[0123] The table creation is complete.

[0124] In some embodiments, a first parameter value corresponding to a preset character string may be extracted from the first script, and a prompt word template corresponding to the preset character string may be obtained; then, a prompt word of the first data may be generated according to the first parameter value and the prompt word template.

[0125] In some embodiments, the processing can be implemented by setting a script recognition task, which can include subtasks corresponding to each preset string, and the type and number of subtasks can vary according to the script type. Taking the table building script as an example, the subtasks of the script recognition task may include a table building script recognition task, a self-increment primary key script recognition task, a string type script recognition task, etc., which are respectively used to recognize the corresponding preset string and generate prompt words. Combining the prompt words generated by each subtask can obtain the output result of the script recognition task, that is, the prompt word of the first data.

[0126] It should be understood that those skilled in the art may set the preset character string, the corresponding prompt word template and the corresponding subtask according to actual conditions, and the present disclosure does not limit this.

[0127] In this example, it is assumed that the first script of the first data is a DDL table creation script in the MySQL database:

[0128] create table `training_table`(

[0129] `trainKey`bigint not null auto_increment,

[0130] `trainCode`varchar(32)not null,

[0131] `trainName`varchar(64),

[0132] primary key(`trainKey`)

[0133] In the example, through the table creation script recognition task in the subtask of the script recognition task, the script field "create table`training_table`" containing the preset string "create table" in the script can be identified and the corresponding first parameter value "training_table" can be extracted, that is, the table name in the prompt word; the prompt word obtained by combining the prompt word template is:

[0134] Create a table, training_table, with the following fields:

[0135] {Reminder for embedded subtasks}

[0136] The table creation is complete.

[0137] In the example, through the self-increment primary key script identification task in the subtask of the script identification task, the script field "primary key (`trainKey`)" containing the preset string "primary key" can be identified and the corresponding first parameter value "trainKey", that is, the primary key name identifier in the prompt word, can be extracted; and the script field "`trainKey`bigint not null auto_increment," containing the preset string "auto_increment" can be identified and the corresponding first parameter value, that is, the self-increment primary key in the prompt word, can be extracted; the prompt word obtained by combining the prompt word template is:

[0138] Auto-increment primary key, trainKey.

[0139] In the example, the string type script recognition task in the subtask of the script recognition task can identify the script fields "`trainCode`varchar(32)not null," and "`trainName`varchar(64)," containing the preset string "varchar", and extract the corresponding first parameter values ​​"trainCode" and "trainName", that is, the string type in the prompt word; the prompt word obtained by combining the prompt word template is:

[0140] String type, trainCode, length 32, not empty

[0141] String type, trainName, length 64.

[0142] In the example, the prompt words of each field of the first script can be concatenated to obtain the prompt words of the first data:

[0143] Create a table, training_table, with the following fields:

[0144] Auto-increment primary key, trainKey;

[0145] String type, trainCode, length 32, not empty;

[0146] String type, trainName, length 64;

[0147] The table creation is complete.

[0148] In this way, by using prompt words that are similar to natural sentences, prompt words for the first data can be generated without learning the specific code of the database, thereby reducing the cost of learning the database. In addition, the prompt word templates corresponding to the preset character strings in different databases are the same, the prompt word templates are independent of the database, and the prompt words corresponding to the generated preset character strings are also independent of the database, which facilitates the subsequent generation of a script for migration to the second database based on the identifier and prompt words of the second database, reducing the complexity of the data migration operation while improving the efficiency of data migration.

[0149] In some embodiments, in step 303, a second script of the first data in the second database may be generated according to the identifier of the second database and the prompt word of the first data. Step 303 includes:

[0150] Extracting a second parameter value corresponding to a preset text from the prompt word of the first data;

[0151] According to the identifier of the second database, obtaining a script template corresponding to the preset text in the second database, wherein the script templates corresponding to the preset text in different databases are different;

[0152] The second script is generated according to the second parameter value and the script template.

[0153] For example, the processing can be implemented by setting a script generation task, which can include subtasks corresponding to each preset text, and the type and number of subtasks can vary according to the script type. For the table creation script in the above example, the subtasks of the script generation task may include a table creation script generation task, a self-increment primary key script generation task, a string type script generation task, etc., which are respectively used to identify the corresponding preset text and generate scripts for the corresponding fields. Combining the scripts generated by each subtask can obtain the output result of the script generation task, that is, the second script of the first data.

[0154] It should be understood that those skilled in the art may set the preset text, the corresponding script template, and the corresponding subtask according to actual conditions, and the present disclosure does not limit this.

[0155] In the example, assuming that the identifier of the second database is a postgresql database, the prompt word of the first data can be expressed as:

[0156] Create a table, training_table, with the following fields:

[0157] Auto-increment primary key, trainKey;

[0158] String type, trainCode, length 32, not empty;

[0159] String type, trainName, length 64;

[0160] The table creation is complete.

[0161] In the example, through the table creation script generation task in the subtask of the script generation task, the preset text "create table" in the prompt word can be identified, and the second parameter value "training_table" corresponding to the preset text "create table" can be extracted as the table name; the script for generating the field in combination with the script template in the second database is:

[0162] create table `training_table`(

[0163] {Embedded Script}

[0164] primary key(`trainKey`)

[0165] In the example, the self-increment primary key script generation task in the subtask of the script generation task can identify the preset text "self-increment primary key" in the prompt word, and generate an embedded self-increment primary key script in combination with the script template in the second database:

[0166] `trainKey`bigserial not null,

[0167] In the example, through the string type script generation task in the subtask of the script generation task, the preset text "string type" in the prompt word can be identified, and the second parameter values ​​"trainCode" and "trainName" after the preset text "string type" are extracted as field names; and the preset text "length" in the prompt word is identified; the second parameter values ​​32 and 64 after the preset text "length" are extracted as field lengths, and the preset text "not empty" in the prompt word is identified as a non-empty description; the script for generating the field in combination with the script template in the second database is:

[0168] `trainCode`varchar(32)not null,

[0169] `trainName`varchar(64),

[0170] In the example, the fields of the second script can be concatenated to obtain the second script of the first data in the second database:

[0171] create table "training_table"(

[0172] "trainKey"bigserial not null,

[0173] "trainCode"varchar(32)not null,

[0174] "trainName"varchar(64),

[0175] primary key("trainKey") )

[0177] In this way, based on the prompt word, the second script of the first data in the second database can be directly generated according to the script template, which simplifies the generation process and reduces the complexity of script generation. In addition, the script templates corresponding to the preset texts in different databases are different, which can be adapted to the corresponding databases, reducing the learning cost of the second database. Only by obtaining the prompt word, the parameter value corresponding to the preset text can be extracted from the prompt word containing the preset text, and the parameter value corresponding to the preset text can be input into the script template corresponding to the preset text in the second database, and the script corresponding to the preset text is generated, and then the script migrated to the second database is generated, which reduces the complexity of the data migration operation and improves the efficiency of data migration.

[0178] In some embodiments, after obtaining the second script in step 303, the second script may be stored in the second database in step 304, thereby completing the migration process of the first data from the first database to the second database.

[0179] In some embodiments, the processing of steps 302 and 303 is implemented in the form of a script recognition task and a script generation task by means of a prompt word template and a script template. Among them, each subtask of the script generation task can correspond to each subtask of the script recognition task, and can also be called a two-way symmetric task. In the two-way symmetric tasks of the same database, the name of each script recognition subtask or each script generation subtask is unique, and the name of each script recognition subtask and the name of each script generation subtask are independent of the database, that is, the name of the same script sub-recognition task in different databases is the same, and the name of the same script generation subtask in different databases is the same. Each script recognition subtask and each script generation subtask have an applicable scope attribute, and the applicable scope is set to the identifier of the database. In the same database, the output of each subtask is used as the input of the symmetric subtask of the subtask, and the output of the symmetric subtask of the subtask is consistent with the input of the subtask. The script recognition subtask and the script generation subtask are mutually symmetric tasks.

[0180] For example, the subtasks of the script recognition task include the table creation script recognition task, the self-increment primary key script recognition task, and the string type script recognition task. The subtasks of the script generation task include the table creation script generation task, the self-increment primary key script generation task, and the string type script generation task. The set of bidirectional symmetric tasks includes the table creation script recognition task, the table creation script generation task, the self-increment primary key script recognition task, the self-increment primary key script generation task, the string type script recognition task, and the string type script generation task. The table creation script recognition task and the table creation script generation task are symmetric tasks, the self-increment primary key script recognition task and the self-increment primary key script generation task are symmetric tasks, and the string type script recognition task and the string type script generation task are symmetric tasks.

[0181] In some embodiments, before the above-mentioned processing is performed, the script recognition task and the script generation task may be trained, and the training process is implemented by a prompt word generation training tool.

[0182] The prompt word generation training tool is an extension of the prompt word generator. It can not only generate prompt words like the prompt word generator, but also needs to generate bidirectional symmetric tasks. The principle of the prompt word generator is mainly based on artificial intelligence and machine learning technology. The prompt word generator usually uses a pre-trained model that analyzes the input text and tries to understand its meaning and context. Then, the model generates a series of possible prompt words based on this understanding. These prompt words are semantically related to the input text and may help users get better results in the next task. The following are some prompt word generators, AI Text Prompt Generator, PromptoMania, PromptPerfect, ai-prompt-generators, Prompts.chat. The present disclosure does not limit the specific type of the prompt word generation training tool.

[0183] In some embodiments, the training process of the bidirectional symmetric task is as follows: obtain the third script of each sample database, wherein the sample database includes the first database and the second database; perform the following operations for any sample database: input the third script of the sample database into the prompt word generation training tool, obtain the prompt word set output by the prompt word generation training tool and the bidirectional symmetric task set corresponding to the identifier of the sample database, wherein the bidirectional symmetric task set corresponding to the identifier of the sample database includes the script recognition task and the script generation task corresponding to the identifier of the sample database; input the third script of the sample database into the script recognition task corresponding to the identifier of the sample database, obtain the target prompt word of the sample database output by the script recognition task corresponding to the identifier of the sample database; input the target prompt word of the sample database into the script generation task corresponding to the identifier of the sample database, obtain the fourth script of the sample database output by the script generation task corresponding to the identifier of the sample database; when it is determined that the third script and the fourth script of any sample database are the same, complete the training process.

[0184] By determining that the third script and the fourth script of any sample database are the same, the accuracy of the target prompt words of the generated sample database and the accuracy of the fourth script of the generated sample database can be verified, which reduces the requirements on the ability of personnel. Moreover, the accuracy of the script migrated to the second database can be improved.

[0185] Figure 4 A schematic diagram of a training process of a data migration method provided in an embodiment of the present disclosure.

[0186] like Figure 4As shown, the third script of the first database can be input into the prompt word generation training tool to obtain the prompt words and bidirectional symmetric tasks (including script recognition tasks and script generation tasks) output by the prompt word generation training tool; then the script generation task is executed by the task execution engine to obtain the fourth script of the first database. Similarly, the third script of the second database is input into the prompt word generation training tool to obtain the prompt words and bidirectional symmetric tasks (including script recognition tasks and script generation tasks) output by the prompt word generation training tool; then the script generation task is executed by the task execution engine to obtain the fourth script of the second database.

[0187] The training process of the two-way symmetric task mainly completes the accumulation of two-way symmetric tasks, covering all task scenarios of multi-database migration. In the database migration process steps 302 and 303, the tasks in the script recognition task corresponding to the identifier of the first database and the script generation task corresponding to the identifier of the second database do not exceed the coverage of the two-way symmetric task corresponding to the identifier of the sample database in the training process of the two-way symmetric task.

[0188] In some embodiments, bidirectional symmetric task training can be performed on four databases (repeatedly, multiple times) to generate bidirectional symmetric tasks applicable to specific databases. The four databases are postgresql database, DAMO database, Renmin University of China Jincang database and MySQL database. The table creation scripts of the four databases are used as input objects, and training is performed using a prompt word training tool.

[0189] The following uses the postgresql database as an example.

[0190] enter:

[0191] create table "training_table"(

[0192] "key"bigserial not null,

[0193] "field1"varchar(10)not null,

[0194] "field2"varchar(20),

[0195] "field3"int4 default 1not null,

[0196] "field4"timestamp,

[0197] primary key("key") );

[0199] Output:

[0200] 1). Ordered set of prompt words:

[0201] Create a table, training_table, with the following fields:

[0202] Self-increment primary key, key

[0203] String type, field1, length 10, not empty

[0204] String type, field2, length 20

[0205] Integer type, field3, default value 1, not empty

[0206] Timestamp type, field4

[0207] End of table creation

[0208] Bidirectional symmetric task (belongs to the postgresql database task):

[0209] Table creation script identification task / table creation script generation task

[0210] Self-increment primary key script identification task / self-increment primary key script generation task

[0211] String type script recognition task / String type script generation task

[0212] Integer type script recognition task / Integer type script generation task

[0213] Timestamp type script identification task / timestamp type script generation task

[0214] Use the task execution engine, input prompt words and bidirectional symmetric tasks, and confirm that the output script is consistent with the input script.

[0215] After the training is completed, a prompt word training tool is used to generate prompt words for the first database.

[0216] For example, enter the mysql database script:

[0217] create table `sample_table`(

[0218] `id`bigint not null auto_increment,

[0219] `code`varchar(64)not null,

[0220] `name`varchar(128),

[0221] primary key (`id`) )

[0223] Get the prompt word:

[0224] Create a table, sample_table, with the following fields:

[0225] Self-increment primary key, id

[0226] String type, code, length 64, not empty

[0227] String type, name, length 128

[0228] The table creation is complete.

[0229] Use the prompt word and the two-way symmetric task of the second database to complete the data migration work to the second database. Specifically, use the task execution engine, input the prompt word of the first database, specify the two-way symmetric task of the postgresql database, and you can get the following postgresql database adaptation script.

[0230] create table "sample_table"(

[0231] "id"bigserial not null,

[0232] "code"varchar(64)not null,

[0233] "name"varchar(128),

[0234] primary key("id") );

[0236] It can be seen that the subtasks in the script recognition task corresponding to the identifier of the first database and the script generation task corresponding to the identifier of the second database do not exceed the coverage of the tasks in the bidirectional symmetric task corresponding to the identifier of the sample database during the training process of the bidirectional symmetric task.

[0237] In the training process of the two-way symmetric task, because the two-way symmetric task is used, the trainer does not need to care about the implementation of the prompt word training tool, nor the specific syntax of the script. Only the prompt word training tool and input data can be used to verify the conformity of the two-way symmetric task and the prompt word, which reduces the ability requirements of the personnel. Moreover, the accuracy of the script migrated to the second database can be improved.

[0238] Figure 5 A schematic diagram of a data migration process provided by an embodiment of the present disclosure. Figure 5 As shown, the task execution engine is started, the identifier of the first database A is obtained, the script recognition task corresponding to the identifier of the first database A is found, the first script of the first data to be migrated (not shown) is input into the script recognition task, and the prompt word of the first data is output; the task execution engine is used to find the script generation task corresponding to the identifier of the second database B according to the identifier of the second database B, the prompt word of the first data is input into the script generation task, the second script of the first data in the second database B is output, and stored in the second database B.

[0239] In some embodiments, the data migration method according to the embodiment of the present disclosure can also be implemented through a network model (hereinafter referred to as the first model), that is, the first model is used to execute the above-mentioned step 302, generate a prompt word for the first data based on the first script, and step 303 generates a processing process of a second script for the first data in the second database based on the identifier of the second database and the prompt word of the first data.

[0240] Among them, the first model can be an intention recognition model based on RNN (Recurrent Neural Network), or it can be any network type such as a convolutional neural network model, a residual network model, etc. The present disclosure does not limit the specific network type of the first model.

[0241] According to the data migration method of the embodiment of the present disclosure, it is possible to generate a prompt word for the data according to a script of the data in the first database, thereby eliminating the need to learn the specific code of the database and directly generating a second script for the data in the second database according to the prompt word and the identifier of the second database; and then migrating the data from the first database to the second database according to the second script, thereby reducing the learning cost of the second database, reducing the complexity of data migration, and improving the efficiency of data migration.

[0242] According to the data migration method of the embodiment of the present disclosure, in the task training stage, because a bidirectional symmetric task is used, the trainer does not need to care about the implementation of the prompt word training tool, nor the specific syntax of the DDL. Only the tool and input data can be used to verify the conformity of the bidirectional symmetric task and the prompt word, which reduces the ability requirements and training costs of the personnel. In the data migration stage, because prompt words similar to natural sentences are used, the DDL language specifications of the specific database are not involved, which reduces the learning cost of the target database, reduces the complexity of the work and improves the adaptation efficiency.

[0243] The data migration method according to the embodiment of the present disclosure is not limited to the database table creation scenario. In the database data synchronization scenario, SQL query statement, and update statement scenario, the adaptation of multiple databases can be completed according to the embodiment of the present disclosure.

[0244] Figure 6 A schematic diagram of a model training method provided in an embodiment of the present disclosure.

[0245] According to an embodiment of the present disclosure, a model training method is also provided, such as Figure 6 As shown, the method includes:

[0246] Step 601, using a first model to perform script recognition on a third script of sample data in a sample database to obtain a prompt word of the sample data; the sample database includes multiple types of databases;

[0247] Step 602, generating a script for the identifier of the sample database and the prompt word of the sample data by using the first model to obtain a fourth script of the sample data;

[0248] Step 603: training the first model according to the third script and the fourth script;

[0249] Wherein, the first model is used to implement the above-mentioned data migration method.

[0250] For example, a training data set may be first established, the training data set including a third script of sample data in a sample database, the sample database including multiple types of databases, at least including the first database and the second database mentioned above. In the example, the Oracle training database table DDL table creation script may be used as the training data set, and the first model adopts an RNN-based intent recognition model.

[0251] In some embodiments, for any sample database, in step 601, the third script can be input into the first model for script recognition to obtain a prompt word for the sample data; in step 602, the identifier of the sample database is input into the first model so that the first model generates a script based on the identifier of the sample database and the prompt word of the sample data to obtain a fourth script for the sample data; then, in step 603, the first model is trained based on the difference between the third script and the fourth script.

[0252] In some embodiments, step 603 includes:

[0253] Determining a model loss of the first model according to the third script and the fourth script;

[0254] According to the model loss, adjusting the parameters of the first model;

[0255] When the training conditions are met, a trained first model is obtained; wherein the training conditions include that the third script and the fourth script under each sample database are the same.

[0256] That is to say, the model loss of the first model can be determined based on the difference between the third script and the fourth script. The model loss can adopt any type of loss function, such as the cross entropy loss function, etc. The present disclosure does not limit the specific type of loss function.

[0257] In some embodiments, the parameters of the first model can be adjusted according to the model loss; after multiple rounds of iterative training, the trained first model can be obtained when the training conditions are met. The training conditions include at least that the third script and the fourth script under each sample database are the same. That is, when the third script and the fourth script under each sample database are the same, it is considered that the accuracy of the first model meets the requirements, and the script recognition and script generation in the data migration process can be realized with high accuracy, and the model training process can be completed.

[0258] In some embodiments, the training conditions may also include that the number of iterations reaches a preset number of rounds, etc. The present disclosure does not limit the specific setting of the training conditions.

[0259] In this way, the model training process can be realized, and the accuracy of script recognition and script generation during data migration can be improved. In addition, by determining that the first script and the second script of any sample database are the same, the accuracy of the target prompt word set of the generated sample database and the accuracy of the second script of the generated sample database can be verified, and the prediction accuracy of the pre-trained script recognition model and the pre-trained script generation model can be improved, which can improve the accuracy of the script migrated to the second database.

[0260] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0261] In addition, the present disclosure also provides a data migration device, a model training device, an electronic device, a computer-readable storage medium, and a computer program product, all of which can be used to implement any data migration method or model training method provided by the present disclosure. The corresponding technical solutions and descriptions are referred to the corresponding records in the method part and will not be repeated here.

[0262] Figure 7 A block diagram of a data migration device provided in an embodiment of the present disclosure.

[0263] Reference Figure 7 , an embodiment of the present disclosure provides a data migration device, the data migration device comprising:

[0264] A determination module 71, used to determine a first script of the first data in the first database and an identifier of the second database;

[0265] A prompt word generating module 72, used for generating a prompt word of the first data according to the first script;

[0266] A script generating module 73, configured to generate a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data;

[0267] The migration module 74 is used to migrate the first data from the first database to the second database according to the second script.

[0268] In some embodiments, the prompt word generation module 72 is used to: extract a first parameter value corresponding to a preset character string from the first script; obtain a prompt word template corresponding to the preset character string, wherein the prompt word templates corresponding to the preset character string in different databases are the same; and generate a prompt word for the first data according to the first parameter value and the prompt word template.

[0269] In some embodiments, the script generation module 73 is used to: extract a second parameter value corresponding to a preset text from the prompt word of the first data; obtain a script template corresponding to the preset text in the second database according to an identifier of the second database, wherein the script templates corresponding to the preset text in different databases are different; generate the second script according to the second parameter value and the script template.

[0270] In some embodiments, the device is implemented by a first model, which is used to: generate a prompt word for the first data based on the first script; and generate a second script for the first data in the second database based on an identifier of the second database and the prompt word for the first data.

[0271] Figure 8 A block diagram of a model training device provided in an embodiment of the present disclosure.

[0272] Reference Figure 8 , the embodiment of the present disclosure provides a model training device, the model training device comprising:

[0273] The sample script recognition module 81 is used to perform script recognition on the third script of the sample data in the sample database by using the first model to obtain the prompt word of the sample data; the sample database includes multiple types of databases;

[0274] A sample script generating module 82, configured to generate a script based on the identifier of the sample database and the prompt word of the sample data by using the first model to obtain a fourth script of the sample data;

[0275] A training module 83, configured to train the first model according to the third script and the fourth script;

[0276] Wherein, the first model is used to implement the above-mentioned data migration method.

[0277] In some embodiments, the training module 83 is used to: determine the model loss of the first model according to the third script and the fourth script; adjust the parameters of the first model according to the model loss; and obtain the trained first model when the training conditions are met; wherein the training conditions include that the third script and the fourth script under each sample database are the same.

[0278] Each module in the above-mentioned data migration device and model training device can be implemented in whole or in part by software, hardware and their combination. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0279] Fig. 9 A block diagram of an electronic device provided in an embodiment of the present disclosure.

[0280] Reference Fig. 9 An embodiment of the present disclosure provides an electronic device, which includes: at least one processor 901; at least one memory 902, and one or more I / O interfaces 903, connected between the processor 901 and the memory 902; wherein the memory 902 stores one or more computer programs that can be executed by at least one processor 901, and the one or more computer programs are executed by at least one processor 901 so that the at least one processor 901 can execute the above-mentioned data migration method or model training method.

[0281] Each module in the above electronic device can be implemented in whole or in part by software, hardware, or a combination thereof. Each module can be embedded in or independent of a processor in a computer device in the form of hardware, or can be stored in a memory in a computer device in the form of software, so that the processor can call and execute operations corresponding to each module.

[0282] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, wherein the computer program implements the above-mentioned data migration method or model training method when executed by a processor / processing core. The computer-readable storage medium may be a volatile or non-volatile computer-readable storage medium.

[0283] An embodiment of the present disclosure also provides a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the above-mentioned data migration method or model training method.

[0284] It will be appreciated by those skilled in the art that all or some of the steps, systems, and functional modules / units in the methods disclosed above may be implemented as software, firmware, hardware, and appropriate combinations thereof. In a hardware implementation, the division between the functional modules / units mentioned in the above description does not necessarily correspond to the division of physical components; for example, a physical component may have multiple functions, or a function or step may be performed by several physical components in cooperation. Some or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application-specific integrated circuit. Such software may be distributed on a computer-readable storage medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium).

[0285] As is known to those of ordinary skill in the art, the term computer storage media includes volatile and nonvolatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable program instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), static random access memory (SRAM), flash memory or other memory technology, portable compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store desired information and can be accessed by a computer. In addition, it is known to those of ordinary skill in the art that communication media typically contain computer-readable program instructions, data structures, program modules or other data in a modulated data signal such as a carrier wave or other transmission mechanism, and may include any information delivery medium.

[0286] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0287] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0288] The computer program product described herein may be implemented in hardware, software, or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0289] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0290] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0291] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0292] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0293] Example embodiments have been disclosed herein, and although specific terms are employed, they are used and should be interpreted only in a general illustrative sense and not for limiting purposes. In some instances, it will be apparent to those skilled in the art that, unless otherwise expressly noted, features, characteristics, and / or elements described in conjunction with a particular embodiment may be used alone or in combination with features, characteristics, and / or elements described in conjunction with other embodiments. Therefore, those skilled in the art will appreciate that various changes in form and detail may be made without departing from the scope of the present disclosure as set forth in the appended claims.

Claims

1. A data migration method, characterized in that: include: Determine a first script of the first data in the first database and an identifier of the second database; Generate a prompt word for the first data according to the first script; generating a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data; According to the second script, the first data is migrated from the first database to the second database.

2. The data migration method according to claim 1, characterized in that: Generating the prompt word of the first data according to the first script includes: Extracting a first parameter value corresponding to a preset character string from the first script; Obtaining a prompt word template corresponding to the preset character string, wherein the prompt word templates corresponding to the preset character strings in different databases are the same; Generate a prompt word for the first data according to the first parameter value and the prompt word template.

3. The data migration method according to claim 2, characterized in that: The step of generating a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data comprises: Extracting a second parameter value corresponding to a preset text from the prompt word of the first data; According to the identifier of the second database, obtaining a script template corresponding to the preset text in the second database, wherein the script templates corresponding to the preset text in different databases are different; The second script is generated according to the second parameter value and the script template.

4. The data migration method according to claim 1, characterized in that: The method is implemented through a first model, which is used to: generate a prompt word for the first data based on the first script; and generate a second script for the first data in the second database based on an identifier of the second database and the prompt word for the first data.

5. A model training method, characterized in that: include: Performing script recognition on a third script of the sample data in the sample database by using the first model to obtain a prompt word of the sample data; The sample database includes multiple types of databases; Generate a script for the identifier of the sample database and the prompt word of the sample data by using the first model to obtain a fourth script of the sample data; Training the first model according to the third script and the fourth script; Wherein, the first model is used to implement the data migration method described in any one of claims 1-4.

6. The model training method according to claim 5, characterized in that: The step of training the first model according to the third script and the fourth script includes: Determining a model loss of the first model according to the third script and the fourth script; According to the model loss, adjusting the parameters of the first model; When the training conditions are met, a trained first model is obtained; wherein the training conditions include that the third script and the fourth script under each sample database are the same.

7. A data migration device, characterized in that: include: A determination module, used to determine a first script of the first data in the first database and an identifier of the second database; A prompt word generating module, used for generating a prompt word of the first data according to the first script; A script generation module, used for generating a second script of the first data in the second database according to the identifier of the second database and the prompt word of the first data; A migration module is used to migrate the first data from the first database to the second database according to the second script.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores one or more computer programs that can be executed by the at least one processor, and the one or more computer programs are executed by the at least one processor so that the at least one processor can execute the data migration method as described in any one of claims 1-4, or execute the model training method as described in claim 5 or 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When executed by a processor, the computer program implements the data migration method as described in any one of claims 1 to 4, or implements the model training method as described in claim 5 or 6.

10. A computer program product, characterized in that It includes a computer-readable code, or a non-volatile computer-readable storage medium carrying a computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the data migration method as described in any one of claims 1 to 4, or executes the model training method as described in claim 5 or 6.

Images