Automatic matching method, device, computer equipment and storage medium for data migration

Abstract

The present invention discloses an automatic matching method, device, computer equipment and storage medium for data migration, the method includes building a universal data processing professional knowledge base; building a personalized data processing auxiliary knowledge base; building a source data model table to be migrated; reading environmental information from a target database system, and using a probability deduction algorithm to calculate the target data model based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base. The method realizes the automatic mapping of field types and names involved in heterogeneous source and target database systems; realizes the automatic generation of a new target database system based on the source database system; realizes the automatic mapping to an existing target database system based on the source database system; realizes the automatic mapping to multiple target database systems based on multiple source database systems; and greatly improves the efficiency of database system matching by leveraging the general industry experience and the experience of actual work of unit entities.

Description

Technical Field

[0001] The present invention relates to the field of database migration, and in particular to an automatic matching method, device, computer equipment and storage medium for data migration. Background Art

[0002] In the information age, data is the most important thing. With the development of technology and the extension of business, data migration is happening all the time. Data is usually stored in files or dedicated databases. File formats include binary structures, specific delimiters, XML, and Json; databases are divided into relational, key-value pair, column, diagram, document, etc. Each type of database is provided by a different database development team. For example, Oracle, DB2, Sybase, Informix, Redis, Casandra, NEO4J, MongoDB, etc. have different mechanisms for storing data, and each type of database has different versions, such as Oracle 9i, Oracle 10g, etc. When migrating data between different data warehouses, data structure mapping (pattern matching) is indispensable.

[0003] The pattern matching methods used by existing data migration tools are mostly implemented manually. Some databases from the same manufacturer, such as Oracle's GoldenGate, provide automatic mapping capabilities between different versions, but they mainly maintain the field type mapping of the database schema and cannot achieve mapping between different data schemas. In addition, they cannot cross databases produced by different companies. Summary of the invention

[0004] The purpose of the present invention is to provide an automatic matching method, device, computer equipment and storage medium for data migration, aiming to solve the problem that the migration of existing database systems requires manual operation, which is relatively troublesome.

[0005] In order to solve the above technical problems, the object of the present invention is to achieve the following technical solutions: to provide an automatic matching method for data migration, which comprises:

[0006] Summarize the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base;

[0007] Perform statistics on log tables in historical migration activities to obtain data mode association information, and build a personalized data processing auxiliary knowledge base based on the data mode association information;

[0008] Acquire the source data schema and actual source data of the source database system, read and record the data types in the source data schema, and form a source data schema table to be migrated, wherein the data types include field types and field names;

[0009] Environmental information is read from the target database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, a probability deduction algorithm is used to calculate the target data model corresponding to the source data model table to be migrated.

[0010] In addition, the technical problem to be solved by the present invention is to provide an automatic matching device for data migration, which includes:

[0011] A universal data processing professional knowledge base unit is used to aggregate the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base;

[0012] A personalized data processing auxiliary knowledge base unit is used to collect statistics on log tables in historical migration activities, obtain data mode association information, and build a personalized data processing auxiliary knowledge base based on the data mode association information;

[0013] The source data schema table unit to be migrated is used to obtain the source data schema and actual source data of the source database system, read and record the data type in the source data schema, and form the source data schema table to be migrated, wherein the data type includes a field type and a field name;

[0014] The target data pattern calculation unit is used to read environmental information from the target database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, use a probability deduction algorithm to calculate the target data pattern corresponding to the source data pattern table to be migrated.

[0015] In addition, an embodiment of the present invention provides a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the automatic matching method for data migration described in the first aspect when executing the computer program.

[0016] In addition, an embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the processor executes the automatic matching method for data migration described in the first aspect above.

[0017] The embodiment of the present invention discloses an automatic matching method, device, computer equipment and storage medium for data migration, wherein the method comprises: aggregating original database systems, and uniformly defining each data type in each of the original database systems to obtain multiple standard data types, mapping the data types in the original database systems with the standard data types, performing migration verification according to the mapping relationship, and generating a universal data processing professional knowledge base; performing statistics on log tables in historical migration activities to obtain data pattern association information, and constructing a personalized data processing auxiliary knowledge base based on the data pattern association information; obtaining the source data pattern and actual source data of the source database system, reading and recording the data types in the source data pattern, and forming a source data pattern table to be migrated, wherein the data types include field types and field names; reading environmental information from the destination database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, using a probability deduction algorithm to calculate the destination data pattern corresponding to the source data pattern table to be migrated. This method realizes the automatic mapping of field types and names involved in heterogeneous source and target database systems; realizes the automatic generation of a new target database system based on the source database system; realizes the automatic mapping to an existing target database system based on the source database system; realizes the automatic mapping to multiple target database systems based on multiple source database systems, thereby reducing manual operations; at the same time, with the help of general industry experience and personalized experience of actual work of unit entities, the efficiency of database system matching is greatly improved, and at the same time, the results of users' daily data migration work are accumulated, a personalized data processing auxiliary knowledge base is established, data migration work is efficiently completed, and a sustainable service experience is provided. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0019] Figure 1 A schematic diagram of a flow chart of an automatic matching method for data migration provided by an embodiment of the present invention;

[0020] Figure 2 A schematic block diagram of an automatic matching device for data migration provided by an embodiment of the present invention;

[0021] Figure 3 A schematic block diagram of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0023] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0024] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0025] It should be further understood that the term "and / or" used in the present description and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0026] See also Figure 1 , Figure 1 A schematic diagram of a flow chart of an automatic matching method for data migration provided by an embodiment of the present invention;

[0027] like Figure 1 As shown, the method includes steps S101 to S104.

[0028] S101, aggregate the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base;

[0029] S102, performing statistics on log tables in historical migration activities to obtain data mode association information, and constructing a personalized data processing auxiliary knowledge base based on the data mode association information;

[0030] S103, obtaining a source data schema and actual source data of a source database system, reading and recording data types in the source data schema, and forming a source data schema table to be migrated, wherein the data types include field types and field names;

[0031] S104, reading environmental information from the target database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, using a probability inference algorithm to calculate the target data model corresponding to the source data model table to be migrated.

[0032] Traditional data migration manually sets the mapping between source and destination data patterns. In order to successfully achieve data migration, it usually depends on the personal experience of the operator, and such work often has to be repeated many times, and it is impossible to give an exact expectation of the final success. In the embodiments of the present application, on the one hand, by combining the comprehensive database model knowledge in the industry, namely the universal data processing professional knowledge base, abstract or full source data, and personalized pattern matching experience data, namely the personalized data processing auxiliary knowledge base, a reasonable probabilistic selection algorithm is applied to automatically generate the destination data pattern, thereby reducing manual operations, ensuring the reliability of data migration, and effectively ensuring that the migration can be implemented as expected; on the other hand, with the help of the general experience of the industry and the personalized experience of the actual work of the unit entity, the efficiency of pattern matching is greatly improved.

[0033] In a specific embodiment, after step S104, the following steps are further included:

[0034] S201, revising and applying the target data model, and generating a corresponding application log;

[0035] S202: Update the personalized data processing auxiliary knowledge base according to the revised application log.

[0036] In this embodiment, the output destination data model is revised by experts in the business field so that the revised destination data model is more in line with actual needs, that is, it is in line with the habitual expression of individuals in the unit. At the same time, the personalized data processing auxiliary knowledge base is updated according to the revised application log, wherein the application log is a log table. The personalized data processing auxiliary knowledge base accumulates a large amount of user daily data migration work results, providing migration experience for subsequent new database migrations. At the same time, as the historical migration data is updated, the personalized data processing auxiliary knowledge base also synchronously adapts to the corresponding changes to meet market demand.

[0037] For example, when a supplier's database is updated, the manufacturer's database must also update the data in the supplier's database accordingly, that is, migrate the supplier's updated database into the manufacturer's database to facilitate the manufacturer's internal personnel to view the latest data information. The traditional approach is to manually enter the data in the supplier's database into the manufacturer's database one by one, which is rather troublesome. However, through the matching method in the embodiment of the present application, since the personalized data processing auxiliary knowledge base has previously recorded the supplier's historical database migration experience, by calling this part of the migration experience, the supplier's new database can be quickly migrated without the need for manual re-entry, which effectively improves the migration efficiency.

[0038] In a specific embodiment, step S101 includes:

[0039] By referring to the data model in the source database system, the corresponding data model of the destination database system is automatically generated and verified. Through a series of migration activities between the source database system and the destination database system, the conversion of each data model is verified, the existing problems are corrected, and a universal data processing professional knowledge base is generated.

[0040] Specifically, first establish a unified data type to achieve consistent expression of all data types. The purpose of unified description of each field type of each database is to achieve a unified interpretation of the data type, to obtain a common standard to design this data type, and to support the definition of a class in a specific programming language. For example, for field types of different types and versions of databases, provide examples and basic descriptions of maximum and minimum lengths, maximum and minimum values, normal values ​​and abnormal values. For example: For the field type Varchar2 of Oralce 12.2, the minimum value of its constituent characters is 0x01, the maximum value is 0x127, the minimum length is 0, the maximum length is 255, an example of a normal value is "abcd1234", and an example of an abnormal value is

[0041] Then, organize the data type mapping relationship table used in the industry category and map them one by one with the unified data type. For example, use Java classes to map different data types of various databases such as NodeJs, Mysql, Oracle, MsSQl, Sybase, Avro, and JSON databases.

[0042] Finally, according to the mapping relationship between the data types of different databases and the unified data type, the corresponding data model of the target database is automatically generated with reference to the data model in the source database system. Through a series of migration activities that serve as source database systems and target database systems, such as migration from Oracle database to MySQL database, and from MySQL database to Oracle database, the conversion of each model and data type is verified, and the basic ability of automatic model generation to achieve one-to-one mapping of data models is confirmed. Existing problems are corrected one by one, and finally a universal data processing professional knowledge base is obtained.

[0043] In a specific embodiment, step S102 includes:

[0044] S301, obtaining a source field type in a historical source database system and a destination data type in a historical destination database system, matching and recording the source field type with the destination data type to form a field type mapping log table, and calculating a field type mapping probability table based on the field type mapping log table;

[0045] S302, obtaining source field names in a historical source database system and destination field names in a historical destination database system, matching and recording the source field names with the destination field names to form a field name mapping log table, and calculating a field name mapping probability table based on the field name mapping log table;

[0046] S303, obtaining a source data schema name in a historical source database system and a destination data schema name in a historical destination database system, associating and recording the source data schema name, destination data schema name, field type, and field name to form a schema field mapping log table, and calculating a schema field mapping probability table based on the schema field mapping log table;

[0047] S304. Obtain a list of associated fields with a relationship from a source field name to a destination field name having a historical migration action, match and record the source data schema name, the destination data schema name, and the associated field list to form a schema association log table, and calculate a schema association probability table based on the schema association log table.

[0048] In step S301, the field type mapping log table is illustrated by the following Table 1.

[0049] Table 1:

[0050]

[0051] As can be seen from Table 1, since the field contents of the source field type Int and the target data type Int are the same, the source field type Int of the historical source data system MySQL 5.6 and the target data type Int in the historical target database system Oracle12.2 can be successfully matched.

[0052] In this embodiment, the field type mapping probability table is illustrated by the following Table 2.

[0053] Table 2:

[0054]

[0055] The probability table is based on the statistics of the log table. From Table 2, we can see that the source field type and the target field type appear 100 times, and there are 97 matches, so the probability rate is 97%. Specifically, the source field type Int of the historical source data system MySQL 5.6 and the target data type Int in the historical target database system Oracle 12.2 successfully matched 97 times, that is, the source field type Int in the database to be migrated will be matched with the target field type Int in the Oracle12.2 database first.

[0056] In step S302, the field name mapping log table is illustrated by the following Table 3.

[0057] Table 3:

[0058] Source field name Source field type Destination field name Destination field type Uin Int Uid Long Uin Int UUID Varchar

[0059] Record the field names of the same-name fields from the source database system to the field names of the target database to form logs and probability tables. The probability table will increase the number of statistical mappings and calculate the probability of the target field name appearing in the source field name.

[0060] In this embodiment, the field name mapping probability table is illustrated by the following Table 4.

[0061] Table 4:

[0062]

[0063] Statistics are performed on the field name mapping log table to obtain the number of combinations with the same source field name, source field type, destination field name, and destination field type; then the total number is obtained by counting the same source field name and source field type; the ratio of the number of combinations to the total number is the combination probability.

[0064] In step S303, the mode field mapping log table is illustrated by the following Table 5.

[0065] Table 5:

[0066] Mode Name direction Field Name Field Type Customer source Uin Int User Purpose Uin Int

[0067] In this embodiment, the pattern field mapping probability table is illustrated by the following Table 6.

[0068] Table 6:

[0069]

[0070] In the source data schema in the source database system and the target data schema in the target database system in the historical migration activity, the field name, field type and schema name are associated to form a schema field mapping log table and a schema field mapping probability table.

[0071] In step S304, the mode association log table is illustrated by the following Table 7.

[0072] Table 7:

[0073] Source Schema Name Destination mode name List of related fields Customer User Uin->Uin… Customer Order Name->CustomerName…

[0074] In this embodiment, the pattern association probability table is illustrated by the following Table 8.

[0075] Table 8:

[0076]

[0077]

[0078] Among them, source schema name: the schema name used as the data source in data migration; target schema name: the schema name used as the destination in data migration; associated field list: the relationship between the source schema field name and the target schema field name with specific migration actions.

[0079] In a specific embodiment, step S103 includes:

[0080] S401, obtaining the source data model and actual source data of the source database system;

[0081] S402, reading and recording the data types in the source data schema to form a source data schema table to be migrated;

[0082] S403: based on the source data pattern table to be migrated, sample or fully traverse the actual source data of the source database system, record identification critical values, and describe the data types in the source data pattern table to be migrated according to the critical values.

[0083] Extract the data model of the source database system, read and record the field data types and field names in the extracted data model, and form the source data model table to be migrated; then, sample or fully traverse the actual data of the corresponding model of the source database system, record the critical values, that is, the maximum and minimum values ​​used for identification, and further enrich the data type description in the source data model table. For example, when the definition of the source data model does not match the actual application, such as when the definition is too broad, the actual data type requirements can be given. For example, a variable-length field varchar is defined as 256, and the actual data length is only up to 128, then the destination field length can be considered to be output as 128.

[0084] In a specific embodiment, the step S104 includes:

[0085] S501, obtaining the environment information of the target database system, wherein the environment information includes the database type, version, and a preset data mode;

[0086] S502. Output a full specification list of data types allowed by the target database system, and record a full capability model table of the target database system for the data model, wherein the full capability model table includes data model naming rules, field naming rules, and data model size constraints.

[0087] When running, some databases will be instantiated based on specific configuration parameters, such as character encoding, which will affect their interpretation of data types. The full-capability data type specification list can clearly indicate whether the source-based data type can be directly migrated successfully, because different databases define different ranges of data types and have different capabilities.

[0088] In a specific embodiment, the step S104 further includes:

[0089] The source data pattern is obtained based on the source database system, the relevant probability table is extracted from the personalized data processing auxiliary knowledge base, and the extracted probability table is calculated based on the Bayesian algorithm to obtain the corresponding posterior probability table to determine the corresponding target data pattern.

[0090] In this embodiment, the personalized data processing auxiliary knowledge base establishes the prior probabilities of field types, field names, mode names, and the relationship between mode field names. In the specific migration suggestions, the Bayesian algorithm is used to calculate the posterior probability based on the prior probability to generate the mapping relationship between the source data mode and the target mode. For example, in actual situations, the type, naming, mode name, and the relationship between mode and type are obtained from the source database system, and the prior probabilities of these and the target are extracted from the personalized data processing auxiliary knowledge base, and the posterior probability is calculated using the Bayesian algorithm, and the optimal solution is given by ranking.

[0091] Specifically, in actual use, the target pattern optimization algorithm is selected, the minimum match, maximum match, and universal data processing professional knowledge base and personalized data processing auxiliary knowledge base are coordinated with rules (including AND, OR, and optimization) and probability deduction algorithms including Bayesian algorithms are adopted. Based on the type probability optimization algorithm, the naming probability optimization algorithm, and the pattern probability optimization algorithm, the highest probability one-to-one, many-to-one, and one-to-many target data patterns are given for the source data pattern.

[0092] Output the target data model. First, provide the user with a recommended target data model to ensure that the data can be migrated according to this model. Guide the user to adjust the data model, that is, the user actively adjusts the model. The system provides deduction capabilities to ensure that the adjusted data model can be implemented. It can help customers generate the target data model with less or even no adjustments, and deduce in real time to ensure that the adjusted model is correct, prompt possible disputes, and ensure that the migration results are predictable. For confirmed adjustments, record modification and application logs to facilitate the establishment of a personalized data processing auxiliary knowledge base.

[0093] In the embodiment of the present application, the method realizes the automatic mapping of field types and names involved in heterogeneous source and target database systems; realizes the automatic generation of a new target database system based on the source database system; realizes the automatic mapping to an existing target database system based on the source database system; realizes the automatic mapping to multiple target database systems based on multiple source database systems; at the same time, with the help of general industry experience and personalized experience of actual work of unit entities, the efficiency of database system matching is greatly improved; at the same time, the reliability of data migration is ensured, the efficiency of database system migration is improved, and it is guaranteed that the migration can be implemented as expected.

[0094] The embodiment of the present invention further provides an automatic matching device for data migration, which is used to execute any embodiment of the automatic matching method for data migration described above. Figure 2 , Figure 2 It is a schematic block diagram of an automatic matching device for data migration provided by an embodiment of the present invention.

[0095] like Figure 2 As shown, the automatic matching device 600 for data migration includes:

[0096] The universal data processing professional knowledge base unit 601 is used to aggregate the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base;

[0097] A personalized data processing auxiliary knowledge base unit 602 is used to collect statistics on log tables in historical migration activities, obtain data mode association information, and build a personalized data processing auxiliary knowledge base based on the data mode association information;

[0098] The source data schema table unit 603 to be migrated is used to obtain the source data schema and actual source data of the source database system, read and record the data type in the source data schema, and form the source data schema table to be migrated, wherein the data type includes the field type and the field name;

[0099] The target data pattern calculation unit 604 is used to read the environment information from the target database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, use a probability deduction algorithm to calculate the target data pattern corresponding to the source data pattern table to be migrated.

[0100] The device and the method realize automatic mapping of field types and names involved in heterogeneous source and target database systems; realize automatic generation of a new target database system based on the source database system; realize automatic mapping to an existing target database system based on the source database system; realize automatic mapping to multiple target database systems based on multiple source database systems; at the same time, with the help of general industry experience and personalized experience of actual work of unit entities, the efficiency of database system matching is greatly improved; at the same time, the reliability of data migration is ensured, the efficiency of database system migration is improved, and it is guaranteed that the migration can be implemented as expected.

[0101] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0102] The automatic matching device for data migration can be implemented in the form of a computer program. Figure 3 Runs on the computer device shown.

[0103] See also Figure 3 , Figure 3 11 is a schematic block diagram of a computer device provided by an embodiment of the present invention. The computer device 1100 is a server, which may be an independent server or a server cluster composed of multiple servers.

[0104] See also Figure 3 The computer device 1100 includes a processor 1102 , a memory, and a network interface 1105 connected via a system bus 1101 , wherein the memory may include a non-volatile storage medium 1103 and an internal memory 1104 .

[0105] The non-volatile storage medium 1103 can store an operating system 11031 and a computer program 11032. When the computer program 11032 is executed, the processor 1102 can execute an automatic matching method for data migration.

[0106] The processor 1102 is used to provide computing and control capabilities to support the operation of the entire computer device 1100 .

[0107] The internal memory 1104 provides an environment for the operation of the computer program 11032 in the non-volatile storage medium 1103. When the computer program 11032 is executed by the processor 1102, the processor 1102 can execute the automatic matching method of data migration.

[0108] The network interface 1105 is used for network communication, such as providing data information transmission, etc. Those skilled in the art will understand that Figure 3 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device 1100 to which the solution of the present invention is applied. The specific computer device 1100 may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0109] Those skilled in the art will understand that Figure 3 The embodiments of the computer device shown in the figure do not constitute a limitation on the specific composition of the computer device. In other embodiments, the computer device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. For example, in some embodiments, the computer device may only include a memory and a processor. In such embodiments, the structure and function of the memory and the processor are the same as those of the embodiment of the present invention. Figure 3 The embodiments shown are consistent and will not be described again here.

[0110] It should be understood that in the embodiment of the present invention, the processor 1102 may be a central processing unit (CPU), and the processor 1102 may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among them, the general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium may be a non-volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the automatic matching method for data migration of the embodiment of the present invention is implemented.

[0112] The storage medium is a physical, non-transient storage medium, for example, it can be a USB flash drive, a mobile hard disk, a read-only memory (ROM), a magnetic disk or an optical disk, etc., which can store program codes.

[0113] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0114] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed by the present invention, and these modifications or replacements should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. An automatic matching method for data migration, characterized in that: include: Summarize the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base; Perform statistics on log tables in historical migration activities to obtain data mode association information, and build a personalized data processing auxiliary knowledge base based on the data mode association information; Acquire the source data schema and actual source data of the source database system, read and record the data types in the source data schema, and form a source data schema table to be migrated, wherein the data types include field types and field names; Reading environmental information from a target database system, and using a probability deduction algorithm to calculate a target data pattern corresponding to the source data pattern table to be migrated based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base; The log table in the historical migration activity is statistically analyzed to obtain data mode association information, and a personalized data processing auxiliary knowledge base is constructed based on the data mode association information, including: obtaining the source field type in the historical source database system and the destination data type in the historical destination database system, matching and recording the source field type with the destination data type to form a field type mapping log table, and calculating the field type mapping probability table based on the field type mapping log table; obtaining the source field name in the historical source database system and the destination field name in the historical destination database system, matching and recording the source field name with the destination field name to form a field name ... Name mapping log table, calculate the field name mapping probability table; obtain the source data mode name in the historical source database system and the destination data mode name in the historical destination database system, associate and record the source data mode name, destination data mode name, field type, and field name to form a mode field mapping log table, and based on the mode field mapping log table, calculate the mode field mapping probability table; obtain the associated field list of the relationship from the source field name to the destination field name with historical migration actions, match and record the source data mode name, destination data mode name and the associated field list to form a mode association log table, and based on the mode association log table, calculate the mode association probability table; The method of reading environmental information from the target database system and using a probability deduction algorithm to calculate the target data pattern corresponding to the source data pattern table to be migrated based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base includes: obtaining the source data pattern based on the source database system, extracting a relevant probability table from the personalized data processing auxiliary knowledge base, and calculating the extracted probability table based on the Bayesian algorithm to obtain a corresponding posterior probability table, and determining the corresponding target data pattern; and giving an optimal solution by ranking the results to provide a recommended target data pattern for the user; The environment information includes database type, version, and pre-set data mode.

2. The automatic matching method for data migration according to claim 1, characterized in that: The process of reading the environment information from the target database system and calculating the target data pattern by using a probability deduction algorithm based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base includes: Revising and applying the target data model, and generating corresponding application logs; The personalized data processing auxiliary knowledge base is updated according to the revised application log.

3. The automatic matching method for data migration according to claim 1, characterized in that: The step of performing migration verification according to the mapping relationship to generate a universal data processing professional knowledge base includes: By referring to the data model in the source database system, the corresponding data model of the destination database system is automatically generated and verified. Through a series of migration activities between the source database system and the destination database system, the conversion of each data model is verified, the existing problems are corrected, and a universal data processing professional knowledge base is generated.

4. The automatic matching method for data migration according to claim 1, characterized in that: The step of obtaining the source data schema and actual source data of the source database system, reading and recording the data types in the source data schema, and forming a source data schema table to be migrated, wherein the data types include field types and field names, includes: Obtain the source data schema and actual source data of the source database system; Read and record the data types in the source data schema to form a source data schema table to be migrated; Based on the source data pattern table to be migrated, sample or fully traverse the actual source data of the source database system, record identification critical values, and describe the data types in the source data pattern table to be migrated according to the critical values.

5. The automatic matching method for data migration according to claim 1, characterized in that: The step of reading the environment information from the target database system includes: Acquire the environment information of the target database system, wherein the environment information includes the database type, version, and a preset data mode; Output a full specification list of data types allowed by the target database system, and record the target database system's full capability model table for data mode, wherein the full capability model table includes data mode naming rules, field naming rules, and data mode size constraints.

6. An automatic matching device for data migration, characterized in that: include: A universal data processing professional knowledge base unit is used to aggregate the original database systems, and uniformly define each data type in each of the original database systems to obtain multiple standard data types, establish a mapping between the data types in the original database systems and the standard data types, perform migration verification according to the mapping relationship, and generate a universal data processing professional knowledge base; A personalized data processing auxiliary knowledge base unit is used to collect statistics on log tables in historical migration activities, obtain data mode association information, and build a personalized data processing auxiliary knowledge base based on the data mode association information; The source data schema table unit to be migrated is used to obtain the source data schema and actual source data of the source database system, read and record the data type in the source data schema, and form the source data schema table to be migrated, wherein the data type includes a field type and a field name; A target data pattern calculation unit is used to read environmental information from a target database system, and based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base, use a probability deduction algorithm to calculate a target data pattern corresponding to the source data pattern table to be migrated; The log table in the historical migration activity is statistically analyzed to obtain data mode association information, and a personalized data processing auxiliary knowledge base is constructed based on the data mode association information, including: obtaining the source field type in the historical source database system and the destination data type in the historical destination database system, matching and recording the source field type with the destination data type to form a field type mapping log table, and calculating the field type mapping probability table based on the field type mapping log table; obtaining the source field name in the historical source database system and the destination field name in the historical destination database system, matching and recording the source field name with the destination field name to form a field name ... Name mapping log table, calculate the field name mapping probability table; obtain the source data mode name in the historical source database system and the destination data mode name in the historical destination database system, associate and record the source data mode name, destination data mode name, field type, and field name to form a mode field mapping log table, and based on the mode field mapping log table, calculate the mode field mapping probability table; obtain the associated field list of the relationship from the source field name to the destination field name with historical migration actions, match and record the source data mode name, destination data mode name and the associated field list to form a mode association log table, and based on the mode association log table, calculate the mode association probability table; The method of reading environmental information from the target database system and using a probability deduction algorithm to calculate the target data pattern corresponding to the source data pattern table to be migrated based on the universal data processing professional knowledge base and the personalized data processing auxiliary knowledge base includes: obtaining the source data pattern based on the source database system, extracting a relevant probability table from the personalized data processing auxiliary knowledge base, and calculating the extracted probability table based on the Bayesian algorithm to obtain a corresponding posterior probability table, and determining the corresponding target data pattern; and giving an optimal solution by ranking the results to provide a recommended target data pattern for the user; The environment information includes database type, version, and pre-set data mode.

7. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the automatic matching method for data migration according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which, when executed by a processor, enables the processor to perform the automatic matching method for data migration according to any one of claims 1 to 5.

Images