Data conversion method and device, equipment, medium and program product

Abstract

The invention provides a data conversion method which can be applied to the technical field of big data. The data conversion method comprises the steps of obtaining metadata and relational statements in a source database; analyzing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model; analyzing the relation statement, extracting semantic levels in the relation statement, and constructing a semantic intermediate model; and associating the metadata intermediate model with the semantic intermediate model, converting the metadata intermediate model into metadata corresponding to a target database according to a preset conversion rule, and converting the semantic intermediate model into a relation statement corresponding to the target database. The invention further provides a data conversion device and equipment, a storage medium and a program product.

Description

Technical Field

[0001] The present application relates to the field of big data technology, and specifically to a data conversion method, apparatus, device, medium, and program product. Background Art

[0002] In the actual application of database technology, there are significant differences in metadata formats and storage methods between different databases. The data conversion process during database migration is complex, and traditional database migration usually adopts a single-table query method, querying the metadata in a data table through a query statement and migrating the metadata in a table. After the metadata migration is completed, it is necessary to add code to associate the data in different tables, resulting in low migration efficiency and prone to problems such as data loss and metadata inconsistency. It is difficult to meet the needs of enterprises for large-scale data migration in a diversified database environment. Summary of the Invention

[0003] In view of the above problems, the present application provides a data conversion method, apparatus, device, medium and program product for improving the accuracy and efficiency of database migration.

[0004] According to a first aspect of the present application, a data conversion method is provided, comprising:

[0005] Obtain metadata and relational statements from the source database;

[0006] Parsing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model;

[0007] Parsing the relational sentences, extracting semantic levels in the relational sentences, and constructing a semantic intermediate model;

[0008] The metadata intermediate model and the semantic intermediate model are associated, and according to a preset conversion rule, the metadata intermediate model is converted into metadata corresponding to the target database, and the semantic intermediate model is converted into a relational statement corresponding to the target database.

[0009] According to an embodiment of the present application, parsing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model include:

[0010] Obtaining the data type and data attributes of the metadata;

[0011] According to a preset data structure, the data types and data attributes are organized to form the metadata intermediate model.

[0012] According to an embodiment of the present application, the data attributes include precision and scale, and converting the metadata intermediate model into metadata corresponding to the target database includes:

[0013] Obtaining the precision and scale of the data type corresponding to the metadata intermediate model, and the precision and scale of the data type corresponding to the target database;

[0014] Match the precision of the data type corresponding to the metadata intermediate model with the precision of the corresponding data type in the target database, and the scale of the data type corresponding to the metadata intermediate model with the scale of the corresponding data type in the target database, and convert the metadata intermediate model into metadata corresponding to the target database.

[0015] According to an embodiment of the present application, if the precision of the data type corresponding to the metadata intermediate model is greater than the precision of the data type corresponding to the target database, the metadata is rounded or truncated.

[0016] According to an embodiment of the present application, if the scale of the data type corresponding to the metadata intermediate model is larger than the scale of the data type corresponding to the target database, the metadata is rounded or truncated.

[0017] According to an embodiment of the present application, parsing the relational statements, extracting the semantic hierarchy in the relational statements, and constructing a semantic intermediate model includes:

[0018] Decomposing the relational statement to obtain a plurality of basic units;

[0019] Constructing an abstract syntax tree based on the basic units obtained by disassembly;

[0020] Perform semantic analysis on the abstract syntax tree to construct the semantic intermediate model.

[0021] According to an embodiment of the present application, associating the metadata intermediate model and the semantic intermediate model includes: associating elements in the metadata intermediate model and the semantic intermediate model through unique identifiers.

[0022] A second aspect of the present application provides a data conversion device, comprising:

[0023] Data acquisition module, used to obtain metadata and relational statements in the source database;

[0024] A metadata parsing module, used to convert the metadata into an intermediate format and construct a metadata intermediate model;

[0025] A relational statement parsing module, used to extract the semantic hierarchy in the relational statement and construct a semantic intermediate model;

[0026] The data conversion module is used to convert the metadata intermediate model into metadata corresponding to the target database and convert the semantic intermediate model into relational statements corresponding to the target database according to preset conversion rules.

[0027] The third aspect of the present application provides an electronic device, comprising: one or more processors; a memory for storing one or more computer programs, wherein the one or more processors execute the one or more computer programs to implement the steps of the above method.

[0028] The fourth aspect of the present application further provides a computer-readable storage medium having a computer program or instructions stored thereon, which implements the steps of the above method when the computer program or instructions are executed by a processor.

[0029] The fifth aspect of the present application further provides a computer program product, comprising a computer program or instructions, which implement the steps of the above method when executed by a processor.

[0030] One or more of the above-mentioned embodiments have the following beneficial effects: In the embodiments of the present application, a metadata intermediate model and a semantic intermediate model are constructed and associated, and the conversion of relational statements is realized on the basis of traditional data conversion, thereby supporting multi-table joint query during database migration. Compared with traditional single-table query, on the one hand, it can realize the simultaneous migration of metadata and relational statements, avoiding the traditional problem of only migrating metadata and re-adding code to associate data, which is inefficient, prone to data loss, and inconsistency, thereby improving the efficiency of database migration; on the other hand, by converting metadata into a unified intermediate format, data conversion can be realized quickly and accurately according to preset conversion rules, thereby improving the accuracy of database migration. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] The above contents and other objects, features and advantages of the present application will become more apparent through the following description of the embodiments of the present application with reference to the accompanying drawings, in which:

[0032] Figure 1 Schematically illustrates an application scenario diagram of the data conversion method, apparatus, device, medium, and program product according to an embodiment of the present application;

[0033] Figure 2 The following schematically shows a flow chart of a data conversion method according to an embodiment of the present application;

[0034] Figure 3 A flowchart of a method for constructing a semantic intermediate model according to an embodiment of the present application is schematically shown;

[0035] Figure 4 A flowchart of a method for constructing a metadata intermediate model according to an embodiment of the present application is schematically shown;

[0036] Figure 5A block diagram schematically illustrates a structure of a data conversion device according to an embodiment of the present application; and

[0037] Figure 6 A block diagram of an electronic device suitable for implementing a data conversion method according to an embodiment of the present application is schematically shown. DETAILED DESCRIPTION

[0038] Hereinafter, embodiments of the present application will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the present application. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present application. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present application.

[0039] The terms used herein are only for describing specific embodiments and are not intended to limit this application. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0040] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0041] When expressions such as "at least one of A, B, and C, etc." are used, they should generally be interpreted in accordance with the meaning commonly understood by those skilled in the art (for example, "a system having at least one of A, B, and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0042] In the technical solution of this application, the user information involved (including but not limited to user personal information, user image information, user device information, such as location information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) are all information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure and application of the relevant data comply with relevant laws, regulations and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0043] In the scenario of using personal information for automated decision-making, the methods, devices, and systems provided in the embodiments of the present application all provide users with corresponding operation portals for users to choose to agree or reject the automated decision-making results; if the user chooses to reject, the expert decision-making process will be entered. The expression "automated decision-making" here refers to the activity of automatically analyzing and evaluating an individual's behavioral habits, interests and hobbies, or economic, health, credit status, etc. through computer programs and making decisions. The expression "expert decision-making" here refers to the activity of making decisions by people who specialize in a certain field, have specialized experience, knowledge and skills, and have reached a certain level of professionalism.

[0044] Figure 1 The application scenario diagram of the data conversion method, apparatus, device, medium and program product according to the embodiments of the present disclosure is schematically shown.

[0045] like Figure 1 As shown, an application scenario 100 according to this embodiment may include a data migration platform 101, a source database 102, a target database 103, a network 104, and a user terminal 105. The network 104 is a medium for providing a communication link between the data migration platform 101, the source database 102, the target database 103, and the user terminal 105. The network 104 may include various connection types, such as wired or wireless communication links or fiber optic cables.

[0046] A user may use a user terminal 105 to interact with the data migration platform 101 via the network 104 to receive or send data migration tasks, etc. Various communication client applications may be installed on the user terminal 105, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, social platform software, etc. (for example only).

[0047] The user terminal 105 may be any electronic device having a display screen and supporting web browsing, including but not limited to a smart phone, a tablet computer, a laptop computer, a desktop computer, and the like.

[0048] The data migration platform 101 can receive data migration tasks from the user terminal 105 and connect the source database 102 and the target database 103 according to the data migration task, thereby migrating the data in the source database 102 to the target database 103. The data migration platform 101 can be a backend management server (for example only). The source database 102 can be stored on a single server or a server cluster, which can include one or more databases. The target database 103 can also be stored on a single server or a server cluster, which can include one or more databases. The source database 102 can include data to be migrated, and the target database 103 can include the migrated data. The data to be migrated can have the same or different structure as the migrated data.

[0049] It should be noted that the data migration method for heterogeneous databases provided in the embodiment of the present disclosure can generally be executed by the data migration platform 101. Accordingly, the data migration device for heterogeneous databases provided in the embodiment of the present disclosure can generally be set in the data migration platform 101. The data migration method for heterogeneous databases provided in the embodiment of the present disclosure can also be executed by a server or server cluster that is different from the data migration platform 101 and can communicate with the source database 102, the target database 103 and / or the user terminal 105. Accordingly, the data migration device for heterogeneous databases provided in the embodiment of the present disclosure can also be set in a server or server cluster that is different from the data migration platform 101 and can communicate with the source database 102, the target database 103 and / or the user terminal 105.

[0050] It should be understood that Figure 1 The number of data migration platforms, user terminals, networks, source databases, and target databases in the embodiment is merely illustrative. Any number of data migration platforms, user terminals, networks, source databases, and target databases may be provided as needed.

[0051] The following will be based on Figure 1 The scene described by Figure 2-Figure 4 The data conversion method according to the embodiment of the present application is described in detail.

[0052] Figure 2 The flowchart of the data conversion method according to an embodiment of the present application is schematically shown.

[0053] like Figure 2 As shown, the data conversion method of this embodiment includes operations S210 to S240.

[0054] In operation S210 , metadata and relational statements in a source database are obtained;

[0055] In operation S220, the metadata is parsed, the metadata is converted into an intermediate format, and a metadata intermediate model is constructed;

[0056] In operation S230, the relational sentence is parsed, the semantic hierarchy in the relational sentence is extracted, and a semantic intermediate model is constructed;

[0057] In operation S240, the metadata intermediate model and the semantic intermediate model are associated, and according to a preset conversion rule, the metadata intermediate model is converted into metadata corresponding to the target database, and the semantic intermediate model is converted into a relational statement corresponding to the target database.

[0058] During the database migration process, there may be significant differences in metadata formats and storage methods between different databases, which makes the database migration process complicated. For example, when converting the data type of a source database to the data type of a target database, the string type of the source database can store a maximum of 2GB of characters, while the corresponding data type of the target database cannot be mapped as a varchar type, but needs to be mapped as a text type with the same storage capacity to ensure data integrity. In addition, in traditional database migration methods, the collection of metadata is mostly a single-table model, that is, metadata queries are directly performed on the current table through instructions, and all metadata of the current single table are returned. However, this method can only achieve the migration of metadata, and cannot achieve the migration of relational statements. After the migration, the database still needs technicians to re-add relational statement code to associate data, resulting in a long database migration cycle and low efficiency. In addition, when re-adding, relational statements are easily omitted, resulting in data loss or inconsistency. Based on this, in the embodiment of the present application, a metadata intermediate model and a semantic intermediate model are constructed and associated, and the conversion of relational statements is achieved on the basis of traditional data conversion, thereby supporting multi-table joint query during the database migration process.

[0059] Because each database has its own unique characteristics, such as field types, index structures, and primary key constraints, metadata is standardized after collection to form a standardized metadata format, resulting in a metadata intermediate model. This metadata intermediate model is constructed based on the metadata characteristics of the source and target databases, covering multiple levels such as databases, schemas, tables, and fields. Based on metadata data types and data attributes, these attributes are organized according to a unified structure to enable cross-database data migration.

[0060] The semantic intermediate model is obtained by extracting the semantic hierarchy of relational statements, recording the complex logical structure in the relational statements, and further mapping it to the metadata of the physical table structure, thereby making up for the missing "logical field" information in the traditional metadata model and providing necessary support for subsequent work such as data analysis, governance, and lineage analysis.

[0061] Compared with traditional single-table queries, the embodiments of the present application can, on the one hand, simultaneously realize the migration of metadata and relational statements, avoiding the inefficiency, data loss, and inconsistency problems of the traditional method of migrating only metadata and re-adding code-related data. On the other hand, by converting metadata into a unified intermediate format, data conversion can be quickly and accurately realized according to preset conversion rules, thereby improving the accuracy of database migration.

[0062] Metadata refers to data that describes other data—in other words, data about data. It provides information about data, enabling users to more effectively find, manage, and use data resources. In a database environment, metadata typically refers to database definition information, such as table structure, column attributes, indexes, views, and stored procedures. This information is crucial for understanding database architecture, query optimization, and data security. Database metadata can be accessed through SQL (Structured Query Language) queries.

[0063] SQL is a standard language for managing data in relational database management systems. It enables functions such as data query, data manipulation (insert, update, and delete), data definition (creating and modifying table structures, etc.), and data access control. In the embodiments of this application, relational statements use SQL statements, including explicit SQL and implicit SQL. Explicit SQL refers to query statements written by developers (such as report queries and business logic), while implicit SQL refers to statements automatically generated by the database (such as foreign key constraints, view definitions, trigger logic, etc.).

[0064] In an embodiment of the present application, before operation S210 , the system is initialized and the source database to be migrated is connected to the target database to implement data migration.

[0065] In operation S210, in a traditional single-table query, metadata can be obtained through the DatabaseMeta method of JDBC (Java Database Connectivity) to obtain various types of metadata information for the table. For databases that do not support the DatabaseMeta method, metadata information for the current table can be obtained from the result set by assembling a select query statement. However, this method can only achieve metadata collection and conversion for a single data table. To achieve multi-table query and conversion, in the embodiments of this application, a metadata intermediate model and a semantic intermediate model are constructed and associated, and the conversion of relational statements is achieved based on traditional data conversion.

[0066] For example, consider four tables: students, courses, scores, and teachers. A multi-table query can simultaneously retrieve student information, selected course information, corresponding teacher information, and the student's grades. To implement a multi-table query, you need to use relational statements to link related information in each table. For example, you can use a student ID to link the student table with the score table, a course ID to link the score table with the course table, and a teacher ID to link the course table with the teacher table.

[0067] Extracting metadata from multi-table queries has the following technical difficulties:

[0068] (1) Complex relational semantics make parsing difficult. Multi-table query SQL statements involve complex joins (such as JOIN, which is used to connect data from two or more tables based on certain association conditions), nested subqueries, field aliases, expression calculations (such as CASE WHEN, CAST, and aggregate functions), and other complex syntax. Traditional metadata interfaces (such as JDBC) are unable to parse this logical structure.

[0069] (2) The source of fields is difficult to trace. In multi-table queries, fields often use aliases or nested expressions rather than coming directly from physical tables, which makes it difficult to directly trace the source of fields.

[0070] (3) Database dialects vary widely. The SQL dialects of different databases vary greatly. For example, one database supports the LIMIT syntax, while another uses the ROWNUM instruction for querying.

[0071] Based on this, in operation S220, the metadata is parsed, converted into an intermediate format, and a metadata intermediate model is constructed. By constructing the metadata intermediate model, the SQL statement is converted, thereby realizing the multi-table query function.

[0072] Figure 3 The flowchart of the method for constructing a semantic intermediate model according to an embodiment of the present application is schematically shown. Figure 3 As shown, parsing the relational statements, extracting the semantic hierarchy in the relational statements, and constructing a semantic intermediate model include:

[0073] In operation S310, the relational statement is disassembled to obtain a plurality of basic units. A syntax analysis tool is used to perform lexical analysis on the SQL statement in the source database and disassemble the statement into basic units such as keywords, table names, field names, and operators.

[0074] In operation S320, an abstract syntax tree is constructed based on the basic units obtained by the disassembly, and syntax analysis is performed to construct an abstract syntax tree to present the grammatical structure of the sentence.

[0075] In operation S330, semantic analysis is performed on the abstract syntax tree to construct the semantic intermediate model. Through semantic analysis, logical semantics are extracted based on the syntax tree and converted into a unified intermediate format to present each part of information in a standardized logical structure.

[0076] In an embodiment of the present application, by parsing the relational statements, extracting the semantic hierarchy in the relational statements, and constructing a semantic intermediate model, metadata elements such as fields, table names, connection relationships, expressions, etc. in the relational statements can be extracted in a structured manner, thereby accurately and fully extracting the semantic hierarchy in the relational statements and ensuring the accuracy of the relational statement conversion during the data conversion process.

[0077] Specifically, in the embodiments of this application, SQL queries are parsed into an abstract syntax tree (AST) using a SQL abstract syntax tree (AST) parser or an integrated open source parser, allowing for the structured extraction of metadata elements such as fields, table names, joins, and expressions within the SQL. The AST parser efficiently identifies the syntax and semantics of SQL statements. By traversing the abstract syntax tree, it can accurately locate the source of fields and identify the semantic structure within SQL, resolving the field tracing problem.

[0078] In the embodiments of this application, to further enable field traceability and field path tracking, the SQL syntax tree is deeply parsed to build field traceability capabilities, thereby generating complete path information for each field. For example, for the field "customer_name", the traceability path in the query is customer_name, b.name, customers. This effectively traces back to the field's original table and field, ensuring the accuracy and completeness of field information.

[0079] In operation S220, since each database has its own characteristics, such as field type, index structure, primary key constraint, etc., after metadata collection, the metadata is standardized and converted into a customized standardized metadata format, and then data conversion is further performed according to preset conversion rules to improve conversion efficiency.

[0080] Figure 4 The flowchart of the method for constructing the metadata intermediate model according to the embodiment of the present application is schematically shown as follows: Figure 4 As shown, in an embodiment of the present application, parsing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model include:

[0081] In operation S410, the data type and data attributes of the metadata are obtained; wherein the data attributes include attributes such as field length, precision, scale, and constraint conditions.

[0082] In operation S420, the data types and data attributes are organized according to a preset data structure to form the metadata intermediate model.

[0083] In an embodiment of the present application, the metadata intermediate model is a uniformly defined standardized format that stores the data type and data attributes of the metadata, thereby enabling fast and accurate conversion of metadata in any format and ensuring the integrity of the data conversion process.

[0084] In the embodiments of the present application, metadata and relational statements are mapped and transformed according to the conversion rules preset in the mapping rule library. Taking metadata conversion as an example, in this process, the field types of the source database are first converted to the field types of the target database according to the mapping rules of the field types. At the same time, structured information such as constraints and indexes is processed. For features that cannot be directly mapped, extended rules are used for supplementary mapping to ensure the integrity of the data structure.

[0085] In embodiments of the present application, metadata attributes include its precision and scale. In database numeric data types, precision refers to the total number of significant digits in a value, while scale refers to the number of digits to the right of the decimal point, i.e., the number of decimal places. In a multidimensional, heterogeneous database environment, different database types support and represent data precision and scale differently. Especially when processing floating data types (such as DECIMAL or NUMERIC), different database systems may define and implement these precision and scale differently. For example, some databases may allow signed or unsigned integers when calculating precision, while others may restrict scaling. Balancing these differences between databases during the conversion process to ensure that data precision and business meaning are not lost is a technical challenge.

[0086] In an embodiment of the present application, the precision and scale of metadata are adaptively converted. The metadata intermediate model stores the precision and scale information of the metadata in the source database. During the adaptive conversion process, the precision and scale of the data type corresponding to the metadata intermediate model and the precision and scale of the corresponding data type in the target database are obtained; the precision of the data type corresponding to the metadata intermediate model is matched with the precision of the corresponding data type in the target database, and the scale of the data type corresponding to the metadata intermediate model is matched with the scale of the corresponding data type in the target database, and the metadata intermediate model is converted into metadata corresponding to the target database.

[0087] In an embodiment of the present application, during the data conversion process, the precision and scale information of the metadata corresponding to the data type in the source database and the target database are matched, thereby ensuring the integrity of the data during cross-database conversion.

[0088] In an embodiment of the present application, the matching of precision and scale is achieved by comparing the precision and scale of corresponding data types in the source database and the target database in metadata, and determining the corresponding supplementation or rounding method according to the difference in size.

[0089] First, standardize data type mapping. For floating data types in different databases, establish a standardized data type mapping mechanism to clarify the same or similar data types and their precision and scale ranges in each database system. For example:

[0090] A DECIMAL(p,s) data type in a source database corresponds to a NUMBER(p,s) data type in a target database, where p is the precision and s is the scale; a NUMERIC(p,s) data type in a source database corresponds to a DECIMAL(p,s) data type in a target database.

[0091] Provides the corresponding maximum range of precision and scale for each data type so that automatic conversion can be performed across databases.

[0092] In an embodiment of the present application, for precision adjustment, when data is migrated from one database to another, it may be encountered that the precision supported by the target database is lower than that of the source database, that is, the precision of the data type corresponding to the metadata intermediate model is greater than the precision of the corresponding data type in the target database. At this time, rounding or truncation is required according to the precision limit of the target database to ensure that the data will not exceed the precision range supported by the target database when stored.

[0093] Based on the database type and data content, you can design automatic adaptation rules to determine how to convert precision. For example, if the target database supports higher precision, you can directly perform type conversion; if the target database supports lower precision, you will need to adjust the data through rounding or truncation. Rounding is rounding the data to the target database's precision, while truncation is truncating data after the decimal point if it exceeds the precision range. You can flexibly choose the appropriate method based on your actual data needs to adapt to different data migration scenarios.

[0094] In the embodiment of the present application, data is processed according to the difference in data precision between the source database and the target database, thereby ensuring the integrity of the data during the data conversion process and avoiding data loss and errors during the data conversion process.

[0095] In an embodiment of the present application, for scale adjustment, when data is migrated from one database to another, it may be encountered that the scale supported by the target database is lower than that of the source database, that is, the scale of the data type corresponding to the metadata intermediate model is larger than the scale of the corresponding data type in the target database. At this time, it is also necessary to round or truncate according to the scale limit of the target database to ensure that the data will not exceed the scale range supported by the target database when stored.

[0096] Based on the database type and data content, automatic adaptation rules can be designed to determine how to convert the scale. For example, if the target database supports a higher scale, direct type conversion can be performed, with trailing zeros padded to ensure data consistency. If the target database supports a lower scale, data can be adjusted through rounding or truncation. Rounding rounds the data to the target database's scale, while truncation removes excess decimal places that would otherwise exceed the scale range. This flexible choice can be tailored to the actual data to suit different data migration scenarios.

[0097] In the embodiment of the present application, data is processed according to the different data scales of the source database and the target database, thereby ensuring the integrity of the data during the data conversion process and avoiding data loss and errors during the data conversion process.

[0098] Furthermore, in the embodiments of the present application, to ensure that the converted data does not suffer from loss of precision, a verification mechanism is designed to check whether the precision and scale of the data before and after the conversion are as expected. For example, by comparing the data values ​​and precision information before and after the conversion, it is determined whether the database rules are met. A difference threshold is pre-designed to compare whether the data values ​​before and after the conversion are within the preset difference threshold range. If so, it indicates that the difference between the converted data and the original data is within the allowable range. If not, the conversion method needs to be adjusted, for example, by changing the truncation of excess decimal places to rounding.

[0099] In the embodiments of this application, when operating across databases, the metadata in the database tables must be updated synchronously to ensure that changes in precision and scale are correctly reflected in the metadata. Furthermore, compatibility checks between different database systems are required to avoid data corruption or inconsistencies caused by different databases supporting different precision and scales.

[0100] In operation S240 , associating the metadata intermediate model with the semantic intermediate model includes associating elements in the metadata intermediate model and the semantic intermediate model through unique identifiers.

[0101] In the embodiments of this application, by associating the unique identifiers in the two models, table names and field names in the semantic intermediate model are linked to the corresponding metadata in the metadata intermediate model, achieving collaborative mapping between the two intermediate models and ensuring the accuracy of data conversion. Using natural language processing and semantic analysis techniques, combined with a mapping rule library, the metadata intermediate model is used as the basis for determining the corresponding relationship and conversion rules for each metadata element in the target database. The semantic intermediate model is also used as a basis for determining the grammatical and semantic conversion methods of relational statements in the target database.

[0102] In the embodiments of this application, dual intermediate models are constructed: a metadata intermediate model and a semantic intermediate model. These models abstract the logical semantics of relational statements and metadata, providing a standardized foundation for database migration. A collaborative mapping mechanism is established between the semantic intermediate model and the metadata intermediate model. Through data association and semantic matching, the collaborative conversion of metadata and relational statements is achieved. This effectively resolves the syntactic and semantic differences between different databases, enabling precise conversion of metadata and relational statements and improving the accuracy of data migration.

[0103] In some implementations, after operation S240, the converted metadata is further verified for integrity and consistency. During this verification process, the metadata structure is first checked to ensure it complies with the specifications of the target database, including the correctness of information such as the number of fields, names, and types. Constraints and indexes are then verified to ensure they are fully preserved and meet expectations. Furthermore, the metadata's availability in the target database is confirmed by simulating the execution of certain operations.

[0104] In some implementations, after operation S240, metadata synchronization and compatibility checking are also performed. When performing cross-database operations, metadata in database tables must be updated synchronously to ensure that changes in precision and scale are correctly reflected at the metadata level. Furthermore, compatibility checks between different database systems are necessary to avoid data corruption or inconsistencies caused by different databases supporting different precision and scales.

[0105] In some embodiments, after operation S240, the process further includes importing the verified metadata into the target database. This metadata is imported into the target database via a data import module. During the import process, a batch import strategy is implemented based on the performance characteristics and compatibility requirements of the target database to avoid performance issues caused by a single import. Import logs are also recorded for traceability, ensuring that the import results are verifiable and supporting subsequent optimization and adjustment.

[0106] Based on the above data conversion method, this application also provides a data conversion device. Figure 5 The device is described in detail.

[0107] Figure 5 The structural block diagram of the data conversion device according to an embodiment of the present application is schematically shown.

[0108] like Figure 5 As shown, the data conversion device 500 of this embodiment includes a data acquisition module 510 , a metadata parsing module 520 , a relational statement parsing module 530 and a data conversion module 540 .

[0109] The data collection module 510 is used to obtain metadata and relational statements in the source database. In one embodiment, the data collection module 510 can be used to perform the operation S210 described above, which will not be repeated here.

[0110] The metadata parsing module 520 is used to convert the metadata into an intermediate format and construct a metadata intermediate model. In one embodiment, the metadata parsing module 520 can be used to perform the operation S220 described above, which will not be repeated here.

[0111] The relational statement parsing module 530 is used to extract the semantic hierarchy in the relational statement and construct a semantic intermediate model. In one embodiment, the relational statement parsing module 530 can be used to perform the operation S230 described above, which will not be described in detail here.

[0112] The data conversion module 540 is configured to convert the metadata intermediate model into metadata corresponding to the target database and the semantic intermediate model into relational statements corresponding to the target database according to preset conversion rules. In one embodiment, the data conversion module 540 can be configured to perform operation S240 described above and will not be further described here.

[0113] According to embodiments of the present application, any multiple modules among the data acquisition module 510, metadata parsing module 520, relational statement parsing module 530, and data conversion module 540 may be combined into a single module, or any one of these modules may be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules may be combined with at least part of the functionality of other modules and implemented in a single module. According to embodiments of the present application, at least one of the data acquisition module 510, metadata parsing module 520, relational statement parsing module 530, and data conversion module 540 may be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application-specific integrated circuit (ASIC), or may be implemented in hardware or firmware through any other reasonable means of circuit integration or packaging, or may be implemented in any one of the three implementation methods of software, hardware, and firmware, or any appropriate combination of any of these. Alternatively, at least one of the data collection module 510, metadata parsing module 520, relational statement parsing module 530 and data conversion module 540 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0114] Figure 6 A block diagram of an electronic device suitable for implementing a data conversion method according to an embodiment of the present application is schematically shown.

[0115] like Figure 6 As shown, an electronic device 600 according to an embodiment of the present application includes a processor 601, which can perform various appropriate actions and processes based on a program stored in a read-only memory (ROM) 602 or a program loaded from a storage unit 608 into a random access memory (RAM) 603. The processor 601 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a dedicated microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 601 may also include onboard memory for caching purposes. The processor 601 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present application.

[0116] Various programs and data required for the operation of the electronic device 600 are stored in the RAM 603. The processor 601, ROM 602, and RAM 603 are connected to each other via a bus 604. The processor 601 performs various operations of the method flow according to the embodiment of the present application by executing the programs in the ROM 602 and / or RAM 603. It should be noted that the programs may also be stored in one or more memories other than the ROM 602 and RAM 603. The processor 601 may also perform various operations of the method flow according to the embodiment of the present application by executing the programs stored in the one or more memories.

[0117] According to an embodiment of the present application, electronic device 600 may further include an input / output (I / O) interface 605, which is also connected to bus 604. Electronic device 600 may also include one or more of the following components connected to I / O interface 605: an input section 606 including a keyboard, mouse, etc.; an output section 607 including devices such as a cathode ray tube (CRT), liquid crystal display (LCD), and speakers; a storage section 608 including a hard disk; and a communication section 609 including a network interface card such as a LAN card or modem. Communication section 609 performs communication processing via a network such as the Internet. A drive 610 is also connected to I / O interface 605 as needed. Removable media 611, such as a magnetic disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed in drive 610 as needed, so that computer programs read from the removable media can be installed into storage section 608 as needed.

[0118] This application also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when the one or more programs are executed, the method according to the embodiments of this application is implemented.

[0119] According to an embodiment of the present application, a computer-readable storage medium may be a non-volatile computer-readable storage medium, and may include, for example, but not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present application, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to an embodiment of the present application, a computer-readable storage medium may include the ROM 602 and / or RAM 603 described above and / or one or more memories other than ROM 602 and RAM 603.

[0120] The embodiments of the present application also include a computer program product, which includes a computer program containing program code for executing the method shown in the flowchart. When the computer program product is run in a computer system, the program code is used to enable the computer system to implement the data conversion method provided in the embodiments of the present application.

[0121] The computer program executes the above functions defined in the system / device of the embodiment of the present application when the computer program is executed by the processor 601. According to the embodiment of the present application, the system, device, module, unit, etc. described above can be implemented by a computer program module.

[0122] In one embodiment, the computer program may be stored on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may be transmitted and distributed in the form of a signal on a network medium, downloaded and installed via the communication portion 609, and / or installed from a removable medium 611. The program code contained in the computer program may be transmitted using any appropriate network medium, including but not limited to wireless, wired, or any suitable combination thereof.

[0123] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 609, and / or installed from a removable medium 611. When the computer program is executed by the processor 601, the above-mentioned functions defined in the system of the embodiment of the present application are performed. According to the embodiment of the present application, the systems, devices, means, modules, units, etc. described above can be implemented by computer program modules.

[0124] According to an embodiment of the present application, the program code for executing the computer program provided by the embodiment of the present application can be written in any combination of one or more programming languages. Specifically, these computer programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C" or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, partially on a remote computing device, or entirely on a remote computing device or server. In the case of a remote computing device, the remote computing device can be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computing device (for example, using an Internet service provider to connect via the Internet).

[0125] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of the boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0126] Those skilled in the art will appreciate that the features described in the various embodiments of this application may be combined and / or coupled in various ways, even if such combinations or couplings are not explicitly described in this application. In particular, the features described in the various embodiments of this application may be combined and / or coupled in various ways without departing from the spirit and teachings of this application. All such combinations and / or couplings fall within the scope of this application.

Claims

1. A data conversion method, characterized in that: include: Obtain metadata and relational statements from the source database; Parsing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model; Parsing the relational sentences, extracting semantic levels in the relational sentences, and constructing a semantic intermediate model; The metadata intermediate model and the semantic intermediate model are associated, and according to a preset conversion rule, the metadata intermediate model is converted into metadata corresponding to the target database, and the semantic intermediate model is converted into a relational statement corresponding to the target database.

2. The method according to claim 1, characterized in that The step of parsing the relational statements, extracting the semantic hierarchy in the relational statements, and constructing a semantic intermediate model includes: Decomposing the relational statement to obtain a plurality of basic units; Constructing an abstract syntax tree based on the basic units obtained by disassembly; Perform semantic analysis on the abstract syntax tree to construct the semantic intermediate model.

3. The method according to claim 1, characterized in that The step of parsing the metadata, converting the metadata into an intermediate format, and constructing a metadata intermediate model includes: Obtaining the data type and data attributes of the metadata; According to a preset data structure, the data types and data attributes are organized to form the metadata intermediate model.

4. The method according to claim 3, characterized in that The data attributes include precision and scale, and the step of converting the metadata intermediate model into metadata corresponding to the target database includes: Obtaining the precision and scale of the data type corresponding to the metadata intermediate model, and the precision and scale of the data type corresponding to the target database; Match the precision of the data type corresponding to the metadata intermediate model with the precision of the corresponding data type in the target database, and the scale of the data type corresponding to the metadata intermediate model with the scale of the corresponding data type in the target database, and convert the metadata intermediate model into metadata corresponding to the target database.

5. The method according to claim 4, characterized in that If the precision of the data type corresponding to the metadata intermediate model is greater than the precision of the data type corresponding to the target database, the metadata is rounded or truncated.

6. The method according to claim 4, characterized in that If the scale of the data type corresponding to the metadata intermediate model is larger than the scale of the data type corresponding to the target database, the metadata is rounded or truncated.

7. The method according to claim 1, characterized in that The associating the metadata intermediate model and the semantic intermediate model includes associating elements in the metadata intermediate model and the semantic intermediate model through unique identifiers.

8. A data conversion device, characterized in that: include: Data acquisition module, used to obtain metadata and relational statements in the source database; A metadata parsing module, used to convert the metadata into an intermediate format and construct a metadata intermediate model; A relational statement parsing module, used to extract the semantic hierarchy in the relational statement and construct a semantic intermediate model; The data conversion module is used to convert the metadata intermediate model into metadata corresponding to the target database and convert the semantic intermediate model into relational statements corresponding to the target database according to preset conversion rules.

9. An electronic device comprising: one or more processors; a memory for storing one or more computer programs, It is characterized in that the one or more processors execute the one or more computer programs to implement the steps of the method according to any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program or instruction stored thereon, characterized in that: When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A computer program product comprising a computer program or instructions, characterized in that When the computer program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images