A data table conversion method, apparatus, electronic device, and storage medium

Abstract

This application provides a data table conversion method, apparatus, electronic device, and storage medium, applied to a data governance platform. The method includes: determining a target field attribute that matches the target field attribute from the field attributes of the fields in the source table based on the similarity between the field attributes of a target field in the governance table and the field attributes of each field contained in the source table; determining the standard field type corresponding to the target field attribute as a target standard field type that matches the target field based on a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; and storing the target field in the governance table in the destination table according to the field type mapped by the target standard field type in the destination table. Therefore, the efficiency of converting a governance table to a destination table can be improved in a data governance scenario.

Description

Technical Field

[0001] This application relates to the field of data governance technology, and in particular to a data table conversion method, apparatus, electronic device and storage medium. Background Technology

[0002] A geographic field can be used to store geographically related data, consisting of spatial data and attribute data; the spatial data is related to the geometric characteristics of spatial features; and the attribute data is used to provide information about spatial features.

[0003] In governance scenarios, since governance databases (such as Hive databases) do not support storing geographic fields, when converting a source table to a governance table, the geographic fields in the source table need to be converted into EWKT format strings. After governance is completed, when converting the governance table to a destination table, the EWKT format strings need to be restored to geographic fields and re-stored in the destination database that supports storing geographic fields. Summary of the Invention

[0004] This application provides a data table conversion method applied to a data governance platform; the data governance platform is used to convert a source table into a governance table and to convert the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; the method includes:

[0005] Based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the target field attribute that matches the field attribute of the target field is determined from the field attributes of each field contained in the source table.

[0006] Based on the first correspondence, the standard field type corresponding to the target field attribute is determined as the target standard field type that matches the target field;

[0007] Based on the field type mapped to the target standard field type in the target table, the target field in the governance table is stored in the target table.

[0008] Optionally, determining the target field attribute that matches the target field attribute from the field attributes of the fields in the source table based on the similarity between the field attributes of the target field in the governance table and the field attributes of the fields contained in the source table includes:

[0009] Calculate the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table;

[0010] From the field attributes of each field contained in the source table, the field attributes whose similarity to the field attributes of the target field reaches a first threshold are determined as the target field attributes that match the field attributes of the target field.

[0011] Optionally, the field attributes include: field name, and / or field comments.

[0012] Optionally, the data governance platform stores a second correspondence relationship associated with each database type; the second correspondence relationship includes the correspondence between standard field types and the field types mapped by the standard field types in the target table;

[0013] Before storing the target field in the governance table in the destination table according to the field type mapped to the target standard field type, the method further includes:

[0014] Based on the second correspondence relationship associated with the database type of the target database corresponding to the target table, the field type mapped by the target standard field type in the target table is determined.

[0015] Optionally, before determining the target field attribute matching the target field attribute from the field attributes of the fields in the source table based on the similarity between the field attributes of the target field in the governance table and the field attributes of the fields contained in the source table, the method further includes:

[0016] Obtain the first mapping relationship pre-configured by the user; or,

[0017] The first correspondence is automatically created based on the source table.

[0018] Optionally, the data governance platform stores a third correspondence relationship associated with each database type; the third correspondence relationship includes the correspondence between the field types of the fields contained in the source table and the standard field types;

[0019] The automatic creation of the first correspondence based on the source table includes:

[0020] Based on the third correspondence relationship associated with the database type of the source database, determine the standard field type corresponding to the field type of each field contained in the source table;

[0021] The first correspondence is created based on the field attributes of each field in the source table and the standard field types that correspond to the field types of each field in the source table.

[0022] Optionally, the field types of the fields contained in the source table include at least the geographic field type; during the process of converting the source table into the governance table, the geographic field types contained in the source table are stored in the governance table as string field types.

[0023] This application also provides a data table conversion device applied to a data governance platform; the data governance platform is used to convert a source table into a governance table and to convert the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; the device includes:

[0024] The first determining unit is configured to determine, based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, a target field attribute that matches the field attributes of the target field.

[0025] The second determining unit is used to determine the standard field type corresponding to the target field attribute as the target standard field type that matches the target field based on the first correspondence relationship;

[0026] A conversion unit is used to store the target field in the governance table in the destination table according to the field type mapped to the target standard field type in the destination table.

[0027] This application also provides an electronic device, including a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus;

[0028] The memory stores machine-readable instructions, and the processor executes the above method by invoking the machine-readable instructions.

[0029] This application also provides a machine-readable storage medium storing machine-readable instructions, which, when called and executed by a processor, implement the above-described method.

[0030] The technical solutions disclosed in the above embodiments first determine the target field attribute matching the target field attribute based on the similarity between the target field attribute in the governance table and the field attributes of each field in the source table. Then, based on the first correspondence between the field attributes of the fields in the source table and the standard field types corresponding to the field types of the fields in the source table, the standard field type corresponding to the target field attribute is determined as the target standard field type matching the target field. Finally, based on the field type mapped to the target standard field type in the destination table, the target field in the governance table can be stored in the destination table. Therefore, during the conversion of the governance table to the destination table, the target standard field type matching the target field in the governance table can be automatically determined based on the first correspondence, and thus, the field type in which the target field should be stored in the destination table can be automatically determined. This allows for automatic conversion of the governance table to the destination table. In other words, in data governance scenarios involving data table conversion between the source table, governance table, and destination table, automatic table creation can be achieved without manual intervention, thereby saving labor and time costs and improving data table conversion efficiency. Attached Figure Description

[0031] Figure 1 This is a schematic diagram illustrating a data governance scenario as an exemplary embodiment;

[0032] Figure 2 This is a flowchart illustrating a data table conversion method as an exemplary embodiment;

[0033] Figure 3 This is an exemplary embodiment illustrating the hardware structure of an electronic device containing a data table conversion device;

[0034] Figure 4 This is a block diagram illustrating a data table conversion device as an exemplary embodiment. Detailed Implementation

[0035] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0036] It should be noted that the steps of the corresponding methods are not necessarily performed in the order shown and described in this specification in other embodiments. In some other embodiments, the methods may include more or fewer steps than described in this specification. Furthermore, a single step described in this specification may be broken down into multiple steps in other embodiments; and multiple steps described in this specification may be combined into a single step in other embodiments.

[0037] In this specification, the data governance platform can be used to convert source tables stored in the source database into governance tables, and can store the governance tables in a governance database; wherein, the data governance platform can store various fields of different field types contained in the source table as the same field type in the governance table.

[0038] This specification does not impose any special restrictions on the database types of the source database and the governance database. For example, the source database may include, but is not limited to, PostgreSQL, Oracle, MySQL, etc.; the governance database may include, but is not limited to, Hive database, etc.

[0039] The field types of the fields in the source table and the field types of the fields in the governance table can correspond to the database types of the source database and the governance database, respectively. For example, if the source database is PostgreSQL, the field types of the fields in the source table can specifically include, but are not limited to, int, varchar, and geometry. If the governance database is a Hive database, the field types of the fields in the governance table can be varchar. In this case, during the conversion of the source table to the governance table, each field of int, varchar, geometry, etc., in the source table can be stored as varchar in the governance table.

[0040] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating a data governance scenario as an exemplary embodiment. For example... Figure 1 As shown, the source and destination databases can support storing geographic fields, while the governance database does not.

[0041] The geographic field can be used to store geographic-related data, which typically consists of spatial data and attribute data; the spatial data is related to the geometric characteristics of spatial features; and the attribute data is used to provide information about spatial features.

[0042] In one embodiment shown, the field types of the fields contained in the source table may include at least a geographic field type; during the process of converting the source table into the governance table, the geographic field types contained in the source table are stored in the governance table as string field types.

[0043] For example, such as Figure 1 As shown, in a governance scenario, when converting a source table into a governance table, the geographic fields in the source table need to be converted into strings in EWKT format. That is, the geographic fields contained in the source table need to be stored as string fields in the governance table.

[0044] WKT (Well-known text) is a text markup language used to represent vector geometric objects, spatial reference systems, and conversions between spatial reference systems. This format was developed by the OGC (Open Geospatial Consortium). EWKT is an extension of WKT, adding support for 3DZ, 3DM, and 4D coordinates, and allowing the embedding of SRID (Spatial Reference Identifier) ​​information.

[0045] In this specification, the data governance platform can also be used to convert the governance table into a target table, and the target table can be stored in the target database.

[0046] For example, such as Figure 1 As shown, when converting the governance table into the destination table after governance is completed, it is necessary to restore the EWKT format strings in the governance table to geographic fields. That is, it is necessary to store the string field type fields contained in the governance table as geographic field types in the destination table.

[0047] For example, in a governance scenario, when converting the source table into the governance table, the integer fields in the source table need to be stored as string fields in the governance table; after governance is completed, when converting the governance table into the destination table, the string fields in the governance table need to be restored to integer fields and stored in the destination table.

[0048] Therefore, in the embodiments shown above, because all fields contained in the source table are stored in the governance table with the same field type, when converting the governance table to the target table, it is impossible to determine in the target table what field type the current field in the governance table should be stored in based solely on the field types of the fields contained in the governance table. Consequently, it is impossible to automatically convert the governance table to the target table after governance is completed, resulting in low data table conversion efficiency in data governance scenarios. In related technologies, manual intervention is usually required to create the target table, which is a cumbersome process with high labor and time costs and is prone to errors.

[0049] In view of this, this specification aims to propose a technical solution to automatically determine the field type in the governance table and convert it into the target table by establishing a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table.

[0050] In implementation, the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table. Based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the data governance platform can determine the target field attribute that matches the target field attribute from the field attributes of each field contained in the source table. Further, based on the first correspondence, the standard field type corresponding to the target field attribute can be determined as the target standard field type that matches the target field. Further, based on the field type mapped to the target standard field type in the destination table, the target field in the governance table can be stored in the destination table.

[0051] Therefore, in the technical solution of this specification, firstly, based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the target field attribute matching the target field attribute can be determined from the field attributes of each field contained in the source table. Then, based on the first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, the standard field type corresponding to the target field attribute can be determined as the target standard field type matching the target field. Finally, based on the field type mapped to the target standard field type in the destination table, the target field in the governance table can be stored in the destination table. Thus, in the process of converting the governance table into the destination table, based on the first correspondence, the target standard field type matching the target field in the governance table can be automatically determined, and consequently, the field type in which the target field should be stored in the destination table can be automatically determined, thereby automatically converting the governance table into the destination table.

[0052] The present application will now be described through specific embodiments and in conjunction with specific application scenarios.

[0053] Please see Figure 2 , Figure 2 This is a flowchart illustrating a data table transformation method as an exemplary embodiment. The method can be applied to the data governance platform; the data governance platform is used to transform a source table into a governance table, and to transform the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table.

[0054] The field type can be the data type stored in the fields of data tables corresponding to different database types; the standard field type can be the actual data type corresponding to different field types. The standard field type can be mapped to different field types in data tables corresponding to different database types.

[0055] For example, if the database type is PostgreSQL, the field types of the corresponding tables can include, but are not limited to, int, varchar, and geometry; if the database type is Oracle, the field types of the corresponding tables can include, but are not limited to, int, varchar, and st_geometry; standard field types can be pre-configured by the user, such as "numeric type," "text type," and "geographic type." Specifically, the standard field type "geographic type" can be mapped to the geometry field type in PostgreSQL tables, while it can be mapped to the st_geometry field type in Oracle tables.

[0056] Optionally, in this specification, the field attributes may specifically include: field name and / or field comments.

[0057] For example, the data governance platform can store a first correspondence between the field names of the fields contained in the source table, the field comments of the fields contained in the source table, and the standard field types corresponding to the field types of the fields contained in the source table; specifically, the first correspondence can be maintained through a data table as shown in Table 1.

[0058]

[0059] Table 1

[0060] As shown in Table 1, the source table contains column_1 with the field name "geo_1" and the field comment "geographic field_1". The standard field type corresponding to column_1 is "geographic type". The source table contains column_2 with the field name "id_1" and the field comment "primary key_1". The standard field type corresponding to column_2 is "numeric type". The source table contains column_3 with the field name "name_1" and the field comment "name_1". The standard field type corresponding to column_3 is "text type".

[0061] It should be noted that Table 1 merely illustrates one example of the first correspondence and does not impose any specific limitations on this specification. For example, the data governance platform may also store the first correspondence between the field names of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, or the first correspondence between the field comments of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, or the first correspondence between other field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, etc.

[0062] The data table conversion method can perform the following steps:

[0063] Step 202: Based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, determine the target field attribute that matches the field attribute of the target field from the field attributes of each field contained in the source table.

[0064] The data governance platform can sequentially determine the target fields from the several fields to be transformed contained in the governance table, and then execute step 202.

[0065] For example, the governance table contains a column_1' with the field name "geo_2", the field comment "geographic field_2", and the field type "varchar"; the governance table contains a column_2' with the field name "id_2", the field comment "primary key_2", and the field type "varchar"; the governance table contains a column_3' with the field name "name_2", the field comment "name_2", and the field type "varchar". If the target field in the governance table is column_1', then based on the similarity between the field name "geo_2" of column_1' in the governance table and the field names "geo_1", "id_1", and "name_1" of each field in the source table as shown in Table 1, it can be determined that the target field name matching the field name "geo_2" of the target field column_1' is "geo_1".

[0066] Using a similar method, if the target field in the governance table is column_2', based on the similarity between the field name of the target field and the field names of each field contained in the source table, the target field name matching the field name "id_2" of the target field column_2' can be determined to be "id_1"; if the target field in the governance table is column_3', based on the similarity between the field name of the target field and the field names of each field contained in the source table, the target field name matching the field name "name_2" of the target field column_3' can be determined to be "name_1".

[0067] For example, if the target field in the governance table is column_1', then based on the similarity between the field annotation "geographic field_2" of column_1' in the governance table and the field annotations "geographic field_1", "primary key_1", and "name_1" of each field in the source table as shown in Table 1, it can be determined that the target field annotation matching the field annotation "geographic field_2" of the target field column_1' is "geographic field_1".

[0068] Using a similar method, if the target field in the governance table is column_2', based on the similarity between the field annotation of the target field and the field annotations of each field contained in the source table, the target field annotation matching "primary key_2" of the target field column_2' can be determined to be "primary key_1"; if the target field in the governance table is column_3', based on the similarity between the field annotation of the target field and the field annotations of each field contained in the source table, the target field annotation matching "name_2" of the target field column_3' can be determined to be "name_1".

[0069] It should be noted that by determining the target field attribute that matches the target field attribute based on the similarity between the field attribute of the target field in the governance table and the field attributes of each field contained in the source table, when the data table structures are similar, the data table can be transformed using the first correspondence created based on the same source table, thereby reducing the workload of creating the first correspondence.

[0070] Optionally, in step 202, the process of determining the target field attribute that matches the target field attribute from the field attributes of each field in the source table based on the similarity between the field attribute of the target field in the governance table and the field attributes of each field in the source table may specifically include: calculating the similarity between the field attribute of the target field in the governance table and the field attributes of each field in the source table; and determining the field attribute whose similarity to the field attribute of the target field reaches a first threshold from the field attributes of each field in the source table as the target field attribute that matches the target field attribute.

[0071] For example, if the target field in the governance table is column_1', the similarity between the field name "geo_2" of the target field column_1' and the field names "geo_1", "id_1", and "name_1" of each field in the source table as shown in Table 1 can be calculated. Furthermore, if the similarity between the field name "geo_2" of the target field column_1' and the field name "geo_1" of the field in the source table reaches a preset first threshold, the field name "geo_1" of the field in the source table can be determined as the target field name that matches the field name "geo_2" of the target field column_1'.

[0072] For example, if the target field in the governance table is column_1', the similarity between the field annotation "geographic field_2" of the target field column_1' and the field annotations "geographic field_1", "primary key_1", and "name_1" of each field in the source table as shown in Table 1 can be calculated. Furthermore, if the similarity between the field annotation "geographic field_2" of the target field column_1' and the field annotation "geographic field_1" of the field in the source table reaches a preset first threshold, the field annotation "geographic field_1" of the field in the source table can be determined as the target field annotation that matches the field annotation "geographic field_1" of the target field column_1'.

[0073] It should be noted that the specific value of the first threshold can be flexibly set by those skilled in the art as needed, and this specification does not impose any restrictions. Regarding the specific implementation method for calculating the similarity between the field attributes of the fields contained in the governance table and the field attributes of the fields contained in the source table, those skilled in the art can implement it without creative effort, and it will not be elaborated here; for example, one can first generate the text vectors corresponding to the field names or field annotations, then calculate the vector similarity between the generated text vectors, and then use the vector similarity to measure the similarity between the field names or field annotations.

[0074] Additionally, it should be noted that in one or more embodiments of this specification, since field names are typically defined when creating fields in a data table, matching by field names can improve the reliability of data table conversion. Furthermore, when field comments are added to the fields contained in the source table, governance table, and destination table, since field comments can represent richer content than field names, matching by field comments can further improve the flexibility and accuracy of data table conversion.

[0075] In practical applications, the first correspondence stored in the data governance platform can be created manually by the user, automatically created by the data governance platform based on the source table, or obtained by the user after adjusting the automatically created first correspondence according to their needs.

[0076] Optionally, before step 202, the method may further include: obtaining the first correspondence pre-configured by the user; or, automatically creating the first correspondence based on the source table.

[0077] For example, based on the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, the user can manually create a list and add a first correspondence as shown in Table 1 to the list; the data governance platform can obtain the list manually created by the user based on the source table corresponding to the governance table to be converted, and then obtain the first correspondence pre-configured by the user, so as to execute the data table conversion method described in this specification based on the first correspondence.

[0078] The list can be used to describe a single data table or multiple data tables with the same table structure. In addition to the first correspondence, the list may also include business attributes such as list identifier and sequence code, which are not specifically limited in this specification.

[0079] It should be noted that the first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, which is pre-configured by the user, is more flexible and better meets the actual needs of the user.

[0080] For example, the data governance platform can automatically create a list based on the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table. The list can include at least the first correspondence shown in Table 1. Furthermore, the data governance platform can execute the data table transformation method described in this specification based on the first correspondence.

[0081] It should be noted that automatically creating a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table can improve the speed of creating the first correspondence and reduce the technical skill requirements for users. Furthermore, in practical applications, users can adjust the automatically created first correspondence according to actual needs, thus combining the advantages of the two methods mentioned above, which not only improves the speed of creating the first correspondence but also makes it more flexible.

[0082] In this context, the data governance platform stores a third correspondence relationship associated with each database type. This third correspondence relationship may include the correspondence between the field types of the fields contained in the source table and standard field types. The process of automatically creating the first correspondence relationship based on the source table specifically includes: determining the standard field types corresponding to the field types of each field contained in the source table according to the third correspondence relationship associated with the database type of the source database; and creating the first correspondence relationship based on the field attributes of each field contained in the source table and the standard field types corresponding to the field types of each field contained in the source table.

[0083] For example, the data governance platform can store a third correspondence relationship associated with each database type. Specifically, if the source database is PostgreSQL, the field types of the fields contained in the source table can include, but are not limited to, int, varchar, geometry, etc. The third correspondence relationship between the field types of the fields contained in the source table associated with PostgreSQL and the standard field types can be maintained through a data table as shown in Table 2.

[0084]

[0085] Table 2

[0086] Based on the third correspondence shown in Table 3, the standard field type corresponding to the field type "geometry" of column_1 in the source table can be determined to be "geographic type", the standard field type corresponding to the field type "int" of column_2 in the source table can be determined to be "numeric type", and the standard field type corresponding to the field type "geometry" of column_3 in the source table can be determined to be "text type". Furthermore, based on the field attributes of the fields column_1, column_2, and column_3 in the source table and the standard field types corresponding to the field types of the above-mentioned fields, the first correspondence shown in Table 1 can be created.

[0087] It should be noted that in the embodiments shown above, the standard field types include "geographic type," "numerical type," and "text type," which is merely an exemplary description. This specification does not impose any special limitations on the number and names of the standard field types, and those skilled in the art can set them as needed. For example, the standard field type "numerical type" can be further divided into "integer type" and "non-integer type."

[0088] Step 204: Based on the first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, determine the standard field type corresponding to the target field attribute as the target standard field type that matches the target field.

[0089] For example, if the target field in the governance table is column_1', after determining that the target field name matching the field name "geo_2" of the target field column_1' is "geo_1", according to the first correspondence shown in Table 1, the standard field type "geographic type" corresponding to the target field name "geo_1" can be determined as the target standard field type matching the target field column_1'.

[0090] For example, if the target field in the governance table is column_1', after determining that the target field annotation "Geographic Field_2" that matches the target field column_1' is "Geographic Field_1", according to the first correspondence shown in Table 1, the standard field type "Geographic Type" that corresponds to the target field annotation "Geographic Field_1" can be determined as the target standard field type that matches the target field column_1'.

[0091] Step 206: Store the target field in the governance table in the target table according to the field type mapped to the target standard field type in the target table.

[0092] For example, if the target field in the governance table is column_1', after determining that the target standard field type matching the target field column_1' is "geographic type", the target field column_1' in the governance table can be stored in the target table as a geographic field type according to the field type mapped by the target standard field type "geographic type" in the target table. That is, the target field column_1' stored as a string in the governance table can be converted into the field column_1 stored as a geographic field type in the target table.

[0093] For example, if the target field in the governance table is column_2', after determining that the target standard field type matching the target field column_2' is "numeric type", and based on the fact that the target standard field type "numeric type" maps to the integer type in the target table, the target field column_2' in the governance table can be stored in the target table as an integer type; that is, the target field column_2' stored as a string type in the governance table can be converted into the field column_2 stored as an integer type in the target table.

[0094] For example, if the target field in the governance table is column_3', after determining that the target standard field type matching the target field column_3' is "text type", and based on the fact that the target standard field type "text type" maps to the string type in the target table, the target field column_3' in the governance table can be stored as a string in the target table; that is, the target field column_3' stored as a string in the governance table can be converted into the field column_3 stored as a string in the target table.

[0095] Optionally, the data governance platform stores a second correspondence relationship associated with each database type; the second correspondence relationship may include a correspondence relationship between a standard field type and the field type mapped by the standard field type in the target table; before step 206, the method may further include: determining the field type mapped by the target standard field type in the target table based on the second correspondence relationship associated with the database type of the target database corresponding to the target table.

[0096] For example, the data governance platform can store a second correspondence relationship associated with each database type. Specifically, if the database type of the target database corresponding to the target table is Oracle, the field types mapped by the standard field types in the target table can include, but are not limited to, int, varchar, st_geometry, etc. The second correspondence relationship between the standard field types associated with Oracle and the field types mapped by the standard field types in the target table can be maintained through a data table as shown in Table 3.

[0097]

[0098] Table 3

[0099] If the target field in the governance table is column_1', after determining that the target standard field type matching the target field column_1' is "geographic type", according to the second correspondence shown in Table 3, it can be determined that the target standard field type "geographic type" maps to the field type "st_geometry" in the destination table, and the target field of the "varchar" field type contained in the governance table can be converted into the field column_1 of the "st_geometry" field type contained in the destination table.

[0100] It should be noted that in the embodiments shown above, since the source database and the destination database may have different database types, the field types of the same field in the governance table may also be different in the source table and the destination table. Therefore, based on the third correspondence between the field types of the fields contained in the source table and the standard field types, and the second correspondence between the standard field types and the field types mapped to the standard field types in the destination table, regardless of whether the source database and the destination database have the same database types, the data table conversion between the "source table - governance table - destination table" can be automatically realized in the governance scenario. For example, the field type of column_1 contained in the source table is "geometry", and the corresponding standard field type can be "geographic type"; the field type of column_1' contained in the governance table can be "varchar"; after storing the field column_1' in the destination table, the field type mapped in the destination table can be "st_geometry".

[0101] Optionally, in step 206, storing the target field in the governance table in the destination table according to the field type mapped to the target standard field type in the destination table may specifically include: generating a table creation statement for storing the target field in the governance table in the destination table according to the field type mapped to the target standard field type in the destination table, and executing the table creation statement.

[0102] For example, if the target field in the governance table is column_1', and the target field name matching the target field name "geo_2" is determined to be "geo_1" based on the similarity between the target field name and the field names of each field in the source table, and the target standard field type matching the target field column_1' is determined to be "geo type" according to the first correspondence as shown in Table 1, if the target database's database type is Oracle, then according to the second correspondence as shown in Table 3, the field type mapped to the target standard field type "geo type" in the target table can be determined to be "st_geometry"; furthermore, it is possible to generate... This is a table creation statement used to store the target field column_1' into the destination table. That is, a table creation statement can be generated to create the field column_1" in the destination table, and then the table creation statement can be executed. The field type of the field column_1" created in the destination table can be "st_geometry", the field name of the field column_1" can be the target field name "geo_1", and the field comment of the field column_1" can be the field comment corresponding to the target field name "geographic field_1", or the field comment of the field column_1" can also be a redefined field comment, such as "geographic field_3".

[0103] For example, if the target field in the governance table is column_1', and based on the similarity between the field annotation of the target field and the field annotations of each field contained in the source table, the target field annotation matching the field annotation "geographic field_2" is determined to be "geographic field_1", and after determining the target standard field type matching the target field column_1' as "geographic type" according to the first correspondence as shown in Table 1, if the database type of the target database is Oracle, then according to the second correspondence as shown in Table 3, the field type mapped by the target standard field type "geographic type" in the target table can be determined to be "st_geometry"; furthermore, it is possible to... A table creation statement can be generated to store the target field column_1' into the destination table. That is, a table creation statement can be generated to create the field column_1" in the destination table, and then the table creation statement can be executed. The field type of the field column_1" created in the destination table can be "st_geometry", the field annotation of the field column_1" can be the same as the annotation of the target field "geographic field_1", the field name of the field column_1" can be the field name "geo_1" corresponding to the annotation of the target field, or the field name of the field column_1" can also be a redefined field name, such as "geo_3".

[0104] It should be noted that in the embodiments shown above, since a target standard field type matching the field attributes of the target field has been determined, the data governance platform can determine which field type the target field in the governance table should be stored in the destination table, thereby automatically converting the governance table into the destination table. Regarding the specific implementation of generating the table creation statement, based on determining the field types of the fields contained in the destination table corresponding to the target standard field type, those skilled in the art can flexibly set the specific generation method of the table creation statement according to the database type of the destination database; this will not be elaborated upon in this specification.

[0105] As can be seen from the above technical solution, firstly, based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the target field attribute matching the target field attribute can be determined from the field attributes of each field contained in the source table. Then, based on the first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table, the standard field type corresponding to the target field attribute can be determined as the target standard field type matching the target field. Finally, based on the field type mapped to the target standard field type in the destination table, the target field in the governance table can be stored in the destination table. Therefore, in the process of converting the governance table to the destination table, the target standard field type matching the target field in the governance table can be automatically determined based on the first correspondence, and thus the field type in which the target field should be stored in the destination table can be automatically determined, thereby automatically converting the governance table to the destination table. That is, in data governance scenarios involving data table conversion between "source table - governance table - destination table", automatic table creation can be achieved without manual intervention, thereby saving labor and time costs and improving data table conversion efficiency.

[0106] Corresponding to the embodiments of the data table conversion method described above, this specification also provides an embodiment of a data table conversion device.

[0107] Please see Figure 3 , Figure 3 This is an exemplary embodiment illustrating the hardware structure of an electronic device housing a data table conversion device. At the hardware level, the device includes a processor 302, an internal bus 304, a network interface 306, memory 308, and non-volatile memory 310, and may also include other hardware required for business operations. One or more embodiments of this specification can be implemented in software, for example, the processor 302 reads the corresponding computer program from the non-volatile memory 310 into memory 308 and then runs it. Of course, besides software implementation, one or more embodiments of this specification do not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution entity of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0108] Please see Figure 4 , Figure 4 This is a block diagram illustrating a data table conversion device according to an exemplary embodiment. This data table conversion device can be applied to... Figure 3 The electronic device shown is used to implement the technical solution described in this specification.

[0109] The data governance platform is used to convert a source table into a governance table and to convert the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; the device includes:

[0110] The first determining unit 402 is used to determine the target field attribute that matches the field attribute of the target field from the field attributes of the fields contained in the source table based on the similarity between the field attribute of the target field in the governance table and the field attributes of the fields contained in the source table.

[0111] The second determining unit 404 is used to determine the standard field type corresponding to the target field attribute as the target standard field type that matches the target field according to the first correspondence relationship;

[0112] The conversion unit 406 is used to store the target field in the governance table in the destination table according to the field type mapped in the destination table by the target standard field type.

[0113] In this embodiment, the first determining unit 402 is specifically used for:

[0114] Calculate the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table;

[0115] From the field attributes of each field contained in the source table, the field attributes whose similarity to the field attributes of the target field reaches a first threshold are determined as the target field attributes that match the field attributes of the target field.

[0116] In this embodiment, the field attributes include: field name and / or field comment.

[0117] In this embodiment, the data governance platform stores a second correspondence relationship associated with each database type; the second correspondence relationship includes the correspondence between standard field types and the field types mapped by the standard field types in the target table;

[0118] The device further includes:

[0119] The third determining unit is used to determine the field type that the target standard field type maps to in the target table based on the second correspondence relationship associated with the database type of the target database corresponding to the target table.

[0120] In this embodiment, the device further includes:

[0121] The acquisition unit is used to acquire the first correspondence relationship pre-configured by the user; or,

[0122] A creation unit is used to automatically create the first correspondence based on the source table.

[0123] In this embodiment, the data governance platform stores a third correspondence relationship associated with each database type; the third correspondence relationship includes the correspondence between the field types of the fields contained in the source table and the standard field types;

[0124] The creation unit is specifically used for:

[0125] Based on the third correspondence relationship associated with the database type of the source database, determine the standard field type corresponding to the field type of each field contained in the source table;

[0126] The first correspondence is created based on the field attributes of each field in the source table and the standard field types that correspond to the field types of each field in the source table.

[0127] In this embodiment, the field types of the fields contained in the source table include at least the geographic field type; during the process of converting the source table into the governance table, the geographic field types contained in the source table are stored in the governance table as string field types.

[0128] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0129] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0130] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer, which can take the form of a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email sending and receiving device, game console, tablet computer, wearable device, or any combination of these devices.

[0131] In a typical configuration, a computer includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0132] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0133] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, disk storage, quantum memory, graphene-based storage media or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0134] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0135] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0136] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0137] It should be understood that although the terms first, second, third, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first information may also be referred to as second information without departing from the scope of one or more embodiments of this specification, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "in response to a determination," or "when," or "in the event of a determination."

[0138] The above description is merely a preferred embodiment of one or more embodiments of this specification and is not intended to limit the scope of one or more embodiments of this specification. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments of this specification should be included within the protection scope of one or more embodiments of this specification.

Claims

1. A data table conversion method, characterized in that, The method is applied to a data governance platform; the data governance platform is used to convert a source table into a governance table, and to convert the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; the method includes: Based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the target field attribute that matches the field attribute of the target field is determined from the field attributes of each field contained in the source table. Based on the first correspondence, the standard field type corresponding to the target field attribute is determined as the target standard field type that matches the target field; Based on the field type mapped to the target standard field type in the target table, the target field in the governance table is stored in the target table.

2. The method according to claim 1, characterized in that, The step of determining the target field attribute that matches the target field attribute from the field attributes of the fields in the source table based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table includes: Calculate the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table; From the field attributes of each field contained in the source table, the field attributes whose similarity to the field attributes of the target field reaches a first threshold are determined as the target field attributes that match the field attributes of the target field.

3. The method according to claim 2, characterized in that, The field attributes include: field name, and / or field comments.

4. The method according to claim 1, characterized in that, The data governance platform stores a second correspondence relationship associated with each database type; the second correspondence relationship includes the correspondence between standard field types and the field types mapped by the standard field types in the target table; Before storing the target field in the governance table in the destination table according to the field type mapped to the target standard field type in the destination table, the method further includes: Based on the second correspondence relationship associated with the database type of the target database corresponding to the target table, the field type mapped by the target standard field type in the target table is determined.

5. The method according to claim 1, characterized in that, Before determining the target field attribute that matches the target field attribute from the field attributes of the fields in the source table based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, the method further includes: Obtain the first mapping relationship pre-configured by the user; or, The first correspondence is automatically created based on the source table.

6. The method according to claim 5, characterized in that, The data governance platform stores a third correspondence relationship associated with each database type; the third correspondence relationship includes the correspondence between the field types of the fields contained in the source table and the standard field types; The automatic creation of the first correspondence based on the source table includes: Based on the third correspondence relationship associated with the database type of the source database, determine the standard field type corresponding to the field type of each field contained in the source table; The first correspondence is created based on the field attributes of each field in the source table and the standard field types that correspond to the field types of each field in the source table.

7. The method according to claim 1, characterized in that, The source table contains fields whose field types include at least geographic field types; during the process of converting the source table into the governance table, the geographic field types contained in the source table are stored in the governance table as string field types.

8. A data table conversion device, characterized in that, The device is applied to a data governance platform; the data governance platform is used to convert a source table into a governance table, and to convert the governance table into a destination table; the data governance platform stores a first correspondence between the field attributes of the fields contained in the source table and the standard field types corresponding to the field types of the fields contained in the source table; the device includes: The first determining unit is configured to determine, based on the similarity between the field attributes of the target field in the governance table and the field attributes of each field contained in the source table, a target field attribute that matches the field attributes of the target field. The second determining unit is used to determine the standard field type corresponding to the target field attribute as the target standard field type that matches the target field based on the first correspondence relationship; A conversion unit is used to store the target field in the governance table in the destination table according to the field type mapped to the target standard field type in the destination table.

9. An electronic device, characterized in that, It includes a communication interface, a processor, a memory, and a bus, wherein the communication interface, the processor, and the memory are interconnected via the bus; The memory stores machine-readable instructions, and the processor executes the method according to any one of claims 1 to 7 by invoking the machine-readable instructions.

10. A machine-readable storage medium, characterized in that, The machine-readable storage medium stores machine-readable instructions, which, when invoked and executed by a processor, implement the method described in any one of claims 1 to 7.

Images