Data model similarity determination method and device, equipment and storage medium

Abstract

The invention provides a data model similarity determination method and device, equipment and a storage medium, and the method comprises the steps: in a plurality of preset data models, obtaining a first data model and a second data model which are associated, the first data model comprises a plurality of first model tables, each first model table comprises a plurality of first model table fields, and the first model table fields comprise a plurality of first model table fields; the second data model comprises a plurality of second model tables, and each second model table comprises a plurality of second model table fields; a preset source table set is obtained, the source table set comprises a plurality of preset source tables, and each preset source table comprises a plurality of preset fields; determining a first reference relationship between the first data model and each preset field in each preset source table; determining a second reference relationship between the second data model and each preset field in each preset source table; and determining a model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship. The efficiency of evaluating whether the data models are similar or not is improved.

Description

Technical Field

[0001] The present application relates to the technical field of data models, and in particular to a method, device, equipment and storage medium for determining similarity of data models. Background Art

[0002] Repeated construction of data models will result in a waste of resources. In order to promptly discover highly similar data models, data warehouse personnel are often required to manually classify data models based on experience, and then compare the number of identical fields in two data models of the same category to evaluate whether the two data models are similar.

[0003] However, this method requires cumbersome manual operations, which makes the efficiency of evaluating whether data models are similar too low. Summary of the invention

[0004] The present application provides a method, apparatus, device and storage medium for determining the similarity of data models, aiming to improve the efficiency of evaluating whether data models are similar.

[0005] In one aspect, the present application provides a method for determining data model similarity, the method comprising:

[0006] From among multiple preset data models, obtain a first data model and a second data model that are associated, wherein the first data model includes multiple first model tables, each of which includes multiple first model table fields, and the second data model includes multiple second model tables, each of which includes multiple second model table fields;

[0007] Acquire a preset source table set, wherein the source table set includes multiple preset source tables, each of the preset source tables includes multiple preset fields, wherein at least one of the multiple preset fields is referenced in at least one of the first model table fields, and at least one of the multiple preset fields is referenced in at least one of the second model table fields;

[0008] Based on the multiple first model tables and the multiple first model table fields, determining a first reference relationship between the first data model and each of the preset fields in each of the preset source tables;

[0009] Based on the plurality of second model tables and the plurality of second model table fields, determining a second reference relationship between the second data model and each of the preset fields in each of the preset source tables;

[0010] Based on the first reference relationship and the second reference relationship, a model similarity between the first data model and the second data model is determined.

[0011] On the other hand, the present application provides a device for determining data model similarity, the device for determining data model similarity comprising:

[0012] A first acquisition unit is used to acquire, from a plurality of preset data models, a first data model and a second data model that are associated, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields;

[0013] A second acquisition unit is used to acquire a preset source table set, wherein the source table set includes multiple preset source tables, each of the preset source tables includes multiple preset fields, wherein at least one of the multiple preset fields is referenced in at least one of the first model table fields, and at least one of the multiple preset fields is referenced in at least one of the second model table fields;

[0014] A first determining unit, configured to determine, based on the plurality of first model tables and the plurality of first model table fields, a first reference relationship between the first data model and each of the preset fields in each of the preset source tables, and to determine, based on the plurality of second model tables and the plurality of second model table fields, a second reference relationship between the second data model and each of the preset fields in each of the preset source tables;

[0015] The second determining unit is used to determine the model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship.

[0016] On the other hand, the present application also provides a computer device, the computer device comprising:

[0017] one or more processors;

[0018] Memory; and

[0019] One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the steps in the above-mentioned method for determining the similarity of data models.

[0020] On the other hand, the present application also provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program is loaded by a processor to execute the steps in the above-mentioned method for determining the similarity of data models.

[0021] The embodiments of the present application provide a method, apparatus, device and storage medium for determining data model similarity. The method for determining data model similarity includes: obtaining a first data model and a second data model that are associated in a plurality of preset data models, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields; obtaining a preset source table set, wherein the source table set includes a plurality of preset source tables, each of which includes a plurality of preset fields, wherein at least one first model table field references at least one preset field among the plurality of preset fields, and at least one second model table field references at least one preset field among the plurality of preset fields; determining a first reference relationship between the first data model and each preset field in each preset source table based on the plurality of first model tables and the plurality of first model table fields; determining a second reference relationship between the second data model and each preset field in each preset source table based on the plurality of second model tables and the plurality of second model table fields; and determining the model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship. Because if two data models are highly similar, then the source data referenced by the two data models must also be highly similar. Therefore, the embodiment of the present application obtains two associated data models, and determines the model similarity between the two data models based on the reference relationship between the two data models and the preset fields in the preset source tables, thereby realizing automatic calculation of the data model similarity, thereby improving the efficiency of evaluating whether the data models are similar. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 It is a schematic diagram of an application scenario provided by an embodiment of the present application;

[0024] Figure 2 It is a flowchart of an embodiment of a method for determining the similarity of data models provided in an embodiment of the present application;

[0025] Figure 3 This is a schematic diagram of an embodiment of a flow chart for obtaining a first data model and a second data model that are associated with each other, provided in an embodiment of the present application;

[0026] Figure 4 It is a flowchart of an embodiment of determining a first reference relationship between a first data model and each preset field in each preset source table provided in an embodiment of the present application;

[0027] Figure 5 It is a schematic diagram of the structure of an embodiment of a device for determining the similarity of data models provided in an embodiment of the present application;

[0028] Figure 6 It is a schematic diagram of the structure of an embodiment of a computer device provided in the embodiments of the present application. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0030] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of this application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0031] In this application, the word "exemplary" is used to mean "used as an example, illustration, or description." Any embodiment described in this application as "exemplary" is not necessarily to be construed as being preferred or advantageous over other embodiments. The following description is given to enable any technician in the field to implement and use the present application. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present application can be implemented without using these specific details. In other instances, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present application with unnecessary details. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present application.

[0032] The embodiments of the present application provide a method, apparatus, device and storage medium for determining the similarity of a data model, which are described in detail below.

[0033] like Figure 1 As shown, Figure 1 is a schematic diagram of an application scenario provided by an embodiment of the present application. The application scenario may include a computer device 100, in which a device for determining data model similarity is integrated, such as Figure 1 A computer device 100 in FIG.

[0034] In the embodiment of the present application, the computer device 100 may be a terminal or a server. When the computer device 100 is a server, it may be an independent server, or a server network or server cluster composed of servers. For example, the computer device 100 described in the embodiment of the present application includes but is not limited to a computer, a network host, a single network server, a plurality of network server sets or a cloud server constructed by a plurality of servers. The cloud server is constructed by a large number of computers or network servers based on cloud computing.

[0035] It is understandable that when the computer device 100 in the embodiment of the present application is a terminal, the terminal used may be a device including both receiving and transmitting hardware, that is, a device having receiving and transmitting hardware capable of performing two-way communication on a two-way communication link. Such a device may include: a cellular or other communication device having a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific computer device 100 may be a desktop terminal or a mobile terminal, and the computer device 100 may also be one of a mobile phone, a tablet computer, a laptop computer, a medical auxiliary instrument, etc.

[0036] Those skilled in the art will understand that Figure 1 The application environment shown in the figure is only an example of the application scenario of the present application and does not constitute a limitation on the application scenario of the present application. Other application environments may also include Figure 1 More or less computer equipment as shown in Figure 1 Only one computer device is shown in the figure. It can be understood that the application scenario may also include one or more other computer devices, which are not specifically limited here.

[0037] In addition, if Figure 1 As shown, the application scenario may also include a data warehouse 200 for storing data, such as storing multiple preset data models, preset source table sets, and the like.

[0038] It should be noted that Figure 1 The application scenario diagram shown is merely an example and does not constitute a limitation on the technical solution provided in the embodiments of the present application. A person skilled in the art may appreciate that, with the evolution of application scenarios and the emergence of new business scenarios, the technical solution provided in the embodiments of the present application is also applicable to similar technical problems.

[0039] Next, a method for determining the similarity of data models provided in an embodiment of the present application is introduced.

[0040] In the embodiment of the method for determining data model similarity of the embodiment of the present application, a device for determining data model similarity is used as an execution subject. For the sake of simplicity and ease of description, the execution subject will be omitted in the subsequent method embodiments. The device for determining data model similarity is applied to a computer device.

[0041] See also Figures 2 to 4 , Figure 2 The following is a flow chart of an embodiment of a method for determining the similarity of data models provided in an embodiment of the present application. The method for determining the similarity of data models includes:

[0042] 201. From a plurality of preset data models, obtain a first data model and a second data model that are associated with each other, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields;

[0043] In an embodiment of the present application, the multiple preset data models may be data models in a preset data warehouse (DW), each preset data model includes multiple preset model tables (tables), each preset model table includes multiple preset model table fields (columns), and the preset data model implements corresponding business functions through preset model tables and preset model table fields. Taking the first data model among the multiple preset data models as an example, the multiple preset model tables in the first data model are multiple first model tables, and each first model table includes multiple first model table fields. Taking the second data model as an example, the multiple preset model tables in the second data model are multiple second model tables, and each second model table includes multiple second model table fields.

[0044] In some embodiments of the present application, in multiple preset data models, obtaining the associated first data model and the second data model may include: determining two preset data models that reference the same preset source table in multiple preset data models, wherein the preset source table refers to a table of the ODS (Operational Data Store) layer in the above-mentioned data warehouse, each preset source table includes multiple preset fields, and the preset model table fields in the preset data model can implement corresponding business functions by referencing the preset fields in the preset source table, so the preset data model and the preset source table may have a reference relationship; the two preset data models that reference the same preset source table are used as the associated first data model and the second data model. It can be understood that the associated first data model and the second data model refer to two preset data models that may be highly similar. For example, if the two preset data models both reference the same preset source table, the two preset data models may reference the same preset field of the same preset source table, so that the two preset data models may be highly similar, so the two preset data models can be used as the associated first data model and the second data model.

[0045] 202. Obtain a preset source table set, where the source table set includes multiple preset source tables, each preset source table includes multiple preset fields, wherein at least one first model table field references at least one preset field among the multiple preset fields, and at least one second model table field references at least one preset field among the multiple preset fields;

[0046] In the embodiment of the present application, the preset source table set includes multiple source tables, and the source table is the table of the ODS layer in the above data warehouse. When obtaining the preset source table set, it can be specifically to obtain all the preset source tables in the ODS layer of the above data warehouse, and take the set of all the preset source tables in the ODS layer as the preset source table set.

[0047] 203. Based on the plurality of first model tables and the plurality of first model table fields, determine a first reference relationship between the first data model and each preset field in each preset source table;

[0048] In the embodiment of the present application, the first reference relationship between the first data model and the preset field in the preset source table can be referenced or unreferenced. For example, when the first reference relationship is referenced, it indicates that the first data model references the preset field in the preset source table, and when the first reference relationship is unreferenced, it indicates that the first data model does not reference the preset field in the preset source table. The specific steps for determining the first reference relationship between the first data model and each preset field in each preset source table based on multiple first model tables and multiple first model table fields are described in detail in Figure 4 The contents of the embodiment shown.

[0049] 204. Determine a second reference relationship between the second data model and each preset field in each preset source table based on the plurality of second model tables and the plurality of second model table fields;

[0050] In the embodiment of the present application, the method for determining the second reference relationship is similar to the method for determining the first reference relationship, and will not be elaborated here.

[0051] 205. Determine a model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship.

[0052] In an embodiment of the present application, the model similarity between the first data model and the second data model can be calculated by cosine similarity. Specifically, based on the first reference relationship and the second reference relationship, determining the model similarity between the first data model and the second data model can include: converting the first reference relationship into a first vector; converting the second reference relationship into a second vector; obtaining the cosine similarity between the first vector and the second vector, and using it as the model similarity between the first data model and the second data model. The calculation formula of cosine similarity is as follows:

[0053]

[0054] Among them, x i is the value of the first vector x in the i-th dimension, y i is the value of the second vector y in the i-th dimension.

[0055] In a further embodiment, converting the first reference relationship into a first vector may include: converting the first reference relationship between the first data model and each preset field in each preset source table into values ​​of different dimensions, for example, if the first data model references the first preset field in the first preset source table, the value of the first dimension is recorded as 1, if the first data model does not reference the second preset field in the first preset source table, the value of the second dimension is recorded as 0, if the first data model references the first preset field in the second preset source table, the value of the third dimension is recorded as 1, if the second data model does not reference the first preset field in the second preset source table, the value of the fourth dimension is recorded as 0, thereby obtaining values ​​of multiple dimensions; combining the values ​​of multiple dimensions to obtain a first vector, taking the values ​​of the multiple dimensions as the values ​​of the above four dimensions as an example, the combined first vector is (1,0,1,0), it can be seen that the value of the first vector x in the first dimension is 1, the value of the first vector x in the second dimension is 0, the value of the first vector x in the third dimension is 1, and the value of the first vector x in the fourth dimension is 0. The steps of converting the second reference relationship into the second vector are similar to the steps of converting the first reference relationship into the first vector, and are not described in detail here.

[0056] In some embodiments of the present application, after determining the model similarity between the first data model and the second data model, the method may further include: determining whether the model similarity is greater than a similarity threshold; if the model similarity is greater than the similarity threshold, determining that the first data model and the second data model are similar data models; if the model similarity is less than or equal to the similarity threshold, determining that the first data model and the second data model are dissimilar data models. This enables automatic determination of similar data models.

[0057] The method for determining the data model similarity provided in the embodiment of the present application is that if two data models are highly similar, then the source data referenced by the two data models must also be highly similar. Therefore, by obtaining two associated data models, the model similarity between the two data models is determined based on the reference relationship between the two data models and each preset field in each preset source table, thereby realizing automatic calculation of data model similarity, thereby improving the efficiency of evaluating whether data models are similar.

[0058] In some embodiments of the present application, Figure 3 As shown, obtaining a first data model and a second data model that are associated with each other in a plurality of preset data models includes:

[0059] 301. Determine, from among a plurality of preset model tables included in a plurality of preset data models, a plurality of target model tables that reference the same preset source table;

[0060] In the embodiment of the present application, since the number of preset source tables included in the preset source table set is large, the number of preset model tables included in the multiple preset data models is also large, and the implementation logic of the business functions of the preset data models is complex. For example, the data warehouse often stores multiple data warehouse levels (data warehouse level refers to the level at which data is stored in the data warehouse, multiple data warehouse levels such as the above-mentioned ODS layer, and DWD (DataWareHouse Detail, data detail) layer, DWM (DataWarehouse Middle, data middle layer) layer, etc.) tables, the reference relationship between the preset data model and the preset field in the preset source table is realized by indirect reference of tables of multiple different data warehouse levels, making the reference relationship between the preset data model and the preset field in the preset source table complex. Therefore, it is difficult to directly determine multiple target model tables that reference the same preset source table based on these complex multi-layer reference relationships. For this reason, the embodiment of the present application can use a graph database to determine multiple target model tables that reference the same preset source table. A graph database such as NebulaGraph, a graph database of NebulaGraph, can be used to process ultra-large-scale data sets of hundreds of billions of vertices and trillions of edges.

[0061] Specifically, in multiple preset model tables included in multiple preset data models, determining multiple target model tables that reference the same preset source table can include: obtaining a preset table set, a preset table reference relationship set, and generating graph data in a graph database based on the table set and the table reference relationship set, wherein the graph data includes multiple first vertices (Vertex) and multiple first edges (Edge), and the first edges are used to connect different first vertices, wherein the multiple first vertices are generated based on the preset table set (the preset table set records multiple different tables in the data warehouse, and each table in the table set serves as a first vertex), and the multiple first edges are generated based on the table reference relationship set. Generate (the preset table reference relationship set includes multiple table reference relationships, each table reference relationship records any two tables in the table set that have a reference relationship, each table reference relationship serves as a first edge, and since each table reference relationship is a unidirectional reference relationship, each first edge is a directed edge), it should be noted that the preset table set includes multiple preset source tables in the preset source table set, and multiple preset model tables in multiple preset data models; in the graph data, determine multiple other first vertices connected to the first vertex where the same preset source table is located; the tables where the multiple other first vertices are located are used as multiple target model tables that reference the same preset source table.

[0062] In a further embodiment, a matching statement in a graph database may be used to query to determine multiple other first vertices in the graph data connected to the first vertex where the same preset source table is located, and the matching statement may be a match statement. Specifically, in the graph data, determining multiple other first vertices connected to the first vertex where the same preset source table is located may include: generating a first matching statement, the starting point of the first matching statement is the first vertex where the preset source table is located, and the maximum length of the matching variable length path of the first matching statement is a first preset value. For example, the first matching statement may include (s:table)-[e:produce*1..31]->(t:table), wherein s:table refers to the starting point as the preset source table, and the maximum length of the matching variable length path is the first preset value, and the first preset value is 31; in the graph data, using the input of the first matching statement, multiple other first vertices connected to the first vertex where the same preset source table is located can be queried. In order to improve the query speed of the matching statement in the graph database, indexes are created for the attributes of each first vertex in advance to use the index for fast query, wherein the attributes of the first vertex include the name of the table of the first vertex and the data warehouse level.

[0063] 302. Determine two target model tables at the same data warehouse level from among the multiple target model tables, where the two target model tables include a first target model table and a second target model table;

[0064] In an embodiment of the present application, since multiple target model tables that reference the same preset source table may be at different data warehouse levels in the data warehouse, and similar data models need to include target model tables at the same data warehouse level, it is necessary to determine two target model tables at the same data warehouse level among multiple target model tables, for example, to determine two target model tables that are at the DWD layer at the same time among multiple target model tables.

[0065] 303. Determine a first data model and a second data model from among a plurality of preset data models, wherein the first data model includes a first target model table, and the second data model includes a second target model table.

[0066] In the embodiment of the present application, the preset data model including the first target model table is used as the first data model, and the preset data model including the second target model table is used as the second data model. It should be noted that if there are multiple preset data models including the first target model table or the preset data model including the second target model table, each preset data model including the first target model table is used as the first data model in turn, and each preset data model including the second target model table is used as the second data model in turn, so as to determine the model similarity between each first data model and each second data model in turn.

[0067] The solution disclosed in the embodiment of the present application determines multiple target model tables that reference the same preset source table, and determines two target model tables at the same data warehouse level among the multiple target model tables, thereby determining a first data model and a second data model, thereby achieving the determination of two preset data models that may be similar.

[0068] In some embodiments of the present application, Figure 4 As shown, based on multiple first model tables and multiple first model table fields, determining a first reference relationship between the first data model and each preset field in each preset source table includes:

[0069] 401. Determine, among multiple preset source tables, a first source table referenced by each first model table;

[0070] In the embodiment of the present application, among multiple preset source tables, source tables referenced by each first model table in the first data model are all used as first source tables, thereby obtaining multiple first source tables.

[0071] In some embodiments of the present application, multiple target model tables that reference the same preset source table determined in step 301 can be used to determine the first source table referenced by each first model table among the multiple preset source tables, so as to make the determination of the first source table faster and more convenient. Specifically, among multiple preset source tables, determining the first source table referenced by each first model table may include: for each first model table in the first data model, left-associate the first model table to multiple target model tables that reference the same preset source table, and the association conditions are that the data warehouse level of the first model table is the same as the data warehouse level of the target model table, and the first model table is the same as the target model table; if the matching result of the left association is successful, it is determined that the first model table also references the same preset source table, and the same preset source table is used as the first source table referenced by the first model table; if the matching result of the left association is failed, it is determined that the first model table does not reference the same preset source table; after left-associating the first model table to multiple target model tables that reference each preset source table in the multiple preset source tables for each first model table in the first data model, the first source table referenced by each first model table can be obtained in the multiple preset source tables.

[0072] 402. Determine, in the first source table referenced by each first model table, a first field referenced by each first model table field in the first model table;

[0073] In the embodiment of the present application, for each first source table referenced by the first model table, in the first source table, the preset fields referenced by each first model table field in the first model table are all used as the first fields, thereby obtaining multiple first fields. It should be noted that in step 402, it is not necessary to perform similar operations as the first source table on other preset source tables except the first source table among the multiple preset source tables, thereby reducing the amount of data calculation and improving the operation efficiency.

[0074] In some embodiments of the present application, a method for determining the first field referenced by each first model table field in the first model table can be implemented using the above-mentioned graph database. The graph data of the above-mentioned graph database may also include multiple second vertices and multiple second edges, and the second edges are used to connect different second vertices, wherein the multiple second vertices are generated based on a preset field set (the preset field set records multiple different fields in the data warehouse, and each field in the field set serves as a second vertex), and the multiple second edges are generated based on a preset field reference relationship set (the preset field reference relationship set includes multiple field reference relationships, each field reference relationship records any two fields in the field set that have a reference relationship, and each field reference relationship serves as a second edge, and since each field reference relationship is a unidirectional reference relationship, each second edge is a directed edge). It should be noted that the field set includes multiple preset fields of the above-mentioned multiple preset source tables and multiple preset model table fields in the above-mentioned multiple preset model tables.

[0075] Specifically, in the first source table referenced by each first model table, determining the first field referenced by each first model table field in the first model table may include: in the graph data, determining multiple other second vertices connected to the second vertex where the same preset field is located, the determination method of the multiple other second vertices is similar to the determination method of the above-mentioned multiple other first vertices, and will not be repeated here. In order to improve the query speed of matching statements in the graph database, indexes are created in advance for the attributes of each second vertex to use the index for fast query, wherein the attributes of the second vertex include the name of the field of the first vertex, the name of the table where the field is located, and the data warehouse level; the fields where the multiple other first vertices are located are used as multiple target model table fields that reference the same preset field; in the first source table referenced by each first model table, the multiple target model table fields that reference the same preset field are used to determine the first field referenced by each first model table field in the first model table, so that the determination of the first field is faster and more convenient.

[0076] In a further embodiment, in the first source table referenced by each first model table, using multiple target model table fields that reference the same preset field, determining the first field referenced by each first model table field in the first model table may include: for each first model table field in the first model table, left-associate the first model table field to multiple target model table fields that reference the same preset field, where the association conditions are that the first model table is the same as the preset model table where the target model table field is located, the data warehouse level of the first model table is the same as the data warehouse level of the preset model table where the target model table field is located, and the first model table field is the same as the target model table field. The same segment; if the matching result of the left association is successful, it is determined that the first model table field also references the same preset field, and the same preset field is used as the first field referenced by the first model table field in the first model table; if the matching result of the left association is failed, it is determined that the first model table field does not reference the same preset field; for the first model table field in each first model table and each preset field in the first source table referenced by the first model table, after left associating the first model table field to multiple target model table fields that reference the preset field, the first field referenced by each first model table field in each first model table can be obtained.

[0077] 403. Determine a first reference relationship between the first data model and each preset field in each preset source table based on the first field referenced by each first model table field in each first model table.

[0078] In the embodiment of the present application, the first source table referenced by each first model table in the first data model is the source table referenced by the first data model; in the source table referenced by the first data model, the first field referenced by each first model table field in each first model table of the first data model is the preset field referenced by the first data model in the source table. Thus, the first reference relationship between the first data model and each preset field in each preset source table is obtained.

[0079] The solution disclosed in the embodiment of the present application first determines the first source table referenced by each first model table in multiple preset source tables, and then determines the first field referenced by each first model table field in each first model table in the first source table, thereby quickly determining the first reference relationship between the first data model and each preset field in each preset source table.

[0080] In order to better implement the method for determining the similarity of data models in the embodiment of the present application, based on the method for determining the similarity of data models, the embodiment of the present application also provides a device for determining the similarity of data models, such as Figure 5 As shown, the data model similarity determination device 500 includes:

[0081] A first acquisition unit 501 is used to acquire associated first data models and second data models from a plurality of preset data models, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields;

[0082] A second acquisition unit 502 is used to acquire a preset source table set, the source table set includes multiple preset source tables, each preset source table includes multiple preset fields, wherein at least one first model table field references at least one preset field among the multiple preset fields, and at least one second model table field references at least one preset field among the multiple preset fields;

[0083] A first determining unit 503 is used to determine a first reference relationship between a first data model and each preset field in each preset source table based on a plurality of first model tables and a plurality of first model table fields, and to determine a second reference relationship between a second data model and each preset field in each preset source table based on a plurality of second model tables and a plurality of second model table fields;

[0084] The second determining unit 504 is configured to determine the model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship.

[0085] The device for determining the data model similarity provided in the embodiment of the present application obtains two associated data models, determines the model similarity between the two data models based on the reference relationship between the two data models and each preset field in each preset source table, and realizes automatic calculation of the data model similarity, thereby improving the efficiency of evaluating whether the data models are similar.

[0086] In addition to the above-described method and device for determining data model similarity, the embodiment of the present application further provides a computer device, which integrates any one of the data model similarity determination devices provided in the embodiment of the present application, and the computer device includes:

[0087] one or more processors;

[0088] Memory; and

[0089] One or more applications, wherein the one or more applications are stored in a memory and are configured to execute, by a processor, the steps of any one of the above-mentioned data model similarity determination method embodiments.

[0090] The present application also provides a computer device that integrates any one of the data model similarity determination devices provided in the present application. Figure 6 As shown, it shows a schematic diagram of the structure of the computer device involved in the embodiment of the present application, specifically:

[0091] The computer device may include one or more processing core processors 601, one or more computer readable storage medium storage units 602, a power supply 603, an input unit 604 and other components. Those skilled in the art will understand that Figure 6 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0092] The processor 601 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. By running or executing software programs and / or modules stored in the storage unit 602 and calling data stored in the storage unit 602, it executes various functions of the computer device and processes data, thereby monitoring the computer device as a whole. Optionally, the processor 601 may include one or more processing cores; preferably, the processor 601 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 601.

[0093] The storage unit 602 can be used to store software programs and modules. The processor 601 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 602. The storage unit 602 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the storage unit 602 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the storage unit 602 may also include a memory controller to provide the processor 601 with access to the storage unit 602.

[0094] The computer device also includes a power supply 603 for supplying power to various components. Preferably, the power supply 603 can be logically connected to the processor 601 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 603 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0095] The computer device may further include an input unit 604, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0096] Although not shown, the computer device may also include a display unit, etc., which will not be described in detail here. Specifically, in the embodiment of the present application, the processor 601 in the computer device will load the executable file corresponding to the process of one or more application programs into the storage unit 602 according to the following instructions, and the processor 601 will run the application program stored in the storage unit 602, thereby realizing various functions, as follows:

[0097] In a plurality of preset data models, a first data model and a second data model that are associated are obtained, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields; a preset source table set is obtained, wherein the source table set includes a plurality of preset source tables, each of which includes a plurality of preset fields, wherein at least one first model table field references at least one preset field among the plurality of preset fields, and at least one second model table field references at least one preset field among the plurality of preset fields; based on the plurality of first model tables and the plurality of first model table fields, a first reference relationship between the first data model and each preset field in each preset source table is determined; based on the plurality of second model tables and the plurality of second model table fields, a second reference relationship between the second data model and each preset field in each preset source table is determined; based on the first reference relationship and the second reference relationship, a model similarity between the first data model and the second data model is determined.

[0098] The method for determining the data model similarity provided in the embodiment of the present application obtains two associated data models, and determines the model similarity between the two data models based on the reference relationship between the two data models and each preset field in each preset source table, thereby realizing automatic calculation of the data model similarity, thereby improving the efficiency of evaluating whether the data models are similar.

[0099] To this end, an embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. The computer-readable storage medium stores a plurality of instructions, which can be loaded by a processor to execute the steps in any method for determining the similarity of a data model provided in an embodiment of the present application. For example, the instruction may execute the following steps:

[0100] In a plurality of preset data models, a first data model and a second data model that are associated are obtained, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields; a preset source table set is obtained, wherein the source table set includes a plurality of preset source tables, each of which includes a plurality of preset fields, wherein at least one first model table field references at least one preset field among the plurality of preset fields, and at least one second model table field references at least one preset field among the plurality of preset fields; based on the plurality of first model tables and the plurality of first model table fields, a first reference relationship between the first data model and each preset field in each preset source table is determined; based on the plurality of second model tables and the plurality of second model table fields, a second reference relationship between the second data model and each preset field in each preset source table is determined; based on the first reference relationship and the second reference relationship, a model similarity between the first data model and the second data model is determined.

[0101] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0102] The above is a detailed introduction to a method, device, equipment and storage medium for determining the similarity of a data model provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope, and the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for determining data model similarity, characterized in that: The method comprises: From among multiple preset data models, obtain a first data model and a second data model that are associated, wherein the first data model includes multiple first model tables, each of which includes multiple first model table fields, and the second data model includes multiple second model tables, each of which includes multiple second model table fields; Acquire a preset source table set, wherein the source table set includes multiple preset source tables, each of the preset source tables includes multiple preset fields, wherein at least one of the multiple preset fields is referenced in at least one of the first model table fields, and at least one of the multiple preset fields is referenced in at least one of the second model table fields; Based on the multiple first model tables and the multiple first model table fields, determining a first reference relationship between the first data model and each of the preset fields in each of the preset source tables; Based on the plurality of second model tables and the plurality of second model table fields, determining a second reference relationship between the second data model and each of the preset fields in each of the preset source tables; Based on the first reference relationship and the second reference relationship, a model similarity between the first data model and the second data model is determined.

2. The method for determining data model similarity according to claim 1, characterized in that: The step of acquiring the associated first data model and the associated second data model from among the plurality of preset data models comprises: Determining, from among the plurality of preset model tables included in the plurality of preset data models, a plurality of target model tables that reference the same preset source table; Among the multiple target model tables, determining two target model tables at the same data warehouse level, the two target model tables comprising a first target model table and a second target model table; Among the multiple preset data models, the first data model and the second data model are determined, the first data model includes the first target model table, and the second data model includes the second target model table.

3. The method for determining data model similarity according to claim 2, characterized in that: The step of determining, from among the plurality of preset model tables included in the plurality of preset data models, a plurality of target model tables that reference the same preset source table comprises: Obtaining a preset table set and a preset table reference relationship set, and generating graph data in a graph database based on the table set and the table reference relationship set, wherein the graph data includes a plurality of first vertices and a plurality of first edges, wherein the first edges are used to connect different first vertices, and the plurality of first vertices are generated based on the table set, and the plurality of first edges are generated based on the table reference relationship set, wherein the table set includes the plurality of preset source tables and the plurality of preset model tables; In the graph data, determining a plurality of other first vertices connected to the first vertex where the same preset source table is located; The tables where the other first vertices are located are used as multiple target model tables that reference the same preset source table.

4. The method for determining data model similarity according to claim 3, characterized in that: The step of determining, in the graph data, a plurality of other first vertices connected to the first vertex where the same preset source table is located comprises: Generate a first matching statement, where the starting point of the first matching statement is a first vertex where the preset source table is located, and the maximum length of the matching variable-length path of the first matching statement is a first preset value; In the graph data, the first matching statement is used to obtain a plurality of other first vertices connected to the first vertex where the same preset source table is located.

5. The method for determining data model similarity according to claim 1, characterized in that: The determining, based on the plurality of first model tables and the plurality of first model table fields, a first reference relationship between the first data model and each of the preset fields in each of the preset source tables comprises: Determine, among the plurality of preset source tables, a first source table referenced by each of the first model tables; In the first source table referenced by each of the first model tables, determining a first field referenced by each of the first model table fields in the first model table; Based on the first field referenced by each of the first model table fields in each of the first model tables, a first reference relationship between the first data model and each of the preset fields in each of the preset source tables is determined.

6. The method for determining data model similarity according to claim 5, characterized in that: The graph data includes a plurality of second vertices and a plurality of second edges, wherein the second edges are used to connect different second vertices, the plurality of second vertices are generated based on a preset field set, and the plurality of second edges are generated based on a preset field reference relationship set, wherein the field set includes the plurality of preset fields and a plurality of preset model table fields in the plurality of preset model tables, and the first source table referenced by each of the first model tables, determining a first field referenced by each of the first model table fields in the first model table, comprises: In the graph data, determining a plurality of other second vertices connected to the second vertex where the same preset field is located; Using the fields where the other first vertices are located as multiple target model table fields that reference the same preset field; In the first source table referenced by each of the first model tables, a first field referenced by each of the first model table fields in the first model table is determined by using a plurality of target model table fields that reference the same preset field.

7. The method for determining data model similarity according to claim 1, characterized in that: The determining, based on the first reference relationship and the second reference relationship, the model similarity between the first data model and the second data model includes: Converting the first reference relationship into a first vector; Converting the second reference relation into a second vector; The cosine similarity between the first vector and the second vector is obtained and used as the model similarity between the first data model and the second data model.

8. A device for determining data model similarity, characterized in that: The data model similarity determination device comprises: A first acquisition unit is used to acquire, from a plurality of preset data models, a first data model and a second data model that are associated, wherein the first data model includes a plurality of first model tables, each of which includes a plurality of first model table fields, and the second data model includes a plurality of second model tables, each of which includes a plurality of second model table fields; A second acquisition unit is used to acquire a preset source table set, wherein the source table set includes multiple preset source tables, each of the preset source tables includes multiple preset fields, wherein at least one of the multiple preset fields is referenced in at least one of the first model table fields, and at least one of the multiple preset fields is referenced in at least one of the second model table fields; A first determining unit, configured to determine, based on the plurality of first model tables and the plurality of first model table fields, a first reference relationship between the first data model and each of the preset fields in each of the preset source tables, and to determine, based on the plurality of second model tables and the plurality of second model table fields, a second reference relationship between the second data model and each of the preset fields in each of the preset source tables; The second determining unit is used to determine the model similarity between the first data model and the second data model based on the first reference relationship and the second reference relationship.

9. A computer device, characterized in that: The computer device comprises: one or more processors; Memory; and One or more applications, wherein the one or more applications are stored in the memory and are configured to be executed by the processor to implement the steps in the method for determining the similarity of data models described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in the method for determining the similarity of data models according to any one of claims 1 to 7.

Images