Data benchmarking method, graph neural network model training method and computer equipment

Abstract

The present application relates to a data benchmarking method, a graph neural network model training method and a computer device. The method comprises: obtaining a data table to be benchmarked, extracting original data information from the data table to be benchmarked, the original data information including field sources and field descriptions, determining preset graph structure data; the graph structure data is constructed according to historical field sources, historical field descriptions and standard data elements corresponding to historical field sources, and according to field sources, field descriptions and preset graph structure data, a target data element corresponding to the original data information is obtained, and the accuracy of data benchmarking is improved by using this method.

Description

Technical Field

[0001] The present application relates to the field of deep learning technology, and in particular to a data benchmarking method, a graph neural network model training method, and a computer device. Background Art

[0002] Data benchmarking is an important part of data governance, which is to benchmark non-standard data item representations to data item representations that meet standard specifications. Specifically, data item benchmarking can be divided into two parts: data element benchmarking (composed of three major elements: object, characteristic word, and representation word) and qualifier benchmarking (modifier of the object).

[0003] At present, data benchmarking methods are mainly divided into two categories. One is manual benchmarking, which is time-consuming and labor-intensive. The other is benchmarking using automatic matching algorithms. At present, the automatic matching algorithm for standard data elements and data items mainly uses the literal similarity of data items to achieve benchmarking. This algorithm has a strong reliance on the naming structure specifications of data items, and the naming of data items of most business data is not standardized, resulting in low accuracy of data benchmarking. Summary of the invention

[0004] Based on this, it is necessary to provide a data benchmarking method, a graph neural network model training method and a computer device to address the above-mentioned technical problems.

[0005] In a first aspect, the present application provides a data benchmarking method. The method comprises:

[0006] Get the data table to be benchmarked;

[0007] Extracting original data information from the to-be-benchmarked data table, the original data information including field source and field description;

[0008] Determine preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source;

[0009] According to the field source, the field description and the preset graph structure data, a target data element corresponding to the original data information is obtained.

[0010] In one embodiment, the preset graph structure data is determined in the following manner:

[0011] Get the historical data table;

[0012] Extracting sample data information from the historical data table; the sample data information includes a historical field source and a historical field description;

[0013] Acquire standard data elements corresponding to the sample data information;

[0014] Acquire a historical standard field name from a standard data element corresponding to the sample data information;

[0015] The preset graph structure data is determined according to the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information.

[0016] In one embodiment, the determining of the preset graph structure data according to the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information includes:

[0017] Converting the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information into initial graph structure data;

[0018] A graph neural network model is trained according to the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

[0019] In one embodiment, the converting of the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information into initial graph structure data further includes:

[0020] Inputting the initial graph structure data into a feature extraction model to obtain a vector corresponding to each node in the initial graph structure data;

[0021] A graph neural network model is trained according to the vectors corresponding to each node in the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

[0022] In one of the embodiments, the number of the historical data tables is at least two, and the at least two historical data tables include a first data table to be trained;

[0023] The method further comprises:

[0024] Determine first graph structure data according to the first to-be-trained data table;

[0025] The graph neural network model is trained according to the first graph structure data to obtain a first prediction result of data benchmarking; the first prediction result of data benchmarking is a prediction data element corresponding to the first data table to be trained;

[0026] Acquire standard data elements corresponding to the first to-be-trained data table;

[0027] Obtaining a loss value according to the first prediction result of the data benchmark and a standard data element corresponding to the first data table to be trained;

[0028] The parameters of the graph neural network model are updated according to the loss value to obtain an updated graph neural network model.

[0029] In one embodiment, at least two of the historical data tables further include a second data table to be trained:

[0030] The updated graph neural network model is obtained, and then includes:

[0031] Determine second graph structure data according to the second data table to be trained;

[0032] The updated graph neural network model is trained according to the second graph structure data to obtain a second prediction result of data benchmarking; the second prediction result of data benchmarking is a predicted data element corresponding to the second data table to be trained.

[0033] In one of the embodiments, the to-be-benchmarked data table includes a plurality of original data information;

[0034] The method further comprises:

[0035] Using the preset graph structure data to obtain a target data element corresponding to each of the original data information;

[0036] Acquire a standard data element corresponding to each of the original data information;

[0037] According to the first identification information in each of the target data elements and the second identification information in each of the standard data elements, the accuracy of the preset graph structure data in predicting the standard data element is determined.

[0038] In one embodiment, obtaining the target data element corresponding to the original data information according to the field source, the field description and the preset graph structure data includes:

[0039] According to the field source, the field description and the preset graph structure data, a plurality of predicted data elements corresponding to the original data information are obtained;

[0040] Obtaining standard data elements corresponding to the original data information;

[0041] Comparing each of the predicted data elements with the standard data element respectively to obtain the benchmarking accuracy rate corresponding to each of the predicted data elements;

[0042] According to the alignment accuracy corresponding to each of the predicted data elements, a target data element corresponding to the original data information is obtained.

[0043] In a second aspect, the present application also provides a graph neural network model training method. The method comprises:

[0044] Get the historical data table;

[0045] Extracting sample data information from the historical data table; the sample data information includes a historical field source and a historical field description;

[0046] Acquire standard data elements corresponding to the sample data information;

[0047] Acquire a historical standard field name from a standard data element corresponding to the sample data information;

[0048] Converting the historical field source, the historical field description, the historical standard field name, and the relationship between the standard data elements corresponding to the sample data information into initial graph structure data;

[0049] The graph neural network model is trained according to the initial graph structure data.

[0050] In a third aspect, the present application further provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0051] Get the data table to be benchmarked;

[0052] Extracting original data information from the to-be-benchmarked data table, the original data information including field source and field description;

[0053] Determine preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source;

[0054] According to the field source, the field description and the preset graph structure data, a target data element corresponding to the original data information is obtained.

[0055] In a fourth aspect, the present application further provides a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0056] Get the data table to be benchmarked;

[0057] Extracting original data information from the to-be-benchmarked data table, the original data information including field source and field description;

[0058] Determine preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source;

[0059] According to the field source, the field description and the preset graph structure data, a target data element corresponding to the original data information is obtained.

[0060] In a fifth aspect, the present application further provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the following steps are implemented:

[0061] Get the data table to be benchmarked;

[0062] Extracting original data information from the to-be-benchmarked data table, the original data information including field source and field description;

[0063] Determine preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source;

[0064] According to the field source, the field description and the preset graph structure data, a target data element corresponding to the original data information is obtained.

[0065] The above-mentioned data benchmarking method, graph neural network model training method and computer device obtain the data table to be benchmarked, extract the original data information from the data table to be benchmarked, and the original data information includes field source and field description. The field source and the field description are input into a pre-trained graph neural network model, and the target data element corresponding to the original data information is obtained using the graph structure data preset in the graph neural network model. The target data element corresponding to the original data information can be obtained more accurately, wherein the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source, thereby solving the problem of low accuracy of data benchmarking. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Figure 1 A schematic diagram of a flow chart of a data benchmarking method in one embodiment;

[0067] Figure 2 The middle figure is a schematic diagram of the structure of an embodiment;

[0068] Figure 3 The middle figure is a schematic diagram of the structure of another embodiment;

[0069] Figure 4 A schematic diagram of a process flow of a neural network model training method in one embodiment;

[0070] Figure 5 A schematic diagram of a scenario of training a neural network model in an embodiment;

[0071] Figure 6 It is a flowchart of a neural network model training method in another embodiment;

[0072] Figure 7 The figure is a schematic diagram of a neural network model training process in one embodiment;

[0073] Figure 8 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0074] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0075] In one embodiment, Figure 1 As shown, a data benchmarking method is provided, comprising the following steps:

[0076] Step 102, obtaining a table of data to be compared.

[0077] Step 104: extracting original data information from the data table to be benchmarked, where the original data information includes field sources and field descriptions.

[0078] In this embodiment, the original data information includes but is not limited to field sources and field descriptions.

[0079] Among them, the data tables to be benchmarked include but are not limited to house rental tables, express delivery registration forms, safety valve data tables, and census data tables. Any data table that includes field-related information can be the data table to be benchmarked. Field-related information includes but is not limited to field description, field source, and field name.

[0080] For example, the data table to be benchmarked is a house rental table. The original data information is extracted from the data table to be benchmarked, and the extraction results are shown in Table 1.

[0081] Table 1 Results of extracting raw data information

[0082] Table name Housing rental table Field Source Shell Search Field Description Landlord's ID

[0083] Step 106, determining preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source.

[0084] in, Figure 2The figure is a schematic diagram of the structure of an embodiment, Figure 2 The standard field names in are obtained from the standard data element corresponding to the historical field source.

[0085] Step 108, obtaining the target data element corresponding to the original data information according to the field source, field description and preset graph structure data.

[0086] Among them, according to the transfer relationship between the historical field source, field description and standard data element in the preset graph structure data, and the field source and field description in the original data information, the target data element corresponding to the original data information is matched, and the target data element is the benchmark data obtained based on the original data information.

[0087] It should be noted that Figure 2 It is just one type of graph structure data. Figure 2 The graph structure data obtained by deformation based on the above are all within the protection scope of this embodiment, for example, Figure 3 As shown, Figure 3 The figure is a schematic diagram of the structure in another embodiment; a data element is a basic unit of data, and a data element can be defined as a data unit whose definition, identification, representation and allowed values ​​are described by a set of attributes. Arranging several related data elements into an overall structure in a certain order is a data model.

[0088] Continuing with the previous example, based on the preset graph structure data, the target data element corresponding to the field source "Shell Search" and the field description "landlord's ID card ID" is obtained, and the target data element corresponding to "landlord's ID card ID" is obtained through the pre-trained graph structure. The target data element includes an object, a characteristic word, and a qualifier. In this embodiment, the object in the target data element is "person", the characteristic word is the benchmark standard "ID card ID", and the qualifier is "landlord".

[0089] It can be understood that in this embodiment, the graph structure data is constructed based on the historical field source, the historical field description and the standard data element corresponding to the historical field source. Therefore, the preset graph structure data can accurately represent the transfer relationship between the historical field description, the historical field source and the standard data element. Furthermore, based on the preset graph structure data and the field source and field description in the original data information, the standard data element corresponding to the original data information can be more accurately predicted, that is, the target data element corresponding to the original data information can be more accurately obtained, which solves the problem of low accuracy of data benchmarking.

[0090] In one embodiment, step 108, obtaining a target data element corresponding to the original data information according to the field source, field description and preset graph structure data, includes:

[0091] According to the field source, field description and preset graph structure data, multiple prediction data elements corresponding to the original data information are obtained;

[0092] Obtaining standard data elements corresponding to the original data information;

[0093] Compare each predicted data element with the standard data element respectively to obtain the benchmark accuracy corresponding to each predicted data element;

[0094] According to the benchmarking accuracy corresponding to each predicted data element, the target data element corresponding to the original data information is obtained.

[0095] In this embodiment, the benchmarking accuracy corresponding to each predicted data element can be obtained based on the similarity between the fields in the predicted data element and the fields in the standard data element. The higher the similarity, the higher the accuracy. The benchmarking accuracy corresponding to each predicted data element can also be obtained based on the similarity between the identification information in the predicted data element and the identification information in the standard data element. This implementation does not make specific limitations on how to obtain the benchmarking accuracy corresponding to the predicted data element based on the predicted data element and the standard data element.

[0096] It should be noted that the data element identification information is a language-independent unique identifier assigned to the data element in the data element directory, using an alphanumeric mixed code and including a two-level structure of a data identifier and a version identifier.

[0097] In addition, in this embodiment, there is no specific limitation on how to obtain the target data element based on the benchmarking accuracy corresponding to each predicted data element. For example, the predicted data element with the highest benchmarking accuracy can be used as the final target data element, or the predicted data elements can be sorted in descending order of the benchmarking accuracy to obtain the top n predicted data elements, and the top n predicted data elements can be used as the final target data elements.

[0098] It can be understood that in this embodiment, each predicted data element is compared with the standard data element respectively to obtain the benchmarking accuracy corresponding to each predicted data element. Furthermore, based on the benchmarking accuracy corresponding to each predicted data element, the target data element corresponding to the original data information can be obtained more accurately.

[0099] In one embodiment, Figure 4 As shown, the preset graph structure data is determined by the following method, including the following steps:

[0100] Step 401, obtaining a historical data table.

[0101] Step 402: extract sample data information from the historical data table; the sample data information includes historical field sources and historical field descriptions.

[0102] Step 403: Acquire the standard data element corresponding to the sample data information.

[0103] Step 404: Obtain the historical standard field name from the standard data element corresponding to the sample data information.

[0104] Step 405, determining the preset graph structure data according to the historical field source, the historical field description, the historical standard field name and the transfer relationship between the standard data elements corresponding to the sample data information.

[0105] Among them, Figure 2 As shown, the nodes in the graph structure data include historical field sources, historical field descriptions, historical standard field names, and standard data elements corresponding to the sample data information. The relationship between each node in the graph structure data represents the transfer relationship between the historical field sources, historical field descriptions, historical standard field names, and standard data elements corresponding to the sample data information.

[0106] It can be understood that in this embodiment, the preset graph structure data is determined based on the transfer relationship between the historical field source, the historical field description, the historical standard field name and the standard data element corresponding to the sample data information, so that the preset graph structure data can accurately represent the transfer relationship between the historical field description, the historical field source and the standard data element. Therefore, based on the preset graph structure data and the field source and field description in the original data information, the standard data element corresponding to the original data information can be more accurately predicted, that is, the target data element corresponding to the original data information can be more accurately obtained, which solves the problem of low accuracy of data benchmarking.

[0107] In one embodiment, the preset graph structure data is determined according to the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information, including:

[0108] Convert the historical field source, historical field description, historical standard field name and the transfer relationship between the standard data elements corresponding to the sample data information into initial graph structure data;

[0109] The graph neural network model is trained according to the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

[0110] In this embodiment, the graph neural network model includes but is not limited to a graph convolutional network model, a graph attention network model, a graph autoencoder model, a graph generation network model, and a graph spatiotemporal network model.

[0111] It can be understood that in this embodiment, the transfer relationship between the historical field source, historical field description, historical standard field name and the standard data element corresponding to the sample data information is converted into initial graph structure data, and the graph neural network model is trained based on the initial graph structure data, so that the graph structure data in the trained graph neural network model can accurately represent the transfer relationship between the historical field description, the historical field source and the standard data element. Therefore, based on the preset graph structure data and the field source and field description in the original data information, the standard data element corresponding to the original data information can be more accurately predicted, that is, the target data element corresponding to the original data information can be obtained more accurately, which solves the problem of low accuracy of data benchmarking.

[0112] It is understandable that the representation of nodes in graph structure data will affect the accuracy of subsequent prediction of standard data elements. The present application further improves the accuracy of predicting standard data elements in the following manner.

[0113] In one embodiment, the historical field source, historical field description, historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information are converted into initial graph structure data, and then the following is further included:

[0114] Input the initial graph structure data into the feature extraction model to obtain the vector corresponding to each node in the initial graph structure data;

[0115] The graph neural network model is trained according to the vectors corresponding to each node in the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

[0116] In this embodiment, the feature extraction model includes but is not limited to the BERT model.

[0117] It can be understood that in this embodiment, the feature extraction model is used to perform vector initialization representation on the nodes in the initial graph structure data, that is, the field source, field description, standard field name and data element information involved in the initial graph structure data are represented by vector initialization representation, which can more fully mine the semantic information contained in the nodes in the initial graph structure data and further improve the accuracy of predicting standard data elements.

[0118] In one embodiment, the number of the historical data tables is at least two, and the at least two historical data tables include a first to-be-trained data table;

[0119] Data benchmarking methods also include:

[0120] Determine first graph structure data according to the first data table to be trained;

[0121] Training the graph neural network model according to the first graph structure data obtains a first prediction result of data benchmarking; the first prediction result of data benchmarking is a prediction data element corresponding to the first data table to be trained;

[0122] Obtaining standard data elements corresponding to the first to-be-trained data table;

[0123] Obtaining a loss value according to the first prediction result of the data benchmark and a standard data element corresponding to a to-be-trained data table;

[0124] The parameters of the graph neural network model are updated according to the loss value to obtain an updated graph neural network model.

[0125] It should be noted that this embodiment uses a data table to be trained to illustrate that the parameters of the graph neural network model can be updated through the data tables to be trained in the previous batch. The previous batch here can be a data table to be trained or multiple historical data tables, which is not limited here.

[0126] It can be understood that in this embodiment, in the process of training the graph neural network model, the graph neural network model parameters are updated based on the loss value calculated based on the first data table to be trained, that is, the graph neural network model parameters are updated based on the loss value calculated based on the previous batch of data tables to be trained, so as to provide better model parameter initial values ​​for subsequent standard data element prediction tasks, and improve the accuracy of predicting standard data elements through the graph neural network model by adjusting the input parameters without modifying the graph structure data.

[0127] In another embodiment, the first graph structure data is determined according to the first data table to be trained;

[0128] Input the graph structure data into the feature extraction model to obtain the vector corresponding to each node in the graph structure data;

[0129] The graph neural network model is trained according to the vectors corresponding to each node in the graph structure data to obtain a first prediction result of the data benchmarking; the first prediction result of the data benchmarking is a prediction data element corresponding to the first data table to be trained;

[0130] Obtaining standard data elements corresponding to the first to-be-trained data table;

[0131] Obtaining a loss value according to the first prediction result of the data benchmark and a standard data element corresponding to a to-be-trained data table;

[0132] The parameters of the graph neural network model and the parameters of the feature extraction model are updated according to the loss value to obtain an updated graph neural network model and an updated feature extraction model.

[0133] It should be noted that this embodiment uses a data table to be trained to illustrate that the parameters of the graph neural network model can be updated through the data tables to be trained in the previous batch. The previous batch here can be a data table to be trained or multiple historical data tables, which is not limited here.

[0134] It can be understood that in this embodiment, in the process of training the graph neural network model, the graph neural network model parameters and feature extraction model parameters are updated based on the loss value calculated based on the first data table to be trained, that is, the graph neural network model parameters and feature extraction model parameters are updated based on the loss value calculated based on the previous batch of data tables to be trained, so as to provide better model parameter initial values ​​for subsequent standard data element prediction tasks, and improve the accuracy of predicting standard data elements through the graph neural network model by adjusting the input parameters without modifying the graph structure data.

[0135] In one embodiment, the at least two historical data tables further include a second to-be-trained data table:

[0136] Get the updated graph neural network model, which then includes:

[0137] Determine the second graph structure data according to the second data table to be trained;

[0138] The updated graph neural network model is trained according to the second graph structure data to obtain a second prediction result of the data benchmarking; the second prediction result of the data benchmarking is a predicted data element corresponding to the second data table to be trained.

[0139] It can be understood that in this embodiment, the updated graph neural network model provides better initial values ​​of model parameters compared to the graph neural network model before the update. Therefore, without modifying the graph structure data, the accuracy of predicting standard data elements through the graph neural network model is improved by adjusting the input parameters.

[0140] In one embodiment, the to-be-benchmarked data table includes a plurality of original data information;

[0141] Data benchmarking methods also include:

[0142] Use the preset graph structure data to obtain the target data element corresponding to each original data information;

[0143] Obtaining standard data elements corresponding to each piece of original data information;

[0144] According to the first identification information in each target data element and the second identification information in each standard data element, the accuracy of the preset graph structure data in predicting the standard data element is determined.

[0145] Among them, the accuracy of the preset graph structure data predicting the standard data element is determined according to whether the first identification information in the target data element is consistent with the second identification information in the standard data element. Assuming that there are 5 original data information, 5 target data elements are obtained according to the 5 original data information, and it is judged whether the first identification information in each target data element is consistent with its standard data element. If they are consistent, it means that the target data element is the standard data element. If the first identification information in 3 target data elements is consistent with their standard data elements, and the first identification information in 2 target data elements is inconsistent with the second identification information in their standard data elements, then the accuracy of the preset graph structure data predicting the standard data element is 60%.

[0146] It can be understood that in this embodiment, the accuracy of the pre-trained graph neural network model in predicting the standard data element can be accurately determined through the first identification information in each target data element and the second identification information in each standard data element.

[0147] The embodiment of the present invention provides a graph neural network model training method. Among them, the model training method can be used in a model training device. The model training device can be integrated in a computer device, and the computer device can be a terminal or a server. Among them, the terminal can be a mobile phone, a tablet computer, a laptop computer, a smart TV, a wearable smart device, a personal computer (PC, Personal Computer) and other devices. The server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms.

[0148] See also Figure 5 , is a schematic diagram of a scenario of training a neural network model in an embodiment; Figure 5 As shown, after the computer device obtains the historical data table, it extracts sample data information from the historical data table; the sample data information includes the historical field source and the historical field description, and then obtains the standard data element corresponding to the sample data information, obtains the historical standard field name from the standard data element corresponding to the sample data information, and then converts the historical field source, historical field description, historical standard field name and the relationship between the standard data elements corresponding to the sample data information into graph structure data; finally, the graph structure data is used to train the graph neural network model, thereby realizing the training process of the graph neural network model.

[0149] It should be noted that Figure 5The schematic diagram of the graph neural network model training scenario shown is only an example. The model training scenario described in the embodiment of this application is to more clearly illustrate the technical solution of this application and does not constitute a limitation on the technical solution provided by this application. It is known to those skilled in the art that with the evolution of model training and the emergence of new business scenarios, the technical solution provided by this application is also applicable to similar technical problems.

[0150] Based on the above implementation scenarios, detailed descriptions are given below.

[0151] The embodiment of the present application will be described from the perspective of a model training device, which can be integrated in a computer device. The computer device can be a terminal or a server. The following is a specific process of the graph neural network model training method:

[0152] Get the historical data table;

[0153] Extract sample data information from the historical data table; the sample data information includes historical field source and historical field description;

[0154] Obtaining standard data elements corresponding to sample data information;

[0155] Obtaining historical standard field names from standard data elements corresponding to sample data information;

[0156] Convert the historical field source, historical field description, historical standard field name, and the relationship between the standard data elements corresponding to the sample data information into initial graph structure data;

[0157] Train the graph neural network model based on the initial graph structure data.

[0158] Among them, see Figure 2 , the nodes in the initial graph structure data include historical field sources, historical field descriptions, historical standard field names, and standard data elements corresponding to the sample data information. The relationship between each node in the initial graph structure data represents the transfer relationship between the historical field sources, historical field descriptions, historical standard field names, and standard data elements corresponding to the sample data information. In addition, in this embodiment, the text information corresponding to the nodes of the initial graph structure data is converted into a vector representation, and then the graph neural network model is trained based on the initial graph structure data in which the nodes are converted into vector representations. During the training process, each node vector is continuously updated based on the node vector information around each node. When the number of model iterations reaches the set upper limit or the loss value converges, the updating of each node vector is stopped, so that the transfer relationship between each node vector after training can well represent the correspondence between the sample data information and the standard data element.

[0159] In addition, graph neural network models include but are not limited to graph convolutional network models, graph attention network models, graph autoencoder models, graph generation network models, and graph spatiotemporal network models.

[0160] It can be understood that in this embodiment, the relationship between historical field sources, historical field descriptions, historical standard field names and standard data elements corresponding to sample data information is converted into graph structure data, and the graph structure data is used to train the graph neural network model, so that the graph structure data in the trained graph neural network model can accurately represent the transfer relationship between historical field descriptions, historical field sources and standard data elements. Therefore, based on the trained graph structure data and the field sources and field descriptions in the original data information, the standard data elements corresponding to the original data information can be more accurately predicted, that is, the target data elements corresponding to the original data information can be more accurately obtained, which solves the problem of low accuracy of data benchmarking.

[0161] In one embodiment, this embodiment provides another graph neural network model training method, such as Figure 6 As shown, the method comprises the following steps:

[0162] Step 602: Obtain graph structure data to be trained.

[0163] Among them, the source of the field, field description and transfer path between standard data elements that need to be benchmarked are defined to obtain the graph structure data to be trained. Secondly, the data benchmarking problem is converted into a link prediction task based on the graph structure data. It should be noted that in some scenarios, it is hoped to predict whether there is an edge between given nodes. Such a task is called a link prediction task. Specifically, the basic information of the data table to be trained, such as the table name, field source, field name, field description, etc., is obtained from the metadata of the storage system of the enterprise source data. According to the transfer relationship between the field source, field description, standard field name and standard data element, the graph structure data to be trained is defined. The graph structure data to be trained is as follows: Figure 2 shown.

[0164] Step 604 , converting the nodes in the graph structure data into vectors to obtain node vectorized graph structure data.

[0165] Among them, the deep learning model is used to perform vector initialization representation on the nodes in the graph structure data to be trained, that is, the field source, field description, standard field name and standard data element involved in the graph structure data to be trained are represented by vector initialization, so as to better extract features of the semantic information of the node, so that the vector representation of each node can comprehensively and accurately characterize the overall information of the text, so that the node information will be better described after extraction. The deep learning model can specifically use an NLP model with word and sentence vector generation effect, such as the BERT model.

[0166] Step 606, using the node vectorized graph structure data to train the graph neural network model.

[0167] Among them, see Figure 7 During the training of the graph neural network, the back propagation simultaneously updates the parameters of the deep learning model and the parameters of the graph neural network model, that is, the gradient back propagation algorithm is used to minimize the loss function to optimize the parameters of the deep learning model and the parameters of the graph neural network model; specifically, during the training of the graph neural network model, the graph neural network model parameters and the deep learning model parameters are updated based on the loss value calculated from the previous batch of graph structure data to be trained, so as to provide better model parameter initial values ​​for the subsequent task of predicting standard data elements, and then the node vector is updated again during the training of the graph neural network model. Without modifying the graph structure, the input parameters are adjusted to improve the accuracy of the trained graph neural network model in predicting standard data elements.

[0168] Step 608: If the number of iterations of the graph neural network model reaches the upper limit or the loss value converges, the accuracy of the trained graph neural network model in predicting the standard data element is output.

[0169] Among them, when the upper limit of the number of iterations is reached or the loss value converges, the training of the graph neural network model is terminated; by judging whether the identifier in the data element predicted by the model is consistent with the identifier in the standard data element, a reference for graph neural network model evaluation indicators is provided; specifically, according to whether the identifier in the data element predicted by the model is consistent with the identifier in the standard data element, the accuracy of the trained graph neural network model in predicting the standard data element is determined. Assuming 4 predicted data elements, it is judged whether the identifier in each predicted data element is consistent with its standard data element. If consistent, it means that the predicted data element is the standard data element. If the identifiers in 3 predicted data elements are consistent with the identifiers in the standard data element, and the identifier in 1 predicted data element is inconsistent with the identifier in the standard data element, then the accuracy of the trained graph neural network model in predicting the standard data element is 75%.

[0170] Through the above method, the data benchmarking problem is transformed into a link prediction task based on graph structure data, so that the standard data elements can be predicted more accurately based on the trained graph structure data. In addition, the loss value calculated iteratively by the graph neural network model is used to update the parameters of the deep learning model and the graph neural network model, generate a more accurate model representation, better understand the semantic information in the text features, and further improve the accuracy of the link prediction task. In addition, in the training process of the graph neural model, each round of iteration initializes the vectors of the nodes in the graph structure data, extracts the features of the field-related data based on the deep learning model, provides high-quality feature input for the graph neural network model, and adjusts the input without modifying the graph structure, making the graph neural network model easier to train.

[0171] It should be understood that, although the steps in the flowcharts involved in the above embodiments are displayed in sequence according to the indication of the arrows, these steps are not necessarily executed in sequence according to the order indicated by the arrows. Unless there is a clear explanation in this article, the execution of these steps is not strictly limited in order, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages, and these steps or stages are not necessarily executed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily carried out in sequence, but can be executed in turn or alternately with other steps or at least a part of the steps or stages in other steps.

[0172] In one embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as follows: Figure 8 As shown. The computer device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, a mobile cellular network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, the above method is implemented. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covered on the display screen, or a button, trackball or touchpad set on the computer device housing, or an external keyboard, touchpad or mouse, etc.

[0173] Those skilled in the art will understand that Figure 8The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0174] In one embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above method embodiments when executing the computer program.

[0175] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments are implemented.

[0176] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0177] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties.

[0178] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetoresistive random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, etc., but are not limited to this.

[0179] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0180] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A data benchmarking method, characterized in that: The method comprises: Get the data table to be benchmarked; Extracting original data information from the to-be-benchmarked data table, the original data information including field source and field description; Determine preset graph structure data; the graph structure data is constructed according to the historical field source, the historical field description and the standard data element corresponding to the historical field source; Obtaining a target data element corresponding to the original data information according to the field source, the field description and the preset graph structure data; The preset graph structure data is determined in the following manner: Get the historical data table; Extracting sample data information from the historical data table; the sample data information includes a historical field source and a historical field description; Acquire standard data elements corresponding to the sample data information; Acquire a historical standard field name from a standard data element corresponding to the sample data information; Converting the historical field source, the historical field description, the historical standard field name, and the transfer relationship between the standard data elements corresponding to the sample data information into initial graph structure data; A graph neural network model is trained according to the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

2. The method according to claim 1, characterized in that The converting of the historical field source, the historical field description, the historical standard field name and the transfer relationship between the standard data elements corresponding to the sample data information into initial graph structure data further includes: Inputting the initial graph structure data into a feature extraction model to obtain a vector corresponding to each node in the initial graph structure data; A graph neural network model is trained according to the vectors corresponding to each node in the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes the preset graph structure data.

3. The method according to claim 1, characterized in that The number of the historical data tables is at least two, and at least two of the historical data tables include a first data table to be trained; The method further comprises: Determine first graph structure data according to the first to-be-trained data table; Training a graph neural network model according to the first graph structure data obtains a first prediction result of data benchmarking; the first prediction result of data benchmarking is a prediction data element corresponding to the first data table to be trained; Acquire standard data elements corresponding to the first to-be-trained data table; Obtaining a loss value according to the first prediction result of the data benchmark and a standard data element corresponding to the first data table to be trained; The parameters of the graph neural network model are updated according to the loss value to obtain an updated graph neural network model.

4. The method according to claim 3, characterized in that At least two of the historical data tables further include a second data table to be trained: The updated graph neural network model is obtained, and then includes: Determine second graph structure data according to the second data table to be trained; The updated graph neural network model is trained according to the second graph structure data to obtain a second prediction result of data benchmarking; the second prediction result of data benchmarking is a predicted data element corresponding to the second data table to be trained.

5. The method according to claim 1, characterized in that The data table to be benchmarked includes multiple original data information; The method further comprises: Using the preset graph structure data to obtain a target data element corresponding to each of the original data information; Acquire a standard data element corresponding to each of the original data information; According to the first identification information in each of the target data elements and the second identification information in each of the standard data elements, the accuracy of the preset graph structure data in predicting the standard data element is determined.

6. The method according to claim 1, characterized in that The step of obtaining a target data element corresponding to the original data information according to the field source, the field description and the preset graph structure data includes: According to the field source, the field description and the preset graph structure data, a plurality of predicted data elements corresponding to the original data information are obtained; Obtaining standard data elements corresponding to the original data information; Comparing each of the predicted data elements with the standard data element respectively to obtain the benchmarking accuracy rate corresponding to each of the predicted data elements; According to the alignment accuracy corresponding to each of the predicted data elements, a target data element corresponding to the original data information is obtained.

7. A graph neural network model training method, characterized in that: The method comprises: Get the historical data table; Extracting sample data information from the historical data table; the sample data information includes a historical field source and a historical field description; Acquire standard data elements corresponding to the sample data information; Acquire a historical standard field name from a standard data element corresponding to the sample data information; Converting the historical field source, the historical field description, the historical standard field name, and the relationship between the standard data elements corresponding to the sample data information into initial graph structure data; The graph neural network model is trained according to the initial graph structure data to obtain a trained graph neural network model; the trained graph neural network model includes preset graph structure data.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images