Data recognition model training method, data recognition method, and apparatus

Abstract

Embodiments of the present specification provide a data recognition model training method, a data recognition method and apparatus, wherein the data recognition model training method comprises: obtaining metadata of a plurality of data nodes respectively, wherein the metadata carries a data label, and the data label represents a category corresponding to data saved by the data node; analyzing a relationship between the data nodes by using the metadata to obtain an analysis result; constructing a data node relationship graph according to the analysis result, wherein the data node relationship graph takes the data nodes as nodes of the graph, and the relationship between the data nodes is an edge of the graph; generating a node vector of a node in the data node relationship graph according to a text representation of the metadata; and training a data recognition model based on a graph neural network by using the node vector and the data node relationship graph to obtain a trained data recognition model. By generating the node vector, the training efficiency is improved, and the accuracy of the trained data recognition model is improved by comprehensively considering the metadata of each node.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a data recognition model training method. One or more embodiments of this specification also relate to a data recognition method, a data recognition model training device, a data recognition apparatus, a computing device, and a computer-readable storage medium. Background Technology

[0002] With the development of information technology and the widespread use of mobile smart devices, people are constantly generating data. Major companies and institutions have collected and accumulated vast amounts of user data, much of which constitutes private user data. Furthermore, people place great importance on the protection of privacy data. Therefore, identifying private data from massive amounts of data has become a crucial issue in the process of privacy data protection.

[0003] Currently, the common approach is to acquire user data, including private information, train a recognition model using this data, and then use the trained model to identify private data. However, because the amount of private data is limited, the accuracy of the trained recognition model is poor. Therefore, there is an urgent need for an accurate data recognition model training scheme. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a data recognition model training method. One or more embodiments of this specification also relate to a data recognition method, a data recognition model training device, a data recognition apparatus, a computing device, a computer-readable storage medium, and a computer program, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a data recognition model training method is provided, comprising:

[0006] Obtain the metadata of each of the multiple data nodes. The metadata carries data tags, which represent the categories of the data stored by the data nodes.

[0007] Analyze the relationships between data nodes using metadata to obtain analysis results;

[0008] Based on the analysis results, a data node relationship graph is constructed, where data nodes are the nodes of the graph and the relationships between data nodes are the edges of the graph.

[0009] For nodes in the data node relationship graph, generate node vectors based on the textual representation of the metadata;

[0010] By using node vectors and data node relationship graphs, a data recognition model based on graph neural networks is trained to obtain the trained data recognition model.

[0011] According to a second aspect of the embodiments of this specification, a data identification method is provided, comprising:

[0012] Obtain the data to be identified;

[0013] The data to be identified is input into the trained data recognition model to obtain the recognition result corresponding to the data to be identified. The data recognition model is trained using the data recognition model training method provided in the first aspect of the embodiments of this specification.

[0014] According to a third aspect of the embodiments of this specification, a data recognition model training apparatus is provided, comprising:

[0015] The first acquisition module is configured to acquire metadata of multiple data nodes, wherein the metadata carries data tags, and the data tags represent the category corresponding to the data stored by the data node;

[0016] The analysis module is configured to analyze the relationships between the data nodes using metadata to obtain analysis results;

[0017] The construction module is configured to construct a data node relationship graph based on the analysis results, wherein the data node relationship graph uses the data nodes as nodes of the graph and the relationships between the data nodes as edges of the graph;

[0018] The generation module is configured to generate node vectors for nodes in the data node relationship graph based on the textual representation of the metadata.

[0019] The training module is configured to use the node vectors and the data node relationship graph to train a data recognition model based on a graph neural network, thereby obtaining the trained data recognition model.

[0020] According to a fourth aspect of the embodiments of this specification, a data identification device is provided, comprising:

[0021] The second acquisition module is configured to acquire the data to be identified.

[0022] The recognition module is configured to input the data to be recognized into the trained data recognition model and obtain the recognition result corresponding to the data to be recognized. The data recognition model is trained using the data recognition model training method provided in the first aspect of the embodiments of this specification.

[0023] According to a fifth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0024] Memory and processor;

[0025] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method provided in the first or second aspect of the embodiments of this specification.

[0026] According to a sixth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the method provided in the first or second aspect of the embodiments of this specification.

[0027] According to a seventh aspect of the embodiments of this specification, a computer program is provided, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the method provided in the first or second aspect of the embodiments of this specification.

[0028] This specification provides a data recognition model training method according to one embodiment. The method involves acquiring metadata for multiple data nodes, where the metadata carries data tags representing the categories of data stored by the data nodes. The method then analyzes the relationships between the data nodes using the metadata to obtain analysis results. Based on these results, a data node relationship graph is constructed, where data nodes are the nodes of the graph, and the relationships between them are the edges. For each node in the graph, a node vector is generated based on the textual representation of the metadata. Finally, the method uses the node vectors and the data node relationship graph to train a graph neural network-based data recognition model, resulting in a trained data recognition model. By generating node vectors based on the textual representation of the metadata, the method eliminates the need to directly acquire training sample data, thus saving time and improving training efficiency. Furthermore, by integrating the metadata of each node, the method enhances the accuracy of the trained data recognition model. Attached Figure Description

[0029] Figure 1 This is a data processing flowchart of a data processing system provided in one embodiment of this specification;

[0030] Figure 2 This is a schematic diagram of a system architecture applied in a payment scenario, provided by one embodiment of this specification;

[0031] Figure 3 This is a flowchart illustrating a data recognition model training method provided in one embodiment of this specification;

[0032] Figure 4 This is a schematic diagram illustrating the generation of relationships between data nodes according to one embodiment of this specification;

[0033] Figure 5 This is a schematic diagram illustrating the generation of a node vector according to one embodiment of this specification;

[0034] Figure 6 This is a schematic diagram illustrating the training process of a data recognition model provided in one embodiment of this specification;

[0035] Figure 7 This is a flowchart illustrating a data identification method provided in one embodiment of this specification;

[0036] Figure 8 This is a data identification system architecture diagram provided in one embodiment of this specification;

[0037] Figure 9 This is a flowchart illustrating a data recognition model training method and a data recognition process, provided in one embodiment of this specification.

[0038] Figure 10 This is a schematic diagram of the structure of a data recognition model training device provided in one embodiment of this specification;

[0039] Figure 11 This is a schematic diagram of the structure of a data identification device provided in one embodiment of this specification;

[0040] Figure 12 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0041] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0042] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0043] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0044] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0045] Personal information refers to various information recorded electronically or otherwise that can, alone or in combination with other information, be used to identify a specific natural person or reflect the activities of a specific natural person.

[0046] Personal sensitive information: Personal sensitive information, also known as private data, refers to personal information that, if leaked, illegally provided, or misused, may endanger personal safety and property, and is highly likely to cause damage to personal reputation, mental and physical health, or discriminatory treatment.

[0047] Supervised learning: Supervised learning is a method of machine learning that uses a set of samples of known classes to adjust the parameters of a classifier to achieve the required performance. It is also called supervised training or teacher-led learning.

[0048] Classification problems: Classification problems are usually considered to belong to supervised learning, that is, the goal of classification problems is to determine which known sample class a new sample belongs to based on certain features of known samples; classification problems can be further divided into binary classification and multi-class classification problems based on the number of classes.

[0049] Graph: In computer science, a graph is an abstract data type used to implement the concepts of undirected and directed graphs in graph theory from mathematics. A graph data structure contains a finite (potentially mutable) set of nodes and a set of unordered pairs (corresponding to undirected graphs) or ordered pairs (corresponding to directed graphs) as the set of edges (also called arcs in directed graphs).

[0050] Graph Neural Networks (GNNs): GNNs typically use a low-level graph as a computational graph and learn neural network primitives to generate single-node embedding vectors by passing, transforming, and aggregating node feature information across the entire graph. The generated node embedding vectors can be used as input to any differentiable prediction layer and for node classification or predicting connections between nodes. A complete model can be trained end-to-end.

[0051] Graph Attention Network (GAT) uses stacked network layers to obtain the neighborhood features of each node and uses an attention mechanism to assign different weights to different nodes in the neighborhood.

[0052] This specification provides a data recognition model training method. One or more embodiments of this specification also relate to a data recognition method, a data recognition model training device, a data recognition device, a computing device, a computer-readable storage medium, and a computer program, which will be described in detail in the following embodiments.

[0053] With the development of information technology and the popularization of mobile smart devices, people are generating data all the time. Major companies and institutions collect and accumulate a large amount of user data, much of which is user privacy data. See Table 1, which is a table of common privacy data.

[0054] Table 1 Privacy Data Sheet

[0055]

[0056]

[0057] Because privacy data breaches can have serious consequences, upholding data security is a bottom line for large enterprises and institutions. Before implementing privacy data protection measures, a crucial first step is identifying which data is private. For large companies or institutions storing massive amounts of user data, identifying private data fields from tens of thousands of records has become a challenging problem. Currently, privacy data identification can be achieved through the following two methods:

[0058] The first approach is to identify private data based on regular expressions. For example, a phone number can be represented by the regular expression "^1[3-9][0-9]{9}$", and fields in the data that match this regular expression can be used as phone number fields.

[0059] The second approach is a multi-class privacy data identification scheme. This scheme uses a large number of pre-labeled privacy data types and learns a multi-class model through machine learning or deep learning algorithms. Then, for the data to be detected, the multi-class model is used to make predictions, and the category with the highest posterior probability is output as the data type of the data to be identified.

[0060] However, the above methods require sampling data from the database, which makes it difficult to guarantee the confidentiality of privacy data, and frequent large-scale sampling may lead to a decrease in the system's scanning efficiency. Therefore, there is an urgent need for an accurate data identification scheme.

[0061] Specifically, one embodiment of this specification provides a data recognition model training method. This method involves acquiring metadata for multiple data nodes, where the metadata carries data tags representing the categories of data stored by the data nodes; analyzing the relationships between the data nodes using the metadata to obtain analysis results; constructing a data node relationship graph based on the analysis results, where the data nodes are the nodes of the graph and the relationships between the data nodes are the edges; generating node vectors for each node in the data node relationship graph based on the textual representation of the metadata; and training a data recognition model based on a graph neural network using the node vectors and the data node relationship graph to obtain the trained data recognition model. By generating node vectors based on the textual representation of the metadata, the time for acquiring training samples is saved, improving training efficiency. Furthermore, by integrating the metadata of each node, the accuracy of the trained data recognition model is improved. Because this method utilizes metadata and performs privacy data identification based on a graph neural network—for example, using SQL statements to establish database lineage and using a graph neural network to fuse upstream and downstream metadata information—it can identify which data belongs to privacy data from a database containing massive amounts of data. In particular, the solution of this embodiment can solve the problem of privacy information identification in scenarios where data sampling is not possible. Furthermore, since it does not require sampling and individual analysis of each sample, this solution is more efficient and provides better protection for private data.

[0062] It should be noted that the application scenarios of the methods provided in the embodiments of this specification are not limited to the identification of privacy data. The specific data types to be identified shall be selected according to the actual situation, and the embodiments of this specification do not impose any limitations on this.

[0063] See Figure 1 , Figure 1 A data processing flowchart of a data processing system provided in one embodiment of this specification is shown.

[0064] like Figure 1 As shown, the data processing system includes a client and a server:

[0065] The client is configured to send metadata from multiple data nodes to the server. The metadata carries data tags, which represent the categories of data stored by the data nodes.

[0066] The server is configured to analyze the relationships between data nodes using metadata to obtain analysis results; construct a data node relationship graph based on the analysis results, where data nodes are the nodes of the graph and the relationships between data nodes are the edges of the graph; generate node vectors for the nodes in the data node relationship graph based on the text representation of the metadata; and train a data recognition model based on a graph neural network using the node vectors and the data node relationship graph to obtain the trained data recognition model.

[0067] Furthermore, the server can send the trained data recognition model to the client so that the client can use the trained data recognition model to make recommendations; alternatively, the server can also receive the data to be recognized sent by the client, use the trained data recognition model to generate recognition results, and send the recognition results to the client.

[0068] The scheme described in this specification involves obtaining metadata for multiple data nodes, where the metadata carries data tags that represent the categories of data stored by the data nodes. The relationships between the data nodes are analyzed using the metadata to obtain analysis results. A data node relationship graph is constructed based on the analysis results, where the data nodes are the nodes of the graph, and the relationships between the data nodes are the edges. For each node in the data node relationship graph, a node vector is generated based on the textual representation of the metadata. Using the node vectors and the data node relationship graph, a data recognition model based on a graph neural network is trained to obtain the trained data recognition model. By generating node vectors based on the textual representation of the metadata, the time for obtaining training samples is saved, training efficiency is improved, and the accuracy of the trained data recognition model is improved by integrating the metadata of each node.

[0069] One or more embodiments provided in this specification can be applied to payment scenarios. See also... Figure 2 , Figure 2 This specification illustrates a system architecture diagram for a payment scenario based on an embodiment of the present invention. The system may include a server 100 and multiple clients 200. The multiple clients 200 can establish communication connections through the server 100. In the payment scenario, the server 100 is used to provide data identification services between the multiple clients 200. Each client 200 can act as a sender or receiver, and data identification is achieved through the server 100.

[0070] Users can interact with server 100 through client 200 to receive data sent by other clients 200, or send data to other clients 200, etc. In a payment scenario, a user can publish data to be identified to server 100 through client 200 to request data identification. Server 100 generates identification results based on the data identification request and pushes the identification results to other clients that have established communication.

[0071] In this setup, client 200 and server 100 establish a connection via a network. The network provides the medium for communication between the client and server. The network can include various connection types, such as wired or wireless communication links or fiber optic cables, etc.

[0072] Client 200 can be a browser, an app (application), a web application such as an H5 (HyperText Markup Language 5) application, a lightweight application (also known as a mini-program), or a cloud application. Client 200 can be developed based on the corresponding service SDK (Software Development Kit) provided by the server, such as based on the RTC SDK. Client 200 can be deployed on electronic devices and depends on the device or certain apps on the device to run. Electronic devices may have displays and support information browsing, such as personal mobile terminals like mobile phones, tablets, and personal computers. Various other types of applications can also be configured on electronic devices, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, and social media platforms.

[0073] Server 100 may include servers that provide various services, such as servers that provide data recognition for multiple clients, servers that support the models used on the clients for background training, and servers that process the data sent by the clients.

[0074] It should be noted that the server-side 100 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The server can also be a server in a distributed system, or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDN), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0075] It is worth noting that the training of the data recognition model provided in the embodiments of this specification is generally performed by the server. However, in other embodiments of this specification, the client may also have similar functions to the server, thereby performing the training of the data recognition model provided in the embodiments of this specification. In other embodiments, the training of the data recognition model provided in the embodiments of this specification may also be performed jointly by the client and the server.

[0076] See Figure 3 , Figure 3 This specification shows a flowchart of a data recognition model training method according to an embodiment, which specifically includes the following steps:

[0077] Step 302: Obtain the metadata of each of the multiple data nodes. The metadata carries data tags, which represent the category of the data stored by the data node.

[0078] In one or more embodiments of this specification, in order to improve the accuracy of the trained data recognition model, metadata of multiple data nodes can be obtained, and the metadata of multiple data nodes can be used to train the model to generate a more accurate data recognition model.

[0079] Specifically, a data node refers to a node that stores data, including but not limited to databases and tables within databases. Metadata from multiple data nodes is used to characterize the data within those nodes, including but not limited to data names, data types, and data comments. The data tags carried by the metadata characterize the category of the data stored by the data node. These tags can be 0 or 1, where 0 indicates non-private data and 1 indicates private data. The choice depends on the specific circumstances, and this specification does not impose any limitations on this. The category of data can be private or non-private data. Alternatively, the category can be a specific category of private data, such as work-related private data or consumer-related private data. The choice depends on the specific circumstances, and this specification does not impose any limitations on this.

[0080] In practical applications, there are multiple ways to obtain metadata from multiple data nodes. This can be done by manually inputting a large amount of metadata or by reading a large amount of metadata from other data acquisition devices or databases. The specific method to choose from depends on the actual situation, and this specification does not limit the specific methods used in the embodiments.

[0081] In one optional embodiment of this specification, a target database can be determined, each data column in the target database can be used as a data node, and the metadata of each data column can be obtained. That is, the above-mentioned step of obtaining the metadata of multiple data nodes can include the following steps:

[0082] Identify the target database, which includes multiple storage units;

[0083] Obtain metadata for each of the multiple storage units, wherein the metadata includes at least one of the following: data name, data type, and data annotation.

[0084] In one or more embodiments of this specification, since major organizations typically store large amounts of data in databases, a target database can be identified, and metadata can be obtained from multiple storage units of the target database.

[0085] Specifically, the target database includes multiple storage units, each storing multiple data items. The attributes of each storage unit constitute its corresponding metadata. The multiple storage units in the target database can be understood as the aforementioned data nodes, including data columns, data tables, etc. When retrieving the metadata of each of the multiple data nodes, the metadata can be directly obtained from the multiple storage units. The metadata includes, but is not limited to, table names, column names, data types, table comments, and column comments, etc., selected according to the actual situation. This specification does not impose any limitations on this embodiment.

[0086] It should be noted that the solutions in the embodiments of this specification can be applied not only to databases, but also to file systems with relationships, and of course to other systems. The specific choice should be made according to the actual situation, and the embodiments of this specification do not impose any limitations on this.

[0087] By applying the solutions in the embodiments of this specification, and by obtaining the metadata of each of the multiple storage units in the target database, the trained data recognition model becomes more closely aligned with the application scenario, thereby further improving the practicality of the trained data recognition model.

[0088] Step 304: Analyze the relationships between data nodes using metadata to obtain analysis results.

[0089] In one or more embodiments of this specification, after obtaining the metadata of each of the multiple data nodes, the metadata can be used to analyze the relationship between the data nodes and obtain analysis results.

[0090] Specifically, the relationships between data nodes can be understood as data lineage. In practical applications, there are multiple ways to analyze the relationships between data nodes, and the relationships between each data node can be analyzed sequentially. Furthermore, to improve analysis efficiency, the relationships between data nodes can also be obtained directly using database operation statements. That is, the steps described above for analyzing the relationships between data nodes using metadata to obtain analysis results can include the following steps:

[0091] The relationships between data nodes are analyzed using the operation statements in the storage unit to obtain the analysis results.

[0092] Specifically, a data node is a storage unit in the database, metadata is the attribute of the storage unit, and operation statements can be understood as SQL statements or query statements. Operation statements are input by the database user and can be used to add, delete, search, and modify information in the database. The information in the database includes data and metadata stored in tables or columns. Furthermore, all operation statements during database use can be parsed to generate a data node relationship diagram between tables and columns. The data node relationship diagram contains the metadata information of the tables and columns themselves, as well as the relationship information between data nodes, such as relationship types and related operation statements. The specific selection depends on the actual situation, and this specification does not impose any limitations on this aspect.

[0093] See Figure 4 , Figure 4 This diagram illustrates the generation of relationships between data nodes according to an embodiment of this specification. Relationships between data nodes can be generated periodically based on an operation statement parsing tool and incrementally synchronized to the graph database. By parsing operation statements in the database, lineage relationships between data tables and columns are established.

[0094] For example, with Figure 4 Taking the operation statement in the example, the specific operation statement is "SELECT user.name, floor(order.price) as price FROM user, order WHERE user.id=order.uid", which means selecting the name column from the user table and the downward approximation of the corresponding price column from the order table, and generating the user column and price column in a new temporary table respectively. The user table (user table) includes a name column and an id column (user ID column), and the order table (order table) includes a price column and an uid column (user ID column). This operation statement can be parsed using an operation statement parsing tool to generate a result like... Figure 4The relationships between the data nodes shown can be used to construct a data node relationship graph. Furthermore, this data node relationship graph can be synchronized to a graph database.

[0095] By applying the solutions in the embodiments of this specification, the relationships between data nodes are analyzed using the operation statements in the storage unit to obtain analysis results, thereby improving analysis efficiency and further enhancing the efficiency and accuracy of constructing the data node relationship graph.

[0096] Step 306: Construct a data node relationship graph based on the analysis results. The data node relationship graph uses data nodes as nodes and the relationships between data nodes as edges.

[0097] In one or more embodiments of this specification, after obtaining the relationship between data nodes, a data node relationship graph can be further constructed based on the analysis results.

[0098] Specifically, when constructing the data node relationship graph, the acquired data nodes are used as nodes in the data node relationship graph, and the relationships between data nodes are used as edges in the data node relationship graph.

[0099] Step 308: For the nodes in the data node relationship graph, generate node vectors based on the text representation of the metadata.

[0100] In one or more embodiments of this specification, metadata of each of multiple data nodes is obtained, the relationships between the data nodes are analyzed using the metadata, and analysis results are obtained. After constructing a data node relationship graph based on the analysis results, further, node vectors of the nodes in the data node relationship graph can be generated based on the text representation of the metadata, and the generated node vectors can be used to train a data recognition model to generate a trained data recognition model.

[0101] In practical applications, there are multiple ways to generate node vectors. One method is to directly generate node vectors based on the text representations of multiple metadata sources. To integrate the information from each node, the metadata of each node can be concatenated to generate a text representation of the metadata. That is, the steps described above for generating node vectors from the text representations of metadata in a data node relationship graph can include the following steps:

[0102] For nodes in the data node relationship graph, the node metadata is concatenated to generate a text representation of the metadata;

[0103] Map the text representation to node vectors.

[0104] For example, see Figure 5 , Figure 5This specification illustrates a schematic diagram of node vector generation according to one embodiment. (Reference) Figure 4 ,by Figure 4 Taking the name column in the user table as an example, Figure 5 The "Text Representation of Metadata" section shows the text representation of the metadata for the `name` column in the `user` table, specifically including "Column Name: Name", "Table Name: User", "Column Comment: Personal Identification Information", "Table Comment: Include Sensitive Information", and "Data Type: String". Further, the different metadata information is concatenated together, resulting in "Column Name: userName [SEP] Table Name: userInformation [SEP] Column Comment: PII [SEP] Table Comment: Include sensitive info [SEP] Data Type: String". Here, "[SEP]" is a separator with a corresponding numerical representation. After generating the text representation of the metadata, this text representation can be mapped to node vectors.

[0105] By applying the scheme of the embodiments in this specification, for nodes in the data node relationship graph, the metadata of the nodes is spliced ​​together to generate a text representation of the metadata, and the text representation is mapped to the node vector of the node. This integrates the information of each node, expands the node vector, and further improves the accuracy and universality of the trained data recognition model.

[0106] Step 310: Use node vectors and data node relationship graphs to train a data recognition model based on graph neural networks to obtain the trained data recognition model.

[0107] In one or more embodiments of this specification, metadata of each of multiple data nodes is obtained, the relationships between the data nodes are analyzed using the metadata, and analysis results are obtained. A data node relationship graph is constructed based on the analysis results. For each node in the data node relationship graph, a node vector is generated based on the text representation of the metadata. Furthermore, the node vector and the data node relationship graph can be used to train a data recognition model based on a graph neural network to obtain the trained data recognition model.

[0108] It should be noted that since the metadata carries data tags, which represent the category of the data stored by the data node, the generated node vectors also carry data tags. Therefore, by inputting the node vectors and the data node relationship graph into the data recognition model based on the graph neural network, the data recognition model can be reverse-trained using the data tags to generate the trained data recognition model.

[0109] The scheme described in this specification involves obtaining metadata for multiple data nodes, where the metadata carries data tags that represent the categories of data stored by the data nodes. The relationships between the data nodes are analyzed using the metadata to obtain analysis results. A data node relationship graph is constructed based on the analysis results, where the data nodes are the nodes of the graph, and the relationships between the data nodes are the edges. For each node in the data node relationship graph, a node vector is generated based on the textual representation of the metadata. Using the node vectors and the data node relationship graph, a data recognition model based on a graph neural network is trained to obtain the trained data recognition model. By generating node vectors based on the textual representation of the metadata, the time for obtaining training samples is saved, training efficiency is improved, and the accuracy of the trained data recognition model is improved by integrating the metadata of each node.

[0110] In practical applications, graph neural network-based data recognition models include an encoder layer, a neighborhood-based entity representation layer, and a classification layer. The steps described above, which utilize node vectors and a metadata graph to train the graph neural network-based data recognition model and obtain the trained data recognition model, can include the following steps:

[0111] Input the node vector into the encoding layer to generate the corresponding encoding vector for the node vector;

[0112] The encoded vector is input into the entity representation layer to generate entity representations of each node in the data node relationship graph;

[0113] Input the entity representation of each node into the classification layer to generate the prediction results corresponding to the metadata;

[0114] Based on the prediction results and the data tags carried by the metadata, the model parameters of the data recognition model are adjusted to obtain the trained data recognition model.

[0115] In one or more embodiments of this specification, see Figure 6 , Figure 6This diagram illustrates the training process of a data recognition model provided in one embodiment of this specification. Node vectors can be understood as numerical text or numerical vectors. Node vectors can be input into an encoding layer, where a transformer model encodes them to generate encoded vectors. Then, an entity representation layer based on neighbor information fuses upstream and downstream metadata information of the nodes to obtain entity representations for each node. Finally, the output entity representations are passed through a classification layer to output privacy-classifying recognition results, i.e., prediction results. After obtaining the prediction results, the model parameters of the data recognition model can be adjusted based on the prediction results and the data labels carried by the metadata to obtain the trained data recognition model.

[0116] It should be noted that the node neighbors input into the data recognition model can be all neighbors of a node, or neighbors connected by one or more relationships. For example, only the metadata information of upstream and downstream nodes that have a replication relationship with this node can be selected and fed into the data recognition model. The model for encoding node vectors is not limited to the transformer model; it can also be a Long Short-Term Memory (LSTM) network or a Recurrent Neural Network (RNN). The specific choice depends on the actual situation, and the embodiments in this specification do not impose any limitations on this.

[0117] Specifically, the process of inputting node vectors into the coding layer and generating corresponding coding vectors for the node vectors can be represented by the following formulas (1), (2), and (3):

[0118]

[0119]

[0120]

[0121] Where i = 0, 1, ..., M-1, M represents the number of nodes in the data node relationship graph, e i Let h represent the i-th node. ei w represents the node vector of a node. i K represents a word (English) or character (Chinese) in the text representation of metadata. i w represents the total length of the metadata text. Ki The last word / character of the metadata text, x wi It is a character / word w i The vectorized representation of X [CLS]It is a node vector placed at the beginning of the sentence to integrate information from the entire sentence. Then, a multi-layer (L layers in total) Transformer is used to encode the node's metadata.

[0122] It's important to note that during encoding, each character not only contains its own meaning but also incorporates information from its preceding and following nodes. In the subsequent model structure, [CLS] is extracted as the representation of the entire sentence for later classification. The model uses gradient descent to reduce the loss, which in turn pushes the trainable parameters in the transformer to be updated to better values. The information in the encoding corresponding to the [CLS] position is more conducive to the subsequent classification of the entire metadata information; therefore, [CLS] can incorporate information from the entire sentence.

[0123] The scheme implemented in this specification involves inputting node vectors into an encoding layer to generate corresponding encoded vectors, inputting the encoded vectors into an entity representation layer to generate entity representations of each node in the data node relationship graph, inputting the entity representations of each node into a classification layer to generate prediction results corresponding to the metadata, and adjusting the model parameters of the data recognition model based on the prediction results and the data labels carried by the metadata to obtain a trained data recognition model. This approach integrates the metadata of each node, thereby improving the accuracy of the trained data recognition model.

[0124] In one optional embodiment of this specification, after obtaining the encoding vector, the encoding vector can be processed using a pre-set adjacency matrix. That is, the steps described above, which involve inputting the encoding vector into the entity representation layer to generate entity representations of each node in the data node relationship graph, may specifically include the following steps:

[0125] Using a pre-set adjacency matrix, a linear transformation is performed on the encoding vector to generate a linearly transformed encoding vector;

[0126] Based on the encoded vector after linear transformation, calculate the attention coefficient of each node in the data node relationship graph;

[0127] Based on the attention coefficients of each node, a weighted average is applied to the node vectors to generate the entity representation of each node.

[0128] Specifically, in the neighborhood-based entity representation layer, the encoded vector of metadata information is first extracted. And use a learnable adjacency matrix Generate by performing a linear transformation on the encoding vector For neighbor node e j The importance of metadata information The attention coefficients of each neighboring node are obtained by measuring and normalizing the vector dot product. It also employs a multi-head attention mechanism, using the attention coefficient as the weight of each node, and weighted averages the node vectors of different neighboring nodes to obtain node e. i Entity representation based on neighbor information

[0129] It should be noted that the above process can be represented by the following formulas (4), (5), (6), and (7):

[0130]

[0131]

[0132]

[0133]

[0134] in, Let N represent the attention coefficient from node j to i, α represent a shared attention mechanism, δ represent the normalization coefficient, and N represent the normalization coefficient. ei Represents node e i The set of all neighboring nodes of node e (excluding node e itself), where k represents node e. i All neighboring nodes and node e i Any node in the set that it constitutes.

[0135] The scheme implemented in this specification uses a pre-set adjacency matrix to perform a linear transformation on the encoding vector, generating a linearly transformed encoding vector. Based on the linearly transformed encoding vector, the attention coefficient of each node in the data node relationship graph is calculated. Based on the attention coefficient of each node, the node vector is weighted and averaged to generate the entity representation of each node, thereby improving the accuracy of the training process.

[0136] In another optional embodiment of this specification, after obtaining the entity representation of each node, the entity representation can be processed using a pre-set classification matrix. That is, the steps described above, which involve inputting the entity representation of each node into the classification layer to generate the prediction result corresponding to the metadata, may include the following steps:

[0137] Using a pre-set classification matrix, a linear transformation is performed on the entity representation to generate a linearly transformed entity representation;

[0138] The linearly transformed entity representation is input into the classification layer to generate prediction results corresponding to the metadata.

[0139] Specifically, in the classification layer, a learnable classification matrix W can be used. classify right Perform a linear transformation, generate a binary prediction result using the sigmoid function, and determine whether the current node (data column) is private data.

[0140] It should be noted that the above process can be represented by the following formula (8):

[0141]

[0142] Among them, y ei Represents node e i Data tags, This indicates the model's prediction results.

[0143] The scheme implemented in this specification uses a pre-set classification matrix to perform a linear transformation on the entity representation, generating a linearly transformed entity representation. The linearly transformed entity representation is then input into the classification layer to generate prediction results corresponding to the metadata, making the prediction results more accurate and improving the efficiency of subsequent model training.

[0144] In practical applications, after obtaining the prediction results, the loss value can be calculated based on the prediction results and data labels. That is, the steps described above, which involve adjusting the model parameters of the data recognition model based on the prediction results and the data labels carried by the metadata, to obtain the trained data recognition model, can include the following steps:

[0145] Calculate the loss value based on the prediction results and data labels;

[0146] If the loss value is greater than the preset threshold, the model parameters of the data recognition model are adjusted, and the process returns to the step of inputting the node vector into the encoding layer to generate the encoding vector corresponding to the node vector.

[0147] If the loss value is less than or equal to the preset threshold, training is stopped, and the trained data recognition model is obtained.

[0148] Specifically, it should be noted that the loss value can be calculated based on the prediction result and the data label. The data label represents the actual result that the data recognition model outputs. That is, the category represented by the data label is the actual result. When the difference between the prediction result and the actual result is small enough, that is, the loss value is small enough, it means that the prediction result is close enough to the actual result. At this time, the data recognition model training is completed, and the trained data recognition model is obtained.

[0149] In one or more embodiments of this specification, the difference between the prediction result and the actual result of the first preset neural network can be intuitively shown by calculating the first loss value. Subsequently, the first preset neural network can be trained in a targeted manner based on the difference, and the parameters of the first preset neural network can be adjusted, which can effectively improve the training rate and the training effect of the first preset neural network.

[0150] In one possible implementation of this specification, the decision to stop training can be based solely on the relationship between the loss value and a preset threshold. Specifically, if the loss value is greater than the preset threshold, it indicates a significant difference between the predicted result and the data label, suggesting poor recognition ability of the data recognition model. In this case, the model parameters of the data recognition model can be adjusted, and the process can return to the step of inputting node vectors into the encoding layer to generate corresponding encoded vectors. Training of the data recognition model can continue until the loss value is less than or equal to the preset threshold, indicating a smaller difference between the predicted result and the data label. Training can then be stopped, and the trained data recognition model can be obtained.

[0151] The preset threshold is a critical value for the loss. If the loss value is greater than the preset threshold, it indicates that there is still a certain deviation between the prediction result and the actual result of the data recognition model, and the model parameters of the data recognition model still need to be adjusted and the model training should continue. If the loss value is less than or equal to the preset threshold, it indicates that the prediction result of the data recognition model is close enough to the actual result, and training can be stopped. The specific value of the preset threshold is selected according to the actual situation, and this specification does not limit it in any way.

[0152] In another possible implementation of this specification, in addition to comparing the relationship between the loss value and the preset threshold, the number of iterations can also be considered to determine whether the current data recognition model has been trained. Specifically, if the loss value is less than or equal to the preset threshold, it indicates that the difference between the prediction result and the data label is small, and training is stopped to obtain the trained data recognition model. That is, when the loss value is less than or equal to the preset threshold, training can be stopped without considering the number of iterations to obtain the trained data recognition model. If the loss value is greater than the preset threshold, it is determined whether the number of iterations at this moment has reached the preset number of iterations. If the number of iterations at this moment has not reached the preset number of iterations, the model parameters of the data recognition model are adjusted, and the step of inputting the node vector into the encoding layer and generating the encoding vector corresponding to the node vector is returned to continue training the data recognition model until the preset number of iterations is reached, at which point the iteration is stopped, and the trained data recognition model is obtained.

[0153] The preset number of iterations is set according to the actual situation. This specification does not limit this in any way. When the preset number of iterations is reached, it means that the training of the data recognition model is sufficient. At this time, the prediction result of the data recognition model is close enough to the actual result, and training can be stopped.

[0154] In practical applications, the loss value can be calculated using the following formula (9). The preset threshold is selected according to the actual situation, and this specification does not limit it in any way.

[0155]

[0156] Among them, e i Let y represent the i-th node. ei Represents node e i Data tags, ε represents the model's prediction result, and ε represents all nodes in the data node relationship graph.

[0157] It is worth noting that after the model training is completed, the performance of the trained data recognition model can be tested. When the effect and efficiency of the trained data recognition model reach the upper limit, it can be used to identify privacy data. In practical applications, the accuracy, recall and precision can be used to evaluate the model. Among them, the accuracy can be calculated using the following formula (10), the precision can be calculated using the following formula (11), and the recall can be calculated using the following formula (12):

[0158]

[0159]

[0160]

[0161] Wherein, TP (True Positive) represents the number of positive samples predicted as positive, FP (False Positive) represents the number of negative samples predicted as positive, TN (True Negative) represents the number of negative samples predicted as negative, and FN (False Negative) represents the number of positive samples predicted as negative.

[0162] The scheme implemented in this specification determines the specific training status of the data recognition model based on the loss value, and adjusts the model parameters of the data recognition model in reverse based on the loss value if the training is unsuccessful, thereby improving the recognition ability of the trained data recognition model. The training rate is high and the training effect is good.

[0163] See Figure 7 , Figure 7This specification shows a flowchart of a data identification method according to an embodiment, which specifically includes the following steps:

[0164] Step 702: Obtain the data to be identified.

[0165] Step 704: Input the data to be identified into the trained data recognition model to obtain the recognition result corresponding to the data to be identified.

[0166] The data recognition model was trained using the aforementioned data recognition model training method.

[0167] Specifically, the data to be identified refers to the data whose category is to be identified, including but not limited to text data, image data, etc. The specific selection is based on the actual situation, and the embodiments in this specification do not impose any limitations on this.

[0168] By applying the solution of the embodiments in this specification, the data to be identified is input into the trained data recognition model to obtain the recognition result corresponding to the data to be identified, thereby improving the accuracy of the recognition result.

[0169] In one optional embodiment of this specification, the data to be identified includes private data and non-private data. The step of inputting the data to be identified into the trained data recognition model to obtain the recognition result corresponding to the data to be identified may include the following steps:

[0170] The data to be identified is input into the trained data recognition model to obtain the identification result of whether the data to be identified is private data.

[0171] In one possible implementation, data to be identified is obtained, including data A, data B, and data C. Data A, data B, and data C are then input into a trained data recognition model to determine that data A is non-privacy data and data B and data C are privacy data.

[0172] In another possible implementation, the data to be identified is obtained, which includes data A, data B, and data C. Data A, data B, and data C are then input into a trained data identification model to determine that data A is non-privacy data, data B is work-related privacy data, and data C is transaction-related privacy data.

[0173] By applying the scheme of the embodiments in this specification, the data to be identified is input into the trained data recognition model to obtain the recognition result of whether the data to be identified is private data, thereby improving the accuracy of the recognition result.

[0174] See Figure 8 , Figure 8 This specification illustrates an embodiment of a data identification system architecture, specifically including:

[0175] Data Layer: This layer aggregates logs, metadata, and SQL statements. Based on SQL parsing tools, it periodically generates relationships between data nodes and updates them offline to the graph database. By parsing the SQL statements in the database, relationships between data nodes in tables and columns can be established.

[0176] Specifically, the logs include information related to the process by which the SQL parsing tool parses SQL statements in the database to generate lineage relationships. When parsing issues arise, the logs in the data layer can be used for troubleshooting. For example, the lineage relationship records two data columns as having a replication relationship, but when checking the data stored in these columns, it's found that the data in these two columns is not completely consistent. In this case, it's possible that some SQL statements were missed during the parsing process (the data in the data columns is updated via SQL statements, but these SQL statements were missed during parsing). Therefore, the logs can be checked to determine whether the SQL statements related to the data in the two columns were parsed and the specific parsing method used, thus revealing the cause of the problem.

[0177] Metadata Relationship Layer: This layer stores a graph of data node relationships. The nodes in the graph represent tables and columns, and the edges represent the relationships between the data nodes. The graph computation engine is used to complete query requests and computations for upstream and downstream nodes defined in the upper layer, and to monitor responses.

[0178] Model layer: The model layer encodes the metadata information of the nodes, generates text representations, and uses labeled data to train the model. After training, the model's performance is tested. Once the model's performance meets the deployment criteria, it is deployed for privacy data identification.

[0179] Application Layer: The application layer uses the models deployed in the model layer to scan the data warehouse, identify privacy data, and determine privacy data.

[0180] The solution described in this specification employs a graph-based neural network to train a corresponding model using metadata information, enabling automated identification of privacy data in a data warehouse. Furthermore, this solution significantly reduces sampling and sample-by-sample scanning and prediction time, effectively improving efficiency and solving the privacy data identification problem in scenarios where sampling of privacy data is not feasible.

[0181] The following is in conjunction with the appendix Figure 9 Taking the application of the data recognition model training method provided in this specification in a transaction scenario as an example, the data recognition model training method will be further explained. Among other things, Figure 9 This specification illustrates a data recognition model training method and a flowchart of the data recognition process according to an embodiment of this specification, specifically including the following steps:

[0182] Step 902: Obtain metadata for multiple transaction nodes.

[0183] Step 904: Analyze the relationships between transaction nodes using metadata to obtain analysis results.

[0184] Step 906: Construct a transaction node relationship diagram based on the analysis results.

[0185] Step 908: For each node in the transaction node relationship graph, generate a node vector based on the text representation of the metadata.

[0186] Step 910: Using node vectors and transaction node relationship graphs, train a data recognition model based on graph neural networks to obtain the trained data recognition model.

[0187] Step 912: Obtain the transaction data to be identified.

[0188] Step 914: Input the transaction data to be identified into the trained data recognition model to obtain the privacy recognition result corresponding to the transaction data to be identified.

[0189] By applying the scheme of the embodiments of this specification, the metadata of each of multiple transaction nodes is obtained, the relationship between transaction nodes is analyzed using the metadata, the analysis results are obtained, a transaction node relationship graph is constructed based on the analysis results, and for the nodes in the transaction node relationship graph, node vectors are generated based on the text representation of the metadata. Using the node vectors and the transaction node relationship graph, a data recognition model based on a graph neural network is trained to obtain the trained data recognition model. The transaction data to be identified is input into the trained data recognition model to obtain the privacy recognition result corresponding to the transaction data to be identified, thereby improving the accuracy of transaction data recognition.

[0190] Corresponding to the above-described embodiments of the data recognition model training method, this specification also provides embodiments of the data recognition model training apparatus. Figure 10 A schematic diagram of a data recognition model training device according to one embodiment of this specification is shown. Figure 10 As shown, the device includes:

[0191] The first acquisition module 1002 is configured to acquire metadata of multiple data nodes, wherein the metadata carries data tags, and the data tags represent the category corresponding to the data stored by the data nodes;

[0192] Analysis module 1004 is configured to analyze the relationships between the data nodes using metadata to obtain analysis results;

[0193] The construction module 1006 is configured to construct a data node relationship graph based on the analysis results, wherein the data node relationship graph uses the data nodes as nodes of the graph and the relationships between the data nodes as edges of the graph;

[0194] The generation module 1008 is configured to generate a node vector for each node in the data node relationship graph, based on the text representation of the metadata.

[0195] The training module 1010 is configured to use the node vector and the data node relationship graph to train a data recognition model based on a graph neural network, and obtain the trained data recognition model.

[0196] Optionally, the generation module 1008 is further configured to, for nodes in the data node relationship graph, concatenate the metadata of the nodes to generate a text representation of the metadata; and map the text representation to node vectors.

[0197] Optionally, a data node is a storage unit in a database, and metadata is the attribute of the storage unit; the analysis module 1004 is further configured to analyze the relationship between data nodes using operation statements in the storage unit to obtain analysis results.

[0198] Optionally, the first acquisition module 1002 is further configured to determine a target database, wherein the target database includes multiple storage units; and acquire metadata for each of the multiple storage units, wherein the metadata includes at least one of data name, data type, and data annotation.

[0199] Optionally, the data recognition model includes an encoding layer, an entity representation layer, and a classification layer; the training module 1010 is further configured to input node vectors into the encoding layer to generate encoding vectors corresponding to the node vectors; input the encoding vectors into the entity representation layer to generate entity representations of each node in the data node relationship graph; input the entity representations of each node into the classification layer to generate prediction results corresponding to the metadata; and adjust the model parameters of the data recognition model based on the prediction results and the data labels carried by the metadata to obtain the trained data recognition model.

[0200] Optionally, the training module 1010 is further configured to perform a linear transformation on the encoding vector using a pre-set adjacency matrix to generate a linearly transformed encoding vector; calculate the attention coefficient of each node in the data node relationship graph based on the linearly transformed encoding vector; and perform a weighted average of the node vectors based on the attention coefficient of each node to generate an entity representation of each node.

[0201] Optionally, the training module 1010 is further configured to perform a linear transformation on the entity representation using a pre-set classification matrix to generate a linearly transformed entity representation; and input the linearly transformed entity representation into the classification layer to generate a prediction result corresponding to the metadata.

[0202] Optionally, the training module 1010 is further configured to calculate the loss value based on the prediction result and data label; if the loss value is greater than a preset threshold, the model parameters of the data recognition model are adjusted, and the step of inputting the node vector into the encoding layer to generate the encoding vector corresponding to the node vector is returned; if the loss value is less than or equal to the preset threshold, training is stopped, and the trained data recognition model is obtained.

[0203] The scheme described in this specification involves obtaining metadata for multiple data nodes, where the metadata carries data tags that represent the categories of data stored by the data nodes. The relationships between the data nodes are analyzed using the metadata to obtain analysis results. A data node relationship graph is constructed based on the analysis results, where the data nodes are the nodes of the graph, and the relationships between the data nodes are the edges. For each node in the data node relationship graph, a node vector is generated based on the textual representation of the metadata. Using the node vectors and the data node relationship graph, a data recognition model based on a graph neural network is trained to obtain the trained data recognition model. By generating node vectors based on the textual representation of the metadata, the time for obtaining training samples is saved, training efficiency is improved, and the accuracy of the trained data recognition model is improved by integrating the metadata of each node.

[0204] The above is an illustrative scheme of a data recognition model training device according to this embodiment. It should be noted that the technical solution of this data recognition model training device and the technical solution of the data recognition model training method described above belong to the same concept. For details not described in detail in the technical solution of the data recognition model training device, please refer to the description of the technical solution of the data recognition model training method described above.

[0205] Corresponding to the above-described data recognition method embodiments, this specification also provides data recognition device embodiments. Figure 11 A schematic diagram of the structure of a data identification device according to one embodiment of this specification is shown. Figure 11 As shown, the device includes:

[0206] The second acquisition module 1102 is configured to acquire data to be identified;

[0207] The recognition module 1104 is configured to input the data to be recognized into the trained data recognition model to obtain the recognition result corresponding to the data to be recognized, wherein the data recognition model is trained using the above-mentioned data recognition model training method.

[0208] Optionally, the data to be identified includes private data and non-private data; the identification module 1104 is further configured to input the data to be identified into the trained data identification model to obtain the identification result of whether the data to be identified is private data.

[0209] By applying the solution of the embodiments in this specification, the data to be identified is input into the trained data recognition model to obtain the recognition result corresponding to the data to be identified, thereby improving the accuracy of the recognition result.

[0210] The above is an illustrative scheme of a data recognition device according to this embodiment. It should be noted that the technical solution of this data recognition device and the technical solution of the data recognition method described above belong to the same concept. For details not described in detail in the technical solution of the data recognition device, please refer to the description of the technical solution of the data recognition method described above.

[0211] Figure 12 A structural block diagram of a computing device 1200 according to one embodiment of this specification is shown. The components of the computing device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and a database 1250 is used to store data.

[0212] The computing device 1200 also includes an access device 1240, which enables the computing device 1200 to communicate via one or more networks 1260. Examples of such networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 1240 may include one or more of any type of wired or wireless network interface (e.g., Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Networks (WLAN) interface, a Wi-MAX (World Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0213] In one embodiment of this specification, the aforementioned components of the computing device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0214] The computing device 1200 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or PCs. The computing device 1200 can also be a mobile or stationary server.

[0215] The processor 1220 is configured to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-mentioned data recognition model training method or data recognition method.

[0216] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the data recognition model training method or data recognition method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data recognition model training method or data recognition method described above.

[0217] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data recognition model training method or data recognition method.

[0218] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the data recognition model training method or data recognition method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data recognition model training method or data recognition method described above.

[0219] An embodiment of this specification also provides a computer program, wherein when the computer program is executed in a computer, it causes the computer to perform the steps of the above-described data recognition model training method or data recognition method.

[0220] The above is an illustrative example of a computer program according to this embodiment. It should be noted that the technical solution of this computer program belongs to the same concept as the technical solution of the data recognition model training method or data recognition method described above. Details not described in detail in the technical solution of the computer program can be found in the description of the technical solution of the data recognition model training method or data recognition method described above.

[0221] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0222] The computer instructions include computer program code, which may be in the form of source code, object code, executable file, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc.

[0223] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0224] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0225] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A data recognition model training method, comprising: Obtaining metadata for each of multiple data nodes includes: obtaining metadata for each of multiple storage units, wherein the metadata carries data tags, and the data tags represent the category corresponding to the data stored by the data nodes; Analyzing the relationships between the data nodes using metadata to obtain analysis results includes: analyzing the relationships between the data nodes using operation statements in the storage unit to obtain analysis results, wherein the data node is a storage unit and the metadata is an attribute of the storage unit; A data node relationship graph is constructed based on the analysis results, wherein the data nodes are the nodes of the graph, and the relationships between the data nodes are the edges of the graph. For each node in the data node relationship graph, a node vector is generated based on the textual representation of the metadata. Using the node vectors and the data node relationship graph, a data recognition model based on a graph neural network is trained to obtain the trained data recognition model.

2. The method according to claim 1, wherein the step of generating a node vector for a node in the data node relationship graph based on the text representation of the metadata includes: For each node in the data node relationship graph, the metadata of the node is concatenated to generate a text representation of the metadata; The text representation is mapped to the node vector of the node.

3. The method according to claim 1, wherein the step of obtaining the metadata of each of the multiple data nodes includes: Identify a target database, wherein the target database includes multiple storage units; Obtain metadata for each of multiple storage units, wherein the metadata includes at least one of data name, data type, and data annotation.

4. The method according to claim 1, wherein the data recognition model comprises an encoding layer, an entity representation layer, and a classification layer; the step of training a graph neural network-based data recognition model using the node vectors and the data node relationship graph to obtain the trained data recognition model comprises: The node vector is input into the encoding layer to generate the encoding vector corresponding to the node vector; The encoded vector is input into the entity representation layer to generate entity representations of each node in the data node relationship graph; The entity representations of each node are input into the classification layer to generate the prediction results corresponding to the metadata; Based on the prediction results and the data tags carried by the metadata, the model parameters of the data recognition model are adjusted to obtain the trained data recognition model.

5. The method according to claim 4, wherein the step of inputting the encoding vector into the entity representation layer to generate entity representations of each node in the data node relationship graph includes: The encoded vector is linearly transformed using a pre-set adjacency matrix to generate a linearly transformed encoded vector. Based on the linearly transformed encoding vector, calculate the attention coefficient of each node in the data node relationship graph; Based on the attention coefficients of each node, a weighted average is applied to the node vectors to generate entity representations for each node.

6. The method according to claim 4, wherein the step of inputting the entity representation of each node into the classification layer to generate the prediction result corresponding to the metadata includes: Using a pre-set classification matrix, the entity representation is linearly transformed to generate a linearly transformed entity representation; The linearly transformed entity representation is input into the classification layer to generate the prediction result corresponding to the metadata.

7. The method according to claim 4, wherein the step of adjusting the model parameters of the data recognition model based on the prediction result and the data tags carried by the metadata to obtain the trained data recognition model includes: Calculate the loss value based on the prediction results and the data labels; If the loss value is greater than a preset threshold, the model parameters of the data recognition model are adjusted, and the process returns to the step of inputting the node vector into the encoding layer to generate the encoding vector corresponding to the node vector. If the loss value is less than or equal to the preset threshold, training is stopped, and the trained data recognition model is obtained.

8. A data identification method, comprising: Obtain the data to be identified; The data to be identified is input into the trained data recognition model to obtain the recognition result corresponding to the data to be identified, wherein the data recognition model is trained using the method described in any one of claims 1-7.

9. The method according to claim 8, wherein the data to be identified includes private data and non-privacy data; the step of inputting the data to be identified into the trained data recognition model to obtain the recognition result corresponding to the data to be identified includes: The data to be identified is input into the trained data recognition model to obtain the recognition result of whether the data to be identified is private data.

10. A data recognition model training device, comprising: The first acquisition module is configured to acquire metadata of multiple data nodes, including: acquiring metadata of multiple storage units, wherein the metadata carries data tags, and the data tags represent the category corresponding to the data stored by the data nodes; The analysis module is configured to analyze the relationships between the data nodes using metadata to obtain analysis results; The analysis module is further configured to analyze the relationship between the data nodes using the operation statements in the storage unit to obtain analysis results, wherein the data node is a storage unit and the metadata is an attribute of the storage unit; The construction module is configured to construct a data node relationship graph based on the analysis results, wherein the data node relationship graph uses the data nodes as nodes of the graph and the relationships between the data nodes as edges of the graph; The generation module is configured to generate node vectors for nodes in the data node relationship graph based on the textual representation of the metadata. The training module is configured to use the node vectors and the data node relationship graph to train a data recognition model based on a graph neural network, thereby obtaining the trained data recognition model.

11. A data identification device, comprising: The second acquisition module is configured to acquire the data to be identified. The recognition module is configured to input the data to be recognized into a trained data recognition model to obtain the recognition result corresponding to the data to be recognized, wherein the data recognition model is trained using the method described in any one of claims 1-7.

12. A computing device, comprising: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7 or claims 8 to 9.

13. A computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 7 or 8 to 9.

Images