A method, device and computer-readable storage medium for predicting entity tag association

Abstract

The present invention provides an entity label association prediction method, device, system and computer-readable storage medium, the method comprising: determining an entity relationship network, a label relationship network and an entity label association network, the entity label association network including unknown entity label association relationships; constructing an entity similarity graph according to the entity relationship network, constructing a label similarity graph according to the label relationship network and the entity label association network; constructing an entity label association bipartite graph according to the entity label association network; extracting entity features, and constructing label features according to the label similarity graph; integrating the entity similarity graph, the label similarity graph and the entity label association bipartite graph into a graph convolutional network to construct a prediction model; inputting entity features and label features into the prediction model for training until the model converges, and outputting the prediction results of the prediction model. Using the above method, the unknown entity label association relationship can be obtained, thereby improving the completeness of the entity label annotation.

Description

Technical Field

[0001] The present invention belongs to the field of neural networks, and specifically relates to an entity tag association prediction method, device and computer-readable storage medium. Background Art

[0002] This section is intended to provide a background or context to embodiments of the invention that are recited in the claims. No description herein is admitted to be prior art by inclusion in this section.

[0003] With the advent of the big data era, massive amounts of data have emerged. The semantic content of objects such as users, articles, and products has become increasingly rich, and the label information that describes their characteristics has become increasingly complex. The size of the label library used by an e-commerce platform to describe user characteristics often reaches tens of thousands, and the average number of labels for a user is also thousands. In order to meet the current refined description of sample objects, the multi-label learning problem has gradually become one of the research hotspots of academic and industrial circles, and has very wide applications in many fields such as image recognition, text classification, information retrieval, and personalized recommendations.

[0004] However, due to the huge size of the label library and the tens of thousands of samples, the existing annotations of samples are often incomplete and contain a lot of omissions. For example, a movie review platform may only label the movie "Enemy at the Gates" as "history" and "war", but miss labels such as "romance" and "action", resulting in an inaccurate description of the movie.

[0005] Incomplete labeling is very common in multi-label classification problems, which will affect real application scenarios such as accurate recommendation. Therefore, how to improve the completeness of sample data labeling is an urgent problem to be solved. Summary of the invention

[0006] In view of the problems existing in the above-mentioned prior art, a method, device and computer-readable storage medium for predicting entity tag association are proposed. By using this method, device and computer-readable storage medium, the above-mentioned problems can be solved.

[0007] The present invention provides the following solutions.

[0008] In a first aspect, a method for predicting entity label associations is provided, comprising: determining an entity relationship network, a label relationship network, and an entity label association network, wherein the entity label association network contains unknown entity label associations; constructing an entity similarity graph according to the entity relationship network, and constructing a label similarity graph according to the label relationship network and the entity label association network; and constructing an entity label association bipartite graph according to the entity label association network; extracting entity features, and constructing label features according to the label similarity graph; integrating the entity similarity graph, the label similarity graph, and the entity label association bipartite graph into a graph convolutional network to construct a prediction model; inputting entity features and label features into the prediction model for training until the model converges, and outputting the prediction results of the prediction model, wherein the prediction results include the association between each entity and each label.

[0009] In one embodiment, the method further includes: determining an entity set An entity relationship network with multiple entities as nodes Where n is the total number of entities, Used to indicate the relationship between entities in an entity set; determine the label set Label relationship network with multiple labels as nodes in Among them, m is the total number of labels, Used to indicate the relationship between each label in the label set; determine the entity label association network based on the existing annotation information Among them, each entity u i (i=1,2,…,n) and each label h j (j=1,2,…,m) The value is 1 or 0.

[0010] In one implementation, the tag set is a hierarchical tag set, the tag relationship network is formed into a tree structure, and there is a parent-child inheritance relationship between the upper and lower tag nodes of the tree structure.

[0011] In one embodiment, the method further includes: updating existing annotation information according to the hierarchical relationship between tags; wherein, if a specified entity is annotated with a specified tag, all ancestor tags of the specified tag are supplemented for the specified entity according to the tag relationship network.

[0012] In one embodiment, constructing an entity similarity graph according to an entity relationship network also includes: u The adjacency matrix A u Determine the entity similarity graph S u ; Among them, the adjacency matrix A u ∈[0,1] n×n Formed into an n-order square matrix, entity relationship network graph G u is an undirected authorized network, entity ui With entity u i′ The edge weight Used to indicate the closeness of the relationship between two entities.

[0013] In one embodiment, constructing a tag similarity graph based on the tag relationship network and the entity tag association network also includes: Calculate the label information volume IC(h) of each label. The label information volume is used to measure the degree of specificity of each label: use the label information volume of each label and the label relationship network to calculate the similarity between labels and obtain the label similarity graph S h ; Among them, the label similarity graph S h ∈[0,1] m×m It forms an m-order square matrix.

[0014] In one embodiment, the similarity between tags is calculated using the following formula:

[0015]

[0016] Among them, h MICA is the label h j With label h j′ The most informative common ancestor of j With label h j′ The common ancestor label node with the highest label information.

[0017] In one implementation, the tag information amount of each tag is calculated using the following formula:

[0018]

[0019] Where n is the total number of entities, n h is the number of entities known to be labeled with label h.

[0020] In one embodiment, constructing an entity tag association bipartite graph according to an entity tag association network includes: constructing an entity tag association bipartite graph A according to an entity tag association matrix R; wherein, 0 is an all-zero matrix.

[0021] In one embodiment, extracting entity features and constructing label features based on label similarity graphs includes: determining each entity u in the entity set by feature mining i Multidimensional entity features According to the label similarity graph S h Perform symmetric normalization to obtain the normalized matrix According to the normalized matrix Compute the positive-valued pointwise mutual information matrix X (h); According to the positive point-by-point mutual information matrix X (h) Construct label features, where the matrix X (h) The jth row of As the jth label h j Label features

[0022] In one embodiment, entity features and label features are input into a prediction model for training, including: using a multi-layer perceptron at the input end to reduce the dimensions of the input entity features and label features respectively; inputting the reduced-dimensional entity features and label features into the prediction model to obtain an entity node embedding representation corresponding to each entity and a label node embedding representation corresponding to each label; using a multi-layer perceptron at the output end to reduce the dimensions of the entity node embedding representation and the label node embedding representation respectively; performing a dot product operation on the reduced-dimensional entity node embedding representation and the label node embedding to obtain a prediction result.

[0023] In one embodiment, the entity features and label features after dimensionality reduction are input into a graph convolutional network, including: the entity features and label features after dimensionality reduction are input into a first graph convolutional network, the first graph convolutional network introduces the topological structure information of the entity label association bipartite graph through a conversion function, and outputs the intermediate embedding representation of the entity node and the label node; the intermediate embedding representation of the entity node is input into a second graph convolutional network, the second graph convolutional network introduces the topological structure information and edge weight information of the entity similarity graph through a conversion function, and outputs the entity node embedding representation; the intermediate embedding representation of the label node is input into a third graph convolutional network, the third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through a conversion function, and outputs the label node embedding representation.

[0024] In one embodiment, the input multilayer perceptron is used to reduce the dimension of the input entity features and label features respectively, and further includes: the entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as:

[0025]

[0026] in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the input end is Represents entity u i The implicit representation generated by the l-1th hidden layer of the multilayer perceptron, and For entity u i The entity characteristics, W u (l) is the weight parameter matrix to be trained, b u (l)is the bias parameter, σ(·) is the activation function;

[0027] Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as:

[0028]

[0029] in, Indicates label h j The implicit representation generated in the lth hidden layer is Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label feature, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

[0030] In one embodiment, the first graph convolutional network introduces the topological structure information of the entity label associated bipartite graph through a conversion function, including: entity node u i The conversion function of the lth layer of the first graph convolutional network is:

[0031]

[0032] Label node h j The conversion function of the lth layer of the first graph convolutional network is:

[0033]

[0034] in, and Respectively refers to entity u i and label h j The embedding representation produced at layer l of the first graph convolutional network, and and They refer to entity features and label features after dimensionality reduction, respectively. Indicates that in the entity label association bipartite graph, i The associated label node set, Indicates that in the entity label association bipartite graph, j The associated entity node collection, and is the weight parameter matrix of the lth layer to be trained in the first graph convolutional network, and σ is the activation function.

[0035] In one embodiment, the second graph convolutional network introduces the topological structure information and edge weight information of the entity similarity graph through a conversion function, including: entity node u i The conversion function in the lth layer of the second graph convolutional network is:

[0036]

[0037] in, Refers to entity u i The embedding representation generated at layer l is Refers to entity u i ′ is the embedding representation generated at layer l-1, S′ u =S u +I,(D′ u ) i,i′ =∑ i′ (S u ′) i,i′ , S u is the entity similarity graph, is the weight parameter matrix of the lth layer to be trained in the second graph convolutional network, and σ(·) is the activation function.

[0038] In one embodiment, the third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through a conversion function, including: label node h j The conversion function in the lth layer of the third graph convolutional network is:

[0039]

[0040] in, Refers to the label h j The embedding representation generated at layer l is Refers to the label h j The embedding representation generated at layer l-1 is S′ h =S h +I,(D′ h ) j,j′ =∑ j′ (S h ′) j,j′ , S h is the label similarity graph, is the weight parameter matrix of the lth layer to be trained in the third graph convolutional network, and σ(·) is the activation function.

[0041] In one implementation, the activation function σ uses a leaky linear rectifier function LeakyReLU.

[0042] In one embodiment, the dimension reduction of the entity node embedding representation and the label node embedding representation is performed by a multilayer perceptron at the output end, and further includes:

[0043] Entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as:

[0044]

[0045] in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the output is Represents entity u i The implicit representation produced by the l-1th hidden layer of the multilayer perceptron at the output, and For entity u i The entity node embedding representation, W u (l0 is the weight parameter matrix to be trained, b u (l) is the bias parameter, σ(·) is the activation function;

[0046] Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as:

[0047]

[0048] in, Indicates label h j The implicit representation generated by the lth hidden layer of the multilayer perceptron at the corresponding output end is, Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label node embedding representation, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

[0049] In one implementation, the activation function uses a linear rectification function ReLU.

[0050] In one embodiment, a dot multiplication operation is performed on the entity node embedding representation after dimension reduction and the label node embedding to obtain a prediction result, including: obtaining the entity u generated by the output multilayer perceptron i Entity node embedding representation and label h j Label node embedding Computational entity u i With label h j The likelihood score of the association between To predict entity u i and label h j The similarity between .

[0051] In one embodiment, the method further includes: associating the entity tag with the network The target value of the positive sample in is raised to a constant greater than 1 to determine the enhanced correlation matrix Using the Augmented Correlation Matrix The prediction model is iteratively trained as the fitting target until the model converges.

[0052] In one embodiment, it also includes: using a minimum square error loss function Training prediction model; least square error loss function Among them, Y is the prediction result matrix, is the enhanced correlation matrix, λ is the attenuation factor; Ω is the mask matrix of the observed elements; ° is the element-by-element product, Θ refers to all the parameters to be trained in the model, ‖·‖ F is the Frobenius norm, and ‖·‖2 is the L2 norm.

[0053] In one embodiment, the method further includes: optimizing the objective function using an RMSProp optimizer; the optimizer continuously reduces the objective function value and optimizes and updates the model parameters through a gradient descent algorithm until the objective function value reaches convergence and the training is terminated.

[0054] In a second aspect, an entity label association prediction device is provided, which is configured to execute the method of the first aspect, including: a network determination module, used to determine an entity relationship network, a label relationship network and an entity label association network, the entity label association network contains unknown entity label association relationships; a graph construction module, used to construct an entity similarity graph according to the entity relationship network, and to construct a label similarity graph according to the label relationship network and the entity label association network; and to construct an entity label association bipartite graph according to the entity label association network; a feature construction module, used to extract entity features and construct label features according to the label similarity graph; a prediction model module, which integrates the entity similarity graph, the label similarity graph and the entity label association bipartite graph into a graph convolutional network to construct a prediction model; the entity features and label features are input into the prediction model for training until the model converges, and the prediction results of the prediction model are output, and the prediction results include the association relationship between each entity and each label.

[0055] According to a third aspect, an entity tag association prediction device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute: the method according to the first aspect.

[0056] In a fourth aspect, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a program, and when the program is executed by a multi-core processor, the multi-core processor executes the method of the first aspect.

[0057] At least one of the above-mentioned technical solutions adopted in the embodiments of the present application can achieve the following beneficial effects: In this embodiment, the graph convolutional network is used to learn the entity relationship network, the label relationship network and the existing entity label association network as well as the constructed entity features and label features, which can effectively predict the degree of association between each sample and each label, and then realize the label filling of the partially labeled samples, thereby improving the completeness of the entity label annotation.

[0058] It should be understood that the above description is only an overview of the technical solution of the present invention, so that the technical means of the present invention can be more clearly understood and implemented according to the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described below by way of example. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] The advantages and benefits described herein and other advantages and benefits will be apparent to those of ordinary skill in the art by reading the detailed description of the exemplary embodiments below. The accompanying drawings are only used for the purpose of illustrating exemplary embodiments and are not to be considered as limiting the present invention. Moreover, the same reference numerals are used throughout the accompanying drawings to represent the same components. In the accompanying drawings:

[0060] Figure 1 A schematic diagram of a flow chart of an entity tag association prediction method according to an embodiment of the present invention;

[0061] Figure 2 is a schematic diagram of the structure of an entity relationship network according to an embodiment of the present invention;

[0062] Figure 3 is a schematic diagram of the structure of a label relationship network according to an embodiment of the present invention;

[0063] Figure 4 is a schematic diagram of the structure of an entity tag association network according to an embodiment of the present invention;

[0064] Figure 5 A schematic diagram of a flow chart of an entity tag association prediction method according to another embodiment of the present invention;

[0065] Figure 6 A schematic diagram of the structure of a prediction model according to an embodiment of the present invention;

[0066] Figure 7 A schematic diagram of a flow chart of an entity tag association prediction method according to another embodiment of the present invention;

[0067] Figure 8 2 is a schematic diagram of the structure of an entity tag association prediction device according to an embodiment of the present invention;

[0068] Fig. 9 4 is a schematic diagram of the structure of an entity tag association prediction device according to an embodiment of the present invention.

[0069] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts. DETAILED DESCRIPTION

[0070] The exemplary embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the present disclosure to those skilled in the art.

[0071] In the description of the embodiments of the present application, it should be understood that terms such as "including" or "having" are intended to indicate the presence of features, numbers, steps, behaviors, components, parts, or a combination thereof disclosed in this specification, and are not intended to exclude the possibility of the presence of one or more other features, numbers, steps, behaviors, components, parts, or a combination thereof.

[0072] Unless otherwise specified, “ / ” means or. For example, A / B can mean A or B. The “and / or” in this article is merely a way to describe the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0073] The terms "first", "second", etc. are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", etc. may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0074] It should also be noted that, in the absence of conflict, the embodiments and features of the embodiments of the present invention can be combined with each other. The present invention will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0075] Figure 1 This is a flow chart of an entity tag association prediction method according to an embodiment of the present application, which is used to predict the association relationship between all entities and tags. In this process, from the device perspective, the execution subject can be one or more electronic devices; from the program perspective, the execution subject can correspondingly be the program installed on these electronic devices.

[0076] like Figure 1 As shown, the method provided in this embodiment may include the following steps:

[0077] S10, determining an entity relationship network, a label relationship network, and an entity label association network;

[0078] Specifically, an entity relationship network is a network with entities as nodes and relationships between entities as edges, which is used to indicate relationships between entities. An entity relationship network is an undirected weighted network. For example, in a trading system, users can be used as entities, and the relationships between entities can be realized based on factors such as historical transactions between users, user locations, and user attributes.

[0079] Specifically, the above-mentioned label relationship network is a network with labels as nodes and the association relationship between labels as edges, which is used to indicate the association relationship between labels. For example, there is a parent-child association relationship between the label "furniture" and the label "bed", and there is also a parent-child association relationship between the label "bed" and the labels "double bed" and "single bed", so that a tree-structured label relationship network can be formed.

[0080] Specifically, the entity label association network contains at least part of the association between entities and labels. It is understandable that due to the large scale of the label library and the number of samples in the tens of thousands, the existing annotations of samples are often very incomplete and there are a lot of omissions. Therefore, in the existing entity label association network of S10, there are only part of the entity-label association relationship, and there are a large number of unknown entity-label association relationships, which need to be obtained in the subsequent prediction model.

[0081] The above entity relationship network and label relationship network are homogeneous node relationship networks, and the entity label association graph is a heterogeneous node relationship network.

[0082] S20, constructing an entity similarity graph according to the entity relationship network, constructing a label similarity graph according to the label relationship network and the entity label association network; and constructing an entity label association bipartite graph according to the entity label association network;

[0083] Among them, the entity relationship network is an undirected weighted network, and the entity similarity graph is a homogeneous similarity graph constructed based on the entity relationship network. For example, the entity similarity graph can be directly determined according to the adjacency matrix of the entity relationship network to record the closeness of the relationship between every two entities.

[0084] Similarly, based on the label relationship network and the entity label association network, we can know the closeness of the relationship between each two labels, and then construct the label similarity graph. Based on the entity label association network, we can know the closeness of the relationship between each entity and each label, and then construct the entity label association bipartite graph. It can be seen that the bipartite graph is also called a bipartite graph, which is a special model in graph theory.

[0085] S30, extracting entity features, and constructing label features according to the label similarity graph;

[0086] The above entity features are derived from the characteristic information of the entity itself. For example, taking the user as an entity, the entity features can be the user's basic information, consumption ability, behavioral characteristics, psychological characteristics, etc. The above label features are derived and constructed from the label similarity graph.

[0087] S40, inputting the entity similarity graph, the label similarity graph, and the entity label association bipartite graph into a graph convolutional network to construct a prediction model;

[0088] This application constructs two categories of three networks: the relationship network between homogeneous nodes (i.e., entity relationship network and label relationship network) and the relationship network between heterogeneous nodes (i.e., entity label association network). The prediction model constructed in this application can be integrated into the entity similarity graph, label similarity graph, and entity label association bipartite graph corresponding to the above three networks, and then the low-order and high-order topological structures in each network can be learned in model training, and the complex nonlinear relationships between objects can be discovered.

[0089] Furthermore, the prediction model constructed in this application can design a variety of graph convolutional network modules for different input network types, such as the Bi-GCN (bipartite graph convolutional network) module running on the entity label association bipartite graph and the S-GCN module running on the similarity graph. Among them, the Bi-GCN module performs message transmission on the unweighted associated edges between entities and labels to achieve full communication and fusion of the information contained in the two objects; the S-GCN (similarity graph convolutional network) module performs message transmission on the weighted edges of the similarity graph, and aggregates the information contained in adjacent nodes according to the degree of similarity. By stacking multiple layers of graph convolutional network layers for repeated iterations, low-order and high-order topological structures in the network can be repeatedly captured, and complex nonlinear relationships between objects can be discovered.

[0090] S50, input entity features and label features into the prediction model for training until the model converges, and output the final prediction result.

[0091] The entity features and label features are input into the prediction model as initial features. After multiple rounds of learning of each network by the graph convolutional network, the final prediction result is output, which includes the association relationship between each entity and each label.

[0092] It can be understood that the purpose of this application is to predict the missing labels for entities that have been partially annotated. Figure 2 , shows an example of an entity relationship network, which consists of multiple entities and the connection relationships between them, where the feature vector corresponding to each entity, i.e., the entity characteristics, can be determined by existing feature mining technology. Figure 3 , shows an example of a label relationship network, where multiple labels are organized in a tree structure to form a hierarchical structure, i.e., a hierarchical label relationship network. The top label is the "root label" and has no actual meaning. Each intermediate label under it has several sub-labels, and the sub-labels and parent labels have a parent-child inheritance relationship (i.e., an "is-a" relationship). Figure 4 , shows an example of an entity label association network, which can also be viewed as an entity label association matrix, where Figure 2 Some entities in the entity relationship network shown are related to Figure 3 Some of the labels in the label relationship network shown have known associations, that is, elements showing "1" in the association matrix, but at the same time there are still a large number of unknown entity-label associations, that is, elements showing "?" in the association matrix. It can be understood that these unknown entity-label associations do not mean that there is no association between the corresponding entity and the label, but that such an association may not have been discovered yet. It can be seen that the purpose of this application is to predict the missing labels of the entity, in other words, to predict the degree of association between each entity and each label.

[0093] It can be understood that, taking users as entities, two users marked with the same label are usually likely to have similar behavioral characteristics and are more likely to have similar labels. For example, users A and B are both "high-frequency consumers at gas stations", and user A is also a "high-frequency consumer at parking lots". Although user B is not marked as a "high-frequency consumer at parking lots" in the current annotation set, there is a high possibility that user B also has this characteristic. Therefore, building a prediction model based on the entity label association network can better discover potential user similar labels. Based on the entity relationship network, it can be seen that users are not independent nodes, but have certain social relationships, which can usually be modeled as a social network. For example, there is a high correlation between users A and B. When user A is a "high-frequency consumer at parking lots", there is a high possibility that user B also has this characteristic. Based on the label relationship network and the entity label association network, it can be seen that labels are not independent nodes, but have certain correlations. For example, the subjects marked by labels C and D are highly overlapped. When user A is "label C", there is a high possibility that user A also has "label D". As such, the above-mentioned entity relationship networks, label relationship networks, and entity label association networks contain a large number of label-label, entity-label, and entity-entity association relationships, which can be learned through graph convolutional networks.

[0094] In order to achieve the above-mentioned purpose, the present application integrates entity similarity graphs, label similarity graphs and entity label association bipartite graphs constructed based on entity relationship networks, hierarchical label relationship networks and entity label association networks into graph convolutional networks, thereby generating a prediction model to be trained. Afterwards, initial feature vectors are established for entity nodes and label nodes, respectively, and sent to the prediction model to be trained for model training. During the model training process, the topological information in the above-mentioned entity relationship networks, hierarchical label relationship networks and entity label association networks is fully learned using graph convolutional network technology to generate high-quality entity node embedding representations and label node embedding representations. Finally, the vector dot product between the final embedding representation of each entity node and the final embedding representation of each label node can be output as the likelihood prediction result of whether there is an association between each entity and each label, that is, the association relationship between each entity and each label is obtained.

[0095] Furthermore, the entities can be supplementally annotated based on the predicted entity-label association. For example, when the predicted association between the entity and the label is greater than a threshold, the two are considered to be associated and can be annotated accordingly.

[0096] Therefore, in view of the actual situation of incomplete entity labeling, the information extraction and utilization of entities and labels themselves and the association network between the two are optimized, so as to achieve label filling for partially labeled samples and improve the completeness of entity labeling.

[0097] In one implementation, in order to more easily obtain the entity relationship network, S10 may further specifically include:

[0098] Determine the entity set Where n is the total number of entities. Then, we can determine the entity relationship network with multiple entities in the entity set as nodes. in, Used to indicate the relationship between entities in an entity collection.

[0099] Among them, the adjacency matrix of the entity relationship network is Since the entity relationship network is an undirected weighted network, the edge weight between entity i and entity i′ is determined It is used to indicate the closeness of the relationship between two entities, where i = 1, 2, ..., n, i′ = 1, 2, ..., n. The closer the edge weight is to 1, the closer the relationship between the two entities is; the closer the edge weight is to 0, the more distant the relationship between the two entities is; an edge weight of 0 indicates that there is no direct relationship between the two entities.

[0100] In one implementation, in order to obtain the label relationship network, S10 may further specifically include:

[0101] Determine the tag set Where m is the total number of tags. Then, we can determine the tag relationship network with multiple tags in the tag set as nodes. in, Used to indicate the relationship between tags in a tag collection.

[0102] Alternatively, if Figure 3 As shown, multiple labels in the above label set obey the hierarchical structure, so the label relationship network actually constitutes a root tree, that is, there is no directed loop in the network, and only the root node has an in-degree of 0, while the in-degrees of the remaining nodes are all 1.

[0103] In one implementation, in order to obtain the entity tag association network, S10 may further specifically include:

[0104] Determine each entity u based on the existing annotation information i And each label h j The relationship between Get entity label association network Among them, each entity u i (i=1,2,…,n) and each label h j (j=1,2,…,m) The value is 1 or 0. The value is 1 or 0. Represents entity u i With label h jThere is a positive correlation (positive sample) between them. Represents entity u i With label h j The relationship between them is unknown.

[0105] In one embodiment, the tag set For a hierarchical tag set, the tag relationship network is formed into a tree structure, such as Figure 3 , where the upper and lower label nodes of the tree structure have a parent-child inheritance relationship. It can be understood that the labels in the label set are often not independent of each other, but have an inherent dependency relationship. In practical applications, according to the semantic relationship between labels, the label set can usually be organized into a hierarchical structure, that is, a hierarchical label set, so that the characterization of the sample becomes more refined.

[0106] In one embodiment, based on the above-mentioned hierarchical tag set, the method also includes: updating existing annotation information according to the hierarchical relationship between tags; wherein, if a specified entity is annotated with a specified tag, all ancestor tags of the specified tag are supplemented for the specified entity according to the hierarchical relationship between tags.

[0107] Specifically, the known association between entities and labels is recorded as in Represents entity u i With label h j There must be a positive correlation between Does not represent u i With h j There is no association between entities, but their association is currently unknown. Since labels follow a hierarchical structure, the association between entities and labels satisfies the "true-path-rule". If an entity is annotated with a certain label, it is annotated with all ancestor labels of the label, that is, the label can be propagated upward along the label hierarchy.

[0108] In one implementation, to construct an entity similarity graph, S20 may further specifically include:

[0109] According to the entity relationship network diagram G u The adjacency matrix A u Determine the entity similarity graph S u For example, the entity similarity graph S u =A u The above adjacency matrix A u ∈[0,1] n×n It is formed into an n-order square matrix, where the entity u i With entity u i′ The edge weight Used to indicate the closeness of the relationship between two entities.

[0110] It can be understood that since the entity relationship network is an undirected weighted network, it already has the characteristics of a similarity graph, so it can be directly based on the adjacency matrix A u Get similarity graph without additional construction.

[0111] In one implementation, to construct a label similarity graph, S20 may further specifically include the following steps ab:

[0112] a. Entity tag association network Calculate the label information content IC(h) of each label. The label information content is used to indicate the degree of concreteness of each label. The higher the information content, the more specific the label reflects. For example, the information content of the label "sports news" is lower than that of "NBA news" because the former is more general and the latter is more specific.

[0113] Optionally, the tag information amount of each tag can be calculated using the following formula:

[0114] Where n is the total number of entities, n h is the number of entities known to be labeled with label h.

[0115] Of course, other formulas or methods can also be used to calculate the amount of label information of each label, as long as they can indicate the degree of concretization of the label, and this application does not impose any specific restrictions on this.

[0116] b. Use the tag information of each tag and the tag relationship network to calculate the similarity between tags and obtain the tag similarity graph S h ; Among them, the label similarity graph S h Formed into an m-order square matrix, that is, S h ∈[0,1] m×m .

[0117] Optionally, the similarity between tags can be calculated using the following formula:

[0118] Among them, h MICA is the label h j With label h j′ The most informative common ancestor of j With label h j′ The common ancestor label node with the highest label information.

[0119] Of course, other formulas or methods can also be used to calculate label similarity, for example, by judging h j ,h j′Whether there is a common ancestor label, and the distance between the two and the common ancestor label are judged, etc. It can be understood that any method or formula that can indicate the implicit closeness between labels can be used, and this application does not make specific restrictions on this.

[0120] Through the above steps a and b, the implicit similarity measurement between labels is excavated, and then a label similarity graph that can express the similarity between labels is constructed, thereby improving the prediction accuracy of the prediction model.

[0121] In one implementation, constructing an entity tag association bipartite graph according to an entity tag association network includes: constructing an entity tag association bipartite graph A according to an entity tag association matrix R.

[0122] in, 0 is an all-zero matrix, that is, an n+m-order square matrix consisting of elements 0 or 1.

[0123] In one embodiment, extracting entity features and constructing label features according to the label similarity graph in S30 includes: determining each entity u in the entity set by feature mining. i Multidimensional entity features According to the label similarity graph S h Perform symmetric normalization to obtain the normalized matrix According to the normalized matrix Compute the positive-valued pointwise mutual information matrix X (h) ; According to the positive point-by-point mutual information matrix X (h) Construct label features, where the matrix X (h) The jth row of As the jth label h j Label features

[0124] The feature vector of the entity node in this application comes from the characteristic information of the entity itself. Taking the e-commerce platform user as an example, its characteristics may include the user's basic information, consumption ability, behavioral characteristics, psychological characteristics, etc. i The eigenvector of where d (u) is the dimension of the entity feature vector.

[0125] The feature vector of the label node in this application is derived from the label similarity graph.

[0126] First, we can calculate the label relationship matrix S h Perform symmetric normalization to obtain the normalized matrix

[0127] Where D = diag(d1, d2, ..., d m), d i =∑ j (S h ) i,j .

[0128] Afterwards, according to Calculate the positive pointwise mutual information (PPMI) matrix X (h) :

[0129] Among them, take X (h) The i-th row As the i-th label h i The feature vector of .

[0130] In this way, features can be mined from the constructed label similarity graph and the initial label features for the input prediction model can be constructed.

[0131] In one embodiment, reference Figure 5 In S40, the entity features and label features are input into the prediction model for training, including:

[0132] S41. Use the input multilayer perceptron (MLP) to reduce the dimension of entity features and label features respectively;

[0133] S42. Input the entity features and label features after dimension reduction into the prediction model to obtain the entity node embedding representation corresponding to each entity and the label node embedding representation corresponding to each label; wherein the entity similarity graph, label similarity graph entity label association bipartite graph constructed in S20 are integrated into the prediction model.

[0134] S43, reducing the dimension of the entity node embedding representation and the label node embedding representation through an output multilayer perceptron;

[0135] S44. Perform a dot product operation on the entity node embedding representation and the label node embedding after dimensionality reduction to obtain a prediction result.

[0136] Specifically, refer to Figure 6First, the constructed entity features and label features are reduced in dimension through the input multilayer perceptron and unified to the same dimension. Among them, the multilayer perceptron is a feedforward neural network that maps a set of input vectors to a set of output vectors. Then, the topological information on the entity similarity graph and the label similarity graph entity label association bipartite graph is integrated into the node representation through the graph convolutional network (GCN). For example, the Bi-GCN module can be used to perform message propagation on the entity label association bipartite graph to enable the information between the entity and the label to interact. After that, the corresponding S-GCN modules are run on their respective similarity graphs to refine the previous implicit representation again. For example, on the S-GCN1 module. The above step can be repeated several times, that is, stacking several layers of GCN modules to fully capture the low-order and high-order topological information in the graph. Finally, the implicit representation obtained by GCN processing is reduced in dimension through the output multilayer perceptron, and the final prediction result is obtained through the dot multiplication operation.

[0137] In one implementation, in order to reduce the dimension of the entity features and the label features and unify them to the same dimension, S41 specifically includes:

[0138] Receive high-dimensional entity features and label features That is, the feature vector mined or constructed in S103 is iteratively reduced in dimension through the multiple hidden layers of the multi-layer perceptron at the input end, and the entity features and label features after the dimension reduction are output.

[0139] The following is for entity node u i and label node h j The corresponding input multilayer perceptron is introduced.

[0140] Entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as:

[0141]

[0142] in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the input end is Represents entity u i The implicit representation produced by the l-1th hidden layer of the multilayer perceptron at the input, and For entity u i The entity characteristics, W u (l) is the weight parameter matrix to be trained, b u(l) is the bias parameter, σ(·) is the activation function;

[0143] Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as:

[0144]

[0145] in, Indicates label h j The implicit representation generated by the lth hidden layer of the multilayer perceptron at the input end is Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label feature, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

[0146] Through the multilayer perceptron set up as above, in S41, by repeatedly stacking the hidden layers as described above and using the output of the previous hidden layer as the input of the next hidden layer, the input high-dimensional feature vector can be reduced in dimension, and the output result after dimensionality reduction can be used as the input of the next graph neural network module.

[0147] In one implementation, the graph convolutional network is described more specifically, and the S42 may specifically include:

[0148] First, the entity features and label features after dimension reduction are input into the first graph convolutional network (e.g. Figure 6 In the Bi-GCN, the first graph convolutional network introduces the topological structure information of the bipartite graph associated with entity labels through a transformation function, and outputs the intermediate embedding representation of entity nodes and label nodes.

[0149] In one embodiment, the first graph convolutional network introduces topological structure information of the entity label associated bipartite graph through a conversion function, including:

[0150] Take entity node u i For example, entity node u i The conversion function of the lth layer of the first graph convolutional network is:

[0151]

[0152] With label node h j For example, label node h j The conversion function of the lth layer of the first graph convolutional network is:

[0153]

[0154] in, and Respectively refers to entity u i and label h j The embedding representation produced at layer l of the first graph convolutional network, and and They refer to the entity features and label features obtained after dimensionality reduction by S41, Indicates that in the entity label association bipartite graph, i The associated label node set, Indicates that in the entity label association bipartite graph, j The associated entity node collection, and is the weight parameter matrix of the lth layer to be trained in the first graph convolutional network, and σ is the activation function.

[0155] Then, the intermediate embedding representation of the entity node is input into the second graph convolutional network (e.g. Figure 6 The second graph convolutional network introduces the topological structure information and edge weight information of the entity similarity graph through the conversion function, and outputs the entity node embedding representation;

[0156] In one embodiment, the second graph convolutional network introduces topological structure information and edge weight information of the entity similarity graph through a conversion function, including:

[0157] Take entity node u i For example, entity node u i The conversion function in the lth layer of the second graph convolutional network is:

[0158]

[0159] in, Refers to entity u i The embedding representation generated at layer l is Refers to entity u i′ The embedding representation generated at layer l-1 is S′ u =S u +I,(D′ u ) i,i′ =∑ i′ (S u ′) i,i′ , S u is the entity similarity graph, is the weight parameter matrix of the lth layer to be trained in the second graph convolutional network, and σ(·) is the activation function.

[0160] Accordingly, the intermediate embedding representation of the label node is input into the third graph convolutional network (e.g. Figure 6 S-GCN2 in ), the third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through the conversion function, and outputs the label node embedding representation.

[0161] In one embodiment, the third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through a conversion function, including:

[0162] With label node h j For example, label node h j The conversion function in the lth layer of the third graph convolutional network is:

[0163]

[0164] in, Refers to the label h j The embedding representation generated at layer l is Refers to the label h j The embedding representation generated at layer l-1 is S′ h =S h +I,(D′ h ) j,j′ =∑ j′ (S h ′) j,j′ , S h is the label similarity graph, is the weight parameter matrix of the lth layer to be trained in the third graph convolutional network, and σ(·) is the activation function.

[0165] For example, the first graph convolutional network, the second graph convolutional network, and the third graph convolutional network can be stacked with multiple layers (the total number of layers is denoted as L GCN ), the output of the previous layer is used as the input of the next layer, so that the low-order and high-order topological structures in the graph can be fully captured.

[0166] In one embodiment, the activation function σ uses a leaky linear rectifier function LeakyReLU, which is defined as:

[0167]

[0168] Among them, α∈(0,1) is a set constant, which represents the slope of the third quadrant ray in the activation function graph.

[0169] In one implementation, in order to reduce the dimension of the node embedding representation generated by the graph convolution calculation and unify it to the same dimension, S43 specifically includes:

[0170] Receive high-dimensional entity node embedding representation and label node embedding representation The dimension reduction is performed through the multiple hidden layers of the multi-layer perceptron at the output end, and the entity features after dimension reduction are output. and label features

[0171] The following is for entity node u i and label node h j The corresponding output end of the multilayer perceptron is introduced.

[0172] Entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as:

[0173]

[0174] in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the output is Represents entity u i The implicit representation generated by the l-1th hidden layer of the multilayer perceptron, and For entity u i The entity node embedding representation, W u (l) is the weight parameter matrix to be trained, b u (l) is the bias parameter, σ(·) is the activation function;

[0175] Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as:

[0176]

[0177] in, Indicates label h j The implicit representation generated by the lth hidden layer of the multilayer perceptron at the corresponding output end is, Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label node embedding representation, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

[0178] Through the multilayer perceptron set up as above, in S43, by repeatedly stacking the hidden layers L as described above MLP Next, the output of the previous hidden layer is used as the input of the next hidden layer, and the node embedding representation generated by the input graph neural network module can be refined by dimensionality reduction. i The output obtained by the multi-layer perceptron on the entity side is Label node h j The output of the multi-layer perceptron of the label layer is And the dimensions of the output vectors of the two multilayer perceptrons should be consistent.

[0179] In one implementation, to improve the model effect, the activation function uses the linear rectification function ReLU, which is defined as

[0180]

[0181] In one embodiment, a dot multiplication operation is performed on the entity node embedding representation after dimension reduction and the label node embedding to obtain a prediction result, including: obtaining the entity u generated by the multilayer perceptron i Entity node embedding representation and label h j Label node embedding Determine entity u i With label h j The likelihood score of the association between To predict entity u i and label h j The similarity between .

[0182] refer to Figure 7 , in the output entity u i With label h j The y between i,j After that, it is necessary to determine whether the trained model has converged. If it has converged, the training is terminated and each entity u is obtained. i With label h j The correlation between them is taken as the prediction result. If there is no convergence, the objective function value is continuously reduced and the model parameters are optimized and updated through the gradient descent algorithm until the objective function value reaches convergence and the training is terminated.

[0183] In one embodiment, in order to alleviate the high sparsity of positive samples and the extreme label imbalance in the entity-label association matrix and enhance the influence of positive samples, the entity-label association matrix can be The target value of the positive sample in is raised to a constant greater than 1 to determine the enhanced correlation matrix Using the Augmented Correlation Matrix The prediction model is iteratively trained as the fitting target until the model converges.

[0184] Specifically, the target matrix can be defined To enhance the correlation matrix:

[0185]

[0186] Among them, ε>1 is a specified hyperparameter, which reflects the influence of the positive sample in the target matrix, and is used to expand the gap and distinguishability between the positive sample and the unobserved sample. Therefore, if the entity u is known i and label h j If there is an association, then the corresponding target value in the target matrix is raised to a constant ε greater than 1 and remains 0 otherwise.

[0187] In one embodiment, in order to allow the prediction model to learn the parameters to be learned, it also includes: using a minimum square error loss function (Minimal Square Error, MSE) Training prediction models;

[0188]

[0189] Where Y = (y i,j ) n×m is the prediction result matrix; is the enhanced correlation matrix; Ω is the mask matrix of the observed elements: if the entity u is known i and label h j There is a positive correlation, that is When Ω i,j =1; otherwise all are 0; ° is the Hadamard product (or element-wise product); ‖·‖ F is the Frobenius norm; the first term The gap between the predicted value and the observed value should be minimized; Θ refers to all the parameters to be trained in the model; λ is the attenuation factor, which is a hyperparameter used to control the balance between the regularization term and the fitting target; ‖·‖2 is the L2 norm; the second term is a regularization term to avoid overfitting.

[0190] In one embodiment, in order to improve the model training effect, it also includes: using the RMSProp optimizer to optimize the objective function; the hyperparameters required include the initial learning rate α and the decay rate ρ, and the optimizer continuously reduces the objective function value and optimizes and updates the model parameters through the gradient descent algorithm until the objective function value reaches convergence and the training is terminated.

[0191] In summary, the entity-label association prediction calculation method proposed in this application focuses on filling in the incomplete label annotations of entities that have been partially labeled. Modeling the input entity set as an entity relationship network is more in line with the characteristics of complex associations rather than independence between samples in the real world. Optionally, the label set involved in this application has a strict and definite hierarchical structure, which takes into account the hierarchical relationship or semantic relationship between labels, so as to achieve a finer division and a more accurate characterization of the samples. Through the input entity relationship network, it is easier to discover the implicit similar features between samples, and use this similarity to improve the accuracy of multi-label classification.

[0192] For the two major objects of entities and labels, this algorithm constructs two categories of three networks: the relationship network between homogeneous nodes (i.e., entity relationship network and hierarchical label relationship network) and the relationship network between heterogeneous nodes (i.e., entity-label association network). For different input network types, this algorithm designs two graph neural network modules: the Bi-GCN module running on the bipartite graph and the S-GCN module running on the similarity graph. Among them, the Bi-GCN module performs message transmission on the unweighted association edge between the entity and the label to achieve full communication and fusion of the information contained in the two objects; the S-GCN module performs message transmission on the weighted edge of the similarity graph to aggregate the information contained in the adjacent nodes according to the similarity. By stacking multiple layers of graph neural network layers and iterating repeatedly, the low-order and high-order topological structures in the network can be repeatedly captured, and the complex nonlinear relationships between objects can be discovered.

[0193] In addition, during the training process, this algorithm improves the influence of positive samples in the original target association matrix and constructs an enhanced association matrix as the final fitting target. This method effectively alleviates the highly unbalanced phenomenon of positive and negative samples in the data set, allowing the model to fully focus on positive sample signals, thereby improving the final prediction accuracy.

[0194] In the description of this specification, the description with reference to the terms "some possible implementations", "some embodiments", "examples", "specific examples", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, unless they are contradictory.

[0195] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0196] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present invention includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present invention belong.

[0197] About the method flow chart of the embodiment of the present application, some operations are described as different steps performed in a certain order.Such flow chart belongs to illustrative and non-restrictive.Some steps described in this article can be grouped together and performed in a single operation, some steps can be divided into multiple sub-steps, and some steps can be performed in an order different from that shown in this article.The various steps shown in the flow chart can be realized in any way by any circuit structure and / or tangible mechanism (for example, by software, hardware (for example, the logical function realized by processor or chip) etc. running on computer equipment and / or any combination thereof).

[0198] Based on the same technical concept, an embodiment of the present invention further provides an entity tag association prediction device, which is used to execute the entity tag association prediction method provided by any of the above embodiments. Figure 8 A schematic diagram of the structure of an entity tag association prediction device provided in an embodiment of the present invention.

[0199] A network determination module 801 is used to determine an entity relationship network, a tag relationship network, and an entity tag association network, wherein the entity tag association network includes unknown entity tag association relationships;

[0200] A graph construction module 802 is used to construct an entity similarity graph according to the entity relationship network, to construct a label similarity graph according to the label relationship network and the entity label association network; and to construct an entity label association bipartite graph according to the entity label association network;

[0201] The feature construction module 803 is used to extract entity features and construct label features according to the label similarity graph;

[0202] The prediction model module 804 integrates the entity similarity graph, the label similarity graph and the entity label association bipartite graph into a graph convolutional network to construct a prediction model; inputs the entity features and the label features into the prediction model for training until the model converges, and outputs the prediction result of the prediction model, which includes the association relationship between each entity and each label.

[0203] It should be noted that the device in the embodiment of the present application can implement the various processes of the embodiment of the aforementioned method and achieve the same effects and functions, which will not be repeated here.

[0204] Fig. 9 A schematic diagram of the structure of an entity tag association prediction device according to an embodiment of the present application is provided. Figure 1 The method shown, the device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in the above embodiment.

[0205] According to some embodiments of the present application, a non-volatile computer storage medium of an entity tag association prediction method is provided, on which computer executable instructions are stored, and the computer executable instructions are configured to execute the method described in the above embodiment when executed by a processor.

[0206] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device, equipment, and computer-readable storage medium embodiments, since they are basically similar to the method embodiments, their descriptions are simplified, and the relevant parts can be referred to the partial description of the method embodiments.

[0207] The apparatus, device and computer-readable storage medium provided in the embodiments of the present application correspond one-to-one to the method. Therefore, the apparatus, device and computer-readable storage medium also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the apparatus, device and computer-readable storage medium will not be repeated here.

[0208] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0209] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0210] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0211] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0212] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0213] The memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0214] Computer-readable media include permanent and non-permanent, removable and non-removable media that can be used to store information by any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, read-only compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. In addition, although the operations of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that these operations must be performed in this particular order, or that all the operations shown must be performed to achieve the desired results. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0215] Although the spirit and principle of the present invention have been described with reference to several specific embodiments, it should be understood that the present invention is not limited to the disclosed specific embodiments, and the division of various aspects does not mean that the features in these aspects cannot be combined to benefit, and such division is only for the convenience of expression. The present invention is intended to cover various modifications and equivalent arrangements included in the spirit and scope of the attached claims.

Claims

1. A method for predicting entity label association, characterized in that: include: Determine an entity relationship network, a tag relationship network, and an entity tag association network, wherein the entity tag association network includes unknown entity tag association relationships, tags in the tag relationship network are used to describe user characteristics, entities in the entity relationship network include users, and association relationships between entities are realized based on one or more of historical transaction elements between users, user location elements, and user attribute elements; Constructing an entity similarity graph according to the entity relationship network, constructing a label similarity graph according to the label relationship network and the entity label association network; and constructing an entity label association bipartite graph according to the entity label association network; Extract entity features and construct label features based on label similarity graph; Integrating the entity similarity graph, the label similarity graph, and the entity label association bipartite graph into a graph convolutional network to construct a prediction model, wherein the graph convolutional network includes a bipartite graph convolutional network and a similarity graph convolutional network; Inputting the entity features and the label features into the prediction model for training until the model converges, and outputting the prediction result of the prediction model, wherein the prediction result includes the association relationship between each entity and each label; Determining the entity relationship network, the label relationship network, and the entity label association network also includes: Determine the entity set An entity relationship network with multiple entities as nodes Where n is the total number of entities, Used to indicate the relationship between entities in the entity set; Determine the tag collection Label relationship network with multiple labels as nodes in Among them, m is the total number of tags, Used to indicate the relationship between the tags in the tag set; Determine entity label association network based on existing annotation information Among them, each entity u i (i=1,2,…,n) and each label h j (j=1,2,…,m) The value is 1 or 0.

2. The method according to claim 1, characterized in that The label set is a hierarchical label set, the label relationship network is formed into a tree structure, and there is a parent-child inheritance relationship between the upper and lower label nodes of the tree structure.

3. The method according to claim 2, further comprising: Update the existing annotation information according to the hierarchical relationship between the tags; If the designated entity is marked with a designated label, all ancestor labels of the designated label are supplemented for the designated entity according to the label relationship network.

4. The method according to claim 1, characterized in that: Constructing an entity similarity graph according to the entity relationship network also includes: According to the entity relationship network diagram G u The adjacency matrix A u Determine the entity similarity graph S u ; Among them, the adjacency matrix A u ∈[0,1] n×n Formed into an n-order square matrix, the entity relationship network graph G u is an undirected authorized network, entity u i With entity u i′ The edge weight Used to indicate the closeness of the relationship between two entities.

5. The method according to claim 2, characterized in that: Constructing a label similarity graph according to the label relationship network and the entity label association network also includes: The label information volume IC(h) of each label is calculated based on the entity label association network R. The label information volume is used to indicate the entity annotation ratio of each label and measure the degree of concretization of each label: The tag similarity graph S is obtained by calculating the tag information of each tag and the tag relationship network. h ; Among them, the label similarity graph S h ∈[0,1] m×m It forms an m-order square matrix.

6. The method according to claim 5, characterized in that The similarity between the tags is calculated using the following formula: Among them, h MICA is the label h j With label h j′ The most informative common ancestor of j With label h j′ The common ancestor label node with the highest label information.

7. The method according to claim 5 or 6, characterized in that: The tag information amount of each tag is calculated using the following formula: Where n is the total number of entities, n h is the number of entities known to be labeled with label h.

8. The method according to claim 1, characterized in that Constructing an entity label association bipartite graph according to the entity label association network includes: According to the entity label association matrix R, an entity label association bipartite graph A is constructed; Among them, the The 0 is an all-zero matrix.

9. The method according to claim 1, characterized in that: Extracting entity features and constructing label features according to the label similarity graph, including: Determine each entity u in the entity set by feature mining i Multidimensional entity features According to the label similarity graph S h Perform symmetric normalization to obtain the normalized matrix According to the normalized matrix Compute the positive-valued pointwise mutual information matrix X (h) ; According to the positive valued pointwise mutual information matrix X (h) Construct the label feature, where the matrix X (h) The jth row of As the jth label h j The label features 10. The method according to claim 1, characterized in that Inputting the entity features and the label features into a prediction model for training, including: Using an input-end multilayer perceptron to reduce the dimensions of the input entity features and the input label features respectively; Inputting the entity features and the label features after dimension reduction into the prediction model to obtain an entity node embedding representation corresponding to each entity and a label node embedding representation corresponding to each label; Performing dimensionality reduction on the entity node embedding representation and the label node embedding representation respectively through an output-end multilayer perceptron; A dot product operation is performed on the entity node embedding representation after dimensionality reduction and the label node embedding to obtain the prediction result.

11. The method according to claim 10, characterized in that Inputting the entity features and the label features after dimension reduction into a graph convolutional network, including: Inputting the entity features and the label features after dimension reduction into a first graph convolutional network, the first graph convolutional network introduces the topological structure information of the entity label association bipartite graph through a conversion function, and outputs an intermediate embedding representation of the entity node and the label node; Inputting the intermediate embedding representation of the entity node into the second graph convolutional network, the second graph convolutional network introduces the topological structure information and edge weight information of the entity similarity graph through a conversion function, and outputs the embedding representation of the entity node; The intermediate embedding representation of the label node is input into the third graph convolutional network, and the third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through a conversion function, and outputs the embedding representation of the label node.

12. The method according to claim 10, characterized in that Using the input-end multilayer perceptron to reduce the dimension of the input entity features and the input label features respectively, further comprising: Entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as: in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the input end is Represents entity u i The implicit representation generated by the l-1th hidden layer of the multilayer perceptron, and For entity u i The entity characteristics, W u (l) is the weight parameter matrix to be trained, b u (l) is the bias parameter, σ(·) is the activation function; Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding input end is defined as: in, Indicates label h j The implicit representation generated in the lth hidden layer is Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label feature, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

13. The method according to claim 11, characterized in that The first graph convolutional network introduces topological structure information of the entity label associated bipartite graph through a conversion function, including: The entity node u i The conversion function of the lth layer in the first graph convolutional network is: The label node h j The conversion function of the lth layer in the first graph convolutional network is: in, and Respectively refers to entity u i and label h j The embedding representation produced at layer l of the first graph convolutional network, and and They respectively refer to the entity features and the label features after dimensionality reduction, Indicates that in the entity label association bipartite graph, i The associated label node set, Indicates that in the entity label association bipartite graph, j The associated entity node collection, and is the weight parameter matrix of the lth layer to be trained in the first graph convolutional network, and σ is the activation function.

14. The method according to claim 11, characterized in that The second graph convolutional network introduces the topological structure information and edge weight information of the entity similarity graph through a conversion function, including: The entity node u i The conversion function in the lth layer of the second graph convolutional network is: in, Refers to entity u i The embedding representation generated at layer l is Refers to entity u i′ The embedding representation generated at layer l-1 is S′ u =S u +I,(D′ u ) i，i′ =∑ i′ (S u ′) i，i′ , S u is the entity similarity graph, is the weight parameter matrix of the lth layer to be trained in the second graph convolutional network, and σ(·) is the activation function.

15. The method according to claim 11, characterized in that The third graph convolutional network introduces the topological structure information and edge weight information of the label similarity graph through a conversion function, including: The label node h j The conversion function in the lth layer of the third graph convolutional network is: in, Refers to the label h j The embedding representation generated at layer l is Refers to the label h j The embedding representation generated at layer l-1 is S′ h =S h +I,(D′ h ) j，j′ =∑ j′ (S h ′) j，j′ , S h is the label similarity graph, is the weight parameter matrix of the lth layer to be trained in the third graph convolutional network, and σ(·) is the activation function.

16. The method according to any one of claims 13 to 15, characterized in that The activation function σ adopts the leaky linear rectifier function LeakyReLU.

17. The method according to claim 10, characterized in that The dimension of the entity node embedding representation and the label node embedding representation are reduced respectively by using an output-end multilayer perceptron, and further comprising: Entity node u i The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as: in, Represents entity u i The implicit representation generated by the lth hidden layer of the multilayer perceptron at the output is Represents entity u i The implicit representation produced by the l-1th hidden layer of the multilayer perceptron at the output, and For entity u i The entity node embedding representation, W u (l) is the weight parameter matrix to be trained, b u (l) is the bias parameter, σ(·) is the activation function; Label node h j The conversion function of the lth hidden layer of the multilayer perceptron at its corresponding output end is defined as: in, Indicates label h j The implicit representation generated by the lth hidden layer of the multilayer perceptron at the corresponding output end is, Indicates label h j The implicit representation generated in the l-1th hidden layer, and For label h j The label node embedding representation, W h (l) is the weight parameter matrix to be trained, b h (l) is the bias parameter and σ(·) is the activation function.

18. The method according to claim 12 or 17, characterized in that The activation function uses the linear rectification function ReLU.

19. The method according to claim 11, characterized in that Performing a dot product operation on the entity node embedding representation after dimension reduction and the label node embedding to obtain the prediction result includes: Get the entity u generated by the multilayer perceptron at the output end i Entity node embedding representation and label h j Label node embedding Computational entity u i With label h j The likelihood score of the association between To predict entity u i and label h j The similarity between .

20. The method according to claim 1, characterized in that Also includes: Associating the entity label with the network The target value of the positive sample in is raised to a constant greater than 1 to determine the enhanced correlation matrix Using the enhanced correlation matrix The prediction model is iteratively trained as a fitting target until the model converges.

21. The method according to claim 20, characterized in that Also includes: Using the least square error loss function l to train the prediction model; The least square error loss function Among them, Y is the prediction result matrix, is the enhanced correlation matrix, λ is the attenuation factor; Ω is the mask matrix of the observed elements; is the element-wise product, Θ refers to all the parameters to be trained in the model, ‖·‖ F is the Frobenius norm, and ‖·‖2 is the L2 norm.

22. The method according to claim 1, characterized in that Also includes: Use RMSProp optimizer to optimize the objective function; The optimizer continuously reduces the objective function value and optimizes and updates the model parameters through the gradient descent algorithm until the objective function value reaches convergence and the training is terminated.

23. An entity tag association prediction device, characterized in that: is configured to perform the method according to any one of claims 1 to 22, comprising: A network determination module, used to determine an entity relationship network, a tag relationship network, and an entity tag association network, wherein the entity tag association network includes unknown entity tag association relationships, tags in the tag relationship network are used to describe user characteristics, entities in the entity relationship network include users, and association relationships between entities are realized based on one or more of historical transaction elements between users, user location elements, and user attribute elements; A graph construction module, configured to construct an entity similarity graph according to the entity relationship network, to construct a label similarity graph according to the label relationship network and the entity label association network; and to construct an entity label association bipartite graph according to the entity label association network; Feature construction module, used to extract entity features and construct label features based on label similarity graph; The prediction model module integrates the entity similarity graph, the label similarity graph and the entity label association bipartite graph into a graph convolutional network to construct a prediction model; inputs the entity features and the label features into the prediction model for training until the model converges, and outputs the prediction result of the prediction model, wherein the prediction result includes the association relationship between each entity and each label.

24. An entity tag association prediction device, characterized in that: include: at least one processor; And, a memory communicatively connected to at least one processor; wherein the memory stores instructions executable by at least one processor, and the instructions are executed by at least one processor to enable the at least one processor to execute: a method as described in any one of claims 1-22.

25. A computer-readable storage medium storing a program, wherein when the program is executed by a multi-core processor, the multi-core processor executes the method according to any one of claims 1 to 22.

Images