A query intent enhancement method based on a search heterogeneous graph neural network and a medium

Abstract

The application relates to a query intention enhancement method and medium based on a search heterogeneous graph neural network, which comprises the following steps: obtaining a query request q of a user and a search heterogeneous graph constructed according to historical search record data of all users, inputting the search heterogeneous graph into a pre-trained search intention enhancement model, and outputting an enhanced query request intention, wherein the search intention enhancement model comprises two convolution modules, each convolution module adopts a multi-head attention mechanism, and each head attention mechanism comprises a vertex type space conversion operation and an aggregation operation of neighborhood information. Compared with the prior art, the application has the advantages of more accurate query intention and improved search accuracy.

Description

Technical Field

[0001] This invention relates to the field of natural language processing technology, and in particular to a method and medium for enhancing query intent based on a search heterogeneous graph neural network. Background Technology

[0002] During online searches, users often submit ambiguous and vague short queries to search engines, making it difficult for them to accurately understand the user's true query intent and quickly return the expected results. For example, a tour guide might submit the query "Java" to a search engine, expecting results about web documents related to "Java Island." However, since "Java" can refer to both "Java Island" and the "Java" programming language, the search engine doesn't know whether the user's search intent is about the island of Java or the Java programming language. Unfortunately, many of the top-ranked web links in the search results might be related to the "Java" programming language. Essentially, this example reflects the problem of ambiguity in search queries. Furthermore, search queries also suffer from uncertainty, lack of personalization, and ambiguity. Therefore, enhancing user query intent or adding search suggestions is both necessary and challenging. By enhancing user query intent, we can improve the accuracy of search relevance, enhance user experience, promote information discovery and exploration, and reduce search costs.

[0003] Traditional query intent enhancement methods mainly include rule-based and template-based methods. These methods, which supplement and expand query keywords using rules or templates, are simple and easy to implement, but the enhanced query intent is not always accurate. Currently, query intent enhancement methods are primarily based on users' historical search behavior and fall into two main categories: one is based on the user's own historical search logs, and the other is based on group historical search logs. Enhancing query intent based on the user's own historical search logs, as the name suggests, uses a single user's previous search behavior or browsing history to infer the user's preferences and intent. However, when a user's historical search activity is limited, or even nonexistent, a cold start problem can occur. Enhancing query intent based on group historical search logs aims to enhance query intent by leveraging the search logs of users with similar search behaviors within a group. Compared to enhancement methods based on individual user search logs, this type of method achieves better results, but it still cannot comprehensively and accurately reflect the intent of each user within the group. Summary of the Invention

[0004] The purpose of this invention is to provide a method and medium for enhancing query intent based on a search heterogeneous graph neural network, which accurately enhances query intent.

[0005] The objective of this invention can be achieved through the following technical solutions:

[0006] A query intent enhancement method based on a heterogeneous graph neural network for search includes the following steps:

[0007] The system obtains the user's query request q and a search heterogeneous graph constructed based on the historical search records of all users. These are input into a pre-trained search intent enhancement model, which outputs the enhanced query request intent. The search intent enhancement model includes two convolutional modules, each of which employs a multi-head attention mechanism. Each attention mechanism includes vertex type space transformation operation and neighborhood information aggregation operation.

[0008] The steps performed by the search intent enhancement model include:

[0009] Based on the search heterogeneous graph, its adjacency matrix A and vertex matrix H are obtained, and the query request q is used as a vertex, which is then converted into a vector h using one-hot encoding. q ;

[0010] Vector h q The adjacency matrix A and the vertex matrix H are input into the two-layer convolutional module to perform a spatial transformation operation on the vertex type, resulting in vertex matrices with the same dimensions. and H^;

[0011] Based on the above H^ and adjacency matrix A are used to aggregate neighborhood information to calculate the attention values ​​between query request q and its neighboring vertices, and the features of the neighboring vertices are aggregated onto query request q to obtain the enhanced representation feature vector h′. q ;

[0012] h′ q The query is converted into the corresponding query enhancement term q′ and merged with the query request q to obtain the enhanced query request intent.

[0013] Furthermore, the process of constructing the search heterogeneous graph includes:

[0014] Obtain historical search records data for all users. The historical search records data contains three different types of objects: online users, query requests, and result documents.

[0015] Let all online users be set U = {u1, u2, ..., u...} z Let all query requests be denoted as set Q = {q1, q2, ..., q}. k Let the set of all online users whose first result documents are clicked be denoted as set}. in A collection of all resulting documents;

[0016] Sets U, Q and All objects are represented as three different types of vertices, and the relationships between objects are represented as edges. A user-query-document relationship graph G = (V, E, X) is constructed as a search heterogeneous graph, where V represents all vertices, E represents all edges, and X = {x1, x2, ..., x...} n} represents the eigenvectors of all vertices.

[0017] Furthermore, during the spatial transformation operation of the vertex type, the set U and The feature dimensions of the corresponding vertices are uniformly transformed to the feature dimensions of the corresponding vertices in set Q.

[0018] Furthermore, the expression for the space transformation operation is:

[0019]

[0020] In the formula, It is the previous hidden state of neighbor vertex i from the space of type φ(i). The projection onto the (l+1)th hidden space of type φ(j), The transformation matrix from user-type vertices to query-type vertices. Let be the transformation matrix from document type vertices to query type vertices. This is the transformation matrix from query type vertex to query type vertex.

[0021] Furthermore, in the process of calculating the attention value between the query request q and its neighboring vertices, the heterogeneous characteristics of the edges in the adjacency matrix A are considered, and the semantic relationships between vertices of different types are added to the calculation.

[0022] Furthermore, the expression for calculating the attention value between the query request q and its neighboring vertices is:

[0023]

[0024] In the formula, e iq Let q be the attention value between q and its neighboring vertices, and α be the attention parameter. For H ^ The vertex adjacent to q in the middle.

[0025] Furthermore, it also includes a normalization operation, which is:

[0026] The attention values ​​of the query request q and its neighboring vertices are normalized using the softmax function. The expression for the normalization operation is as follows:

[0027] α iq =softmax(e iq )

[0028] In the formula, e iq Let α be the attention value between q and its neighboring vertices. iq This is the normalized attention value.

[0029] Furthermore, the enhanced representation feature vector h′ q The expression is:

[0030]

[0031] In the formula, σ is the LeakyReLU activation function, M is the number of attention heads, and N is the number of attention heads. q For vertex v q The set of neighbors, α iq This is the result of normalizing the attention values ​​of q and its neighboring vertices. For H ^ The vertex adjacent to q in the middle.

[0032] Furthermore, the search intent enhancement model also includes a multilayer perceptron and a softmax layer connected sequentially to the two convolutional modules.

[0033] The present invention also provides a computer-readable storage medium including one or more programs executable by one or more processors of an electronic device, the one or more programs including instructions for performing the query intent enhancement method based on the search heterogeneous graph neural network as described above.

[0034] Compared with the prior art, the present invention has the following beneficial effects:

[0035] (1) The present invention uses the search heterogeneous graph constructed by the friend network and the user's historical search behavior as the feature data of the user search. The vertex feature data generated in this way has more prominent similarity in the semantic space, which makes the subsequent enhancement of the user's query intent more accurate.

[0036] (2) The multi-head attention mechanism in the convolution module of the present invention transforms the features of different types of vertices into the feature space of the same dimension, and adjusts the contribution of the neighboring vertices of different vertices through the attention mechanism and assigns different attention weights, thus overcoming the problem that the attribute features of neighboring vertices cannot be directly aggregated.

[0037] (3) In the process of aggregating neighbor vertex information in this invention, the heterogeneous characteristics of edges in the search graph are taken into account, that is, different types of edges represent different semantic relationships. The semantic relationships of the edges are also included in the calculation, which helps to improve the accuracy of the query intent. Attached Figure Description

[0038] Figure 1 This is a schematic diagram of the method flow of the present invention;

[0039] Figure 2 This is a framework diagram of the SIER-HetGAT search intent enhancement model of the present invention. Detailed Implementation

[0040] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments. These embodiments are based on the technical solution of the present invention and provide detailed implementation methods and specific operating procedures. However, the scope of protection of the present invention is not limited to the following embodiments.

[0041] This embodiment provides a query intent enhancement method based on a search heterogeneous graph neural network. This method considers that the creation of social circles is often based on shared backgrounds or interests; therefore, utilizing friends' historical search logs can help enhance user query intent. However, the search log information in social circles is interconnected and complex, and effectively utilizing friends' historical search log information is not a simple task. It requires designing a neural network model to mine the features and interrelationships of query keywords and search result documents in the logs. Therefore, this method uses the friend network and the user's historical search behavior to generate a "user-query-document" search heterogeneous graph as feature data for user searches. To characterize the embedding representation of vertex attributes in the search heterogeneous graph, a heterogeneous attention mechanism is designed to transform the features of different types of vertices into a feature space of the same dimension. To obtain an enhanced representation of the user's query request, i.e., adding query keywords, a search intent enhancement model is designed and trained. For any newly submitted search query by the user, the corresponding query intent enhancement representation can be directly generated using the trained model without retraining the model, thus accelerating search keyword expansion and suggestions. Specifically, as shown... Figure 1 As shown, the method includes the following steps:

[0042] Combination Figure 2 The method in this embodiment mainly includes four parts:

[0043] (1) Construction of search heterogeneous graphs

[0044] The historical search records of all users contain three different types of objects: online users, query requests, and result documents. We denote all online users as a set U = {u1, u2, ..., u...} z Let all query requests be denoted as set Q = {q1, q2, ..., q}. k Let the set of all the first result documents clicked by users be denoted as set}. in This is a collection of all resulting documents. We will use sets U, Q, and... All objects are viewed as vertices in the graph, and the relationships between them are viewed as edges. From this, a user-query-document relationship graph G = (V, E, X) can be constructed, which is a heterogeneous graph of historical search behavior and serves as feature data of user searches.

[0045] (2) Search for the embedded representation of vertex attributes in heterogeneous graphs

[0046] Vertex Attribute Feature Space Transformation: The heterogeneous search graph G=(V,E,X) in this patent has three types of vertices: query type Q, user type U, and document type D, with feature vector lengths of α, β, and γ, respectively. Clearly, their vector dimensions are different and cannot be uniformly calculated. This patent focuses on query type vertices Q, requiring accurate calculation of their embedding representations; therefore, they need to be transformed to a feature dimension space of the same size. We utilize different matrices... Transform the eigenvectors of any vertex i to α. ^ 1D eigenvector, this integer α ^ This refers to the dimension of the vector corresponding to the vertex embedding representation of the query type. In other words, the dimension of the feature vectors of all neighboring vertices of vertex j after matrix transformation is α. ^ The conversion formula is shown below. Other types of space conversions follow the same principle.

[0047]

[0048] in It is the previous hidden state of neighbor vertex i from the space of type φ(i). The projection onto the (l+1)th level hidden space of type φ(j). That is, we transform the neighboring vertices of vertex j to the same dimension vector space of the query class Q vertices for neighborhood aggregation.

[0049] Neighbor vertex information aggregation: In searching a heterogeneous graph, the importance of different neighbors to each vertex varies. Therefore, when calculating the embedding representation of each vertex in the graph, different weights should be assigned to each vertex. This patent takes into account the heterogeneous characteristics of edges in the search graph when calculating the attention weights of vertices, i.e., different types of edges represent different semantic relationships. Therefore, the semantic relationships of edges are also included in the calculation.

[0050] (3) Enhanced representation of search intent

[0051] By utilizing historical search logs and preprocessing them, a large amount of training data is obtained, which can be used to train the search intent enhancement model SIER-HetGAT. The training input consists of a heterogeneous graph G = (V, E, X) of the search history and the feature vectors X = {x1, x2, ..., x} of all vertices. n The output is an enhanced representation H = {h′1, h′2, ..., h′} of all query class vertices. p}, H∈R α′×p The intermediate hidden layer consists of two convolutional modules, HetAL. Each convolutional module employs a multi-head attention mechanism, which primarily includes vertex type space transformation and neighborhood information aggregation operations. The SIER-HetGAT model in this embodiment also includes an MLP (Multilayer Perceptron) and a softmax layer sequentially connected after the HetAL module. The MLP learns a nonlinear function during model training.

[0052] (4) Add intent extension for new query requests

[0053] Through training, the optimal vertex weight matrix W and attention parameter α in the search intent enhancement model SIER-HetGAT were obtained. Since the parameters are shared during the design process, the embedding representation of newly added query vertices can be directly calculated and encoded. Assume a user submits a new query request as follows: The corresponding attribute feature vector is h q Given that the neighboring vertices of the current query q in the search heterogeneous graph are known, this data is input into the pre-trained SIER-HetGAT model. SIER-HetGAT uses an adaptive attention mechanism to weight the neighboring vertices and outputs the feature vector h′ corresponding to the enhanced representation of the current query q. q , h′ q The corresponding extended query keywords are q′={w′1,w′2,…,w′ p}, where q′ is the enhanced representation of the query intent of q. With q′={w′1,w′2,…,w′ p} merged into q * This represents the final query request intent for q.

[0054] The implementation process of the above method is as follows:

[0055] First, a search heterogeneity graph is constructed using users' historical search logs from their social circles, and a search intent enhancement model, SIER-HetGAT, is designed and trained.

[0056] Then, when a user submits a new query request to the search engine When, the corresponding initial vector h qThe adjacency matrix of the original search heterogeneous graph is input into the SIER-HetGAT model to obtain the enhanced representation feature vector h′ of the query. q And convert it into the corresponding query enhancement term q′.

[0057] In this embodiment, the initial query q is merged with the corresponding query enhancement term q′, and the union result is... As a real request from a user after a new query has been expanded and enhanced, the specific algorithm implementation steps are as follows:

[0058] Input: Query request The adjacency matrix A of the heterogeneous graph, and the vertex feature matrix H = {h1, h2, ..., h} n};

[0059] Output: The expanded and enhanced real request q * ;

[0060] Step 1: Convert the input query q into a vector h using One-hot encoding. q And further transformed using the transformation matrix

[0061] Step 2: Calculate the attention between query q and neighboring vertices.

[0062] Step 3: Normalize the attention weights using the softmax function to obtain α. iq =softmax(e iq );

[0063] Step 4: Repeat steps 2 and 3 until the attention values ​​of all neighboring vertices are obtained;

[0064] Step 5: Aggregate the features of all neighboring vertices of the query vertex q onto vertex q.

[0065] Step 6: For h′ q Perform the conversion by executing Convert(h′) q ), thus obtaining the enhanced q′={w′1,w′2,…,w′ p};

[0066] Step 7: Output q * =q∪q′.

[0067] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0068] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. The solutions in the embodiments of the present invention can be implemented using various computer languages, such as the object-oriented programming language Java and the interpreted scripting language JavaScript.

[0069] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0070] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0071] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0072] Although preferred embodiments of the invention have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including both the preferred embodiments and all changes and modifications falling within the scope of the invention.

[0073] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Therefore, if these modifications and variations fall within the scope of the claims of this invention and their equivalents, this invention also intends to include these modifications and variations.

Claims

1. A query intent enhancement method based on a search heterogeneous graph neural network, characterized in that, Includes the following steps: Get the user's query request The search heterogeneous graph constructed based on the historical search records of all users is input into a pre-trained search intent enhancement model, which outputs an enhanced query request intent. The search intent enhancement model includes two convolutional modules, each of which employs a multi-head attention mechanism. Each attention mechanism includes vertex type space transformation operation and neighborhood information aggregation operation. The steps performed by the search intent enhancement model include: Based on the search heterogeneous graph, its adjacency matrix A and vertex matrix H are obtained, and the query request is then processed. As vertices, they are converted into vectors using one-hot encoding. ; vector The adjacency matrix A and the vertex matrix H are input into the two-layer convolutional module to perform a spatial transformation operation on the vertex type, resulting in vertex matrices with the same dimensions. and H ^ ; Based on the above H ^ Aggregate neighborhood information with adjacency matrix A to calculate the query request. The attention values ​​of the vertices are used to aggregate the features of the neighboring vertices into the query request. The above yields the enhanced representation feature vector. ; Will Convert to corresponding query enhancement terms and with the query request The query intent is then merged to obtain the enhanced query request intent.

2. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, The process of constructing the search heterogeneous graph includes: Obtain historical search records data for all users. The historical search records data contains three different types of objects: online users, query requests, and result documents. Let all online users be a set All query requests are denoted as a set. The set of the first result documents clicked by all online users is denoted as set. , ,in A collection of all resulting documents; set , and All objects are represented as three different types of vertices, and the relationships between objects are represented as edges, constructing a user-query-document relationship graph. As a search heterogeneous graph, in For all vertices, For all edges, represents the eigenvectors of all vertices.

3. The query intent enhancement method based on a heterogeneous graph neural network according to claim 2, characterized in that, During the spatial transformation operation of the vertex type, the set and The feature dimensions of the corresponding vertices are uniformly converted to a set. On the feature dimension of the corresponding vertex.

4. The query intent enhancement method based on a heterogeneous graph neural network according to claim 3, characterized in that, The expression for the space transformation operation is: In the formula, It is a neighboring vertex From type Previously hidden state in space , to type The l + The projection of the hidden space on the 1st floor, The transformation matrix from user-type vertices to query-type vertices. This is a transformation matrix from document type vertices to query type vertices. This is the transformation matrix from query type vertex to query type vertex.

5. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, The calculation query request In the process of calculating the attention value of its neighboring vertices, the heterogeneity of the edges in the adjacency matrix A is considered, and the semantic relationships between vertices of different types are added to the calculation.

6. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, The calculation query request The expression for the attention value of a vertex to its neighboring vertices is: = ( , ) In the formula, for Attention value with neighboring vertices For attention parameters, For H ^ Zhongyu Adjacent vertices.

7. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, It also includes a normalization operation, which is: Using the softmax function to process query requests The attention values ​​of the vertices are normalized, and the expression for the normalization operation is as follows: =softmax( ) In the formula, for Attention value with neighboring vertices This is the normalized attention value.

8. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, The enhanced representation feature vector The expression is: In the formula, The LeakyReLU activation function is used. For the number of attention heads, As vertex The neighborhood group, for The result of normalizing the attention values ​​with those of neighboring vertices. For H ^ Zhongyu Adjacent vertices.

9. The query intent enhancement method based on a heterogeneous graph neural network according to claim 1, characterized in that, The search intent enhancement model also includes a multilayer perceptron and a softmax layer connected sequentially to two convolutional modules.

10. A computer-readable storage medium, characterized in that, Includes one or more programs executable by one or more processors of an electronic device, said one or more programs including instructions for performing the query intent enhancement method based on a search heterogeneous graph neural network as described in any one of claims 1-9.

Images