An information retrieval method and system based on user implicit retrieval intention

Abstract

The present invention relates to an information retrieval method and system based on the user's implicit retrieval intention, and belongs to the field of computer information retrieval technology. In the process of searching using a search engine, the method sorts the documents that meet the search conditions according to their relevance to the query keywords, introduces the semantic convex hull in the Euclidean semantic space, and forms a visual comparison scheme of the retrieval intention and the document intention. The system constructs the user's retrieval intention to reflect the answers the user is looking for and quantify the diversity of documents. At the same time, the system introduces a timeline to graphically display the retrieval intentions of multiple rounds of queries, and organizes them serially from first to last through the timeline. The present invention effectively integrates the user's retrieval intention into the traditional relevance ranking search engine, allowing users to mine various documents, facilitating users to retrieve some exploratory questions, improving the diversity of information retrieval results, enhancing the interaction between users and search engines, and effectively improving retrieval efficiency.

Description

Technical Field

[0001] The present invention relates to an information retrieval method and system, and in particular to an information retrieval method and system based on semantic modeling and visualization of user implicit retrieval intentions, belonging to the technical field of computer information retrieval. Background Art

[0002] Information retrieval is the primary way for users to query and obtain information. It is a method and means of finding information. With the continuous development of computer technology and information retrieval technology, search engines have become the primary tool for users to conduct retrieval, providing great convenience for people's lives.

[0003] Information retrieval methods involve computers comparing and matching input search expressions with the characteristic identifiers of document information stored in the retrieval system and their logical relationships. The principle of information retrieval is to ensure the full exchange and effective utilization of information. Based on the collection of large amounts of dispersed information, indexers construct document / information libraries using document / information descriptions, extract / select identifiers that express the characteristics and subject matter of the documents / information, and organize them in a sequenced manner to construct various retrieval systems. Based on the unified search and name specifications used in the storage and retrieval process, the identifiers of the user's query are matched with the identifiers of the document or information content and form characteristics in the retrieval system. If the identifiers match, the documents or information with these identifiers are output as required.

[0004] Search engines typically use document similarity as their primary ranking factor. Relevance is used to calculate and return the degree of relevance between the query and search results, specifically whether the content in the search result documents satisfies the user's query. In reality, users' search targets are often ambiguous. For example, when a user enters the search term "apple," the actual search target could be fruit or a mobile phone. Therefore, appropriate use of document diversity indexing can effectively address user intent and prevent duplicate results. However, traditional search engines only use document relevance as their primary ranking factor, ignoring the diversity of retrieved content.

[0005] Document relevance and diversity are two commonly used ranking metrics in information retrieval. Due to model bias, overreliance on relevance ranking can easily lead to a lack of breadth in search results, which can lead to the "information cocoon" problem. Overreliance on diversity ranking can easily lead to overly fragmented search results, making it more difficult for users to explore the information space and thus deviating from the original search purpose. In addition to the semantics expressed by individual words, queries submitted by users to search engines also include implicit semantics arising from possible word combinations, known as implicit search intent. While this difficulty can be addressed to some extent by combining relevance and diversity metrics in a weighted manner, it is difficult to fully utilize users' implicit search intent.

[0006] Currently, some search result diversification methods have initially utilized document semantic information and measured its diversity scores with a range of documents, demonstrating the feasibility of using implicit user search intent to improve search engine retrieval efficiency. However, there is still a lack of methods to construct the relationship between users' implicit search intent and the document intent in the search results, and a lack of intuitive representation.

[0007] Since traditional information retrieval technology is based on keyword matching, it often suffers from incomplete and inaccurate searches and low search quality, especially when the user's search target is unclear. Current search engines have not yet fully utilized the user's implicit search intentions in the information retrieval process to optimize the diverse ranking of search results. Summary of the Invention

[0008] The purpose of the present invention is to address the defects and shortcomings of the existing technology, and to solve technical problems such as how to effectively formalize the user's implicit retrieval intention and the differences between documents during information retrieval to further determine the user's retrieval intention and the diversity ranking between documents. This invention creatively proposes an information retrieval method and system based on user's implicit retrieval intention.

[0009] The present invention is implemented by adopting the following technical solutions.

[0010] In one aspect, the present invention proposes an information retrieval method based on a user's implicit retrieval intention, comprising the following steps:

[0011] Step 1: The user submits a query (keyword or sentence), and the search engine retrieves content (documents) from the corpus that meet the user's requirements. The search engine ranks the content by relevance and returns the ranking results.

[0012] Specifically, step 1 includes the following steps:

[0013] Step 1.1: The user enters a keyword or sentence to be searched, and the query is submitted to the search engine of the server.

[0014] Step 1.2: The search engine calculates the search content corresponding to the query and returns the results ranked by relevance.

[0015] The returned search results are sorted from high to low according to the relevance between the user-submitted query and the document when no diversification of search results is performed.

[0016] Step 2: On the server side, use the intent engine to build the user query intent and use the document title to build the document intent.

[0017] Specifically, step 2 includes the following steps:

[0018] Step 2.1: The user summarizes his / her intention into a sentence (i.e., consecutive words) as a query and provides it to the search engine.

[0019] Since queries are usually in the form of word sequences, recurrent neural networks (RNNs), which are widely used to process text sequences, can be used to construct the user intent representation hidden behind the query.

[0020] Specifically, on the server side, the semantic convex hull method in the n-dimensional semantic Euclidean space can be used to fully construct the user's implicit retrieval intention in the information retrieval process. The word vector representation is obtained by means of word embedding models and pre-trained language models, and natural language vocabulary can be mapped to a numerical vector representation. Each point (word vector) in the n-dimensional Euclidean space is mapped one-to-one with a point (semantic point) in the n-dimensional semantic Euclidean space. In the n-dimensional semantic Euclidean space, in addition to these specific semantic points that directly correspond to specific semantics, there are also abstract semantic points that do not directly correspond to specific semantics. Therefore, using the convex hull to express the semantics of the sequence includes not only specific semantic points, but also the implicit meanings generated by possible combinations of specific semantic points. The convex hull is represented by Formula 1, that is: for each point x in the semantic space i , using coefficient α i The weighted average of all possible choices constitutes the entire convex hull.

[0021]

[0022] Among them, Conv() represents the complete query intent.

[0023] Step 2.2: Similarly, construct the intent of each candidate document based on the document title.

[0024] Step 3: The server participates in the diversity evaluation based on the generated user query intent and document intent.

[0025] The semantic coverage and central idea shift between user intent and document intent reflect the semantic relationships between queries and documents, as well as between documents. For example, for two different documents, semantic coverage refers to the area of ​​the intersection of their semantic convex hulls, and central idea shift refers to the Euclidean distance between the centroids of the semantic convex hulls. The higher the semantic coverage and the lower the central idea shift, the closer the intent expressed by the documents. Therefore, semantic coverage and central idea shift can be used to assess diversity.

[0026] Step 4: The search engine returns the search results to the user and displays the user's intent to the user.

[0027] Specifically, step 4 includes the following steps:

[0028] Step 4.1: The search engine will rank the documents after evaluating the diversity and return the results to the client.

[0029] Step 4.2: The visualization engine completes the dimensionality reduction and visualization of the data and displays the visualized part to the user.

[0030] On the client side, the interface displays the query results returned by the search engine, as well as a visual graph of the user's query intent.

[0031] Step 5: If the user is interested in a document in the query results, they click on the document title to further explore its content. Furthermore, when the user is about to click on the document title, the visualization engine displays a visual representation of the document's intent. This document's intent is overlaid on the user's current query intent, allowing the user to compare query-document intent similarity and document novelty. If the user clicks away from the location, the document's intent remains displayed. If the user reselects a document and hovers over the title, the document's intent is updated.

[0032] Step 6: During the retrieval session, each query submitted by the user to the search engine will have a graph of the user's query intent added along the timeline and returned to the client for the user to view.

[0033] A search session is the process by which a user continuously modifies the query they submit to a search engine within a short period of time to ultimately satisfy their search intent. After browsing several documents ranked high in the result list, if their needs are not met, they will abandon the search or start a new round of queries.

[0034] On the other hand, the present invention also proposes an information retrieval system based on user implicit retrieval intention, including deploying a search engine module, an intention engine module, and a visualization engine module on the server side.

[0035] The search engine module is used to meet the user's search requirements. After the user submits the keywords they want to search for on the interface, the search engine module retrieves content that meets the user's requirements, performs a diversified evaluation, and returns the search results to the user.

[0036] The intent engine module is responsible for constructing user query intent and document intent. When a user summarizes their intent as a sentence and submits it to the search engine, the intent engine module constructs the user's implicit search intent using the semantic convex hull method in semantic Euclidean space. For each candidate document in the search results, the intent engine module uses the document title to construct the document intent for each candidate document. When a user selects a document to view, the intent engine module uses the document title to construct the document intent for the selected document.

[0037] The visualization engine module is used to restore and visualize data dimensions. Based on the user's query, a visualization graph of the user's query intent is displayed. Based on the document selected by the user, the document intent is displayed or updated and overlaid with the user's current query intent. During a retrieval session, the intent of each round of user query is added along the timeline and returned to the user.

[0038] The connection relationship between the above modules is:

[0039] The output end of the search engine module is connected to the input end of the intention engine module, and the output end of the intention engine module is connected to the input ends of the search engine module and the visualization engine module.

[0040] Beneficial effects

[0041] Compared with the prior art, the present invention has the following beneficial effects:

[0042] 1. In the process of searching using a search engine, the retrieval method of the present invention ranks documents that meet the search criteria according to their relevance to the query keywords. At the same time, a semantic convex hull is introduced in the Euclidean semantic space to form a visual comparison scheme for retrieval intent and document intent. The system constructs the user's retrieval intent to reflect the answers the user is looking for and to quantify the diversity of documents. At the same time, the system introduces a timeline to graphically display the retrieval intent of multiple rounds of queries and organize them serially from the first to the last through the timeline. The present invention effectively integrates the user's retrieval intent into the traditional relevance ranking search engine. The new user interface and retrieval mechanism enable users to explore various documents, facilitate users to search for some exploratory questions, and enhance the diversity of information retrieval results. Compared with plain text, adding visual elements to the search engine intuitively feeds back the user's implicit retrieval intent to the user in a graphical manner, which enhances the interaction between the user and the search engine, effectively improves the retrieval efficiency and enhances the retrieval fun. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 It is an information retrieval process based on the user's implicit retrieval intention;

[0044] Figure 2 This is a system framework diagram based on the user's implicit retrieval intention;

[0045] Figure 3 It is the overall topology diagram of the system and web server based on the user's implicit search intention;

[0046] Figure 4 This is the first screenshot when a user uses an information retrieval system based on the user's implicit retrieval intention;

[0047] Figure 5 This is the second screenshot when a user uses the information retrieval system based on the user's implicit retrieval intention. DETAILED DESCRIPTION

[0048] The present invention is further described in detail below with reference to the accompanying drawings.

[0049] An information retrieval method based on a user's implicit retrieval intention comprises the following steps:

[0050] Step 1: The user submits a query (keyword or sentence), and the search engine retrieves content (documents) that meet the user's requirements from the corpus, sorts them by relevance, and returns the sorted results.

[0051] Step 1.1: The user enters a keyword or sentence to be searched, and the query is submitted to the search engine of the server.

[0052] The client used by the user includes a browser, such as Firefox, IE, Chrome, etc.

[0053] Step 1.2: The server-side search engine creates an inverted index for existing documents in advance. When the search engine receives the query submitted by the client, it sorts the documents by their TF-IDF and PageRank values ​​to obtain the search results, and returns the relevance ranking results.

[0054] For the returned search content, when no search result diversification is performed, it is sorted from high to low according to the relevance between the query submitted by the user and the document.

[0055] Step 2: On the server side, use the intent engine to build the user query intent and use the document title to build the document intent.

[0056] Step 2.1: Users usually summarize their intent into a sentence (continuous words) as a query and provide it to the search engine.

[0057] Since queries are usually made in the form of word sequences, recurrent neural networks (RNNs), which are widely used to process text sequences, are naturally used to construct the user intention representation hidden behind the query. A recurrent neural network is a type of recurrent neural network that takes sequence data as input, recursively in the direction of sequence evolution, and all nodes (recurrent units) are connected in a chain. Whether it is a query or a document, it is natural language sequence data, and the long short-term memory network (LSTM) is a neural network suitable for processing text sequence data. It solves the gradient vanishing problem of the original recurrent neural network and often achieves better results in various tasks. Its hidden layer can memorize the context information of the sequence, and the expression of semantics requires the context information of the sequence. It should be noted that the long short-term memory network is only a specific implementation method, and the present invention is not limited to the method mentioned above.

[0058] On the server side, the semantic convex hull method in the n-dimensional semantic Euclidean space is used to fully construct the user's implicit retrieval intention in the information retrieval process. In the semantic Euclidean space, the user query generates two n-dimensional vector sequences (χ and H). Each point in χ is a specific semantic point that can correspond to an actual word, and each point in H is an abstract semantic point. The explicit query intention can be expressed as Conv(χ), while Conv(H) can be used to indicate the user's implicit query intention. Each semantic point that constitutes the convex hull is a distributed representation method (word embedding). Each queried document can also be represented as an n-dimensional vector sequence (χ d and H d ).

[0059] Step 2.2: Similarly, construct the intent of each candidate document based on the document title.

[0060] Step 3: The server participates in the diversity evaluation based on the generated user query intent and document intent.

[0061] Since the semantic coverage and central idea shift between user intent and document intent reflect the relationship between query-document and document-document semantics, they can be used to evaluate diversity.

[0062] The central idea of ​​a natural language sequence is another important aspect of measuring sequence semantics. The central idea of ​​a sequence refers to the core summary of the meaning expressed by the sequence and should be located at the center of the semantic convex hull. The meaning of the entire sequence is closely centered around the central idea. For example, the central idea of ​​"Beijing Institute of Technology" is "Beijing Institute of Technology."

[0063] Semantic coverage represents the similarity in intent between the query and the document. A larger semantic coverage indicates a more similar intent between the query and the document; a smaller semantic coverage indicates a more dissimilar intent. Central idea shift represents the relevance of intent between the query and the document. A smaller shift indicates a closer central idea between the query and the document; a smaller shift indicates a greater dissimilarity.

[0064] Step 4: The search engine returns the search results to the user and displays the user's intent to the user.

[0065] Step 4.1: The search engine will rank the documents after evaluating the diversity and return the results to the client.

[0066] Step 4.2: The visualization engine completes the dimensionality reduction and visualization of the data and presents the visualized part to the user.

[0067] By reducing the convex hull in n-dimensional semantic Euclidean space to a two-dimensional space, it can be plotted on the page, resulting in a graphical representation of the search intent. This dimensionality reduction method can employ the t-SNE algorithm. t-SNE is a nonlinear dimensionality reduction algorithm that preserves the topological structure of high-dimensional space during dimensionality reduction, making it ideal for reducing high-dimensional data to two or three dimensions.

[0068] Step 4.3: On the client side, the left side of the interface displays the query results returned by the search engine, and the right side of the interface displays a visualization of the user's query intent.

[0069] Step 5: If a user is interested in a document in the query results, they typically click on the document title to further explore its content. When the mouse hovers over the document title, the visualization engine presents a visual representation of the document's intent to the user.

[0070] This document intent is displayed overlaid on the user's current query intent, allowing users to compare query-document intent similarity and document novelty. The document intent graphic remains visible when the mouse moves away from the document, and updates when the user reselects a document and hovers over the title.

[0071] Step 6: During the retrieval session, each query submitted by the user to the search engine will be added with a user query intent along the timeline and returned to the client for the user to view.

[0072] A search session is the process by which a user continuously modifies the query they submit to a search engine within a short period of time to ultimately satisfy their search intent. After browsing the top few documents in the result list, if their needs are not met, they often abandon the search or start a new search.

[0073] Figure 1This article demonstrates the information retrieval process based on the user's implicit search intent. After a user submits a query to the search engine, the search engine uses relevant technologies, such as the vector space model to calculate text similarity and the TF-IDF algorithm to calculate document weights, to calculate the document set corresponding to the user's query. The search engine then constructs the user's query intent using the intent engine and the document intent using the document titles. Based on the generated user query intent and document intent, the search engine then evaluates the diversity of the resulting document set and returns it to the user as search results. A visual representation of the user's intent is also presented to the user.

[0074] Figure 2 The following diagram shows the system framework structure based on the user's implicit retrieval intention. Figure 2 As shown, the following modules are deployed on the server side, including the search engine module, the intent engine module, and the visualization engine module.

[0075] The search engine module is used to meet the user's search requirements. After the user submits the keywords they want to search for on the interface, the search engine module generates query completion prompts, retrieves content that meets the user's requirements, performs a diversified evaluation, and returns the ranked search results to the user.

[0076] The intent engine module is used to construct user query intent and document intent. When a user summarizes their intent as a sentence and submits it to the search engine, the intent engine module constructs the user's implicit search intent using the semantic convex hull method in semantic Euclidean space. For each candidate document in the search results, the intent engine module uses the document title to construct the document intent for each candidate document. When a user selects a document to view, the intent engine module uses the document title to construct the document intent for the selected document.

[0077] The visualization engine module is used to restore and visualize data dimensions. Based on the user's query, it displays search results and a visual graph of the user's query intent. Based on the document selected by the user, the document intent is displayed or updated and overlaid with the user's current query intent. During a retrieval session, a graph of the user's query intent is added along a timeline and returned to the user.

[0078] The connection relationship between the above modules is:

[0079] The output end of the search engine module is connected to the input end of the intention engine module, and the output end of the intention engine module is connected to the input ends of the search engine module and the visualization engine module.

[0080] Figure 3The overall topology diagram of the system based on the user's implicit retrieval intention and the web server is shown. The tools used in the social interaction-based information retrieval system during implementation are specifically shown. Among them, the client mainly uses vue.js to complete the user interface. Vue.js only focuses on the view layer and is easy to integrate with other libraries. The web server uses the Python Tornado framework, which has the properties of asynchronous and non-blocking network I / O. The search engine module uses the Python Whoosh toolkit to index existing documents and provide search support. The intent engine module uses PyTorch to calculate related models. The visualization engine module uses the t-SNE algorithm for dimensionality reduction and the open source JavaScript visualization tool ECharts5 to achieve graphical visualization.

[0081] Figure 4 and Figure 5 It is a screenshot of a user using an information retrieval system based on the user’s implicit retrieval intention. Figure 4 In the search box, the user enters the query "Harry Potter and the Philosopher's Stone" to search, and the system returns a variety of search results to the user on the left side of the interface. At the same time, the graphic part on the right side of the interface shows the user a graphic of the query intent. Next, the user is interested in a document in the query results and is ready to click on the document title to further view its content. When the user hovers the mouse over the document title, the visualization engine shows the user a visualization graphic of the document intent "Harry Potter and the Philosopher's Stone - Encyclopedia". This document intent is displayed in superimposed colors with the user's current query intent, making it easier for users to compare the similarity of intent between queries and documents and the novelty of documents, such as Figure 5 As shown. When the mouse moves away from the document, the graphic of the document's intent continues to display. When the user reselects a document and hovers over the title, the graphic of the document's intent is updated. Compared to reading the plain text in the search results, adding visual elements to the search engine allows users to intuitively judge the relevance of the user's intent to the document, as well as the differences between different documents. When the user modifies the query and resubmits it, the search engine recalculates the search results and displays them on the left side of the system. On the right side, a new graphic of the user's query intent is added along the timeline based on the previous one and returned to the client for the user to view.

[0082] The above is a preferred embodiment of the present invention, and the present invention should not be limited to the contents disclosed in the embodiment and the drawings. Any equivalent or modification completed without departing from the spirit disclosed in the present invention shall fall within the scope of protection of the present invention.

Claims

1. An information retrieval method based on user implicit retrieval intention, characterized in that: The following steps are involved: Step 1: The user submits a query, and the search engine retrieves content that meets the user's requirements from the corpus; Sort the relevance of the content and return the sorted results; The returned search results are sorted from high to low based on the relevance between the user-submitted query and the document when no search result diversification is performed; Step 2: On the server side, use the intent engine to construct the user query intent and use the document title to construct the document intent; Step 2.1: The user summarizes their intent into a sentence as a query and provides it to the search engine; On the server side, the semantic convex hull method in n-dimensional semantic Euclidean space is used to fully construct the user's implicit search intent during the information retrieval process. Through word embedding models and pre-trained language models, word vector representations are obtained, mapping natural language vocabulary to numerical vector representations. Each point in n-dimensional Euclidean space is mapped one-to-one to a point in n-dimensional semantic Euclidean space. The convex hull is used to express the semantics of the sequence, including not only the specific semantic points but also the implicit meaning generated by the combination of specific semantic points; the convex hull is expressed by Formula 1, that is: for each point x in the semantic space i , using coefficient α i All the choices obtained by weighted average constitute the entire convex hull; Among them, Conv() represents the complete query intent; Step 2.2: Construct the intent of each candidate document based on the document title; Step 3: The server participates in the diversity evaluation based on the generated user query intent and document intent; Step 4: The search engine returns the search results to the user and displays the user's intent to the user; Step 5: If the user is interested in a document in the query results, he or she will click on the document title to further view its content; When the user is about to click on the document title, the visualization engine displays a visualization of the document intent to the user, and this document intent is superimposed on the user's current query intent. When the user clicks away from the location, the document's intent continues to be displayed; When the user reselects a document and hovers over the title, the document intent is updated; Step 6: During the retrieval session, each query submitted by the user to the search engine will have a graph of the user's query intent added along the timeline and returned to the client for the user to view; Among them, a retrieval session refers to the process in which a user continuously modifies the query submitted to the search engine in a short period of time around a retrieval target to ultimately satisfy his or her retrieval intention; after browsing several documents ranked high in the result list, if the user's needs are not met, he or she will give up the search or start a new round of queries.

2. The information retrieval method based on user implicit retrieval intention according to claim 1, characterized in that: Step 1 includes the following steps: Step 1.1: The user enters the keywords or phrases they wish to search for, and the query is submitted to the server's search engine. Step 1.2: The search engine on the server side calculates the search content corresponding to the query and returns the results ranked by relevance.

3. The information retrieval method based on user implicit retrieval intention according to claim 2, characterized in that: In step 1.2, the search engine builds an inverted index for existing documents in advance. When the search engine receives the query submitted by the client, it sorts the documents by their TF-IDF values ​​and PageRank values ​​to obtain the search results, and returns the relevance ranking results.

4. The information retrieval method based on user implicit retrieval intention according to claim 1, characterized in that: Step 4 includes the following steps: Step 4.1: The search engine ranks the documents after evaluating their diversity and returns the results to the client. Step 4.2: The visualization engine completes the dimensionality reduction and visualization of the data and displays the visualized part to the user; On the client side, the interface displays the query results returned by the search engine, as well as a visual graph of the user's query intent.

5. An information retrieval system based on user implicit search intent, implemented according to the information retrieval method of claim 1, characterized in that the system includes a search engine module, an intent engine module, and a visualization engine module deployed on the server side; The search engine module is used to meet the user's search requirements. After the user submits the keywords they want to search for on the interface, the search engine module retrieves the content that meets the user's requirements, performs a diversified evaluation, and returns the search results to the user. The intent engine module is used to build user query intent and document intent; When a user summarizes his / her intention into a sentence as a query and submits it to the search engine, the intention engine module constructs the user's implicit retrieval intention using the semantic convex hull method in the semantic Euclidean space; For each candidate document in the search results, the intent engine module uses the document title to construct the document intent of each candidate document; When a user selects a document to view, the intent engine module uses the document title to construct the document intent of the selected document; The visualization engine module is used to achieve dimensionality reduction and visualization of data. Based on the user's query, it displays a visual graph of the user's query intent. Based on the document selected by the user, it displays or updates the document intent and overlays it with the user's current query intent. In the retrieval session, the intent of each round of user query is added along the timeline and returned to the user. The connection relationship between the above modules is: The output end of the search engine module is connected to the input end of the intention engine module, and the output end of the intention engine module is connected to the input ends of the search engine module and the visualization engine module.

Images