Enhanced AI search system

Abstract

The invention discloses an enhanced AI search system which comprises an interaction module, an approximate recommendation problem generation module, a retrieval module and a summary feedback module. The interaction module is used for providing a query interaction interface, and a user inputs a query problem and obtains a retrieval result through the query interaction interface; the approximate recommendation question generation module is used for generating an approximate recommendation question according to a query question input by a user and sending the generated approximate recommendation question to the retrieval module; the retrieval module is used for performing content retrieval by utilizing a search engine according to a query question input by a user and / or an approximate recommendation question generated by the approximate recommendation question generation module; and the summary feedback module is used for receiving a retrieval result of the retrieval module, generating a retrieval summary by utilizing a generative AI model and returning the retrieval summary to the interaction module. By means of the enhanced AI search system, the intelligence, individuation and high efficiency of search can be improved, the accuracy of search results is improved, and better information search service is provided for users.

Description

Technical Field

[0001] The present invention relates to the technical field of search engines, and in particular to an enhanced AI search system. Background Art

[0002] As an important part of the Internet, traditional search engines have greatly changed the way people obtain information. From the initial simple keyword matching to the current intelligent search based on semantic understanding, search engines have undergone tremendous changes. However, while developing rapidly, search engines are also facing some challenges, which mainly come from changes in user needs and technical limitations.

[0003] Users usually want to find the information they need in a short time, so they tend to browse the first few records on the search results page. With the application of big data and machine learning technology, search engines are able to provide personalized recommendation results based on users' search history and preferences, and users are increasingly relying on these recommendations. However, this recommendation mechanism sometimes leads to the "information cocoon" effect, that is, the range of information received by users becomes narrower and narrower, because the system always tends to display content similar to the user's past behavior, which may affect the user's ability to obtain comprehensive information.

[0004] In addition, although search engines are quite mature, users still encounter many problems when using them. First, faced with a huge amount of information, users often find it difficult to filter out the most relevant content. This information overload phenomenon troubles many users. Second, commercial advertisements may occupy a prominent position in the search results, affecting the user's search experience. Furthermore, since search engines rely on big data analysis to make recommendations, this mechanism may cause users to encounter irrelevant results when searching, thereby affecting the user's retrieval quality and efficiency.

[0005] Although traditional search engines have made progress in many aspects, there are still many areas for improvement. With the advancement of technology and the continuous changes in user needs, search engines need to continue to innovate in order to better meet user needs. Summary of the invention

[0006] In view of the deficiencies of the prior art, the present invention aims to provide an enhanced AI search system.

[0007] In order to achieve the above object, the present invention adopts the following technical solution:

[0008] An enhanced AI search system, comprising an interaction module, an approximate recommendation question generation module, a retrieval module and a summary feedback module;

[0009] The interactive module is used to provide a query interactive interface, through which users input query questions and obtain search results;

[0010] The approximate recommendation question generation module is used to generate an approximate recommendation question according to the query question input by the user, and send the generated approximate recommendation question to the retrieval module;

[0011] The retrieval module is used to perform content retrieval using a search engine according to a query question input by a user and / or an approximate recommendation question generated by an approximate recommendation question generation module;

[0012] The summary feedback module is used to receive the search results of the search module, and use the generative AI model to generate a search summary and return it to the interaction module.

[0013] Furthermore, the interaction module provides the user with a choice of query modes, including simple and deep; if the query mode selected by the user is the simple mode, the interaction module directly sends the query question to the retrieval module for processing, otherwise the query question is sent to the approximate recommendation question generation module to further generate an approximate recommendation question.

[0014] Furthermore, the interactive module also provides users with options for query types, including full-network query and academic query. When the user selects full-network query, the retrieval module uses a comprehensive search engine for comprehensive search. When the user selects academic query, the retrieval module uses an academic search engine for search.

[0015] Furthermore, the approximate recommendation question generation module generates a recommendation question that is similar to the original query question by using the generative AI model 2.

[0016] Furthermore, the retrieval module accesses services of multiple search engines and sends query requests to all search engines that meet the query type requirements in parallel.

[0017] Furthermore, before generating the search summary, the summary feedback module pre-processes the search results obtained by each search engine as follows:

[0018] De-duplication: Since the same query may return the same search results in different search engines, the summary feedback module deduplicates the same link addresses in the search results to eliminate duplicate content;

[0019] Content detection: After deduplication, the summary feedback module uses regular matching to detect and filter the content of the search results;

[0020] Retrieval result verification: The summary feedback module checks the response of each request. If the request returns a 200OK status code within the set timeout period, it means that the link is valid; if the request times out or returns an error status code, the link is considered invalid.

[0021] Furthermore, the summary feedback module performs post-processing on the pre-processed search results:

[0022] (1) Content segmentation: The linked page content is divided into multiple short paragraphs according to the set first number of words, and the set second number of words is reserved between each paragraph as contextual transition to ensure the coherence and integrity of the content;

[0023] (2) Vectorization: Use the embedding model to vectorize each paragraph;

[0024] (3) Similarity calculation: Calculate the cosine similarity between the vector of the original query question and the vector of each paragraph obtained in step (2);

[0025] (4) Result sorting: Sort each paragraph according to the calculated similarity score, and select the top N paragraphs with the highest scores as the final retrieval results.

[0026] Furthermore, the summary feedback module uses the generative AI model 1 and the final retrieval result obtained by post-processing to generate a retrieval summary using different generation strategies according to different query modes to meet the specific needs of users; the generative AI model 1 is deployed on a third-party server, and the summary feedback module integrates the interface of the generative AI model 1 to realize the remote call of the generative AI model 1. The summary feedback module uses the server-sent event SSE mechanism to support streaming response returns. When a user initiates a query request, the summary feedback module begins to gradually return the retrieval summary without waiting until the entire retrieval process is completed.

[0027] Furthermore, the retrieval module stores the web page information and documents retrieved by each search engine into a document retrieval library. When the retrieval module receives a query question or a recommendation question, it simultaneously retrieves the corresponding response results in the document retrieval library according to the retrieval question input by the user.

[0028] The document retrieval library first parses the received web page information and documents, and then preprocesses the parsed content. For unstructured text, it uses rule cutting to divide the text content into smaller blocks and retains a certain amount of text overlap between blocks to improve the accuracy of retrieval; for structured text, it divides the text according to its organizational structure and hierarchical logic; finally, the preprocessed document content is converted into document vectors and stored in the document retrieval library.

[0029] Furthermore, the process of searching in the document retrieval library by the retrieval module specifically includes query processing, vector matching and reverse ranking fusion: the retrieval module forms a question list with the original query question and the recommended question, matches each question in the question list in the document retrieval library respectively, and recalls the N document vectors that are most similar to the target question respectively. Finally, the retrieval module uses the RRF algorithm to merge the recall results of different questions; the RRF algorithm calculates the comprehensive score of each document vector by ranking the weighted document vector in the recall results of different questions, and then comprehensively sorts the document vectors in the recall results of all questions according to the comprehensive score of each document vector, removes duplicate document vectors, ensures that each document appears only once, and filters out document vectors with scores lower than the set similarity threshold to improve the quality of the results, and finally merges the recall results of different questions; finally, the summary feedback module calls the language model, generates detailed and coherent response content for the final recall result and returns it to the interaction module for the user to view.

[0030] The beneficial effect of the present invention is that by utilizing the enhanced AI search system of the present invention, the intelligence, personalization and efficiency of the search can be improved, the accuracy of the search results can be improved, and better information retrieval services can be provided to users. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a schematic diagram of the architecture of an enhanced AI search system in an embodiment of the present invention;

[0032] Figure 2 This is an example diagram of a query interaction interface in an embodiment of the present invention. DETAILED DESCRIPTION

[0033] The present invention will be further described below in conjunction with the accompanying drawings. It should be noted that this embodiment is based on the technical solution and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to this embodiment.

[0034] This embodiment provides an enhanced AI search system, such as Figure 1 As shown, it includes an interaction module, an approximate recommendation question generation module, a retrieval module and a summary feedback module.

[0035] The interactive module is used to provide a query interactive interface, through which users input query questions and obtain search results;

[0036] The approximate recommendation question generation module is used to generate an approximate recommendation question according to the query question input by the user, and send the generated approximate recommendation question to the retrieval module;

[0037] The retrieval module is used to perform content retrieval using a search engine according to a query question input by a user and / or an approximate recommendation question generated by an approximate recommendation question generation module;

[0038] The summary feedback module is used to receive the search results of the search module, and use the generative AI model to generate a search summary and return it to the interaction module.

[0039] In this embodiment, the interactive module provides the user with a choice of query modes, which may include simple, general, and detail, or may only include simple and detail; if the query mode selected by the user is the simple mode, the interactive module directly sends the query question to the retrieval module for execution, otherwise the query question is sent to the approximate recommendation question generation module to further generate an approximate recommendation question. In this way, the efficiency of retrieval in the simple mode will be faster, but the range of retrieval results will be narrower, while in the general or deep mode, by generating approximate recommendation questions, the range of retrieval can be expanded to obtain more comprehensive and accurate retrieval results, but the retrieval efficiency will be relatively reduced, and users can make more flexible choices according to their own needs.

[0040] In this embodiment, the interactive module also provides the user with the option of query type, including full network query and academic query. When the user selects full network query, the search module uses the comprehensive search engine to perform comprehensive search, and when the user selects academic query, the search module uses the academic search engine to perform search. This ensures that the search is more professional and academic, and is more suitable for researchers, students and other users with academic needs.

[0041] like Figure 2 As shown, the query interaction interface may include elements such as a query input box, a mode selector, a type selector, and a send button. The query input box is where the user enters the query question, the mode selector is a drop-down menu for selecting the query mode, the type selector is a switch selection box on the right side of the page for selecting the query type, and the send button is used to initiate a query request after entering the query question and selecting the query mode and query type.

[0042] Furthermore, in practical applications, the user's original query may only cover one aspect or angle, and this one-sidedness may result in incomplete or low relevance retrieval results. In order to improve the quality of search results, the approximate recommendation question generation module of this embodiment uses the generative AI model 2 to implement user expansion and extension of query questions, generate recommendation questions similar to the original query questions, and provide a more comprehensive and multi-angle perspective, thereby making the retrieved information more valuable.

[0043] In this embodiment, the search module accesses services of multiple search engines, such as Google, Bing, DuckDuckG, etc., so as to cover a wide range of Internet resources, such as web pages, news, and academic literature, etc. The integration of multiple search engines can ensure that users can obtain information from multiple perspectives to ensure the relevance and accuracy of the search results.

[0044] Furthermore, based on access to multiple search engines, in order to improve query efficiency, the retrieval module will send query requests to all search engines that meet the query type requirements in parallel. This means that each search question will be sent to every search engine that meets the query type requirements, so that the search results can be obtained quickly. Specifically, a timeout (such as 2 seconds) can be set for each query request to avoid overall blocking due to unresponsive websites. Query requests can be managed by using coroutines to ensure that the server is not burdened by excessive concurrency.

[0045] Furthermore, in this embodiment, the summary feedback module performs the following preprocessing on the search results obtained by each search engine before generating the search summary:

[0046] De-duplication: Since the same query may return the same search results in different search engines, the summary feedback module deduplicates the same link addresses in the search results to eliminate duplicate content.

[0047] Content detection: After deduplication, in the search results, sometimes the search engine returns unclear values ​​or vague information, such as "xx yuan", "** billion", etc. In order to filter out meaningless information, in this embodiment, the summary feedback module uses regular matching to perform content detection and filtering on the search results to ensure that the information displayed to the user is meaningful and reliable.

[0048] Retrieval result verification: Ensuring that all links returned to users are valid and accessible is the key to improving user experience. Therefore, the summary feedback module checks the response to each query request. If the query request returns a 200OK status code within the set timeout period, it means that the link is valid; if the request times out or returns an error status code, the link is considered invalid.

[0049] Furthermore, in order to further improve the relevance and accuracy of the retrieval results, the summary feedback module post-processes the pre-processed retrieval results:

[0050] (1) Content segmentation: Divide the linked page content into sections of 500 words each, leaving 100 words between each section as contextual transitions to ensure the coherence and integrity of the content.

[0051] (2) Vectorization: Use the embedding model to vectorize each paragraph.

[0052] (3) Similarity calculation: Calculate the cosine similarity between the vector of the original query question and the vector of each paragraph obtained in step (2).

[0053] (4) Result sorting: Sort each paragraph according to the calculated similarity score, and select the top 8 paragraphs with the highest scores as the final retrieval results.

[0054] In this embodiment, the summary feedback module uses a generative AI model 1, combined with the final search results obtained by post-processing, to generate a search summary using different generation strategies based on different query modes to meet the specific needs of users. The prompt words and number of generated words in each query mode will be different, but all query modes use the search results as contextual basis to guide the generative AI model 1 to generate the search summary content. This process is designed to reduce the illusion phenomenon in AI-generated content, improve the accuracy of information, and ensure that the final search summary can draw specific conclusions, so that the entire search content is more focused on answering the user's query questions.

[0055] In this embodiment, the generation strategy specifically includes a word count strategy. Taking the query mode including simple, general and deep as an example, when the query mode is simple, a search summary of 200-300 words is generated, and when the query mode is general or deep, a search summary of 500-800 words is generated. By limiting the degree of freedom of generating content and increasing the reliance on search results, the accuracy and reliability of information can be significantly improved.

[0056] In this embodiment, the user can define the retry interval in case of network problems or other abnormal situations by setting the countdown parameter. The retry mechanism helps to improve the fault tolerance of the system and ensures that reliable search results can be obtained even in unstable network conditions.

[0057] In this embodiment, the generative AI model 1 can be deployed on a third-party server, and the summary feedback module integrates the interface of the generative AI model 1 to realize the remote call of the generative AI model 1.

[0058] Furthermore, in this embodiment, the generative AI model usually supports streaming data transmission, and the system of this embodiment can use this feature to provide a better user experience. By using the Server-Sent Events (SSE) mechanism, real-time data updates are achieved, allowing users to see the gradually generated content while waiting for the search results.

[0059] Server-Sent Events (SSE) are a technology that allows a server to push real-time updates to a browser. In this embodiment, the SSE mechanism is used to support streaming response returns, which means that when a user initiates a query request, the summary feedback module can begin to gradually return the search summary without waiting until the entire search process is completed. This approach improves the user's interactive experience, especially when processing large amounts of data or complex queries, and can prevent users from getting impatient due to long waits. Moreover, the streaming response mechanism can save bandwidth resources because it only transmits necessary data instead of transmitting a large amount of data at one time. If an error occurs during the transmission process, the SSE mechanism allows the retransmission of part of the data, thereby reducing the possibility of complete failure due to network problems. Therefore, by adopting streaming response returns, not only the user experience is improved, but also it can better adapt to various network conditions, ensuring that stable and reliable services can be provided even in poor network conditions.

[0060] In this embodiment, the retrieval module stores the web page information and documents (PDF, Word, etc.) retrieved by various search engines into a document retrieval library. When the retrieval module receives a query question or a recommendation question, it simultaneously retrieves the corresponding response results in the document retrieval library based on the retrieval question entered by the user.

[0061] Specifically, the document retrieval library supports processing and parsing documents in multiple formats, including PDF, Word documents, JSON files, and web pages, converting documents from different sources into a format that is easy to retrieve.

[0062] Furthermore, the document retrieval library preprocesses the parsed content. For unstructured text, the text content is divided into smaller chunks by regular cutting, and a certain amount of text overlap is retained between chunks to improve the accuracy of retrieval. For structured text, the focus is on understanding the organizational structure and hierarchical logic of the text to facilitate more accurate information retrieval. Specifically, for texts with different title levels such as word and pdf, they are parsed into structures of different levels according to the titles. For data such as JSON files, they are parsed and preprocessed according to structural information. Finally, the preprocessed document content is converted into document vectors (using NLP technology such as TF-IDF, Word2Vec or BERT) and stored in the document retrieval library to speed up retrieval and improve retrieval efficiency.

[0063] In this embodiment, the process of the retrieval module performing retrieval in the document retrieval library specifically includes query processing, vector matching and reverse ranking fusion (RRF). The retrieval module further forms a question list with the original query question and the recommended question, matches each question in the question list in the document retrieval library, and recalls the N document vectors that are most similar to the target question (for example, the similarity can be calculated using cosine similarity), and finally the retrieval module uses the RRF algorithm to merge the recall results of different questions. The RRF algorithm calculates the comprehensive score of each document vector by weighting the ranking of the document vector in the recall results of different questions, usually using the formula 1k+rankk+rank1, where k is a constant and rank is the ranking of the document vector. Then, according to the comprehensive score of each document vector, the document vectors in the recall results of all questions are comprehensively sorted, duplicate document vectors are removed, and each document appears only once, and document vectors with scores lower than the set similarity threshold are filtered out to improve the quality of the results, and finally the recall results of different questions are merged. Finally, the summary feedback module calls the language model, generates detailed and coherent response content for the final recall result and returns it to the interactive module for the user to view.

[0064] Furthermore, based on the preliminary recall, the merged recall results can be re-evaluated and screened. The relevance of the recall results can be further optimized by setting a higher similarity threshold or adjusting the RRF weight parameter. In addition, a deep learning model based on semantic understanding can be introduced to improve the quality of the recalled content.

[0065] For those skilled in the art, various corresponding changes and modifications can be made according to the above technical solutions and concepts, and all of these changes and modifications should be included in the protection scope of the claims of the present invention.

Claims

1. An enhanced AI search system, characterized in that: It includes an interactive module, an approximate recommendation question generation module, a retrieval module, and a summary feedback module; The interactive module is used to provide a query interactive interface, through which users input query questions and obtain search results; The approximate recommendation question generation module is used to generate an approximate recommendation question according to the query question input by the user, and send the generated approximate recommendation question to the retrieval module; The retrieval module is used to perform content retrieval using a search engine according to a query question input by a user and / or an approximate recommendation question generated by an approximate recommendation question generation module; The summary feedback module is used to receive the search results of the search module, and use the generative AI model to generate a search summary and return it to the interaction module.

2. The enhanced AI search system according to claim 1, characterized in that: The interactive module provides the user with a choice of query modes, including simple and deep; if the query mode selected by the user is the simple mode, the interactive module directly sends the query question to the retrieval module for execution, otherwise the query question is sent to the approximate recommendation question generation module to further generate an approximate recommendation question.

3. The enhanced AI search system according to claim 1, characterized in that: The interactive module also provides users with options for query types, including full-network query and academic query. When the user selects full-network query, the retrieval module uses a comprehensive search engine to perform a comprehensive search. When the user selects academic query, the retrieval module uses an academic search engine to perform a search.

4. The enhanced AI search system according to claim 1, characterized in that: The approximate recommendation question generation module generates recommendation questions that are similar to the original query question by using the generative AI model 2.

5. The enhanced AI search system according to claim 1, characterized in that: The retrieval module accesses the services of multiple search engines and sends query requests to all search engines that meet the query type requirements in parallel.

6. The enhanced AI search system according to claim 1, characterized in that: Before generating the search summary, the summary feedback module first pre-processes the search results obtained by each search engine as follows: De-duplication: Since the same query may return the same search results in different search engines, the summary feedback module deduplicates the same link addresses in the search results to eliminate duplicate content; Content detection: After deduplication, the summary feedback module uses regular matching to detect and filter the search results; Retrieval result verification: The summary feedback module checks the response of each request. If the request returns a 200OK status code within the set timeout period, it means that the link is valid; if the request times out or returns an error status code, the link is considered invalid.

7. The enhanced AI search system according to claim 6, characterized in that: The summary feedback module performs post-processing on the pre-processed retrieval results: (1) Content segmentation: The linked page content is divided into multiple short paragraphs according to the set first number of words, and the set second number of words is reserved between each paragraph as contextual transition to ensure the coherence and integrity of the content; (2) Vectorization: Use the embedding model to vectorize each paragraph; (3) Similarity calculation: Calculate the cosine similarity between the vector of the original query question and the vector of each paragraph obtained in step (2); (4) Result sorting: Sort each paragraph according to the calculated similarity score, and select the top N paragraphs with the highest scores as the final retrieval results.

8. The enhanced AI search system according to claim 7, characterized in that: The summary feedback module uses the generative AI model 1 and the final retrieval result obtained by post-processing to generate a retrieval summary using different generation strategies according to different query modes to meet the specific needs of users; the generative AI model 1 is deployed on a third-party server, and the summary feedback module integrates the interface of the generative AI model 1 to realize the remote call of the generative AI model 1. The summary feedback module uses the server-sent event SSE mechanism to support streaming response returns. When a user initiates a query request, the summary feedback module begins to gradually return the retrieval summary without waiting until the entire retrieval process is completed.

9. The enhanced AI search system according to claim 1, characterized in that: The retrieval module stores the web page information and documents retrieved by each search engine into a document retrieval library. When the retrieval module receives a query question or a recommendation question, it simultaneously retrieves the corresponding response results in the document retrieval library according to the retrieval question input by the user; The document retrieval library first parses the received web page information and documents, and then preprocesses the parsed content. For unstructured text, it uses rule cutting to divide the text content into smaller blocks and retains a certain amount of text overlap between blocks to improve the accuracy of retrieval; for structured text, it divides the text according to its organizational structure and hierarchical logic; finally, the preprocessed document content is converted into document vectors and stored in the document retrieval library.

10. The enhanced AI search system according to claim 9, characterized in that: The process of retrieval in the document retrieval library by the retrieval module specifically includes query processing, vector matching and reverse ranking fusion: the retrieval module forms a question list with the original query question and the recommended question, matches each question in the question list in the document retrieval library respectively, and recalls the N document vectors that are most similar to the target question respectively. Finally, the retrieval module uses the RRF algorithm to merge the recall results of different questions; the RRF algorithm calculates the comprehensive score of each document vector by ranking the weighted document vector in the recall results of different questions, and then comprehensively sorts the document vectors in the recall results of all questions according to the comprehensive score of each document vector, removes duplicate document vectors, ensures that each document appears only once, and filters out document vectors with scores lower than the set similarity threshold to improve the quality of the results, and finally merges the recall results of different questions; finally, the summary feedback module calls the language model, generates detailed and coherent response content for the final recall result and returns it to the interaction module for users to view.

Images