Network search enhanced question answering method, system, device, medium and program product

Abstract

The present application discloses a network search enhanced question-answering method, system, device, medium and program product, which belongs to the field of language model retrieval technology. The method includes: responding to a question request initiated by a target terminal, determining a first prompt word according to the question statement included in the question request and the environmental information corresponding to the question request; obtaining a question search statement by inputting the first prompt word and a predefined network search function into a pre-trained language model; determining search results according to the search engine corresponding to the question search statement and the network search function; determining a second prompt word according to the search results, and obtaining the answer content corresponding to the question statement by inputting the second prompt word into the pre-trained language model. In this way, useful information in the web page content in the search engine can be obtained, and the answer quality of the pre-trained language model can be improved.

Description

Technical Field

[0001] The present application relates to the field of language model retrieval technology, and in particular to a network retrieval enhanced question and answer method, system, device, medium and program product. Background Art

[0002] The training data of Large Language Model (LLM) is delayed, which may cause the model to generate inaccurate information when answering questions in certain unknown fields. Therefore, the combination of large language model and search engine can improve the quality of content generation.

[0003] Currently, the main ways to provide additional information to large language models are fine-tuning and retrieval-augmented generation (RAG). For time-sensitive questions, RAG technology is usually used. However, due to factors such as the messy layout of web page content and the high amount of noise information, the proportion of useful content on the web page is low, and it is difficult to clean irrelevant content, which ultimately makes it impossible for LLM to provide enhanced answers based on the content of these web pages. Summary of the invention

[0004] The embodiments of the present application provide a network retrieval enhanced question and answer method, system, device, medium and program product to at least solve the problem that the existing retrieval enhancement generation technology is difficult to obtain useful information in the web page content and the answer quality of the large language model is not ideal.

[0005] In order to solve the above technical problems, this application is implemented as follows: In a first aspect, an embodiment of the present application provides a network retrieval enhanced question and answer method, comprising: responding to a question request initiated by a target terminal, determining a first prompt word based on a question statement included in the question request and environmental information corresponding to the question request; obtaining a question search statement by inputting the first prompt word and a predefined network search function into a pre-trained language model; determining search results based on a search engine corresponding to the question search statement and the network search function; determining a second prompt word based on the search results, and obtaining a reply content corresponding to the question statement by inputting the second prompt word into the pre-trained language model.

[0006] In a second aspect, an embodiment of the present application provides a network retrieval enhanced question and answer method, comprising: obtaining a question search statement, wherein the question search statement is obtained by inputting a first prompt word and a pre-defined network search function into a pre-trained language model, wherein the first prompt word is determined according to the question statement included in a question request initiated by a target terminal and environmental information corresponding to the question request; determining at least one web page content according to a search engine corresponding to the question search statement and the network search function; determining a target query condition according to the at least one web page content, and retrieving the at least one web page content according to the target query condition to obtain a search result; wherein the search result is used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, obtaining a reply content corresponding to the question statement generated by the pre-trained language model according to the second prompt word.

[0007] In a third aspect, an embodiment of the present application provides a network retrieval enhanced question-answering method, comprising: obtaining at least one web page content; wherein the at least one web page content is determined according to a search engine corresponding to a question search statement and a network search function, the question search statement is obtained by inputting a first prompt word and a pre-defined network search function into a pre-trained language model, and the first prompt word is determined according to the question statement included in a question request initiated by a target terminal and environmental information corresponding to the question request; vectorizing the at least one web page content to obtain a plurality of text vectors; in response to the obtained target query condition, vectorizing the target query condition to obtain a query vector; determining a search result according to the similarity between the query vector and the plurality of text vectors, wherein the search result is used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, obtaining a reply content corresponding to the question statement generated by the pre-trained language model according to the second prompt word.

[0008] In a fourth aspect, an embodiment of the present application provides a language model calling module, including: a prompt word determination unit, used to respond to a question request initiated by a target terminal, and determine a first prompt word according to a question sentence included in the question request and environmental information corresponding to the question request; a first sentence acquisition unit, used to obtain a question search sentence by inputting the first prompt word and a predefined network search function into a pre-trained language model; a search result acquisition unit, used to determine a search result according to the question search sentence and a search engine corresponding to the network search function; a reply content acquisition unit, used to determine a second prompt word according to the search result, and obtain a reply content corresponding to the question sentence by inputting the second prompt word into the pre-trained language model.

[0009] In a fifth aspect, an embodiment of the present application provides an online search module, comprising: a second sentence acquisition unit, used to acquire a question search sentence, wherein the question search sentence is obtained by inputting a first prompt word and a pre-defined online search function into a pre-trained language model, wherein the first prompt word is determined based on the question sentence included in the question request initiated by the target terminal and the environmental information corresponding to the question request; a web page content determination unit, used to determine at least one web page content based on the question search sentence and the search engine corresponding to the online search function; a web page content retrieval unit, used to retrieve the at least one web page content according to the target query condition to obtain a search result; wherein the search result is used to determine a second prompt word, by inputting the second prompt word into the pre-trained language model to obtain a reply content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

[0010] In a sixth aspect, an embodiment of the present application provides a vector storage and retrieval module, comprising: a web page content acquisition unit, configured to acquire at least one web page content; wherein the at least one web page content is determined according to a search engine corresponding to a question search statement and an online search function, wherein the question search statement is obtained by inputting a first prompt word and a predefined online search function into a pre-trained language model, wherein the first prompt word is determined according to the question statement included in a question request initiated by a target terminal and environmental information corresponding to the question request; a vectorization processing unit, configured to vectorize the at least one web page content to obtain a plurality of text vectors; a query vector acquisition unit, configured to respond to the acquired target query condition, vectorize the target query condition, and obtain a query vector; a search result determination unit, configured to determine a search result according to the similarity between the query vector and the plurality of text vectors, wherein the search result is used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, a reply content corresponding to the question statement generated by the pre-trained language model according to the second prompt word is obtained.

[0011] In the seventh aspect, an embodiment of the present application provides a network retrieval enhanced question and answer system, comprising a language model calling module as described in the fourth aspect, a networked search module as described in the fifth aspect, and a vector storage retrieval module as described in the sixth aspect; the language model calling module and the networked search module are communicatively connected to each other, and the networked search module and the vector storage retrieval module are communicatively connected to each other.

[0012] In an eighth aspect, an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the programs or instructions are executed by the processor, the steps of the method described in the first aspect, the second aspect, or the third aspect above are implemented.

[0013] In the ninth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the steps of the method described in the first aspect, the second aspect, or the third aspect are implemented.

[0014] In the tenth aspect, an embodiment of the present application provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer executes the steps of the method described in the first aspect, the second aspect, or the third aspect above.

[0015] In an embodiment of the present application, in response to a question request initiated by a target terminal, a first prompt word is determined according to the question statement included in the question request and the environmental information corresponding to the question request; the question search statement is obtained by inputting the first prompt word and a predefined network search function into a pre-trained language model; the search result is determined according to the question search statement and the search engine corresponding to the network search function; the second prompt word is determined according to the search result, and the reply content corresponding to the question statement is obtained by inputting the second prompt word into the pre-trained language model. In this way, by determining the first prompt word through the question statement and the environmental information in the question request, the generated query statement can be more accurately matched with the user's intention, the first prompt word is optimized by the pre-trained language model to obtain the question search statement, and the search engine corresponding to the network search function is used for searching, so that useful information in the web page content in the search engine can be obtained to obtain the search result, and then the pre-trained language model is guided to generate a more accurate answer according to the search result, thereby improving the answer quality of the pre-trained language model.

[0016] It should be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0018] Figure 1 A schematic diagram of a flow chart of a network search enhanced question-answering method provided in an embodiment of the present application is shown; Figure 2 A schematic diagram of the execution logic of the network retrieval enhanced question-answering method provided in an embodiment of the present application is shown; Figure 3 Another schematic diagram of the process of the network search enhanced question answering method provided by the embodiment of the present application is shown; Figure 4An example diagram of the structure of a search result page provided by an embodiment of the present application is shown; Figure 5 Another schematic diagram of a flow chart of a network search enhanced question-answering method provided in an embodiment of the present application is shown; Figure 6 A schematic diagram of the execution logic of web page storage provided by an embodiment of the present application is shown; Figure 7 A schematic diagram of the execution logic of web page content retrieval provided by an embodiment of the present application is shown; Figure 8 A schematic diagram of the structure of a language model calling module provided in an embodiment of the present application is shown; Fig. 9 A schematic diagram of the structure of the network search module provided in an embodiment of the present application is shown; Fig.10 A schematic diagram of the structure of a vector storage and retrieval module provided in an embodiment of the present application is shown; Fig.11 An interactive flow chart of the network search enhanced question-answering system provided by an embodiment of the present application is shown; Fig.12 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present application. Instead, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0020] At present, the way to implement online question answering with a large language model can be roughly divided into the following specific steps: (1) Search extraction: Input the user's question into the search application programming interface (API) to obtain a list of Uniform Resource Locators (URLs) of pages related to the question. The search API can be an API opened by the search engine manufacturer or a crawler framework implemented by the developer. Select several URLs with high relevance to crawl the corresponding Hypertext Markup Language (HTML) content. Next, based on the logic of HTML to text conversion, extract the text content from the HTML page; (2) Segmentation and embedding: The webpage text content obtained in the previous step is segmented according to certain rules (for example, by word count or paragraph) to obtain multiple text blocks. Then, the Embedding model is used to encode each text block into an embedded vector and save it into the vector database together with its original text. (3) Content retrieval: Use the Embedding model to vectorize the user's question and use it as the query condition of the vector database to retrieve the document blocks that match the user's semantics. (4) Enhanced generation: The question input by the user and the document block obtained in the previous step are combined and sent to the large language model, so that the large language model has the additional knowledge obtained from the web page and generates the final response.

[0021] In traditional open question answering systems, the system usually only retrieves information from reliable sources (such as Wikipedia or internal company documents). The quality of this information is often very high, without much noise interference. However, it is challenging to build a web-enhanced question answering system based on a large language model. For some queries, the network information obtained is rich but noisy, and it is difficult to clean and purify wild web pages. This directly makes the knowledge content obtained in the "content retrieval" step irrelevant to the question, or even completely irrelevant, resulting in the inability of the large language model to answer the question correctly.

[0022] To address the above issues, relevant researchers proposed a series of designs and strategies in the open source project WebGLM, including the following three parts: (1) LLM-enhanced retriever: WebGLM is designed with two stages: coarse-grained web search and fine-grained LLM-enhanced dense retrieval to find relevant citations based on the query.

[0023] (2) Bootstrapped Generator: WebGLM derives WebGLM-QA through contextual learning and corresponding strategies, which is an LLM-bootstrapped reference and long-form question-answering dataset to clean and refine web content. It includes 45k high-quality samples after filtering and 83k noisy but diverse samples before filtering. The core of the WebGLM system is a GLM model trained on this dataset.

[0024] (3) Human preference-aware scorer: WebGLM learns the majority preferences of humans from the likes in online question-and-answer forums and trains a human preference-aware scorer for the best choice.

[0025] Although the emergence of WebGLM points out a direction for LLM-based web page enhanced question-answering systems, its LLM enhanced retriever will use the first few most relevant pages as reference sources, let the bootstrap generator generate multiple answers, and finally the scorer selects the one that is most likely to meet human preferences as the final output. However, its implementation is very complicated and requires a lot of underlying mathematical theoretical knowledge, and it is difficult for general development teams to replicate its case.

[0026] In response to the problems existing in the above-mentioned network retrieval enhanced question and answer process, an embodiment of the present application provides a network retrieval enhanced question and answer method, which determines a first prompt word through a question statement and environmental information in a question request so that the generated query statement can more accurately match the user's intention, optimizes the first prompt word through a pre-trained language model to obtain a question search statement, uses a search engine corresponding to a networked search function to search, obtains useful information in the web page content in the search engine, obtains search results, and then guides the pre-trained language model to generate a more accurate answer based on the search results.

[0027] Figure 1 A flow chart of the network retrieval enhanced question-and-answer method provided in an embodiment of the present application is shown. The execution subject of the method may be a terminal device or a server, wherein the terminal device may be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device may be a terminal device used by a user. The server may be an independent server, or a server cluster composed of multiple servers, and the server may be a background server of a certain business, or a background server of a certain platform or application (for example, an intelligent question-and-answer system, a voice assistant, a medical health consultation platform, an online learning platform, etc.). In the embodiment of the present application, the execution subject is taken as an example to illustrate. For the case of a terminal device, it can be processed according to the following relevant content, which will not be repeated here. As shown in the figure, the method 100 may include the following steps: Step 101: responding to a question request initiated by a target terminal, and determining a first prompt word according to a question statement included in the question request and environmental information corresponding to the question request.

[0028] In a specific implementation, the user initiates a question request through the target terminal, and the question request includes a question statement. For example, the user's question is "How is the weather today?" The server determines the user's geographical location information based on the question statement and the environmental information corresponding to the question request. The environmental information can be the time information of the current system or the Internet Protocol (IP) address corresponding to the question request. The first prompt word Prompt is determined based on the current system's time information, geographical location information, and question statement. For example, the first prompt word Prompt is finally determined as: "Today is 2024-05-13, Monday, location: Guangzhou, please answer the user's question, the user's question is: How is the weather today?".

[0029] In this way, some imperceptible real-time information can be provided to the pre-trained language model, such as time information or the user's geographic location information, so that the pre-trained language model can generate response content that is more in line with the user's intention.

[0030] Step 102: Obtain a question search statement by inputting the first prompt word and a predefined online search function into a pre-trained language model.

[0031] In a specific implementation, a search engine or an online search function for information query may be predefined, and the first prompt word and the online search function obtained in the above step 101 may be input into a pre-trained language model, such as an LLM, to obtain a question search statement for the search engine corresponding to the online search function generated by the pre-trained language model according to the first prompt word. For example, the first prompt word Prompt is: "Today is 2024-05-13, Monday, location: Guangzhou, please answer the user's question, the user's question is: How is the weather today?", the online search function is "Questions that need to be searched using a search engine, please extract and optimize from the user's question statement to obtain better search results, for real-time questions, please add time", and the obtained question search statement is "Weather forecast for Guangzhou in the next few days". For another example, for example, the question statement is "Please recommend a new router for me this year, requiring strong performance and touching price", and the Prompt and the online search function are sent to the LLM together to obtain an optimized question, which is used as the query condition of the search engine corresponding to the online search function. The question optimized by the LLM may be "Recommendation of cost-effective routers in 2024". The optimized question simplifies the description of the question and adds the current time, which will obtain better search results than using the question statement as the search condition.

[0032] Step 103: Determine search results based on the question search statement and the search engine corresponding to the network search function.

[0033] In a specific implementation, the question search statement obtained in the above step 102 can be input into a search engine corresponding to the network search function, and the search results can be obtained through the search engine.

[0034] Step 104: Determine a second prompt word according to the search result, and obtain a response content corresponding to the question statement by inputting the second prompt word into the pre-trained language model.

[0035] In a specific implementation, the first prompt word and the network search function are input into the pre-trained language model, and after the pre-trained language model generates a question search sentence, the corresponding search result is searched according to the question search sentence. For example, the search result is "Guangzhou's weather forecast for the past 15 days". The second prompt word is determined according to the search result, and the second prompt word is input into the pre-trained language model to generate a reply content corresponding to the question sentence through the pre-trained language model, for example: "Today's weather is sunny, and the temperature is between 10°C and 18°C. The temperature is expected to gradually rise in the next few days, and there will be light rain on Thursday and Friday." In an exemplary embodiment, if Figure 2 As shown in , the above-mentioned network retrieval enhanced question answering method includes: Step 201: responding to a question request initiated by a target terminal, and obtaining a question statement included in the question request; Step 202: Obtaining environmental information corresponding to the question request, and determining a first prompt word according to the question statement and the environmental information, for example, obtaining information such as system time and user geographic location to generate a prompt; Step 203: define a network search function; Step 204: input the first prompt word and the network search function into the large language model LLM; Step 205: obtaining a question search statement for a search engine corresponding to the online search function generated by the LLM according to the first prompt word; Step 206: Perform an online query based on the question search statement to obtain search results. For example, through online query, obtain the first N web page contents, search for relevant texts in the web pages, obtain web page text contents with high relevance, and use the web page text contents as search results; Step 207: Determine a second prompt word according to the search results, input the second prompt word into the LLM, and obtain the answer content corresponding to the question statement.

[0036] In this way, by gradually refining the query, users can get more accurate responses, thereby improving the answer quality of the pre-trained language model.

[0037] In a possible implementation, the environmental information includes time information and geographic location information; in the above step 101, determining the first prompt word according to the question statement included in the question request and the environmental information corresponding to the question request includes: The question statement, the time information and the geographical location information included in the question request are filled into a predefined prompt word template to obtain a first prompt word.

[0038] In an embodiment of the present application, the question statement: "How is the weather today?", time information: "2024-05-13", and geographic location information "Guangzhou" can be filled into a predefined prompt word template to obtain the first prompt word: "Today is 2024-05-13, Monday, location: Guangzhou, please answer the user's question, the user's question is: How is the weather today?".

[0039] In this way, by providing structured input to the pre-trained model, it can help the pre-trained language model better understand the user's query requirements, thereby making subsequent answers more accurate.

[0040] In a possible implementation, in the above step 102, the question search statement is obtained by inputting the first prompt word and the predefined online search function into the pre-trained language model, including: Determine an online search function according to a search engine name, a search question, and a parameter description of the search question; input the first prompt word and the online search function into a pre-trained language model so that the pre-trained language model optimizes the first prompt word based on the online search function to obtain a question search statement; and obtain the question search statement generated by the pre-trained language model.

[0041] In an embodiment of the present application, an online search function Function that can be called by a pre-trained language model is pre-defined. Here, the Function that needs to be defined is an online query information tool. First, define a function "XXX_search" that calls a search engine to obtain the results of a search question. This function has only one parameter, which is the search question "question". The parameter description can be: "Questions that need to be searched using a search engine, please extract and optimize from the user's questions to obtain better search results. For real-time questions, please add time." The pre-trained language model decides whether to call this Function based on the search engine name, parameter description, and search question entered by the user.

[0042] For example, according to the specification of Function Call, the specific definition of the network search function can be as follows: { "name": "XXX _search", "description": "Call the search engine to get the results of the search question", "parameters": { "type": "object", "properties": { "question": { "type": "string", "description": "Questions that need to be searched using search engines, please extract and optimize from user questions to get better search results. For real-time questions, please add time." } }, "required": ["question"] } } It should be noted that the description of the search question here is extremely important. The networked search function is used to guide the pre-trained language model to extract and optimize the question statement, and obtain the question search statement for the search engine, so that the pre-trained language model can search for more accurate and relevant search results through the search engine.

[0043] In this way, by defining the online search function Function, the description of what the pre-trained language model needs to do is added to the parameter, such as the description of the question parameter: "Questions that need to be searched using a search engine, please extract and optimize from the user's questions to obtain better search results. For real-time questions, please add time", so that the pre-trained language model can optimize the question statement requested by the user to reduce the retrieval of invalid content and obtain better search conditions for the search engine. At the same time, it also avoids calling the pre-trained language model twice, requiring it to optimize the question, reducing the number of calls to the pre-trained language model, and improving the overall response time.

[0044] In a possible implementation, after obtaining the answer content corresponding to the question statement in the above step 104, the following is further included: Sending a question response to the target terminal, wherein the question response includes the reply content.

[0045] In an embodiment of the present application, after obtaining the reply content generated by the pre-trained language model, a question response is sent to the target terminal, and the question response includes the reply content, so that the target terminal displays it to the user in a preset manner (such as text, voice, etc.).

[0046] Figure 3 Another flow chart of the network retrieval enhanced question-and-answer method provided in an embodiment of the present application is shown. The execution subject of the method may be a terminal device or a server, wherein the terminal device may be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device may be a terminal device used by a user. The server may be an independent server, or a server cluster composed of multiple servers, and the server may be a background server of a certain business, or a background server of a certain platform or application (for example, an intelligent question-and-answer system, a voice assistant, a medical health consultation platform, an online learning platform, etc.). In the embodiment of the present application, the execution subject is taken as an example to illustrate. For the case of a terminal device, it can be processed according to the following relevant content, which will not be repeated here. As shown in the figure, the method 300 may include the following steps: Step 301: Obtain a question search statement, where the question search statement is obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model, where the first prompt word is determined based on the question statement included in a question request initiated by a target terminal and environmental information corresponding to the question request.

[0047] In a specific implementation, a question search statement is obtained, and the question search statement can be obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model as in the above method 100, and the first prompt word is determined according to the question statement included in the question request initiated by the target terminal and the environmental information corresponding to the question request. For example, the obtained question search statement is "Guangzhou's weather forecast for the past 15 days".

[0048] Step 302: Determine at least one web page content according to the question search statement and the search engine corresponding to the network search function.

[0049] In a specific implementation, a question search statement is input into a search box of a search engine corresponding to an online search function, and at least one webpage content is determined by the search engine, such as Figure 4 As shown in , web page content 401 and web page content 402 are searched through a search engine.

[0050] Step 303: Determine a target query condition based on the at least one web page content, and retrieve the at least one web page content based on the target query condition to obtain search results; wherein the search results are used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, obtain the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

[0051] The target query condition may be obtained by concatenating the summaries corresponding to at least one webpage content, or may be determined according to a question search statement.

[0052] In a specific implementation, the webpage content 401 and the webpage content 402 obtained in the above step 302 are retrieved by the target query condition to obtain search results. A second prompt word is determined according to the search results, and the second prompt word is input into the pre-trained language model to obtain the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

[0053] In this way, searching through target query conditions instead of using question search statements is because question search statements are relatively short and there is more noise data in the web page content. The text recalled by searching through question search statements is likely to contain more irrelevant content. Searching through target query conditions is beneficial to improving the quality of the recalled text.

[0054] In a possible implementation, in the above step 302, determining at least one webpage content according to the search engine corresponding to the question search statement and the network search function includes: The question search statement is input into the search engine corresponding to the networked search function, and a search result page is obtained through the search engine; page content information is obtained from the search result page according to a preset category name; and at least one web page content is determined from multiple web page contents on the search result page according to the page content information.

[0055] In the embodiment of the present application, by inputting the question search statement into the search engine corresponding to the network search function, the search result page is obtained through the search engine. In the search result page, the CSS selector can be used to locate the HTML tag with a preset class name on the page, for example, the CSS class "result c-container xpath-log new-pmd", so that the web page information such as the hyperlink, introduction, and title of the page can be obtained, and the related content such as advertising recommendations can be avoided to improve the relevance of the web page crawling. Then select the preset number of web page contents ranked at the top to open and obtain the HTML content of the page.

[0056] The step of inputting the problem search statement into the search engine corresponding to the network search function includes: According to the website address of the search engine corresponding to the networked search function, the search page of the search engine is accessed; the position information of the question input box in the search page is determined; according to the position information, the question search statement is input into the search engine.

[0057] In the embodiment of the present application, since most search engines do not have open search engine related interfaces, you need to write your own crawler to crawl web page content related to user questions. However, search engines usually have a powerful anti-crawler mechanism. The commonly used crawler frameworks such as Jsoup and Scrapy to crawl web page information will be intercepted and return a secure authentication page. Therefore, Playwright can be selected. Playwright is an open source automated testing framework that can simulate user interactions with browsers, such as clicks, input, navigation, etc. It supports all modern rendering engines, including Chromium, WebKit and Firefox. Specifically: according to the search engine URL https: / / www.XXX.com corresponding to the network search function, visit the search page of the search engine, use Playwright to locate the position of the question input box in the search engine, and according to the position information of the question input box, enter the question search statement into the search engine, and then click the "Search" button to imitate people's daily habits of using search engines to avoid being recognized by the anti-crawler mechanism.

[0058] In a possible implementation, in the above step 303, determining the target query condition according to the at least one webpage content includes: Obtain a content brief corresponding to the at least one webpage content; and perform splicing processing on the content brief to obtain a target query condition.

[0059] In this way, since the content summary of the web page content includes an introduction to the core content of the page, and the content summary is highly relevant to the user's question, by splicing the content summaries of the crawled web page content as target query conditions, the web page content related to the user's question can be obtained, which greatly improves the quality of the recalled text.

[0060] Figure 5 Another flow chart of the network retrieval enhanced question-and-answer method provided in an embodiment of the present application is shown. The execution subject of the method may be a terminal device or a server, wherein the terminal device may be a device such as a personal computer, or a mobile terminal device such as a mobile phone or a tablet computer, and the terminal device may be a terminal device used by a user. The server may be an independent server, or a server cluster composed of multiple servers, and the server may be a background server of a certain business, or a background server of a certain platform or application (for example, an intelligent question-and-answer system, a voice assistant, a medical health consultation platform, an online learning platform, etc.). In the embodiment of the present application, the execution subject is taken as an example to illustrate. For the case of a terminal device, it can be processed according to the following relevant content, which will not be repeated here. As shown in the figure, the method 500 may include the following steps: Step 501: Obtain at least one webpage content.

[0061] Among them, the at least one web page content is determined according to a search engine corresponding to a question search statement and an online search function, the question search statement is obtained by inputting a first prompt word and a pre-defined online search function into a pre-trained language model, and the first prompt word is determined according to the question statement included in the question request initiated by the target terminal and the environmental information corresponding to the question request.

[0062] In a specific implementation, after obtaining the HTML content of the page, the HTML can be converted into plain text, and the HMTL tag can be removed to obtain at least one web page content.

[0063] Step 502: vectorize the at least one web page content to obtain a plurality of text vectors.

[0064] In a specific implementation, at least one web page content may be preprocessed, such as segmented into blocks to obtain a plurality of text blocks, and the plurality of text blocks may be vectorized using an Embedding model to obtain a plurality of text vectors.

[0065] Step 503: In response to the acquired target query condition, the target query condition is vectorized to obtain a query vector.

[0066] The target query condition may be obtained by concatenating the summaries corresponding to at least one webpage content, or may be determined according to a question search statement.

[0067] In a specific implementation, when the target query condition is obtained, the same Embedding model can be used to vectorize the target query condition to obtain a query vector.

[0068] Step 504: Determine search results based on the similarity between the query vector and the multiple text vectors, wherein the search results are used to determine a second prompt word, and obtain a response content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word by inputting the second prompt word into the pre-trained language model.

[0069] In a specific implementation, the search results may be determined based on the similarity between the query vector in step 503 and the multiple text vectors in step 502, for example, the text vectors with similarity higher than a preset similarity threshold are determined as the search results. The second prompt word is determined based on the search results, and the answer content corresponding to the question sentence generated by the pre-trained language model based on the second prompt word is obtained by inputting the second prompt word into the pre-trained language model.

[0070] In a possible implementation, in the above step 502, the at least one webpage content is vectorized to obtain a plurality of text vectors, including: Preprocessing the at least one web page content to obtain text content; dividing the text content into a plurality of text blocks according to a preset text segmentation rule; and vectorizing the plurality of text blocks to obtain a plurality of text vectors.

[0071] In the present application embodiment, Figure 6 As shown, after obtaining the web page, the HTML can be converted into plain text, and the HMTL tag can be removed to obtain at least one web page content. Then, the at least one web page content is segmented. For example, according to the word count segmentation rule, 300 words can be divided into a paragraph to obtain multiple text blocks. The multiple text blocks are vectorized using the Embedding model to obtain multiple text vectors. Then, the text vectors and the corresponding original text are stored together in a preset vector database.

[0072] In a possible implementation, in the above step 504, determining the search results according to the similarity between the query vector and the multiple text vectors includes: The similarity between the query vector and the multiple text vectors is obtained; a target text vector whose similarity meets a preset similarity threshold is determined among the multiple text vectors; and a text block corresponding to the target text vector is determined as a search result.

[0073] In a specific implementation, the similarity between the query vector and multiple text vectors in the vector database can be calculated to determine a target text vector whose similarity meets a preset similarity threshold among the multiple text vectors, wherein the similarity threshold can be set according to actual needs, and the multiple text vectors can be sorted according to the similarity, and the similarity of a preset number of text vectors with the top rankings is determined as the similarity threshold, and the text block corresponding to the target text vector is determined as the search result.

[0074] In an exemplary embodiment, if Figure 7 As shown, the web page introduction obtains a query vector through the Embedding model, and the query vector is used to query the vector database to obtain the relevant text block. The question statement and the relevant text block are input into a pre-trained language model, such as LLM, to obtain the answer content corresponding to the question statement.

[0075] Figure 8 The structure diagram of the language model calling module provided in the embodiment of the present application is shown. The language model calling module can implement the following Figure 1 In the embodiment shown, all or part of the content, the language model calling module 800 includes: The prompt word determination unit 810 is used to respond to the question request initiated by the target terminal and determine the first prompt word according to the question statement included in the question request and the environment information corresponding to the question request; A first sentence acquisition unit 820 is used to acquire a question search sentence by inputting the first prompt word and a predefined online search function into a pre-trained language model; A search result acquisition unit 830, configured to determine search results according to the question search statement and the search engine corresponding to the network search function; The reply content acquisition unit 840 is used to determine a second prompt word according to the search result, and obtain the reply content corresponding to the question sentence by inputting the second prompt word into the pre-trained language model.

[0076] In a possible implementation, the environmental information includes time information and geographic location information; The prompt word determining unit 810, when used to determine the first prompt word according to the question statement included in the question request and the environment information corresponding to the question request, is specifically used to: The question statement, the time information and the geographical location information included in the question request are filled into a predefined prompt word template to obtain a first prompt word.

[0077] In a possible implementation, the answer content acquisition unit 840, when used to obtain the question search statement by inputting the first prompt word and the predefined network search function into the pre-trained language model, is specifically used to: Determine an online search function according to a search engine name, a search question, and a parameter description of the search question; Inputting the first prompt word and the online search function into a pre-trained language model, so that the pre-trained language model optimizes the first prompt word based on the online search function to obtain a question search statement; Obtain the question search sentence generated by the pre-trained language model.

[0078] In a possible implementation, the language model calling module 800 further includes: The sending unit is used to send a question response to the target terminal, wherein the question response includes the reply content.

[0079] The embodiment of the present application provides a language model calling module, including a prompt word determination unit, a first sentence acquisition unit, a search result acquisition unit and a reply content acquisition unit; the prompt word determination unit responds to the question request initiated by the target terminal, and determines the first prompt word according to the question sentence included in the question request and the environmental information corresponding to the question request; the first sentence acquisition unit obtains the question search sentence by inputting the first prompt word and a predefined network search function into a pre-trained language model; the search result acquisition unit determines the search result according to the search question sentence and the search engine corresponding to the network search function; the reply content acquisition unit determines the second prompt word according to the search result, and obtains the reply content corresponding to the question sentence by inputting the second prompt word into the pre-trained language model. In this way, useful information in the web page content in the search engine can be obtained, and the answer quality of the pre-trained language model can be improved.

[0080] Fig. 9 The schematic diagram of the structure of the network search module provided in the embodiment of the present application is shown. The network search module can implement the following Figure 3 In the embodiment shown, all or part of the contents, the network search module 900 includes: A second sentence acquisition unit 910 is used to acquire a question search sentence, wherein the question search sentence is obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model, wherein the first prompt word is determined according to the question sentence included in the question request initiated by the target terminal and the environment information corresponding to the question request; A web page content determination unit 920, configured to determine at least one web page content according to the question search statement and the search engine corresponding to the network search function; The web page content retrieval unit 930 is used to determine a target query condition based on the at least one web page content, and retrieve the at least one web page content based on the target query condition to obtain search results; wherein the search results are used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, obtain the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

[0081] In a possible implementation, the webpage content determination unit 920, when used to determine at least one webpage content according to the question search statement and the search engine corresponding to the networked search function, is specifically used to: Inputting the question search statement into the search engine corresponding to the network search function, and obtaining a search result page through the search engine; Obtaining page content information from the search result page according to a preset category name; At least one web page content is determined from a plurality of web page contents on the search result page according to the page content information.

[0082] In a possible implementation, the webpage content determination unit 920, when used to input the question search statement into the search engine corresponding to the network search function, is specifically used to: Accessing a search page of the search engine according to the URL of the search engine corresponding to the network search function; Determine the location information of the question input box in the search page; The question search sentence is input into the search engine according to the position information.

[0083] In a possible implementation, the webpage content retrieval unit 930, when used to determine the target query condition according to the at least one webpage content, is specifically used to: Obtaining a content introduction corresponding to the at least one webpage content; The content brief introductions are spliced ​​together to obtain target query conditions.

[0084] The embodiment of the present application provides a network search module, including a second sentence acquisition unit, a web page content determination unit and a web page content retrieval unit; the second sentence acquisition unit acquires a question search sentence, the question search sentence is obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model, the first prompt word is determined according to the question sentence included in the question request initiated by the target terminal and the environmental information corresponding to the question request; the web page content determination unit determines at least one web page content according to the search engine corresponding to the question search sentence and the network search function; the web page content retrieval unit determines a target query condition according to the at least one web page content, and retrieves the at least one web page content according to the target query condition to obtain a search result; wherein the search result is used to determine the second prompt word, and by inputting the second prompt word into the pre-trained language model, the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word is obtained. In this way, useful information in the web page content in the search engine can be obtained, and the answer quality of the pre-trained language model can be improved.

[0085] Fig.10 The schematic diagram of the structure of the vector storage and retrieval module provided in the embodiment of the present application is shown. The vector storage and retrieval module can be implemented as follows: Figure 5 In the embodiment shown, all or part of the content, the vector storage retrieval module 1000 includes: The webpage content acquisition unit 1010 is used to acquire at least one webpage content; wherein the at least one webpage content is determined according to a search engine corresponding to a question search statement and an online search function, the question search statement is obtained by inputting a first prompt word and a predefined online search function into a pre-trained language model, and the first prompt word is determined according to a question statement included in a question request initiated by a target terminal and environment information corresponding to the question request; A vectorization processing unit 1020, configured to perform vectorization processing on the at least one webpage content to obtain a plurality of text vectors; A query vector acquisition unit 1030 is used to vectorize the target query condition in response to the acquired target query condition to obtain a query vector; The search result determination unit 1040 is used to determine the search result according to the similarity between the query vector and the multiple text vectors, wherein the search result is used to determine the second prompt word, and the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word is obtained by inputting the second prompt word into the pre-trained language model.

[0086] In a possible implementation, the vectorization processing unit 1020, when used to vectorize the at least one webpage content to obtain a plurality of text vectors, is specifically used to: Preprocessing the at least one webpage content to obtain text content; According to a preset text segmentation rule, the text content is divided into multiple text blocks; Vectorization is performed on the multiple text blocks to obtain multiple text vectors.

[0087] In a possible implementation, the search result determining unit 1040, when used to determine the search result according to the similarity between the query vector and the multiple text vectors, is specifically used to: Obtaining similarities between the query vector and the multiple text vectors; Determine a target text vector whose similarity among the multiple text vectors meets a preset similarity threshold; The text block corresponding to the target text vector is determined as the search result.

[0088] The embodiment of the present application provides a vector storage and retrieval module, including a web page content acquisition unit, a vectorization processing unit, a query vector acquisition unit and a search result determination unit; the web page content acquisition unit acquires at least one web page content; wherein the at least one web page content is determined according to a search engine corresponding to a question search statement and an online search function, wherein the question search statement is obtained by inputting a first prompt word and a predefined online search function into a pre-trained language model, wherein the first prompt word is determined according to the question statement included in the question request initiated by the target terminal and the environmental information corresponding to the question request; the vectorization processing unit vectorizes the at least one web page content to obtain a plurality of text vectors; the query vector acquisition unit responds to the acquired target query condition and vectorizes the target query condition to obtain a query vector; the search result determination unit determines the search result according to the similarity between the query vector and the plurality of text vectors, wherein the search result is used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, the answer content corresponding to the question statement generated by the pre-trained language model according to the second prompt word is obtained. In this way, useful information in the web page content in the search engine can be obtained, and the answer quality of the pre-trained language model can be improved.

[0089] The present application also provides a network search enhanced question answering system, including: Figure 8 The language model calling module shown in Fig. 9 The network search module shown and Fig.10 The vector storage and retrieval module shown; the language model calling module and the network search module are connected to each other for communication, and the network search module and the vector storage and retrieval module are connected to each other for communication.

[0090] In an exemplary embodiment, if Fig.11 As shown, the network retrieval enhanced question answering method of the network retrieval enhanced question answering system includes the following steps: Step 1101: a question request initiated by a target terminal; Step 1102: the language model calling module 800 generates a first prompt word Prompt according to the question sentence included in the question request and the environment information corresponding to the question request; Step 1103: the language model calling module 800 defines an online search function; Step 1104: the language model calling module 800 initiates a request to the pre-trained language module, inputs the first prompt word and the pre-defined online search function into the pre-trained language model, and obtains a question search sentence for a search engine corresponding to the online search function generated by the pre-trained language model according to the first prompt word; Step 1105: If there is no need to call the search engine, the language model calling module 800 returns the content generated by the pre-trained language model to the target terminal; Step 1106: the language model calling module 800 calls the search engine and sends the question search statement to the network search module 900; Step 1107: the network search module 900 obtains the first N webpage contents (i.e. the above-mentioned search results); Step 1108: the network search module 900 saves the webpage content to the vector storage and retrieval module 1000; Step 1109: the vector storage and retrieval module 1000 processes the webpage content into text form and saves it in segments into the vector database; Step 1110: the network search module 900 uses the target query condition, such as the page introduction, to retrieve the content, and obtains the search results returned by the vector storage retrieval module 1000, such as the first N related contents; Step 1111: the network search module 900 returns the first N relevant contents to the network search module 900; Step 1112: the online search module 900 generates a second prompt word according to the N related contents, initiates a request to the pre-trained language model, the pre-trained language model generates a result using the webpage content, and returns the request result to the online search module 900; Step 1113: The network search module 900 returns the content generated by the pre-trained language model to the target terminal.

[0091] Through the above method, the problem of too much noise in web page data, resulting in too much invalid information in text retrieval results, can be effectively solved. This method does not need to clean and purify the web page data to improve the quality of the content. It is simple to implement, does not require developers to have the necessary underlying theoretical knowledge, and has better results.

[0092] Fig.12 The hardware structure diagram of the electronic device provided by the embodiment of the present application is shown. Referring to the figure, at the hardware level, the electronic device 1200 includes a processor 1210, and optionally, includes an internal bus 1220, a network interface 1230, and a memory 1240. Among them, the memory 1240 may include a memory 1241, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory 1242 (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device 1200 may also include hardware required for other services.

[0093] The processor 1210, the network interface 1230 and the memory may be interconnected via an internal bus 1220, which may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one bidirectional arrow is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0094] The memory 1240 stores programs. Specifically, the programs may include program codes, and the program codes include computer operation instructions. The memory 1240 may include a memory 1241 and a non-volatile memory 1242 , and provide instructions and data to the processor 1210 .

[0095] The processor 1210 reads the corresponding computer program from the non-volatile memory 1242 into the memory and then runs it, forming a device for locating the target user at the logical level. The processor 1210 executes the program stored in the memory and specifically performs: Figure 1 or Figure 3 or Figure 5 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0096] The above application Figure 1 or Figure 3 or Figure 5The method disclosed in the illustrated embodiment can be applied to the processor 1210, or implemented by the processor 1210. The processor 1210 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 1210. The above processor 1210 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor for execution, or a combination of hardware and software modules in the decoding processor for execution. The software module can be located in a mature storage medium in the field such as random access memory, flash memory, read-only memory, programmable read-only memory or electrically erasable programmable memory, register, etc. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0097] The computer device can also execute the methods described in the foregoing method embodiments, and realize the functions and beneficial effects of the methods described in the foregoing method embodiments, which will not be repeated here.

[0098] Of course, in addition to software implementation, the electronic device 1200 of the present application does not exclude other implementation methods, such as logic devices or a combination of software and hardware, etc., that is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0099] The embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable medium stores one or more programs, and when the one or more programs are executed by an electronic device including multiple application programs, the electronic device executes Figure 1 or Figure 3 or Figure 5 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0100] The computer-readable storage medium includes a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0101] Furthermore, an embodiment of the present application also provides a computer program product, the computer program product comprising a computer program stored on a non-transitory computer-readable storage medium, the computer program comprising program instructions, and when the program instructions are executed by a computer, the following process is implemented: Figure 1 or Figure 3 or Figure 5 The method disclosed in the illustrated embodiment realizes the functions and beneficial effects of each method described in the foregoing method embodiments, which will not be described in detail here.

[0102] The embodiments of the present application can be applied to various electronic device collaboration or interconnection scenarios, including: collaboration and interconnection between mobile phones and laptops / tablet computers; collaboration and interconnection between mobile terminals and smart TVs / displays; collaboration and interconnection between mobile phones or tablet computers and in-car entertainment systems; collaboration and interconnection between mobile terminals and smart conference systems, etc., so as to meet the needs of users in various scenarios such as smart homes, smart offices, and smart travel.

[0103] In short, the above is only a preferred embodiment of the present application and does not limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

[0104] The systems, devices, modules or units described in the above embodiments may be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer may be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0105] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic tape disk storage or other magnetic storage devices or any other non-transmission media that can store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0106] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0107] Each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

Claims

1. A network retrieval enhanced question answering method, characterized in that: include: In response to a question request initiated by a target terminal, determining a first prompt word according to a question statement included in the question request and environmental information corresponding to the question request; Obtaining a question search statement by inputting the first prompt word and a predefined online search function into a pre-trained language model; Determine search results according to the problem search statement and the search engine corresponding to the network search function; A second prompt word is determined according to the search result, and a reply content corresponding to the question statement is obtained by inputting the second prompt word into the pre-trained language model.

2. The method according to claim 1, characterized in that The environmental information includes time information and geographical location information; The determining of the first prompt word according to the question statement included in the question request and the environment information corresponding to the question request includes: The question statement, the time information and the geographical location information included in the question request are filled into a predefined prompt word template to obtain a first prompt word.

3. The method according to claim 1, characterized in that The step of inputting the first prompt word and a predefined online search function into a pre-trained language model to obtain a question search statement includes: Determine an online search function according to a search engine name, a search question, and a parameter description of the search question; Inputting the first prompt word and the online search function into a pre-trained language model, so that the pre-trained language model optimizes the first prompt word based on the online search function to obtain a question search statement; Obtain the question search sentence generated by the pre-trained language model.

4. The method according to any one of claims 1 to 3, characterized in that: After obtaining the answer content corresponding to the question statement, the method further includes: Sending a question response to the target terminal, wherein the question response includes the reply content.

5. A network retrieval enhanced question answering method, characterized in that: include: Obtaining a question search statement, where the question search statement is obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model, where the first prompt word is determined according to a question statement included in a question request initiated by a target terminal and environment information corresponding to the question request; Determining at least one webpage content according to the question search statement and the search engine corresponding to the network search function; A target query condition is determined according to the at least one web page content, and the at least one web page content is retrieved according to the target query condition to obtain a search result; wherein the search result is used to determine a second prompt word, and the second prompt word is input into the pre-trained language model to obtain a reply content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

6. The method according to claim 5, characterized in that The step of determining at least one webpage content according to the search engine corresponding to the question search statement and the network search function includes: Inputting the question search statement into the search engine corresponding to the network search function, and obtaining a search result page through the search engine; Obtaining page content information from the search result page according to a preset category name; At least one web page content is determined from a plurality of web page contents on the search result page according to the page content information.

7. The method according to claim 6, characterized in that The step of inputting the problem search statement into a search engine corresponding to the network search function includes: Accessing a search page of the search engine according to the URL of the search engine corresponding to the network search function; Determine the location information of the question input box in the search page; The question search sentence is input into the search engine according to the position information.

8. The method according to claim 5, characterized in that The determining of the target query condition according to the at least one webpage content includes: Obtaining a content introduction corresponding to the at least one webpage content; The content brief introductions are spliced ​​together to obtain target query conditions.

9. A network retrieval enhanced question answering method, characterized in that: include: Acquire at least one webpage content; wherein the at least one webpage content is determined according to a search engine corresponding to a question search statement and an online search function, the question search statement is obtained by inputting a first prompt word and a predefined online search function into a pre-trained language model, and the first prompt word is determined according to a question statement included in a question request initiated by a target terminal and environmental information corresponding to the question request; Vectorizing the at least one webpage content to obtain a plurality of text vectors; In response to the acquired target query condition, vectorizing the target query condition to obtain a query vector; A search result is determined based on the similarity between the query vector and the multiple text vectors, wherein the search result is used to determine a second prompt word, and a response content corresponding to the question sentence generated by the pre-trained language model based on the second prompt word is obtained by inputting the second prompt word into the pre-trained language model.

10. The method according to claim 9, characterized in that The vectorizing the at least one webpage content to obtain a plurality of text vectors includes: Preprocessing the at least one webpage content to obtain text content; According to a preset text segmentation rule, the text content is divided into multiple text blocks; Vectorization is performed on the multiple text blocks to obtain multiple text vectors.

11. The method according to claim 9, characterized in that The step of determining search results according to the similarity between the query vector and the plurality of text vectors includes: Obtaining similarities between the query vector and the multiple text vectors; Determine a target text vector whose similarity among the multiple text vectors meets a preset similarity threshold; The text block corresponding to the target text vector is determined as the search result.

12. A language model calling module, characterized in that: include: A prompt word determination unit, configured to respond to a question request initiated by a target terminal and determine a first prompt word according to a question statement included in the question request and environment information corresponding to the question request; A first sentence acquisition unit, configured to acquire a question search sentence by inputting the first prompt word and a predefined online search function into a pre-trained language model; A search result acquisition unit, configured to determine search results according to the question search statement and the search engine corresponding to the networked search function; The answer content acquisition unit is used to determine a second prompt word according to the search result, and obtain the answer content corresponding to the question sentence by inputting the second prompt word into the pre-trained language model.

13. A network search module, characterized in that: include: a second sentence acquisition unit, configured to acquire a question search sentence, wherein the question search sentence is obtained by inputting a first prompt word and a predefined network search function into a pre-trained language model, wherein the first prompt word is determined according to a question sentence included in a question request initiated by a target terminal and environment information corresponding to the question request; A web page content determination unit, configured to determine at least one web page content according to the question search statement and the search engine corresponding to the network search function; A web page content retrieval unit is used to determine a target query condition based on the at least one web page content, and retrieve the at least one web page content based on the target query condition to obtain a search result; wherein the search result is used to determine a second prompt word, and by inputting the second prompt word into the pre-trained language model, a reply content corresponding to the question sentence generated by the pre-trained language model based on the second prompt word is obtained.

14. A vector storage and retrieval module, characterized in that: include: A web page content acquisition unit, configured to acquire at least one web page content; wherein the at least one web page content is determined according to a search engine corresponding to a question search statement and an online search function, wherein the question search statement is obtained by inputting a first prompt word and a predefined online search function into a pre-trained language model, wherein the first prompt word is determined according to a question statement included in a question request initiated by a target terminal and environment information corresponding to the question request; A vectorization processing unit, used for performing vectorization processing on the at least one webpage content to obtain a plurality of text vectors; A query vector acquisition unit, configured to respond to the acquired target query condition and vectorize the target query condition to obtain a query vector; A search result determination unit is used to determine the search result according to the similarity between the query vector and the multiple text vectors, wherein the search result is used to determine a second prompt word, and the second prompt word is input into the pre-trained language model to obtain the answer content corresponding to the question sentence generated by the pre-trained language model according to the second prompt word.

15. A network retrieval enhanced question answering system, characterized in that: It comprises the language model calling module as claimed in claim 11, the networked search module as claimed in claim 12 and the vector storage retrieval module as claimed in claim 13; the language model calling module and the networked search module are connected to each other for communication, and the networked search module and the vector storage retrieval module are connected to each other for communication.

16. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the method described in any one of claims 1 to 11 are implemented.

17. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program or instruction, and when the program or instruction is executed by a processor, the steps of the method according to any one of claims 1 to 11 are implemented.

18. A computer program product, characterized in that The computer program product comprises a computer program stored on a non-transitory computer-readable storage medium, wherein the computer program comprises program instructions, and when the program instructions are executed by a computer, the computer is caused to perform the steps of the method according to any one of claims 1 to 11.

Images