Document recall method and device, electronic equipment and storage medium
By rewriting and matching user query information, the problem of low accuracy of document recall in traditional methods is solved, the ability to recall more correct documents is realized, and the quality of reply is improved.
Patent Information
- Application Number
- CN202510031316.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-08
- Publication Date
- 2025-06-10
AI Technical Summary
The traditional search-enhanced generation method has shortcomings in document recall accuracy, which leads to the inability to recall the correct document, affecting the quality of the reply in the subsequent generated part.
By obtaining user query information and rewriting it, using different candidate rewriting schemes and rewriting examples, multiple rewriting query information are generated, thereby performing multiple search matching and recalling more correct documents.
Improve the correct document type and number of document recalls, enhance the model's ability to recall correct documents, and improve the quality of reply.
Smart Images
Figure CN120123456A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of artificial intelligence technology, and in particular, to a document recall method, apparatus, electronic device, storage medium, and computer program product. Background Art
[0002] As one of the artificial intelligence technologies, in the application of large language models, in order to overcome problems such as lack of domain knowledge, untimely model knowledge update, and generation of false content, the Retrieval-augmented generation (RAG) technology can be adopted. By introducing additional information sources such as external knowledge bases, effective mitigation strategies are provided, which are particularly effective in knowledge-intensive scenarios or specific domain applications that require continuous knowledge update.
[0003] In related technologies, traditional retrieval-augmented generation methods require a large amount of data annotation to train a rewriting model, so that when retrieving and using, retrieval results are usually generated for one type of query. This will limit the types and quantities of correct documents that can be recalled, resulting in the inability to recall correct documents in the retrieval enhancement part, and further affecting the subsequent generation part's inability to generate responses based on the correct document content. Summary of the Invention
[0004] The present disclosure provides a document recall method, apparatus, electronic device, storage medium, and computer program product to at least solve the problem of low accuracy of document recall in related technologies. The technical solution of the present disclosure is as follows:
[0005] According to the first aspect of the embodiments of the present disclosure, a document recall method is provided, including:
[0006] Obtain user query information input to a target response model; the target response model includes a query information rewriting model for retrieval enhancement; the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme;
[0007] Rewrite the user query information according to the different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme to obtain a plurality of rewritten query information;
[0008] Respectively use each rewritten query information to perform retrieval matching on a plurality of documents to determine the recalled documents of each rewritten query information; the recalled documents are used for the target response model to output a query response result for the user query information.
[0009] In a possible implementation manner, the method further includes:
[0010] Obtain the rewritten solution configuration information, configure the different candidate rewritten solutions in the rewritten solution library, and generate rewritten examples of each candidate rewritten solution in the rewritten example library; the rewritten examples of the candidate rewritten solutions are used to indicate the rewriting process of the corresponding candidate rewritten solutions.
[0011] In a possible implementation manner, the method further includes:
[0012] Obtain the rewritten solution new information, configure the newly added candidate rewritten solutions in the rewritten solution library, and generate rewritten examples of the newly added candidate rewritten solutions in the rewritten example library;
[0013] Or, obtain the rewritten solution reduction information, delete the reduced candidate rewritten solutions in the rewritten solution library, and delete the rewritten examples of the reduced candidate rewritten solutions in the rewritten example library.
[0014] In a possible implementation manner, the rewriting of the user query information according to the different candidate rewritten solutions and the rewritten examples of each candidate rewritten solution to obtain multiple rewritten query information includes:
[0015] Obtain the rewriting prompt text; the rewriting prompt text is used to indicate the data structure of the query information input to the rewriting model.
[0016] According to the rewriting prompt text, perform data integration on the user query information, the different candidate rewritten solutions, and the rewritten examples of each candidate rewritten solution to obtain information integration data;
[0017] Input the information integration data into the query information rewriting model, and output multiple rewritten query information.
[0018] In a possible implementation manner, the inputting the information integration data into the query information rewriting model and outputting multiple rewritten query information includes:
[0019] Input the information integration data into the query information rewriting model, and determine multiple target rewritten solutions applicable to the user query information from the multiple candidate rewritten solutions according to the rewritten examples of each candidate rewritten solution;
[0020] Generate multiple rewritten query information by using each target rewritten solution and the user query information.
[0021] In a possible implementation manner, the retrieving and matching multiple documents respectively by using each rewritten query information to determine the retrieved documents of each rewritten query information includes:
[0022] Determine the non-public document database configured for the user account to which the user query information belongs;
[0023] Obtain, from the non-public document database, non-public documents that are retrieved and matched with each of the rewritten query information as the recall documents for each of the rewritten query information.
[0024] In a possible implementation manner, the method further includes:
[0025] Use the document ranking model in the target reply model to perform relevance ranking on the recall documents of any one of the rewritten query information;
[0026] Output the recall documents of any one of the rewritten query information whose relevance ranking meets a preset threshold to the reply generation model in the target reply model; the reply generation model is used to output a query reply result for the user query information corresponding to any one of the rewritten query information.
[0027] According to a second aspect of the embodiments of the present disclosure, there is provided a document recall device, including:
[0028] A query information acquisition unit configured to execute acquiring user query information input to a target reply model; the target reply model includes a query information rewriting model for retrieval enhancement; the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each of the candidate rewriting schemes;
[0029] A query information rewriting unit configured to execute rewriting the user query information according to the different candidate rewriting schemes and the rewriting examples of each of the candidate rewriting schemes to obtain a plurality of rewritten query information;
[0030] A document retrieval and matching unit configured to execute respectively using each of the rewritten query information to perform retrieval and matching on a plurality of documents to determine the recall documents of each of the rewritten query information; the recall documents are used for the target reply model to output a query reply result for the user query information.
[0031] According to a third aspect of the embodiments of the present disclosure, there is provided an electronic device, including:
[0032] A processor;
[0033] A memory for storing executable instructions of the processor;
[0034] Wherein, the processor is configured to execute the instructions to implement the document recall method as described in any one of the above.
[0035] According to a fourth aspect of the embodiments of the present disclosure, there is provided a computer-readable storage medium. When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the document recall method as described in any one of the above.
[0036] According to a fifth aspect of the embodiments of the present disclosure, there is provided a computer program product. The computer program product includes instructions that, when executed by a processor of an electronic device, enable the electronic device to execute the document recall method as described in any one of the above.
[0037] The technical solutions provided by the embodiments of the present disclosure at least bring the following beneficial effects:
[0038] In the solution of the present disclosure, by obtaining user query information input to a target response model, the target response model includes a query information rewriting model for retrieval enhancement, and the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme. Then, according to different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme, the user query information is rewritten to obtain multiple rewritten query information. Furthermore, each rewritten query information is respectively used to retrieve and match multiple documents to determine the recalled documents of each rewritten query information, and the recalled documents are used for the target response model to output a query response result for the user query information. In this way, the user query information can be adaptively rewritten based on different candidate rewriting schemes and their rewriting examples, enriching the diversity of query information entering the retrieval enhancement generation system, being able to improve the types and quantities of correct documents recalled, and effectively improving the ability of the model to recall correct documents.
[0039] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0040] The accompanying drawings herein are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present disclosure, and are used together with the specification to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.
[0041] Figure 1 is an application environment diagram of a document recall method shown according to an exemplary embodiment.
[0042] Figure 2 is a flowchart of a document recall method shown according to an exemplary embodiment.
[0043] Figure 3 is a schematic diagram of a target response model architecture shown according to an exemplary embodiment.
[0044] Figure 4It is a flowchart of another document recall method shown according to an exemplary embodiment.
[0045] Figure 5 It is a block diagram of a document recall device shown according to an exemplary embodiment.
[0046] Figure 6 It is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners
[0047] In order to enable those of ordinary skill in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings.
[0048] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily need to be used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the present disclosure described here can be implemented in an order other than those illustrated or described here. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present disclosure.
[0049] It should also be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data for display, data for analysis, etc.) involved in the present disclosure are all information and data authorized by the user or fully authorized by all parties.
[0050] The document recall method provided by the present disclosure can be applied to, for example Figure 1In the application environment shown. Among them, the terminal 102 communicates with the server 104 through the network. The data storage system can store the data that the server 104 needs to process. The data storage system can be integrated on the server 104, or can be placed on the cloud or other network servers. The terminal 102 can send user query information to the server 104. The server 104 can obtain the user query information and input the user query information into the target reply model. The target reply model can include a query information rewriting model for retrieval enhancement. The query information rewriting model can be pre-configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme. Through the query information rewriting model, according to different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme, the user query information can be rewritten to obtain multiple rewritten query information. Furthermore, in the preset document database, each rewritten query information can be used to retrieve and match multiple documents respectively to determine the retrieved documents of each rewritten query information. The target reply model can perform subsequent reply generation processing based on the retrieved documents and output a query reply result for the user query information. The server 104 can feedback the query reply result to the terminal 102. Among them, the terminal 102 can be but is not limited to various personal computers, laptop computers, smart phones, tablet computers, Internet of Things devices and portable wearable devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc. The portable wearable devices can be smart watches, smart bracelets, head-mounted devices, etc. The server 104 can be implemented by an independent server or a server cluster composed of multiple servers.
[0051] Retrieval-Augmented Generation (RAG) can be divided into two parts. One part is retrieval-augmented. By given a user query, relevant documents to the user query can be found from external knowledge sources. The other part is generation. By giving the retrieved documents to a large model, the large model can generate a reply to the user query based on the retrieved documents.
[0052] In actual use, in the retrieval-augmented part of the traditional Retrieval-Augmented Generation (RAG) method, there is a problem that the correct documents cannot be retrieved, which will affect that the subsequent generation part cannot refer to the appropriate correct document content to generate a reply. In view of the problems existing in the retrieval-augmented part of the Retrieval-Augmented Generation (RAG), by optimizing the user query, rewritten queries can be obtained based on multi-way query rewriting, so as to be able to increase the types and quantities of the retrieved correct documents. And through few-shot demonstration, the adaptive selection of multi-way query rewriting can be realized, so that the whole rewriting process does not require model training. The document retrieval method of the present disclosure can effectively improve the ability of the model to retrieve correct documents.
[0053] Figure 2 is a flowchart of a document recall method shown according to an exemplary embodiment. This method can be used in Figure 1 server 104 and includes the following steps.
[0054] In step S210, obtain the user query information input to the target reply model.
[0055] As an example, the user query information can be a user query. For example, based on the query conditions or keywords input by the user in the terminal, a user query can be generated, which can be used to find relevant information required by the user.
[0056] Among them, the target reply model can include a query information rewriting model for retrieval enhancement. This query information rewriting model can be configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme.
[0057] As an example, multiple candidate rewriting schemes can be stored in a rewriting scheme library. Different candidate rewriting schemes can be used to rewrite the user query information. For example, the rewriting scheme library can include but is not limited to general rewriting, keyword rewriting, pseudo-answer rewriting, question abstraction rewriting, multi-sub-question rewriting, and can also include other rewriting methods, which are not specifically limited in this embodiment.
[0058] In practical applications, as Figure 3 shown, in a user query scenario, the user query information input to the target reply model can be obtained, such as a user query. Then, the target reply model can perform retrieval enhancement and reply generation processing based on this user query information.
[0059] Specifically, as Figure 3 shown, the target reply model can include a retrieval enhancement module and a reply generation module. The retrieval enhancement module can include a query information rewriting model for retrieval enhancement and a document sorting model for sorting the recalled documents. The reply generation module can include a reply generation model for generating a reply result.
[0060] In step S220, rewrite the user query information according to different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme to obtain multiple rewritten query information.
[0061] As an example, the rewriting examples of different candidate rewriting schemes can be stored in a rewriting example library, such as examples of general rewriting, examples of keyword rewriting, examples of pseudo-answer rewriting, examples of question abstraction rewriting, examples of multi-sub-question rewriting, etc.
[0062] In a specific implementation, by querying the information rewriting model, the user query information can be rewritten according to different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme to obtain multiple rewritten query information, such as Figure 3 the rewritten multi-way query in Figure 3 . Thus, based on the multi-way query rewriting, the query diversity entering the Retrieval-Augmented Generation (RAG) system can be improved, and further, the types and quantities of correctly retrieved documents can be increased; based on few-shot demonstration, the adaptive selection of multi-way query rewriting can be realized. While ensuring the reduction of irrelevant documents, the whole process does not require model training, reducing the training investment cost.
[0063] In step S230, each rewritten query information is respectively used to retrieve and match multiple documents to determine the retrieved documents of each rewritten query information.
[0064] Among them, the retrieved documents can be used for the target response model to output the query response result for the user query information.
[0065] After obtaining multiple rewritten query information, such as Figure 3 shown, based on the rewritten multi-way query, for each way of query (i.e., each rewritten query information), the document most matching the current query (i.e., the retrieved document of each rewritten query information) can be selected in the preset document database, and then the retrieved documents of multiple ways can be merged and returned to the retrieved document set. Thus, by adaptively rewriting the user query into multiple possible queries, more matching documents can be searched, which helps to improve the response quality processed by the downstream model. Optionally, the preset document database can include a public document database (such as public Internet data obtained through a search engine) and a non-public document database (such as the user's privatized local files).
[0066] In an example, after obtaining the retrieved document set, such as Figure 3 shown, through the document ranking model in the target response model, all the documents included in the retrieved document set can be ranked according to relevance, and then the documents whose relevance ranking meets the preset threshold (such as the top N of the relevance ranking) can be output to the response generation model. Optionally, the document ranking model can be a model based on the reciprocal rank fusion of RRF or an embedding-based model, or a ranking model trained through relevance ranking. Thus, based on the document ranking model, it can not only ensure that as many correct documents as possible are given to the downstream response model, but also reduce the influence of irrelevant documents.
[0067] In another example, such as Figure 3As shown, based on the response generation model in the target response model, appropriate prompt words can be used to structurally organize the user query and the top N documents sorted by relevance. Then, corresponding response results can be generated based on the structurally organized information, such as generating a specific response to the user query and returning it to the user terminal. Optionally, the response generation model can adopt a pre-trained large model or a fine-tuned model.
[0068] In an alternative embodiment, the technical solution of this embodiment can be used to improve the performance of Retrieval-Augmented Generation (RAG) in the large model scenario to recall more correct documents, thereby improving the response quality. It can also be used to understand the user query intention in the Internet resource transfer scenario to better recall the resource objects that best match the user intention.
[0069] In the above document recall method, by obtaining the user query information input to the target response model, and then according to different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme, the user query information is rewritten to obtain multiple rewritten query information. Then, each rewritten query information is used to retrieve and match multiple documents respectively to determine the recalled documents of each rewritten query information. In this way, the user query information can be adaptively rewritten based on different candidate rewriting schemes and their rewriting examples, enriching the diversity of query information entering the retrieval-augmented generation system, and being able to improve the type and quantity of correctly recalled documents, effectively enhancing the ability of the model to recall correct documents.
[0070] In an exemplary embodiment, it further includes: obtaining rewriting scheme configuration information, configuring different candidate rewriting schemes in the rewriting scheme library, and generating rewriting examples of each candidate rewriting scheme in the rewriting example library.
[0071] Among them, the rewriting example of the candidate rewriting scheme is used to indicate the rewriting process of the corresponding candidate rewriting scheme.
[0072] In a specific implementation, as Figure 3 shown, initialization offline configuration can be performed based on the system configuration module, which specifically includes the following configurations:
[0073] 1. Configure the rewriting scheme library: Different candidate rewriting schemes can be pre-stored in the rewriting scheme library. For example, the query rewriting schemes can include but are not limited to general rewriting, keyword rewriting, pseudo-answer rewriting, question abstraction rewriting, and multi-sub-question rewriting. The query rewriting schemes can be custom-configured according to downstream tasks (i.e., obtaining rewriting scheme configuration information), and new query rewriting schemes can be added.
[0074] 2. Configure the rewritten example library (such as the Few-shot Demo pool): In the rewritten example library, rewritten examples of each candidate rewritten scheme can be pre-stored, such as examples of different rewritten schemes for different queries. For example, for the user query "What is the stock price of xx today?" in Example 1, the query rewritten scheme and its rewritten example can be "keyword rewriting": "xx stock price", or "pseudo-answer rewriting": "The stock price of xx today is xxxx". Optionally, the rewritten examples can be custom-configured or generated by a large artificial intelligence model. By configuring the rewritten examples, the large language model can better understand the rewritten schemes suitable for different queries, which helps the model to adaptively select the most appropriate rewritten scheme from the rewritten example library when encountering real user problems, rather than only being able to select some fixed schemes, thus increasing the flexibility of the scheme.
[0075] The technical solution of this embodiment provides data support for adaptive rewriting based on different candidate rewritten schemes and their rewritten examples by obtaining rewritten scheme configuration information, configuring different candidate rewritten schemes in the rewritten scheme library, and generating rewritten examples of each candidate rewritten scheme in the rewritten example library.
[0076] In an exemplary embodiment, it further includes: obtaining rewritten scheme addition information, configuring the newly added candidate rewritten scheme in the rewritten scheme library, and generating the rewritten example of the newly added candidate rewritten scheme in the rewritten example library; or, obtaining rewritten scheme reduction information, deleting the reduced candidate rewritten scheme in the rewritten scheme library, and deleting the rewritten example of the reduced candidate rewritten scheme in the rewritten example library.
[0077] In practical applications, for the rewritten scheme library, the query rewritten scheme can be increased or decreased according to the business situation (that is, obtaining rewritten scheme addition information or obtaining rewritten scheme reduction information). In the rewritten scheme library, not only the scheme based on the prompt can be added or deleted, but also the scheme based on model training can be added or deleted, so that various rewritten schemes can be flexibly increased or decreased continuously; after adding a rewritten scheme in the rewritten scheme library, new rewritten examples (that is, the rewritten examples of the newly added candidate rewritten scheme) also need to be added correspondingly in the rewritten example library to show how to use the newly added query rewritten scheme; or after reducing the rewritten scheme in the rewritten scheme library, the reduced rewritten examples (that is, the rewritten examples of the reduced candidate rewritten scheme) also need to be deleted correspondingly in the rewritten example library.
[0078] The technical solution of this embodiment can increase the flexibility of the rewriting scheme by obtaining new information about the rewriting scheme, configuring new candidate rewriting schemes in the rewriting scheme library, and generating rewriting examples of the new candidate rewriting schemes in the rewriting example library; or, obtaining reduced information about the rewriting scheme, deleting the reduced candidate rewriting schemes in the rewriting scheme library, and deleting the rewriting examples of the reduced candidate rewriting schemes in the rewriting example library.
[0079] In an exemplary embodiment, user query information is rewritten according to different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme to obtain multiple rewritten query information, including: obtaining a rewriting prompt text; the rewriting prompt text is used to indicate the data structure of the input query information rewriting model; according to the rewriting prompt text, the user query information, different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme are integrated to obtain information integration data; the information integration data is input into the query information rewriting model to output multiple rewritten query information.
[0080] As an example, the query information rewriting model may adopt an artificial intelligence large model or a fine-tuned specific rewriting model.
[0081] In a specific implementation, the user query (i.e., user query information), different candidate rewriting schemes, and each candidate rewriting scheme can be structured and organized according to the rewriting prompt word prompt (i.e., rewriting prompt text), such as performing data integration to obtain information integration data, and then the information integration data can be input into the query information rewriting model to generate a rewritten query (rewritten query information) based on each rewriting scheme.
[0082] In one example, the rewriting prompt can be a piece of text designed to guide the large language model to complete a specific task (such as query rewriting, text generation, etc.), which can contain clear instructions for the task, and can also include the format of input data, the format of expected output, and required context or example information. By designing a rewriting prompt, the model can understand the task requirements more accurately and generate output that meets expectations.
[0083] For example, in a scenario of user query rewriting, the rewriting prompt may be based on one or more specific examples, showing how to rewrite the original user query into a data form that can be input into the query information rewriting model.
[0084] For example, through the structured organization of information, the model can identify the various parts of the input data (different information) and the relationship between the information, thereby ensuring the consistency and coherence of the data input to the model through structured organization.
[0085] In the technical solution of this embodiment, by obtaining the rewritten prompt text, and then integrating the user query information, different candidate rewritten solutions, and the rewritten examples of each candidate rewritten solution according to the rewritten prompt text to obtain the information integration data, and then inputting the information integration data into the query information rewriting model to output multiple rewritten query information, it can help the query information rewriting model perform the rewriting process.
[0086] In an exemplary embodiment, inputting the information integration data into the query information rewriting model to output multiple rewritten query information includes: inputting the information integration data into the query information rewriting model, and determining multiple target rewritten solutions applicable to the user query information from multiple candidate rewritten solutions according to the rewritten examples of each candidate rewritten solution; using each target rewritten solution and the user query information to generate multiple rewritten query information.
[0087] In an example, the query information rewriting model can adaptively select the candidate rewritten solution most suitable for the current user query (i.e., the user query information) from multiple candidate rewritten solutions according to the rewritten examples of each candidate rewritten solution, and then can generate a path of rewritten query (i.e., the rewritten query information) for each candidate rewritten solution.
[0088] In the technical solution of this embodiment, by inputting the information integration data into the query information rewriting model, determining multiple target rewritten solutions applicable to the user query information from multiple candidate rewritten solutions according to the rewritten examples of each candidate rewritten solution, and then using each target rewritten solution and the user query information to generate multiple rewritten query information, it can realize the adaptive selection of multi-path query information rewriting and ensure the reduction of irrelevant documents.
[0089] In an exemplary embodiment, using each rewritten query information to retrieve and match multiple documents respectively to determine the retrieved documents of each rewritten query information includes: determining the non-public document database configured by the user account to which the user query information belongs; obtaining the non-public documents retrieved and matched with each rewritten query information from the non-public document database as the retrieved documents of each rewritten query information.
[0090] In practical applications, the preset document database may include a non-public document database. For the scenario of using the non-public document database, based on the user account to which the user query information belongs, the non-public document database configured by the user account can be determined, and then the non-public documents retrieved and matched with each rewritten query information can be obtained from the non-public document database as the retrieved documents of each rewritten query information for the target reply model to output the query reply result for the user query information.
[0091] The technical solution of this embodiment can provide users with personalized document recall services by determining the non-public document database configured for the user account to which the user's query information belongs, and then obtaining non-public documents that are retrieved and matched with each rewritten query information from the non-public document database as the recall documents for each rewritten query information.
[0092] In an exemplary embodiment, it further includes: using the document ranking model in the target response model to perform relevance ranking on the recall documents of any rewritten query information; outputting the recall documents of any rewritten query information whose relevance ranking meets the preset threshold to the response generation model in the target response model; the response generation model is used to output the query response result of the user query information corresponding to any rewritten query information.
[0093] After obtaining the recall document set (i.e., the recall documents of any rewritten query information), through the document ranking model in the target response model, the relevance ranking of all the documents included in the recall document set can be performed, and then the documents whose relevance ranking meets the preset threshold (such as the top N of the relevance ranking) can be output to the response generation model.
[0094] In an example, the document ranking model can be a model based on the fusion of reciprocal rank fusion (RRF) or an embedding-based model, or a ranking model trained through relevance ranking.
[0095] The technical solution of this embodiment can, by using the document ranking model in the target response model, perform relevance ranking on the recall documents of any rewritten query information, and then output the recall documents of any rewritten query information whose relevance ranking meets the preset threshold to the response generation model in the target response model. Thus, based on the document ranking model, it can not only ensure that as many correct documents as possible are provided to the downstream response model, but also reduce the influence of irrelevant documents.
[0096] Figure 4 It is a flowchart of another document recall method shown according to an exemplary embodiment. As Figure 4 shown, this method is used in computer devices such as servers and includes the following steps.
[0097] In step S410, obtain the rewritten scheme configuration information, configure different candidate rewritten schemes in the rewritten scheme library, and generate rewritten examples of each candidate rewritten scheme in the rewritten example library. In step S420, obtain the user query information input to the target response model. In step S430, obtain the rewritten prompt text, and integrate the user query information, different candidate rewritten schemes, and the rewritten examples of each candidate rewritten scheme according to the rewritten prompt text to obtain the integrated information data. In step S440, input the integrated information data into the query information rewriting model, and determine multiple target rewritten schemes applicable to the user query information from multiple candidate rewritten schemes according to the rewritten examples of each candidate rewritten scheme. In step S450, generate multiple rewritten query information by using each target rewritten scheme and the user query information. In step S460, respectively use each rewritten query information to perform retrieval matching on multiple documents to determine the retrieved documents of each rewritten query information. It should be noted that the specific limitations of the above steps can refer to the specific limitations of a document retrieval method described above, and will not be elaborated here.
[0098] It should be understood that although the steps in the flowcharts involved in the above-described embodiments are shown in sequence according to the arrows, these steps do not necessarily need to be executed in the order indicated by the arrows. Unless there is a clear description in this article, there is no strict order limit for the execution of these steps, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above-described embodiments may include multiple steps or multiple stages. These steps or stages do not necessarily need to be executed at the same time, but can be executed at different times. The execution order of these steps or stages does not necessarily need to be sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.
[0099] It can be understood that the same / similar parts among the various embodiments of the above methods in this specification can be referred to each other. Each embodiment focuses on the differences from other embodiments. For the relevant parts, refer to the descriptions of other method embodiments.
[0100] Based on the same inventive concept, the embodiments of the present disclosure also provide a document retrieval device for implementing the above-mentioned document retrieval method.
[0101] Figure 5 It is a block diagram of a document retrieval device shown according to an exemplary embodiment. Refer to Figure 5 , the device includes:
[0102] The query information acquisition unit 501 is configured to obtain the user query information input to the target response model; the target response model includes a query information rewriting model for retrieval enhancement; the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme;
[0103] The query information rewriting unit 502 is configured to rewrite the user query information according to the different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme to obtain multiple rewritten query information;
[0104] The document retrieval and matching unit 503 is configured to separately retrieve and match multiple documents with each rewritten query information to determine the retrieved documents of each rewritten query information; the retrieved documents are used for the target response model to output a query response result for the user query information.
[0105] In a possible implementation manner, the document retrieval device further includes:
[0106] The rewriting scheme configuration unit is specifically configured to obtain the rewriting scheme configuration information, configure the different candidate rewriting schemes in the rewriting scheme library, and generate the rewriting examples of each candidate rewriting scheme in the rewriting example library; the rewriting examples of the candidate rewriting schemes are used to indicate the rewriting process of the corresponding candidate rewriting schemes.
[0107] In a possible implementation manner, the document retrieval device further includes:
[0108] The new configuration unit is specifically configured to obtain the new rewriting scheme information, configure the newly added candidate rewriting schemes in the rewriting scheme library, and generate the rewriting examples of the newly added candidate rewriting schemes in the rewriting example library;
[0109] Alternatively, the reduction configuration unit is specifically configured to obtain the rewriting scheme reduction information, delete the reduced candidate rewriting schemes in the rewriting scheme library, and delete the rewriting examples of the reduced candidate rewriting schemes in the rewriting example library.
[0110] In a possible implementation manner, the query information rewriting unit 502 is specifically configured to obtain the rewriting prompt text; the rewriting prompt text is used to indicate the data structure of the data input to the query information rewriting model; according to the rewriting prompt text, perform data integration on the user query information, the different candidate rewriting schemes, and the rewriting examples of each candidate rewriting scheme to obtain information integration data; input the information integration data into the query information rewriting model to output multiple rewritten query information.
[0111] In a possible implementation manner, the query information rewriting unit 502 is further specifically configured to input the information integration data into the query information rewriting model, and determine, from multiple candidate rewriting schemes, multiple target rewriting schemes applicable to the user query information according to the rewriting examples of the candidate rewriting schemes; and generate multiple rewritten query information by using each of the target rewriting schemes and the user query information.
[0112] In a possible implementation manner, the document retrieval and matching unit 503 is specifically configured to determine a non-public document database configured for the user account to which the user query information belongs; and obtain, from the non-public document database, non-public documents that are retrieved and matched with each of the rewritten query information as the recall documents for each of the rewritten query information.
[0113] In a possible implementation manner, the document recall device further includes:
[0114] A document sorting unit, which is specifically configured to perform a relevance sorting on the recall documents of any one of the rewritten query information by using the document sorting model in the target reply model;
[0115] A sorting output unit, which is specifically configured to output the recall documents of any one of the rewritten query information whose relevance sorting meets a preset threshold to the reply generation model in the target reply model; and the reply generation model is used to output a query reply result of the user query information corresponding to any one of the rewritten query information.
[0116] Regarding the device in the above embodiments, the specific manners in which each module performs operations have been described in detail in the embodiments related to the method, and will not be elaborated herein.
[0117] Each module in the above document recall device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the computer device in hardware form or be independent of it, or be stored in the memory in the computer device in software form so that the processor can call and execute the operations corresponding to the above modules.
[0118] Figure 6 It is a block diagram of an electronic device 600 for implementing a document recall method according to an exemplary embodiment. For example, the electronic device 600 may be a server. Refer to Figure 6, the electronic device 600 includes a processing component 620, which further includes one or more processors, and memory resources represented by a memory 622 for storing instructions executable by the processing component 620, such as application programs. The application programs stored in the memory 622 may include one or more modules each corresponding to a set of instructions. In addition, the processing component 620 is configured to execute instructions to perform the above-described method.
[0119] The electronic device 600 may further include: a power component 624 configured to perform power management of the electronic device 600, a wired or wireless network interface 626 configured to connect the electronic device 600 to a network, and an input / output (I / O) interface 628. The electronic device 600 may operate based on an operating system stored in the memory 622, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, or the like.
[0120] In an exemplary embodiment, there is also provided a computer-readable storage medium including instructions, such as the memory 622 including instructions, and the above instructions can be executed by a processor of the electronic device 600 to complete the above method. The storage medium may be a computer-readable storage medium. For example, the computer-readable storage medium may be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, and an optical data storage device, etc.
[0121] In an exemplary embodiment, there is also provided a computer program product, which includes instructions, and the above instructions can be executed by a processor of the electronic device 600 to complete the above method.
[0122] It should be noted that the above-described device, electronic device, computer-readable storage medium, computer program product, etc. may also include other implementation manners according to the description of the method embodiments. The specific implementation manners may refer to the description of the relevant method embodiments and will not be elaborated herein one by one.
[0123] After considering the specification and practicing the invention disclosed herein, those skilled in the art will readily conceive of other embodiments of the present disclosure. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common general knowledge or conventional technical means in the technical field not disclosed in the present disclosure. The specification and embodiments are only regarded as exemplary, and the true scope and spirit of the present disclosure are pointed out by the claims.
[0124] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.
Claims
1. A document recall method, characterized in that: The method comprises: Obtaining user query information input to a target reply model; the target reply model includes a query information rewriting model for retrieval enhancement; the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme; rewriting the user query information according to the different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme to obtain a plurality of rewritten query information; The rewritten query information is respectively used to search and match multiple documents to determine the recalled documents of each rewritten query information; the recalled documents are used by the target response model to output the query response results for the user query information.
2. The method according to claim 1, characterized in that The method further comprises: Rewriting scheme configuration information is obtained, the different candidate rewriting schemes are configured in a rewriting scheme library, and rewriting examples of the candidate rewriting schemes are generated in a rewriting example library; the rewriting examples of the candidate rewriting schemes are used to indicate the rewriting process of the corresponding candidate rewriting schemes.
3. The method according to claim 2, characterized in that The method further comprises: Acquire newly added information of the rewriting scheme, configure newly added candidate rewriting schemes in the rewriting scheme library, and generate rewriting examples of the newly added candidate rewriting schemes in the rewriting example library; Or, obtaining rewriting scheme reduction information, deleting reduced candidate rewriting schemes in the rewriting scheme library, and deleting rewriting examples of the reduced candidate rewriting schemes in the rewriting example library.
4. The method according to claim 1, characterized in that: The user query information is rewritten according to the different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme to obtain a plurality of rewritten query information, including: Obtaining a rewriting prompt text; the rewriting prompt text is used to indicate the data structure of the query information rewriting model input; According to the rewriting prompt text, data integration is performed on the user query information, the different candidate rewriting schemes, and rewriting examples of each candidate rewriting scheme to obtain information integration data; The information integration data is input into the query information rewriting model, and a plurality of the rewritten query information is output.
5. The method according to claim 4, characterized in that The step of inputting the information integration data into the query information rewriting model and outputting a plurality of the rewritten query information includes: Inputting the information integration data into the query information rewriting model, and determining multiple target rewriting schemes applicable to the user query information from multiple candidate rewriting schemes according to rewriting examples of each candidate rewriting scheme; A plurality of rewritten query information is generated by using each of the target rewriting schemes and the user query information.
6. The method according to claim 1, characterized in that The step of respectively using the rewritten query information to search and match multiple documents to determine the recalled documents of the rewritten query information includes: Determine a non-public document database configured for the user account to which the user query information belongs; From the non-public document database, non-public documents that match the rewritten query information are retrieved and obtained as the recalled documents of the rewritten query information.
7. The method according to claim 1, characterized in that The method further comprises: Using the document ranking model in the target response model, relevance ranking is performed on any recalled document of the rewritten query information; The recalled documents of any of the rewritten query information whose relevance ranking meets the preset threshold are output to the reply generation model in the target reply model; the reply generation model is used to output the query reply result of the user query information corresponding to any of the rewritten query information.
8. A document recall device, characterized in that: The device comprises: A query information acquisition unit is configured to acquire user query information input to a target reply model; the target reply model includes a query information rewriting model for retrieval enhancement; the query information rewriting model is configured with different candidate rewriting schemes and rewriting examples of each candidate rewriting scheme; a query information rewriting unit, configured to rewrite the user query information according to the different candidate rewriting schemes and the rewriting examples of each candidate rewriting scheme, to obtain a plurality of rewritten query information; The document retrieval and matching unit is configured to perform retrieval and matching of multiple documents using each of the rewritten query information respectively, and determine the recalled document for each of the rewritten query information; the recalled document is used for the target response model to output the query response result for the user query information.
9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the document recall method as described in any one of claims 1 to 7.
10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the document recall method as described in any one of claims 1 to 7.
11. A computer program product, comprising instructions, characterized in that: When the instruction is executed by a processor of an electronic device, the electronic device is enabled to execute the document recall method as described in any one of claims 1 to 7.