Retrieval enhancement generation method and device, equipment and storage medium

Abstract

The embodiment of the invention discloses a retrieval enhancement generation method and device, equipment and a storage medium. According to the technical scheme, firstly, type classification is carried out on received initial query questions, target question types are determined, the target question types comprise at least one of document questions and non-document questions, then target models corresponding to the target question types are used, target answers corresponding to the initial query questions are output, and the target answers corresponding to the initial query questions are obtained. The target model is one of preset models, the preset models comprise a document model and a non-document model, the document model is used for obtaining answers to document questions, and the non-document model is used for obtaining answers to non-document questions. According to the retrieval enhancement generation method, the initial query question is subjected to type classification, and then the target answer corresponding to the initial query question is output by using the corresponding target model, so that the accuracy of the answer generated by retrieval enhancement can be improved, and the user experience is improved.

Description

Technical Field

[0001] This application relates to the fields of natural language processing and information retrieval technology, and to, but is not limited to, a method, apparatus, device, and storage medium for retrieval enhancement generation. Background Technology

[0002] With the development of artificial intelligence and big data technologies, retrieval-augmented generation (RAG)-based question answering systems have been widely used in various application scenarios.

[0003] However, existing RAG systems have several shortcomings in use, leading to low accuracy in the output answers. Therefore, improving the accuracy of retrieval-enhanced answers is an urgent problem to be solved. Summary of the Invention

[0004] In view of this, the retrieval enhancement generation method, apparatus, device, and storage medium provided in the embodiments of this application can improve the accuracy of the retrieval enhancement generation answers. The retrieval enhancement generation method, apparatus, device, and storage medium provided in the embodiments of this application are implemented as follows:

[0005] The retrieval enhancement generation method provided in this application includes: classifying a received initial query question by type to determine a target question type, wherein the target question type includes at least one of document-type questions and non-document-type questions; using a target model corresponding to the target question type to output a target answer corresponding to the initial query question, wherein the target model is one of a preset model, wherein the preset model includes a document model and a non-document model, wherein the document model is used to obtain the answer to the document-type question, and the non-document model is used to obtain the answer to the non-document question.

[0006] The retrieval enhancement generation apparatus provided in this application includes: a determining module, configured to classify a received initial query question into types and determine a target question type, wherein the target question type includes at least one of document-type questions and non-document-type questions; and an output module, configured to use a target model corresponding to the target question type to output a target answer corresponding to the initial query question, wherein the target model is one of preset models, the preset models include a document model and a non-document model, the document model is used to obtain the answer to the document-type question, and the non-document model is used to obtain the answer to the non-document-type question.

[0007] The computer device provided in this application includes a memory and a processor. The memory stores a computer program that can run on the processor. When the processor executes the program, it implements the method described in this application.

[0008] The computer-readable storage medium provided in this application embodiment stores a computer program thereon, which, when executed by a processor, implements the method described in this application embodiment.

[0009] The computer program product provided in this application includes a computer program that, when executed by a processor, implements the method described in this application.

[0010] In the retrieval enhancement generation method, apparatus, device, and storage medium provided in this application embodiment, the electronic device first classifies the received initial query question into types to determine the target question type. The target question type includes at least one of document-type questions and non-document-type questions. Then, using a target model corresponding to the target question type, it outputs the target answer corresponding to the initial query question. The target model is one of the preset models, including document models and non-document models. The document model is used to obtain answers to document-type questions, and the non-document model is used to obtain answers to non-document-type questions. This retrieval enhancement generation method, by classifying the initial query question into types and then using the corresponding target model to output the target answer corresponding to the initial query question, can improve the accuracy of the retrieval enhancement generation answer, improve the user experience, and solve the technical problems mentioned in the background art. Attached Figure Description

[0011] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the technical solutions of this application.

[0012] Figure 1 A schematic diagram of the implementation flow of an existing RAG system provided in one embodiment of this application;

[0013] Figure 2 A schematic diagram of the system architecture of an electronic device provided in one embodiment of this application;

[0014] Figure 3 A schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in one embodiment of this application;

[0015] Figure 4 A schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in another embodiment of this application;

[0016] Figure 5 This is a schematic diagram of the coarse flow splitting architecture provided in one embodiment of this application;

[0017] Figure 6 A schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in yet another embodiment of this application;

[0018] Figure 7 This is a schematic diagram of the architecture for fine stream splitting processing provided in one embodiment of this application;

[0019] Figure 8 A schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in yet another embodiment of this application;

[0020] Figure 9 A schematic diagram of the workflow architecture for problem retrieval provided in one embodiment of this application;

[0021] Figure 10 A schematic diagram of an overall exemplary process framework for a method of generating search enhancements provided in one embodiment of this application;

[0022] Figure 11 A structural schematic diagram of a retrieval enhancement generation apparatus provided in one embodiment of this application;

[0023] Figure 12 This is a structural schematic diagram of a computer device provided in one embodiment of this application. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the specific technical solutions of this application will be further described in detail below with reference to the accompanying drawings of the embodiments of this application. The following embodiments are used to illustrate this application, but are not intended to limit the scope of this application.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing embodiments of this application only and is not intended to limit this application.

[0026] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0027] It should be noted that the terms "first, second, third" used in the embodiments of this application are used to distinguish similar or different objects and do not represent a specific order of objects. It can be understood that "first, second, third" can be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0028] With the development of artificial intelligence and big data technologies, retrieval-augmented generation (RAG)-based question answering systems have been widely used in various application scenarios. RAG is a technical framework that combines retrieval techniques and generative models, aiming to improve the performance of natural language processing (NLP) tasks, especially for tasks such as open-domain dialogue, question answering systems, and text generation.

[0029] For example, please refer to Figure 1 This is a schematic diagram illustrating the implementation flow of an existing RAG system according to an embodiment of this application, as shown below. Figure 1 As shown, firstly, the user-input query is vectorized. For example, an embedded model can be used to convert the query into a text vector. Then, the vectorized query is searched in a vector database using Top N similarity, which finds the top N most matching terms. The vector database is a database system used to store and retrieve vector data, obtained by extracting, parsing, and vectorizing text from data in a knowledge base. Next, the query and the top N most matching terms are integrated into a context to obtain prompt words. For example, the query and the top N most matching terms can be embedded together into a preset prompt word template. Finally, the prompt words are input into a large language model (LLM) to output the corresponding answer.

[0030] However, existing RAG systems have several shortcomings in use, such as missing knowledge base content, truncating useful documents when searching TopN, loss of contextual integration information, failure to identify useful information, problems with prompt word format, insufficient accuracy of answers, and incomplete answers, resulting in low accuracy of the output answers.

[0031] Therefore, how to improve the accuracy of the answers generated by search enhancement is an urgent problem to be solved.

[0032] In view of this, embodiments of this application provide a method for enhanced retrieval generation, applied to an electronic device. The method specifically includes: classifying a received initial query question by type; determining a target question type, where the target question type includes at least one of document-type questions and non-document-type questions; then using a target model corresponding to the target question type to output a target answer corresponding to the initial query question. The target model is one of preset models, including document models and non-document models. The document model is used to obtain answers to document-type questions, and the non-document model is used to obtain answers to non-document-type questions. In the enhanced retrieval generation method of this application, by classifying the initial query question by type and then using the corresponding target model to output a target answer corresponding to the initial query question, the accuracy of the enhanced retrieval answer can be improved, thus enhancing the user experience.

[0033] It should be noted that the execution subject of the search enhancement generation method provided in this application can be an electronic device such as a smartphone, tablet computer, smart speaker, smart TV, computer, server or server cluster, or a specially designed knowledge or document question-and-answer device, or a question-and-answer device set in the above-mentioned electronic device. The question-and-answer device can be implemented by software, hardware or a combination of both, and the embodiments of this application are not limited in this regard.

[0034] For example, Figure 2 This is a schematic diagram of the system architecture of an electronic device provided in one embodiment of this application. Figure 2 As shown, the electronic device includes components such as a processor 210, a memory 220, a transceiver 230, a display unit 240, an input unit 250, and a power module 260.

[0035] The processor 210 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs and / or modules stored in the memory 220, and by calling data stored in the memory 220, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 210 may include one or more processing units; optionally, the processor 210 may integrate an application processor, which mainly handles operating devices, user interfaces, and application programs. Of course, it may also include other processors, which are not listed here.

[0036] The memory 220 can be used to store software programs and modules. The processor 210 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 220. The memory 220 mainly includes a program storage area and a data storage area. The program storage area can store the operating device and application programs required for at least one function (such as sound playback function, image playback function, etc.); the data storage area can store data created according to the use of the electronic device (such as audio data, telephone book, etc.). In addition, the memory 220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0037] Transceiver 230 can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies. Transceiver 230 can be one or more devices integrating at least one communication processing module; for example, it can integrate an antenna with a baseband processor, or it can integrate an antenna with a modem processor, etc., without limitation.

[0038] The display unit 240 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device. The display unit 240 can be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like, and there is no limitation herein.

[0039] The input unit 250 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Specifically, the input unit 250 can collect user operations on or near it and drive corresponding connected devices according to a pre-set program. Furthermore, the input unit 250 may include a touch panel, which can be implemented using various types such as resistive, capacitive, infrared, and surface acoustic wave touch panels. In addition to the touch panel, the input unit 250 may also include other input devices. Specifically, other input devices may include, but are not limited to, one or more of the following: function keys (such as volume control buttons, power buttons, etc.), trackballs, joysticks, etc.

[0040] The electronic device also includes a power module 260 that supplies power to the various components. Optionally, the power module 260 can be logically connected to the processor 210 through a power management device, thereby enabling the power management device to manage functions such as charging, discharging, and power consumption.

[0041] Although not shown, the electronic device may also include a camera. Optionally, the camera may be positioned in the front or rear of the electronic device, and this application embodiment does not limit this.

[0042] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0043] To make the objectives and technical solutions of this application clearer and more intuitive, the search enhancement generation method, apparatus, device, and storage medium provided in the embodiments of this application will be described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are merely illustrative of this application and are not intended to limit this application.

[0044] Please refer to Figure 3 This is a schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in one embodiment of this application. This method can be applied to, for example... Figure 2 The electronic devices shown, such as Figure 3 As shown, the method may include the following steps 301 and 302:

[0045] Step 301: Classify the received initial query questions by type and determine the target question type. The target question type includes at least one of document-type questions and non-document-type questions.

[0046] In some embodiments, the electronic device can first perform text preprocessing on the received initial query question, such as removing redundant spaces, punctuation marks, and special characters. Then, it can perform word segmentation and part-of-speech tagging based on the input language to better understand the role of words, such as nouns, verbs, and adjectives. Further, it can perform keyword extraction, syntactic analysis, and named entity recognition on the initial query question. Finally, it can classify the query question using machine learning or rule-based methods. For example, document-based questions typically involve specific information requests, file searches, and text queries, such as "How do I write a paper?" or "What is quantum mechanics?". Non-document-based questions involve other types of needs, such as behavioral, perceptual, or general questions, such as "What's the weather like today?" or "How do I get to the nearest hospital?".

[0047] Optionally, the context of the initial query can be used to help determine the question type. If the question is within a specific service or application scenario, the classification may be dynamically adjusted based on the context. A rule-based classifier can also be designed, which sets a set of rules to determine the question type. For example, if the question contains keywords such as "how" or "how to," it may be considered a document-based question; if it contains keywords such as "where" or "who," it may be considered a non-document-based question. Alternatively, a text classification model can be trained to automatically identify the question type, etc. This application does not limit the scope of this approach.

[0048] In other words, electronic devices can preprocess the initial query question, extract features, analyze question intent, apply classification models, and classify it to determine the target question type.

[0049] Step 302: Using the target model corresponding to the target question type, output the target answer corresponding to the initial query question. The target model is one of the preset models, which include document models and non-document models. The document model is used to obtain the answer to document-type questions, and the non-document model is used to obtain the answer to non-document-type questions.

[0050] In some embodiments, based on the target question type determined in step 301, a target model corresponding to the target question type, such as a document model or a non-document model, is selected and invoked to generate the answer to the question. The document model is suitable for questions that require extracting information from a large number of documents, as well as some long text questions, while the non-document model focuses on handling behavioral, contextual, and common-sense questions, typically generating answers through common-sense reasoning or dialogue systems.

[0051] In some embodiments, for document-related questions, the document model is invoked, and the system retrieves relevant documents from a knowledge base or document database to generate an answer. For non-document-related questions, the non-document model is invoked, and the system uses common-sense reasoning, a dialogue system, or an external service interface to generate an answer.

[0052] In this embodiment, the electronic device first classifies the received initial query question to determine the target question type. The target question type includes at least one of document-type questions and non-document-type questions. Then, using a target model corresponding to the target question type, it outputs the target answer corresponding to the initial query question. The target model is one of a set of preset models, including document models and non-document models. The document model is used to obtain answers to document-type questions, and the non-document model is used to obtain answers to non-document-type questions. This retrieval enhancement generation method, by classifying the initial query question and then using the corresponding target model to output the target answer corresponding to the initial query question, can improve the accuracy of the retrieval enhancement generated answer and enhance the user experience.

[0053] Based on the above embodiments, Figure 4 A schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in another embodiment of this application is shown below. Figure 4 As shown, the method may include steps 401 and 402:

[0054] Step 401: Perform coarse splitting on the initial query question to determine the target question type. The coarse splitting is used to determine the target question type based on the intent classification and recognition algorithm and the relevance judgment method.

[0055] Among them, the intent classification and recognition algorithm is used to identify the intent of the initial query question, and the relevance judgment method is used to match the initial query question with documents in the knowledge base to obtain the relevance.

[0056] In some embodiments, the electronic device uses an intent classification and recognition algorithm to identify the intent behind the initial query question, determining whether the initial query question is an intent-based question. The goal of the intent classification and recognition algorithm is to identify the intent behind the query question, that is, to determine the type of answer the user wants to obtain or the task they want to perform.

[0057] Optionally, intent classification and recognition algorithms include rule-based algorithms and model-based algorithms. Rule-based algorithms quickly determine the category of the question using specific rules, such as keyword matching and regular expressions, especially in standardized scenarios. Model-based algorithms can be based on text classification models or deep learning models. Text classification models use pre-trained classifiers, such as Support Vector Machines (SVM), XGBoost, and Bidirectional Encoder Representations from Transformers (BERT), to classify the initial query question. The system inputs the initial query question into the text classification model and outputs category labels, such as "information query" or "operation guidance." Deep learning models use neural networks, such as Transformers, to perform end-to-end text understanding and automatically determine the user's intent.

[0058] In some embodiments, the electronic device uses a relevance assessment method to determine the initial relevance between the query question and documents in the knowledge base. Non-document-type questions include intent-type questions. The relevance assessment method matches the query question with documents in the knowledge base to evaluate the relevance of the query question to existing documents.

[0059] Optionally, common implementations of relevance assessment methods include summary-based relevance assessment methods and historical question-answering-based relevance assessment methods. Summary-based relevance assessment methods primarily match the initial query question with the summaries of documents in the knowledge base to obtain the corresponding relevance score. Historical question-answering-based relevance assessment methods primarily perform semantic matching between the initial query question and related historical questions and answers to obtain the corresponding relevance score. Of course, relevance assessment methods can also include keyword matching methods and deep learning model-based methods; this application does not limit these methods.

[0060] Based on the above description of coarse splitting processing, please refer to... Figure 5 This is a schematic diagram of the coarse flow splitting architecture provided in one embodiment of this application, as shown below. Figure 5 As shown, for the initial query question, the target question type is determined by an intent classification and identification algorithm and a relevance judgment method. The intent classification and identification algorithm includes rule-based algorithms and model-based algorithms, and the relevance judgment method includes a summary-based relevance judgment method and a history-based question answering relevance judgment method.

[0061] Furthermore, if the initial query question is determined to be an intent-based question and the first relevance is less than the first relevance threshold, the target question type is determined to be a non-document-based question. The first relevance threshold can be set based on experience or preset in advance; this application does not impose any restrictions on this.

[0062] Step 402: Using the target model corresponding to the target question type, output the target answer corresponding to the initial query question. The target model is one of the preset models, which include document models and non-document models. The document model is used to obtain the answer to document-type questions, and the non-document model is used to obtain the answer to non-document-type questions.

[0063] In some embodiments, after coarsely sorting the initial query to determine the target question type, a target model corresponding to the target question type can be selected and invoked, such as a document model or a non-document model, to generate the answer to the question. Document models are suitable for questions that require extracting information from a large number of documents, as well as some long text questions, while non-document models focus on handling behavioral, contextual, and common-sense questions, typically generating answers through common-sense reasoning or dialogue systems.

[0064] In this embodiment, the electronic device performs coarse-streaming processing on the initial query question to determine the target question type. This coarse-streaming process uses an intent classification and recognition algorithm and a relevance assessment method to determine the target question type. Then, it uses the target model corresponding to the target question type to output the target answer corresponding to the initial query question. By performing intent and relevance assessment on the user's question based on coarse-streaming processing, the target question type can be comprehensively determined. This type is then input into the corresponding target model to ultimately obtain the answer, which can improve the accuracy of the enhanced response generated by the search.

[0065] Based on the above embodiments, Figure 6 This is a schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in another embodiment of this application, as shown below. Figure 6 As shown, the method may include the following steps 601 and 602:

[0066] Step 601: Perform fine-tuning on the initial query question to determine the target question type. Fine-tuning is used to determine the target question type based on model judgment method and relevance judgment method.

[0067] Among them, the model judgment method is used to classify the initial query question based on the summary model, translation model and table model, and the relevance judgment method is used to match the initial query question with the documents in the knowledge base to obtain the relevance.

[0068] It should be noted that document-related problems include multiple document types, such as summary documents, translation documents, and table documents, each with a different model.

[0069] In some embodiments, the electronic device uses a model-based judgment method to determine whether the initial query question is a document-type question corresponding to the model, and the corresponding first confidence level. For example, based on a summary model, such as the BERT model, it determines whether the initial query question is a summary document type question and the corresponding confidence level; based on a translation model, it determines whether the initial query question is a translation document type question and the corresponding confidence level; based on a table model, it determines whether the initial query question is a table document type question and the corresponding confidence level.

[0070] Furthermore, when the initial query question is a document-type question that does not correspond to the model, a relevance judgment method is used to determine the second relevance between the initial query question and the documents in the knowledge base. The document-type question that does not correspond to the model is used to indicate that the first confidence score is less than the first confidence score threshold. Based on the second relevance score, it is determined whether the initial query question is a document-type question and the corresponding second confidence score. Finally, if the second relevance score is greater than the second relevance threshold and the second confidence score is greater than the second confidence score threshold, the target question type is determined to be a document-type question. The first confidence score threshold and the second relevance threshold can be set based on experience or preset in advance; this application does not impose any limitations on this.

[0071] The relevance assessment methods include a text feature matching method based on TopN retrieval and a relevance assessment method based on historical question-answering. TopN retrieval selects the N most relevant documents from a large number of documents based on certain ratings or relevance scores. The text feature matching method based on TopN retrieval matches the initial query with the retrieved documents to determine the second relevance. The relevance assessment method based on historical question-answering semantically matches the initial query with related historical questions and answers to obtain the second relevance.

[0072] Furthermore, based on the second relevance, it is determined whether the initial query question is a document-type question and the corresponding second confidence level. Optionally, if the second relevance is greater than the second relevance threshold, the initial query question is determined to be a document-type question. The confidence level for determining that the initial query question is a document-type question can be achieved by combining text similarity calculation, a deep learning-based semantic understanding model, and a classification and regression model.

[0073] Based on the above description of the fine-tuning process, please refer to... Figure 7 This is a schematic diagram of the fine stream splitting processing architecture provided in one embodiment of this application, as shown below. Figure 7As shown, for the initial query question, the following methods are used in sequence: a summary model-based judgment method, a translation model-based judgment method, a table model-based judgment method, a TopN retrieval-based text feature matching judgment method, and a historical question-answer relevance judgment method to classify the initial query question into types, thereby obtaining the target question type and the corresponding confidence level.

[0074] Step 602: Using the target model corresponding to the target question type, output the target answer corresponding to the initial query question. The target model is one of the preset models, which include document models and non-document models. The document model is used to obtain the answer to document-type questions, and the non-document model is used to obtain the answer to non-document-type questions.

[0075] In some embodiments, after performing fine-tuning on the initial query question to determine the target question type, a target model corresponding to the target question type can be selected and invoked, such as a document model or a non-document model, to generate the answer to the question. Document models are suitable for questions that require extracting information from a large number of documents, as well as some long text questions, while non-document models focus on handling behavioral, contextual, and common-sense questions, typically generating answers through common-sense reasoning or dialogue systems.

[0076] In this embodiment, the electronic device performs fine-tuning on the initial query question to determine the target question type. Fine-tuning is used to determine the target question type based on model judgment and relevance judgment methods. Then, using the target model corresponding to the target question type, it outputs the target answer corresponding to the initial query question. By performing model judgment and relevance judgment on the user question based on fine-tuning, the target question type can be comprehensively determined. Then, by inputting the corresponding target model, the answer is finally obtained, which can improve the accuracy of the enhanced response generated by the search.

[0077] Based on the above embodiments, Figure 8 This is a schematic diagram illustrating the implementation flow of a retrieval enhancement generation method provided in another embodiment of this application, as shown below. Figure 8 As shown, the method may include the following steps 801 to 809:

[0078] Step 801: Use an intent classification and recognition algorithm to identify the intent of the initial query question, determine whether the initial query question is an intent-type question, and use a relevance judgment method to determine the first relevance between the initial query question and the documents in the knowledge base. Non-document-type questions include intent-type questions.

[0079] In some embodiments, intent classification and recognition algorithms include rule-based algorithms and model-based algorithms. Rule-based algorithms quickly determine the category of the question using specific rules, such as keyword matching or regular expressions, especially in standardized scenarios. Model-based algorithms can be based on text classification models or deep learning models.

[0080] Relevance assessment methods include summary-based relevance assessment methods and historical question-answering-based relevance assessment methods. The summary-based relevance assessment method mainly matches the initial query question with the summaries of documents in the knowledge base to obtain the corresponding relevance score. The historical question-answering-based relevance assessment method mainly performs semantic matching between the initial query question and historical questions and answers related to the initial query question to obtain the corresponding relevance score.

[0081] Step 802: If the first relevance is greater than the first relevance threshold, the initial query question is rewritten based on the historical questions and answers associated with the initial query question and the preset question rewriting model to obtain the target query question.

[0082] In some embodiments, after determining the first relevance using a relevance judgment method, if the first relevance is greater than the first relevance threshold, the initial query question is rewritten using historical questions and answers. For example, based on selected relevant historical question and answer pairs, the keywords, expressions, and grammatical structures can be analyzed to rewrite the query question, making it more precise in expression and easier to match relevant answers. Alternatively, rules or models can be used to extract expression patterns from historical questions and apply them to the initial query question. Furthermore, key information related to the query can be found in the answers to historical questions, and the query question can be rewritten accordingly.

[0083] Furthermore, building upon question rewriting based on historical question-answer pairs, a pre-defined question rewriting model plays a crucial role. Question rewriting models are typically deep learning-based natural language generation models, capable of rewriting queries with higher quality based on historical data and context. Common rewriting models include: Seq2Seq-based models, which use sequence-to-sequence models for question rewriting; these models are usually trained on large amounts of question-answer data and can generate rewritten questions relevant to the input query. Transformer-based models, such as BERT, Text-to-Text Transfer Transformer (T5), or Generative Pretrained Transformer (GPT), can capture the contextual information of the query and generate related rewritten questions. Models like T5, in particular, perform exceptionally well in various text generation tasks. Custom question rewriting models, trained specifically for question rewriting, are fine-tuned using a large number of historical question-answer pairs to generate more accurate and context-aware target query questions.

[0084] In this embodiment, the target query question is obtained by rewriting the initial query question. The target query question can more accurately express the user's intent and reduce ambiguity. The target query question will optimize the use of keywords or adopt more common question phrasing, making it easier to retrieve relevant answers. By introducing the context of historical questions and answers, the target query question can better match the actual information.

[0085] Step 803: Based on the initial query question and the target query question, a search is performed in the preset knowledge base to obtain the second search result, which includes at least three different types of terms related to the target query question.

[0086] The preset knowledge base includes a keyword library, a text vector library, and a knowledge graph (KG) library.

[0087] Keyword libraries are typically used to store and manage keywords, phrases, or concepts related to specific topics. They help systems quickly identify, match, or classify information in text processing. Keyword libraries accelerate the search and retrieval process, especially with large-scale text data. Keyword libraries are usually built by extracting keywords or phrases from existing documents, articles, reports, question-and-answer pairs, etc. Natural language processing techniques, such as term frequency-inverse document frequency or word embedding models, can be used to extract important words. These keywords are then stored in a database according to topics or categories, supporting rapid retrieval. The keyword library is updated regularly or through triggering mechanisms to ensure it stays up-to-date with the latest content. Web crawling techniques can be used to extract newly emerging keywords from the internet.

[0088] A text vector library is a repository that converts text data into vectors (numerical representations) and stores them, typically using machine learning models such as BERT, Word2Vec, and FastText. These vectors can be used for various tasks, such as text similarity calculation, semantic matching, and classification. For example, pre-trained language models such as BERT, GPT, and T5, or custom-trained models, can be used to convert text into fixed-dimensional vectors, and sentence-level or paragraph-level vector representations can also be employed. The generated vectors are stored in databases that efficiently support the storage and similarity search of large-scale vectors. Similarity retrieval is performed by calculating methods such as cosine similarity or Euclidean distance between text vectors. For instance, when a user enters a query, the system can quickly calculate the similarity between the input text and all texts in the library and return the most relevant text.

[0089] Knowledge graphs are used to store and represent relationships between different entities, such as people, places, events, and concepts. These entities and relationships are typically represented in the form of a graph, where nodes represent entities and edges represent relationships. Knowledge graphs help systems understand complex semantic relationships. For example, entity and relationship graphs can be constructed using techniques such as knowledge extraction, entity recognition, and relationship extraction. Existing tools like OpenIE and spaCy can also be used to extract entities and relationships from structured or unstructured text. Knowledge graphs are typically stored using graph databases such as Neo4j and GraphDB, which support efficient graph data storage, querying, and graph algorithms. Knowledge graphs support reasoning capabilities, allowing new knowledge to be derived from known relationships. For instance, a system can infer a relationship not explicitly present in the data based on known facts in the graph. Combined with natural language processing techniques, the system can achieve intelligent question answering based on knowledge graphs.

[0090] In some embodiments, keyword libraries help quickly identify topics and domains, typically used for information retrieval and filtering; text vector libraries perform semantic matching by vectorizing text data, enabling them to handle more complex natural language understanding tasks; and knowledge graph libraries provide a deeper understanding of entities and relationships at a higher level, supporting reasoning and knowledge graph-driven applications such as intelligent question answering systems and recommendation systems. When these three are used in combination, efficient semantic search, recommendation, and intelligent reasoning can be performed on large-scale data, helping AI systems improve their understanding capabilities and application effectiveness.

[0091] In some embodiments, keywords in the target query can be matched with terms in a keyword library to obtain relevant terms; text vectorization techniques (such as BERT, Word2Vec, etc.) can be used to calculate the similarity between the target query and terms in the knowledge base to obtain semantically related terms; entities and relationships related to the target query can be inferred from the knowledge graph to find directly or indirectly related terms; finally, the retrieved terms of different types can be combined into a second search result. These terms not only cover concepts and technical methods, but also tools and platforms, providing users with multi-dimensional information.

[0092] Based on the above description of searching in the preset knowledge base, please refer to... Figure 9 This is a schematic diagram of the workflow architecture for problem retrieval provided in one embodiment of this application, as shown below. Figure 9 As shown, for the initial query question, the question is first rewritten, and then an index retrieval is performed. The index retrieval includes keyword retrieval, text vector retrieval, and knowledge graph retrieval, respectively obtaining the corresponding top N relevant terms.

[0093] In one possible implementation, without rewriting the initial query question, a search can be performed in a preset knowledge base based on the initial query question to obtain a first search result, which includes at least three different types of terms related to the initial query question.

[0094] In this embodiment, by introducing a keyword library and a knowledge graph library into the knowledge base, the certainty of knowledge can be improved, knowledge can be dynamically and efficiently integrated, and vector / knowledge graph / keyword retrieval based on user queries can be performed to supplement rich relation representations and semantic hierarchical context, further improving the accuracy and professionalism of model output, realizing in-depth mining and global grasp of information structure, and enhancing the system's ability to handle complex semantic relationships and multi-topic issues.

[0095] Step 804: Based on a preset reordering algorithm, sort at least three different types of terms related to the target query question by relevance to obtain the target ranking result.

[0096] It should be understood that reordering algorithms are typically a process of further sorting search results. Their purpose is to optimize the order of search results through more refined evaluation and algorithmic adjustments, so that the content ultimately displayed to users better meets their needs and expectations. Reordering algorithms can be categorized based on their strategies and model types, such as rule-based reordering, machine learning-based reordering, deep learning-based reordering, and hybrid methods.

[0097] Rule-based reordering adjusts the ranking of results by manually setting rules; for example, a higher score is assigned to a title containing the keyword "performance." This method is simple and easy to understand, but it struggles with complex situations and often requires frequent rule adjustments. Machine learning-based reordering automatically learns the impact of different features on ranking through training data. Common machine learning methods include linear models, decision trees, random forests, and ranking learning methods. Decision trees and random forests use tree structures to classify and rank results. Deep learning-based reordering typically requires large amounts of training data and high computational resources, but it can provide more accurate ranking results. Hybrid methods combine multiple algorithms. For example, simple rule-based filtering can be used first, followed by machine learning methods for fine-tuning the candidate ranking, or a combination of deep learning and traditional methods can be used to achieve the best ranking results.

[0098] For example, the preset reordering algorithms may include RankNet, LambdaMART, BERT, and Transformer, etc.

[0099] In this embodiment, at least three different types of terms related to the target query question are sorted by relevance based on a preset reordering algorithm, which can enhance the relevance of contextual information and construct richer and more accurate contextual information.

[0100] Step 805: Based on the initial query question, the target query question, and the target ranking result, perform fine-tuning on the initial query question to determine the target question type.

[0101] In some embodiments, the fine-grained flow processing is used to determine the target question type based on a model-based judgment method and a relevance-based judgment method. The model-based judgment method is used to classify the initial query question based on a summary model, a translation model, and a table model, while the relevance-based judgment method is used to match the initial query question with documents in the knowledge base to obtain a relevance score.

[0102] In some embodiments, the electronic device determines the target question type based on the initial query question, the target query question, and the target ranking result using a model judgment method and a relevance judgment method. The specific implementation process can be found in step 601 and will not be repeated here. In this embodiment, the refined stream processing also includes the target query question and the target ranking result, resulting in a more accurate target question type and confidence level.

[0103] In one possible implementation, without rewriting the initial query question, the initial query question can be finely segmented to determine the target question type. That is, based on the first search result mentioned above, the initial query question is finely segmented to determine the target question type.

[0104] Step 806: Construct target prompts. Target prompts are obtained based on the target query question, target sorting results, target question type, and preset type prompt templates.

[0105] The preset type prompt word templates include at least one of the following: general document type prompt word template, table type prompt word template, translation type prompt word template, and summary type prompt word template.

[0106] For example, a general document type prompt template might be: Using a document model, here is my question: XXX (followed by some retrieved and sorted terms), please answer my question based on this information. A summary type prompt template might be: Using a document model, please output a summary based on the following question: XXX (followed by some retrieved and sorted terms).

[0107] In some embodiments, the electronic device first selects the corresponding type prompt word template according to the target question type, and then fills the target query question and target ranking results into the template to obtain the target prompt words, so that the subsequent model can better understand the user's question and improve the accuracy of question answering.

[0108] Step 807: Input the target prompt words into the target model to obtain the target answer.

[0109] In some embodiments, when the target question type is a document-type question, the target model is a document model; one way to obtain the target answer by inputting the target prompt words into the target model is to input the target prompt words into the document model to obtain the target answer.

[0110] When the target question type is a non-document question, the target model is a non-document model. One way to obtain the target answer by inputting the target prompt words into the target model is to input the target prompt words into the non-document model to obtain the target answer.

[0111] Step 808: Determine the confidence level of the target answer.

[0112] It should be understood that confidence score is typically used to measure how confident a model is in a particular prediction (such as text classification or entity recognition). For example, in a question-answering system, a higher confidence score indicates that the model believes its answer is more accurate. The calculation method for confidence score usually depends on the model's output. The representation and calculation method of confidence score will differ for different types of models. Generally, confidence score is obtained based on the model's predicted probabilities.

[0113] In some embodiments, the confidence level of the target answer can be determined based on the predicted probability of the target model.

[0114] Step 809: If the confidence level of the target answer is lower than the third confidence level threshold, recall the target answer based on the preset recall model.

[0115] It should be understood that pre-defined recall models typically refer to recall strategies designed based on pre-defined rules, features, and algorithms. Unlike dynamically trained models, pre-defined recall models do not rely on online training or real-time updates during execution; instead, they perform retrieval and candidate selection according to fixed strategies. Recall models usually utilize simple rules, keyword matching, vector retrieval, and other methods for initial screening to ensure that the candidate set covers a sufficiently large amount of potentially relevant content.

[0116] For example, recall models can include rule-based recall models, vector space model-based recall models, sparse vector-based recall methods, and learning-based recall models. Rule-based recall models rely on manually defined rules or logic to filter candidate options. These rules are typically simple conditional judgments, such as keyword matching, phrase matching, and category filtering. Vector space models are a common information retrieval method where documents and queries are represented as vectors. Recall models based on vector space models filter candidate options based on the similarity between the query and document vectors. Sparse vector-based recall methods represent documents and queries as sparse vectors, such as the bag-of-words model, and perform matching based on these vectors; common recall techniques include inverted indexes and locality-sensitive hashing. Learning-based recall models leverage machine learning models to filter candidate options. For example, by analyzing a user's historical behavior and the behavior of other users, they recommend content similar to the user's interests; by learning patterns in historical data, decision trees or random forests can help identify candidate options related to the user's query or interests.

[0117] In some embodiments, when the target question type is a document-type question, the target prompt words are input into the document model to obtain the target answer. Further, if the confidence level of the obtained target answer is lower than the third confidence threshold, the target answer is recalled based on a preset recall model, and the target prompt words are input into the non-document model to obtain the target answer.

[0118] In other words, if the confidence level of the output target answer is low, for example, if the target answer is completely irrelevant to the target query question, then a pre-defined recall model is used to recall the target answer, and a general non-document model, such as the LLM model, is used to output a new answer to ensure the accuracy and completeness of the response.

[0119] In this embodiment, the electronic device uses an intent classification and recognition algorithm to identify the intent of the initial query question, determining whether the initial query question is an intent-type question. It also uses a relevance judgment method to determine the first relevance between the initial query question and documents in the knowledge base. If the first relevance is greater than a first relevance threshold, the initial query question is rewritten based on historical questions and answers associated with the initial query question and a preset question rewriting model to obtain the target query question. Based on the initial query question and the target query question, a search is performed in the preset knowledge base to obtain a second search result. The second search result includes at least three different types of documents related to the target query question. The system first identifies the terms and then ranks at least three different types of terms related to the target query question based on a pre-defined reordering algorithm to obtain the target ranking result. Based on the initial query question, the target query question, and the target ranking result, the initial query question is further processed by fine-tuning to determine the target question type. Then, target prompt words are constructed based on the target query question, the target ranking result, the target question type, and a pre-defined type prompt word template. Finally, the target prompt words are input into the target model to obtain the target answer, and the confidence level of the target answer is judged. If the confidence level of the target answer is lower than the third confidence level threshold, the target answer is recalled based on a pre-defined recall model.

[0120] The retrieval enhancement generation method provided in this application rewrites the initial query question based on historical questions and answers and a preset question rewriting model to form the target query question. The use of historical questions and answers guides the system to understand the patterns and context of similar questions, thereby generating query questions that better meet user needs, enhancing the system's contextual understanding ability, and helping to rewrite the question more accurately and improve the relevance of subsequent searches. After rewriting, the initial query question is searched in a preset knowledge base. The second search result returned by this search includes at least three different types of terms related to the target query question. These terms cover relevant content from different angles, ensuring that the user's question can be answered from multiple dimensions. Simultaneously, through diversified term retrieval, the system can generate richer candidate answers, improving its responsiveness to diverse queries. A preset reordering algorithm is used to reorder terms related to the target query question. Ranking related terms by relevance significantly improves the accuracy of search results and ensures that the system provides answers that better match user needs. Based on the target query and the reordered results, the system performs fine-tuning to further determine the specific type of the target question. This helps the system to deeply analyze the query, refining it into more precise types, thus generating more accurate answers. After receiving target prompts, the target model can generate more accurate answers. Finally, after obtaining the target answer, the system assesses its confidence level. If the confidence level is lower than a preset third confidence threshold, a recall model is used for supplementation and optimization. This step further enhances the reliability and accuracy of the answer by re-searching related content and combining it with additional recall strategies. The recall mechanism provides additional support when the initial answer generation is unsatisfactory, ensuring that users receive more accurate answers.

[0121] In summary, through multi-stage intent recognition, question rewriting, retrieval, ranking, and refined processing, more precise target query questions can be generated, and highly relevant search results can be provided, thereby improving the accuracy of the answers. Utilizing the correlation between historical questions and answers, coupled with a question rewriting model, the context of the user's query can be better understood, improving the depth of question understanding. Different types of relevance ranking and reordering algorithms can efficiently filter the most relevant answers and avoid information redundancy. Even when the initial answer confidence is low, the system can still further improve the reliability of the answer through a recall mechanism, preventing users from receiving inaccurate answers and thus improving the overall experience. In conclusion, this embodiment, through a refined question understanding and answer generation process, can provide more accurate, efficient, and highly reliable question-answering services, especially suitable for handling complex questions and long-tail query scenarios.

[0122] Based on the above embodiments, Figure 10This is a schematic diagram of the overall exemplary process framework of a retrieval enhancement generation method provided in one embodiment of this application, as shown below. Figure 10 As shown, for the initial query question, a coarse splitting process is first performed. If the coarse splitting process determines that the initial query question is a non-document question, then the initial query question is input into the non-document model to output the target answer. If the first relevance obtained after the coarse splitting process is greater than the first relevance threshold, the question is rewritten to obtain the target query question. Then, the target query question is searched in the preset knowledge base using TopN search to obtain at least three different types of terms related to the target query question. For the knowledge base, the data in the knowledge base is first extracted into text, then parsed and segmented, and then keyword extraction, text vectorization, and knowledge graph generation are performed to obtain the corresponding keyword library, text vector library, and knowledge graph library. Furthermore, after performing a TopN search on the target query question in a pre-defined knowledge base to obtain at least three different types of terms related to the target query question, these terms are then mixed and rearranged to obtain a target ranking result. Based on the initial query question, the target query question, and the target ranking result, the initial query question undergoes fine-tuning to determine the target question type. If the target question type is determined to be a non-document question, the target query question and at least three different types of terms related to the target query question are input into the non-document model to obtain the corresponding answer. If the target question type is determined to be a document question, target prompts are constructed based on the target query question, the target ranking result, the target question type, and a pre-defined type prompt template. These prompts are then input into the document model to obtain the target answer. If the confidence level of the target answer is lower than the third confidence threshold, the target answer is recalled based on a pre-defined recall model and re-input into the non-document model to obtain the target answer. If the confidence level of the target answer is higher than the third confidence threshold, the target answer is output.

[0123] It should be noted that, Figure 10 For detailed information on the methods involved, please refer to the above. Figure 3 , Figure 4 , Figure 6 and Figure 8 The methods described in the embodiments shown are not further elaborated herein.

[0124] In summary, the retrieval enhancement generation method in this application embodiment can improve the accuracy and completeness of enhanced knowledge retrieval and generation, and improve user experience. It introduces a variety of optimization techniques, including question rewriting, knowledge graph introduction, question type classification, hybrid rearrangement, and prompt word construction, which improve the system's response quality from multiple levels.

[0125] It should be understood that although the steps in the above flowcharts are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the above flowcharts may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0126] Based on the foregoing embodiments, this application provides a retrieval enhancement generation device, which includes the included modules and the units included in each module, and can be implemented by a processor; of course, it can also be implemented by specific logic circuits; in the implementation process, the processor can be a central processing unit, a microprocessor, a digital signal processor or a field programmable gate array, etc.

[0127] Figure 11 This is a structural schematic diagram of a retrieval enhancement generation apparatus provided in one embodiment of this application, as shown below. Figure 11 As shown, the device 1100 includes a determining module 1101 and an output module 1102, wherein:

[0128] The determining module 1101 is used to classify the received initial query question into types and determine the target question type, wherein the target question type includes at least one of document-type questions and non-document-type questions; the output module 1102 is used to output the target answer corresponding to the initial query question using a target model corresponding to the target question type, wherein the target model is one of the preset models, wherein the preset models include a document model and a non-document model, the document model is used to obtain the answer to the document-type question, and the non-document model is used to obtain the answer to the non-document-type question.

[0129] In some embodiments, the determining module 1101 is specifically used to: perform coarse splitting processing on the initial query question to determine the target question type. The coarse splitting processing is used to determine the target question type based on an intent classification and recognition algorithm and a relevance judgment method. The intent classification and recognition algorithm is used to identify the intent of the initial query question. The relevance judgment method is used to match the initial query question with documents in the knowledge base to obtain relevance.

[0130] In some embodiments, the determining module 1101 is specifically configured to: use the intent classification and recognition algorithm to perform intent recognition on the initial query question, determine whether the initial query question is an intent-type question, and use the relevance judgment method to determine the first relevance between the initial query question and documents in the knowledge base, wherein the non-document-type question includes the intent-type question; and if the initial query question is the intent-type question and the first relevance is less than the first relevance threshold, determine that the target question type is a non-document-type question.

[0131] In some embodiments, the document-type question includes multiple document types, including summary document types, translation document types, and table document types. Each document type corresponds to a different model. The determining module 1101 is specifically used to: perform fine-segmentation processing on the initial query question to determine the target question type. The fine-segmentation processing is used to determine the target question type based on a model judgment method and a relevance judgment method. The model judgment method is used to classify the initial query question based on a summary model, a translation model, and a table model. The relevance judgment method is used to match the initial query question with documents in the knowledge base to obtain relevance.

[0132] In some embodiments, the determining module 1101 is specifically used to: use the model judgment method to determine whether the initial query question is a document class question corresponding to the model, and the corresponding first confidence level;

[0133] If the initial query question is a document-type question that does not correspond to the model, the relevance judgment method is used to determine the second relevance between the initial query question and the documents in the knowledge base. The document-type question that does not correspond to the model is used to indicate that the first confidence score is less than the first confidence score threshold. Based on the second relevance score, it is determined whether the initial query question is a document-type question and the corresponding second confidence score. If the second relevance score is greater than the second relevance threshold and the second confidence score is greater than the second confidence score threshold, the target question type is determined to be a document-type question.

[0134] In some embodiments, the determining module 1101 is further configured to perform fine-tuning of the initial query question to determine the target question type when the first relevance is greater than the first relevance threshold.

[0135] In some embodiments, the apparatus further includes a retrieval module. The retrieval module is configured to perform a retrieval in a preset knowledge base based on the initial query question to obtain a first retrieval result. The preset knowledge base includes a keyword database, a text vector database, and a knowledge graph database. The first retrieval result includes at least three different types of terms related to the initial query question. The determining module 1101 is specifically configured to: perform fine-tuning processing on the initial query question based on the first retrieval result to determine the target question type.

[0136] In some embodiments, the apparatus further includes: a rewriting module. The rewriting module is configured to rewrite the initial query question based on historical questions and answers associated with the initial query question and a preset question rewriting model to obtain a target query question; the retrieval module is specifically configured to: perform a retrieval in the preset knowledge base based on the initial query question and the target query question to obtain a second retrieval result, the second retrieval result including at least three different types of terms related to the target query question.

[0137] In some embodiments, the apparatus further includes a sorting module. The sorting module is configured to sort the at least three different types of terms related to the target query question based on a preset rearrangement algorithm to obtain a target sorting result; the determining module 1101 is specifically configured to: perform fine-tuning processing on the initial query question based on the initial query question, the target query question, and the target sorting result to determine the target question type.

[0138] In some embodiments, the apparatus further includes a construction module. The construction module is configured to construct target prompt words, which are obtained based on the target query question, the target ranking result, the target question type, and a preset type prompt word template. The preset type prompt word template includes at least one of a general document type prompt word template, a table type prompt word template, a translation type prompt word template, and a summary type prompt word template. The output module 1102 is specifically configured to input the target prompt words into the target model to obtain the target answer.

[0139] In some embodiments, when the target question type is the document-type question, the target model is the document model; the apparatus further includes a processing module. The processing module is specifically configured to: input the target prompt word into the document model to obtain the target answer.

[0140] In some embodiments, the apparatus further includes a judgment module and a recall module. The judgment module is used to judge the confidence level of the target answer; the recall module is used to recall the target answer based on a preset recall model if the confidence level of the target answer is lower than a third confidence threshold.

[0141] In some embodiments, the processing module is further configured to input the target prompt word into the non-document model to obtain the target answer.

[0142] The descriptions of the above device embodiments are similar to those of the above method embodiments, and have similar beneficial effects. For technical details not disclosed in the device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0143] It should be noted that, in the embodiments of this application... Figure 11 The module division of the retrieval enhancement generation device shown is illustrative and represents only one logical functional division; in actual implementation, other division methods may be used. Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, exist as separate physical entities, or be integrated into one unit by two or more units. The integrated units described above can be implemented in hardware, as software functional units, or in a combination of software and hardware.

[0144] It should be noted that, in the embodiments of this application, if the above-described methods are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks. Thus, the embodiments of this application are not limited to any specific hardware and software combination.

[0145] This application provides a computer device, which may be a server, and its internal structure diagram may be as follows: Figure 12As shown, the computer device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data. The network interface communicates with external terminals via a network connection. When the computer program is executed by the processor, it implements the methods described above.

[0146] This application provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the method provided in the above embodiments.

[0147] This application provides a computer program product containing instructions that, when run on a computer, cause the computer to perform the steps in the method provided in the above-described method embodiments.

[0148] Those skilled in the art will understand that Figure 12 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0149] In one embodiment, the retrieval enhancement generation apparatus provided in this application can be implemented as a computer program, which can be implemented in, for example... Figure 12 The device operates on the computer device shown. The memory of the computer device can store the various program modules that make up the above-described apparatus. The computer program, composed of the various program modules, causes the processor to execute the steps of the methods in the various embodiments of this application described in this specification.

[0150] It should be noted that the descriptions of the storage medium and device embodiments above are similar to the descriptions of the method embodiments above, and have similar beneficial effects. For technical details not disclosed in the storage medium, storage medium, and device embodiments of this application, please refer to the descriptions of the method embodiments of this application for understanding.

[0151] It should be understood that the phrases "one embodiment," "an embodiment," or "some embodiments" mentioned throughout the specification mean that a specific feature, structure, or characteristic related to an embodiment is included in at least one embodiment of this application. Therefore, "in one embodiment," "in one embodiment," or "in some embodiments" appearing throughout the specification do not necessarily refer to the same embodiment. Furthermore, these specific features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that in the various embodiments of this application, the sequence numbers of the above-described processes do not imply a sequential order of execution; the execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application. The sequence numbers of the above-described embodiments are merely for descriptive purposes and do not represent the superiority or inferiority of the embodiments. The descriptions of the various embodiments above tend to emphasize the differences between the various embodiments; their similarities or commonalities can be referred to mutually, and for the sake of brevity, they will not be repeated here.

[0152] In this article, the term "and / or" is merely a description of the relationship between related objects, indicating that there can be three kinds of relationships. For example, object A and / or object B can represent three situations: object A exists alone, object A and object B exist simultaneously, and object B exists alone.

[0153] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes that element.

[0154] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The embodiments described above are merely illustrative. For example, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple modules or components can be combined, or integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed can be through some interfaces, and the indirect coupling or communication connection between devices or modules can be electrical, mechanical, or other forms.

[0155] The modules described above as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules. They may be located in one place or distributed across multiple network units. Some or all of the modules may be selected to achieve the purpose of this embodiment according to actual needs.

[0156] In addition, each functional module in the various embodiments of this application can be integrated into one processing unit, or each module can be a separate unit, or two or more modules can be integrated into one unit; the integrated modules can be implemented in hardware or in the form of hardware plus software functional units.

[0157] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, read-only memory (ROM), magnetic disks, or optical disks.

[0158] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROMs, magnetic disks, or optical disks.

[0159] The methods disclosed in the several method embodiments provided in this application can be arbitrarily combined without conflict to obtain new method embodiments.

[0160] The features disclosed in the several product embodiments provided in this application can be arbitrarily combined without conflict to obtain new product embodiments.

[0161] The features disclosed in the several method or device embodiments provided in this application can be arbitrarily combined without conflict to obtain new method or device embodiments.

[0162] The above description is merely an embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for enhancing retrieval generation, characterized in that, include: The received initial query questions are classified by type to determine the target question type, which includes at least one of document-type questions and non-document-type questions; Using a target model corresponding to the target question type, output the target answer corresponding to the initial query question. The target model is one of the preset models, which includes a document model and a non-document model. The document model is used to obtain the answer to the document-type question, and the non-document model is used to obtain the answer to the non-document-type question.

2. The method according to claim 1, characterized in that, The process of classifying the received initial query questions and determining the target question type includes: The initial query question is subjected to coarse splitting to determine the target question type. The coarse splitting is used to determine the target question type based on an intent classification and recognition algorithm and a relevance judgment method. The intent classification and recognition algorithm is used to identify the intent of the initial query question, and the relevance judgment method is used to match the initial query question with documents in the knowledge base to obtain the relevance.

3. The method according to claim 2, characterized in that, The coarse-streaming process for the initial query question to determine the target question type includes: The intent classification and recognition algorithm is used to identify the intent of the initial query question, determine whether the initial query question is an intent-type question, and the relevance judgment method is used to determine the first relevance between the initial query question and the documents in the knowledge base. The non-document-type questions include the intent-type questions. If the initial query question is an intent-type question and the first relevance is less than the first relevance threshold, the target question type is determined to be a non-document-type question.

4. The method according to claim 1, characterized in that, The document-related questions include multiple document types, such as summary documents, translation documents, and table documents. Each document type corresponds to a different model. The process of classifying the received initial query questions to determine the target question type includes: The initial query question is subjected to fine-tuning to determine the target question type. The fine-tuning is used to determine the target question type based on a model judgment method and a relevance judgment method. The model judgment method is used to classify the initial query question based on a summary model, a translation model, and a table model. The relevance judgment method is used to match the initial query question with documents in the knowledge base to obtain the relevance.

5. The method according to claim 4, characterized in that, The process of performing fine-tuning on the initial query question to determine the target question type includes: The model judgment method is used to determine whether the initial query question is a document class question corresponding to the model, and the corresponding first confidence level; If the initial query question is a document class question that does not correspond to the model, the relevance judgment method is used to determine the second relevance between the initial query question and the documents in the knowledge base. The document class question that does not correspond to the model is used to indicate that the first confidence score is less than the first confidence threshold. Based on the second relevance, determine whether the initial query question is a document-type question and the corresponding second confidence level; If the second relevance is greater than the second relevance threshold and the second confidence is greater than the second confidence threshold, the target question type is determined to be a document-type question.

6. The method according to claim 3, characterized in that, The method further includes: If the first relevance is greater than the first relevance threshold, the initial query question is subjected to fine-tuning to determine the target question type.

7. The method according to claim 4, characterized in that, Before performing fine-tuning on the initial query problem, the method further includes: The search is performed in a preset knowledge base based on the initial query question to obtain a first search result. The preset knowledge base includes a keyword library, a text vector library, and a knowledge graph library. The first search result includes at least three different types of terms related to the initial query question. The process of performing fine-tuning on the initial query question to determine the target question type includes: Based on the first search result, the initial query question is further segmented to determine the target question type.

8. The method according to claim 7, characterized in that, Before retrieving the first search result from a preset knowledge base based on the initial query question, the method further includes: Based on the historical questions and answers associated with the initial query question, and a preset question rewriting model, the initial query question is rewritten to obtain the target query question; The step of searching a preset knowledge base based on the initial query question to obtain a first search result includes: Based on the initial query question and the target query question, a search is performed in the preset knowledge base to obtain a second search result. The second search result includes at least three different types of terms related to the target query question.

9. The method according to claim 8, characterized in that, After obtaining the second search result, the method further includes: Based on a preset reordering algorithm, the at least three different types of terms related to the target query question are sorted by relevance to obtain the target ranking result; The process of performing fine-tuning on the initial query question to determine the target question type includes: Based on the initial query question, the target query question, and the target ranking result, the initial query question is subjected to fine-tuning to determine the target question type.

10. The method according to claim 9, characterized in that, Before using the target model corresponding to the target question type to output the target answer corresponding to the initial query question, the method further includes: Construct target prompt words, which are obtained based on the target query question, the target sorting result, the target question type, and a preset type prompt word template. The preset type prompt word template includes at least one of a general document type prompt word template, a table type prompt word template, a translation type prompt word template, and a summary type prompt word template. The step of using a target model corresponding to the target question type to output the target answer corresponding to the initial query question includes: The target prompt is input into the target model to obtain the target answer.

11. The method according to claim 10, characterized in that, When the target problem type is the document-type problem, the target model is the document model; The step of inputting the target prompt words into the target model to obtain the target answer includes: The target prompt is input into the document model to obtain the target answer.

12. The method according to claim 11, characterized in that, After inputting the target prompt word into the document model to obtain the target answer, the method further includes: Perform a confidence assessment on the target answer; If the confidence level of the target answer is lower than the third confidence level threshold, the target answer is recalled based on a preset recall model.

13. The method according to claim 12, characterized in that, After recalling the target answer based on the preset recall model, the method further includes: The target prompt is input into the non-document model to obtain the target answer.

14. A retrieval enhancement generation apparatus, characterized in that, include: The determination module is used to classify the received initial query questions by type and determine the target question type, wherein the target question type includes at least one of document-type questions and non-document-type questions; The output module is used to output the target answer corresponding to the initial query question using a target model corresponding to the target question type. The target model is one of the preset models, which includes a document model and a non-document model. The document model is used to obtain the answer to the document-type question, and the non-document model is used to obtain the answer to the non-document-type question.

15. A computer device comprising a memory and a processor, the memory storing a computer program executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the method according to any one of claims 1 to 13.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 13.

Images