Retrieval method and apparatus, and computer-readable storage medium

Abstract

The embodiments of the present application relate to the technical field of artificial intelligence. Provided are a retrieval method and apparatus, and a computer-readable storage medium, which can increase the relevance between retrieval-augmented knowledge and a user query during retrieval-augmented generation, thereby improving the accuracy of generated content. The method comprises: receiving a user query inputted by a user; and in response to the user query, displaying an answer corresponding to the user query, wherein the answer is obtained on the basis of a first prompt; the first prompt is a prompt constructed on the basis of at least one piece of first knowledge; the at least one piece of first knowledge comprises at least one of the following: knowledge which is associated with the user query and is retrieved from among first-type knowledge in a knowledge base on the basis of a first retrieval mode, or knowledge which is associated with the user query and is retrieved from among second-type knowledge in the knowledge base on the basis of a second retrieval mode; and the first retrieval mode and the second retrieval mode are different retrieval modes. The embodiments of the present application are applied to a retrieval-augmented generation process.

Description

Search methods, apparatus, and computer-readable storage media

[0001] This application claims priority to Chinese Patent Application No. 202411445823.5, filed on October 15, 2024, entitled "Search Method, Apparatus, and Computer-Readable Storage Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of artificial intelligence technology, and in particular to a retrieval method, apparatus, and computer-readable storage medium. Background Technology

[0003] With the development of large language model (LLM) technology, the application of artificial intelligence generated content (AIGC) is becoming increasingly widespread. AIGC can generate richer and more complete answers based on user-input questions. However, due to limitations in the domain knowledge used by AIGC, the timeliness of information, and the authenticity of content, the content generated by AIGC may not be accurate enough.

[0004] Retrieval augmented generation (RAG) technology retrieves relevant knowledge information from an external knowledge base (also known as a domain library) before generating content. The user question and the retrieved knowledge information are then input into a large language model, which integrates and summarizes the user input and the retrieved knowledge information to output the final generated content. In this way, more accurate content can be generated based on the user input and the retrieved knowledge information.

[0005] However, in the current retrieval enhancement generation process, the retrieval methods are relatively simple, and the retrieved knowledge may have low relevance to the user's question, which in turn leads to low accuracy of the content generated by the large language model. Summary of the Invention

[0006] This application provides a retrieval method, apparatus, and computer-readable storage medium that can improve the relevance of retrieval enhancement knowledge to user questions during the retrieval enhancement generation process, thereby improving the accuracy of the generated content.

[0007] To achieve the above objectives, this application provides the following technical solution:

[0008] Firstly, a retrieval method is provided. This method can be executed by a first device, or by a component of the first device, such as its processor, chip, or chip system, or by a logic module or software capable of implementing all or part of the first device. The following description uses the execution of this method by the first device as an example. The retrieval method includes: after receiving a question input by a user, the first device first retrieves knowledge related to the user's question from a first type of knowledge in a knowledge base based on a first retrieval method, and / or retrieves knowledge related to the user's question from a second type of knowledge in the knowledge base based on a second retrieval method. Afterward, the first device constructs a first prompt word based on the retrieved knowledge, determines the answer corresponding to the user's question based on the first prompt word, and displays the answer related to the user's question.

[0009] In this embodiment, when the first device searches for a user's question, it searches for different types of knowledge using different search methods. This allows the first device to use a search method more suitable for the knowledge type, resulting in more accurate search results. Subsequently, the first device constructs prompts based on the retrieved knowledge and determines the answer based on the prompts, which can also make the determined answer more accurate.

[0010] In one possible implementation, the first retrieval method is vector retrieval; the second retrieval method is keyword retrieval.

[0011] Based on this, the first device employs both vector retrieval and keyword retrieval methods to search for different types of knowledge. In this way, the first device can perform vector retrieval for knowledge suitable for vector retrieval and keyword retrieval for knowledge suitable for keyword retrieval, thereby improving the relevance of the retrieved knowledge to the user's question.

[0012] In one possible implementation, the first type of knowledge and the second type of knowledge include knowledge related to at least one of the text, image, and table.

[0013] Based on this, the first device can use appropriate retrieval methods to search for text-type knowledge, image-type knowledge, and table-type knowledge, rather than being limited to retrieval based on single information such as text, making the retrieval more comprehensive and improving the accuracy of the retrieval.

[0014] In one possible implementation, the first type of knowledge includes at least one of the following: a question field in knowledge of question-and-answer type, a semantic description field in knowledge of table type, an image context information field in knowledge of image type, and a title field and content field in knowledge of text type. In other words, the first device retrieves knowledge related to the user's question from the knowledge base using vector retrieval, based on the following: question field in knowledge of question-and-answer type, semantic description field in knowledge of table type, image context information field in knowledge of image type, and title field and content field in knowledge of text type. Optionally, the semantic description field is used to characterize the complete semantics of each row (or column) of data in the table, generated by combining the table header data.

[0015] Based on this, since the question field, semantic description field, image context information field, title field, and content field mentioned above are all fields with complete semantics, the first device can perform vector retrieval on the above devices to retrieve knowledge with higher semantic similarity to the user's question, thereby improving the relevance of the retrieved knowledge to the user's question.

[0016] In one possible implementation, the second type of knowledge includes at least one of the following: answer fields in question-and-answer type knowledge, table text fields in table type knowledge, image content fields in image type knowledge, and title and content fields in text type knowledge. In other words, the first device retrieves knowledge related to the user's question from the knowledge base using keyword retrieval, based on the following: answer fields in question-and-answer type knowledge, table text fields in table type knowledge, image content fields in image type knowledge, and title and content fields in text type knowledge. Optionally, the image content field is the image content obtained after image recognition.

[0017] Therefore, since the answer, title, and content fields mentioned above are all pre-summarized high-quality and high-precision knowledge, the first device can perform keyword retrieval on these types of knowledge to ensure that the search results are more relevant to the user's question and that the quality and accuracy of the retrieved knowledge are also higher. However, the original text fields in tables and the image content fields mentioned above are usually semantically incomplete and cannot be retrieved using vector retrieval. Therefore, keyword retrieval can be used to retrieve knowledge in these types of fields.

[0018] In one possible implementation, the process of constructing a first prompt word based on at least one first knowledge includes: determining the second knowledge with the highest relevance to the user's question from at least one first knowledge; and constructing a prompt word for the user's question based on the second knowledge.

[0019] Based on this, during the process of constructing the first suggestion term, the first device sorts the search results obtained from searching different types of knowledge using different search methods according to their relevance to the user's question, and selects the second knowledge with the highest relevance to the user's question. Subsequently, the first device constructs suggestion terms for the user's question based on the second knowledge, thereby maximizing the relevance of the constructed suggestion terms to the user's question.

[0020] In one possible implementation, constructing prompt words for the user question based on second knowledge includes: determining the type of second knowledge; selecting a prompt word template that matches the type of second knowledge and / or at least one first prompt word example; and determining the prompt words for the user question by filling the prompt word template based on the user question, the second knowledge, and / or at least one first prompt word example.

[0021] Based on this, when constructing prompts for user questions, the first device can select prompt templates and / or prompt examples that best match the knowledge type most relevant to the user question. This ensures that the constructed prompts are adapted to the knowledge type, thereby further improving the accuracy of the generated content.

[0022] In one possible implementation, determining the prompt word for the user question by filling a prompt word template based on the user question, second knowledge, and / or at least one first prompt word example includes: selecting a second prompt word example that is most relevant to the user question from at least one first prompt word example; and filling the prompt word template with the user question, second knowledge, and the second prompt word example to obtain the prompt word for the user question.

[0023] Based on this, when constructing prompts for user questions, the first device selects prompt examples that are not only compatible with the knowledge type but also with the user question. This allows the first device to select more accurate prompt examples, thereby improving the accuracy of the constructed prompts.

[0024] In one possible implementation, the process of determining the answer based on the first prompt word includes: inputting the prompt word into a large language model and obtaining the output of the large language model; and determining the answer to the user's question based on the output of the large language model.

[0025] Based on this, the first device inputs the retrieved and enhanced knowledge into the large language model to generate answers to user questions, which can make the generated answers more accurate.

[0026] Secondly, a retrieval device is provided for implementing the various methods described above. This retrieval device can be the first device described in the first aspect, or any implementation thereof, or a device including the first device, or a device included in the first device, such as a chip. The retrieval device includes modules, units, or means that implement the methods described above. These modules, units, or means can be implemented in hardware, software, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the functions described above.

[0027] In some possible designs, the retrieval device may include a processing module and a transceiver module. The transceiver module, also referred to as a transceiver unit, is used to implement the transmission and / or reception functions in any of the above aspects and any possible implementations. The transceiver module may consist of transceiver circuits, transceivers, transceivers, or communication interfaces. The processing module can be used to implement the processing functions in any of the above aspects and any possible implementations.

[0028] In some possible designs, the transceiver module includes a sending module and a receiving module, which are used to implement the sending and receiving functions in any of the above aspects and any possible implementation methods.

[0029] Thirdly, a retrieval apparatus is provided, comprising: a processor and a memory; the memory is used to store computer instructions, which, when executed by the processor, cause the retrieval apparatus to perform the methods of any of the above aspects. The retrieval apparatus may be the first device described in the first aspect, or any implementation thereof, or an apparatus including the first device, or an apparatus included in the first device, such as a chip.

[0030] Fourthly, a retrieval device is provided, comprising: a processor and a communication interface; the communication interface being used to communicate with a module outside the retrieval device; the processor being used to execute computer programs or instructions such that the retrieval device can be a first device as described in the first aspect above, or any implementation thereof, or a device including the first device, or a device included in the first device, such as a chip.

[0031] Fifthly, a retrieval apparatus is provided, comprising: at least one processor; the processor being configured to execute a computer program or instructions stored in a memory to cause the retrieval apparatus to perform the methods of any of the above aspects. The memory may be coupled to the processor, or may be independent of the processor. The retrieval apparatus may be the first device of the first aspect, or any implementation thereof, or an apparatus including the first device, or an apparatus included in the first device, such as a chip.

[0032] In a sixth aspect, a computer-readable storage medium is provided, which stores a computer program or instructions that, when executed on a retrieval device, enable the retrieval device to perform the methods of any of the above aspects or any implementation thereof.

[0033] In a seventh aspect, a computer program product containing instructions is provided, which, when run on a retrieval device, enables the retrieval device to perform the method of any of the above aspects or any implementation thereof.

[0034] Eighthly, a retrieval device (e.g., a chip or chip system) is provided, the retrieval device including a processor for implementing the functions involved in any of the above aspects or any implementation thereof.

[0035] In some possible designs, the retrieval device includes a memory for storing necessary program instructions and data.

[0036] In some possible designs, when the device is a chip system, it can be composed of chips or contain chips and other discrete components.

[0037] It is understood that when the retrieval device provided by any of the third to sixth aspects is a chip, the aforementioned transmitting action / function can be understood as an output, and the aforementioned receiving action / function can be understood as an input.

[0038] Ninthly, a retrieval system is provided, which includes the first device described above.

[0039] In a tenth aspect, a computer program product is provided that, when run on a retrieval device, enables the retrieval device to execute the method of any of the above aspects or any implementation thereof.

[0040] The technical effects of any of the implementation methods in aspects two through ten can be found in the technical effects of the corresponding implementation methods in aspect one, and will not be repeated here.

[0041] Among these, any possible implementation methods of any one of the above aspects can be combined, provided that the solutions do not contradict each other. Attached Figure Description

[0042] Figure 1 is a schematic diagram of a large model retrieval enhancement generation method based on a retrieval enhancement generation framework provided in an embodiment of this application, and a hierarchical index structure in the vector index block of the large model retrieval enhancement generation method;

[0043] Figure 2 is a schematic diagram of a multi-level knowledge retrieval enhanced large language model reasoning method provided in an embodiment of this application;

[0044] Figure 3 is a schematic diagram of a multimodal, multi-scale, multi-path recall method for enhancing the generation of large language model retrieval, provided in an embodiment of this application.

[0045] Figure 4 is a flowchart illustrating a retrieval method provided in an embodiment of this application;

[0046] Figure 5 is a schematic diagram of a retrieval device provided in an embodiment of this application;

[0047] Figure 6 is a flowchart illustrating a retrieval method provided in an embodiment of this application;

[0048] Figure 7 is a flowchart illustrating another retrieval method provided in an embodiment of this application;

[0049] Figure 8 is a flowchart illustrating another retrieval method provided in an embodiment of this application;

[0050] Figure 9 is a flowchart illustrating another retrieval method provided in an embodiment of this application;

[0051] Figure 10 is a schematic diagram of segmenting table type knowledge and configuring indexing strategies according to an embodiment of this application;

[0052] Figure 11 is a schematic diagram of constructing prompt words based on table type knowledge according to an embodiment of this application;

[0053] Figure 12 is a schematic diagram of an embodiment of this application providing a method for inputting the prompt word into a large language model to obtain the output result of the large language model;

[0054] Figure 13 is a schematic diagram of constructing prompt words based on image type knowledge according to an embodiment of this application;

[0055] Figure 14 is a comparative diagram of a search enhancement generation based on existing technology and a search enhancement generation based on the search method provided in this application embodiment;

[0056] Figure 15 is a schematic diagram of another retrieval device provided in an embodiment of this application. Detailed Implementation

[0057] To facilitate understanding of the technical solutions provided in the embodiments of this application, a brief introduction to the relevant technologies of this application is given first. The brief introduction is as follows:

[0058] 1. Frequently Asked Questions (FAQ): In the field of content creation, FAQ knowledge typically includes frequently asked questions summarized in advance by humans, as well as the answers to these questions. FAQ knowledge can help users quickly find the answers to their questions.

[0059] 2. Prompt: Prompts are the input text used by the Large Language Model (LLM) for content generation. Inputting prompts into the LLM helps guide it to generate the corresponding output content.

[0060] 3. Prompt template: The prompt template includes instructions that guide the large language model to generate content, such as instructions describing the task to be completed. The prompt template also includes variable slots related to the content generation task, such as variable slots for filling user questions or variable slots for filling retrieved knowledge.

[0061] 4. Prompt word examples: Prompt word examples are used to provide examples of user input and output from the large language model based on the given task, so that the large language model can generate content similar in format to the prompt word examples.

[0062] 5. Markdown: Markdown is a lightweight markup language that allows users or devices to add formatting elements to plain text documents. Markdown can format text on web pages, such as adding bold, italics, numbered lists, bullet points, headings, tables, etc.

[0063] 6. Vector indexing: Vector indexing refers to the technique of converting text into semantic vectors (i.e., vectors composed of high-dimensional continuous floating-point numbers) and using the semantic vectors as indexes of the text. Vector indexing can match texts with high semantic vector similarity.

[0064] 7. Vector retrieval: Vector retrieval refers to the process of converting user questions into semantic vectors during the retrieval process, calculating the similarity between these vectors and the vector indexes of text in the knowledge base, and retrieving knowledge with a high semantic similarity to the user question. Through vector retrieval, the retrieved knowledge content can be returned in descending order of semantic similarity.

[0065] 8. Keyword indexing: Keyword indexing refers to converting text into a sequence of words after word segmentation, and determining the index relationship between keywords in the text and the text by associating words that appear in the text with corresponding segments.

[0066] 9. Keyword search: Keyword search refers to a search method that segments the user's question into words during the search process and returns documents that match the keywords from the knowledge base in descending order of the number of keywords matched.

[0067] 10. Retrieval-enhanced generation: This is an artificial intelligence technique that combines retrieval and generative technologies. Before inputting content into a large language model, relevant knowledge is retrieved from a knowledge domain base. The content and knowledge are then input together into the large language model, which processes the content and knowledge to obtain the final result. Retrieval-enhanced generation is commonly used in content-generative artificial intelligence.

[0068] Retrieval-enhanced generation techniques typically involve three processes: retrieval, enhancement, and generation. In the retrieval process, relevant knowledge is retrieved from a knowledge base based on the user's input. For example, the user's input is converted into a vector using an embedding model and matched against pre-converted domain knowledge in the knowledge base. Knowledge fragments with sufficient similarity are considered relevant to the content. In the enhancement process, the user's input and the retrieved relevant knowledge are used to populate a prompt word template, and the prompt words are supplemented with relevant instructions and context to obtain the prompt words. In the generation process, the retrieved and enhanced prompt words are input into a large language model, and the model's output is used as the answer to the user's input.

[0069] Some retrieval enhancement generation schemes provide a framework that employs a hierarchical index structure. The first-level index contains summary information for all documents, used to quickly filter out potentially relevant documents. The second-level index covers detailed aspects of each document, allowing for more refined and in-depth searches after the initial screening of relevant documents. The index is constructed using a vectorized approach, and online retrieval also converts user queries into vectors and matches document vectors based on their similarity. The retrieved results are scored using a cross-encoder, rearranged according to the scores, and results with low similarity are filtered out. Only the top K most relevant document blocks after the final ranking are added to a prompt word template and fed into a large language model for final answer generation.

[0070] Figure 1 illustrates the large-model retrieval enhancement generation method based on this retrieval enhancement generation framework, and the hierarchical index structure in the vector index block of the large-model retrieval enhancement generation method. In the method shown in Figure 1, a block-based algorithm is first designed to improve the current vector block-based strategy, performing vector indexing, constructing two indexes, and conducting in-depth searches. Then, the post-processing module is designed, using a cross-encoder to rearrange the search results, intelligently re-ranking them, and finally synthesizing them accordingly. The resulting answers are integrated or refined to form the final response.

[0071] However, the retrieval enhancement and generation scheme shown in Figure 1 uses a uniform processing flow for knowledge data with different structures, such as FAQs, tables, images, and text. This fails to adapt well to the retrieval of knowledge with different structures, and it cannot select the most suitable strategy for answer generation based on different knowledge structures. For example, during the retrieval process, the content of tables may have semantic incompleteness, resulting in the inability to accurately retrieve table content, and images cannot be directly retrieved via text. Furthermore, during the enhancement and generation process, the large language model cannot accurately recognize table data, nor can it accurately understand the semantics of images.

[0072] In some retrieval enhancement generation solutions, a multi-level knowledge retrieval enhancement method using a large language model for reasoning is provided, the workflow of which is shown in Figure 2. This method constructs an operational knowledge base based on FAQs. After obtaining the user's input question, it first compares the question with the FAQs in the operational knowledge base. If the question and FAQ match successfully, the answer from the FAQ is directly output. If the comparison fails, a retrieval is performed in an industry knowledge base (vector database). If the search results are highly relevant, they are added to a prompt word template and input into the model to generate the final answer. If the search results are not highly relevant, the large language model automatically generates the answer without using the search results. While this multi-level knowledge retrieval enhancement method uses a specific approach for FAQ data, its processing only involves directly returning the retrieved results for processing. Furthermore, this solution is only applicable to FAQ data and cannot be adapted to other types of knowledge data.

[0073] In some retrieval enhancement generation schemes, a multimodal, multi-scale, multi-path recall method using a large language model is provided, as shown in Figure 3. This method first constructs a text library and a vector library based on ordinary medical text (or medical text), image description text (or medical images), and tabular text (or medical tables). Then, based on the user's question, medical knowledge text blocks are retrieved from the text library and vector library respectively, and the final target knowledge text block is determined based on the text blocks obtained from the above two retrieval paths. Finally, the question and target knowledge text block are input into a large language model to generate the answer. While the above retrieval enhancement generation method constructs text and vector libraries for ordinary text, image description text, and tabular text, it does not generate language descriptions for images and tables, only obtaining the text content. This fails to effectively represent semantic vectors, affecting the relevance of vector retrieval. The tables were not processed for generation, meaning the large model could not perceive the table's structural information when generating answers. In addition, the fixed prompts could not be dynamically changed for different intentions and search results, affecting the accuracy and diversity of the answer generation intent. For example, it could not provide answers in the form of images and text.

[0074] The above is a brief introduction to the relevant technologies of this application.

[0075] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0076] To facilitate understanding of the embodiments of this application, the following points will be explained before introducing the embodiments of this application.

[0077] 1. In the embodiments of this application, for ease of description, when numbering is involved, it can start from 1 and be numbered consecutively, or it can start from 0 and be numbered from any parameter. It should be understood that the above are settings made for the convenience of describing the technical solutions provided in the embodiments of this application, and are not intended to limit the scope of the embodiments of this application.

[0078] 2. In the embodiments of this application, the descriptions such as "when," "under the circumstances," "if," and "if" all refer to the fact that the device (e.g., the first device) will make corresponding processing under certain objective circumstances. They are not time limits, nor do they require the device (e.g., the first device) to have a judgment action when implementing it, nor do they mean that there are other limitations.

[0079] 3. In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can represent A or B. The "and / or" in the embodiments of this application is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. A and B can be singular or plural. Furthermore, in the description of the embodiments of this application, unless otherwise stated, "multiple" refers to two or more than two. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, a to b, a to c, b to c, or a to b to c, where a, b, and c can be single or multiple. Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0080] 4. In this application, "at least one" means one or more. "More" means two or more. "At least two" means two or three or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of single or multiple items. For example, at least one of a, b, or c can mean: a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be single or multiple.

[0081] 5. In this application, "instruction" may include: direct instruction, or, indirect instruction, or, explicit instruction, or, implicit instruction.

[0082] In this application, "including" can include: direct inclusion, indirect inclusion, explicit inclusion, or implicit inclusion.

[0083] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or apparatus that includes a series of steps or units is not limited to the steps or units listed, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to such process, method, product, or apparatus.

[0084] It should be understood that existing technologies may change as technical solutions evolve, and the technical solutions provided in this application are not limited to the existing technologies provided.

[0085] In this application, different embodiments or parts of steps (e.g., any one or more steps) can be combined to form new embodiments. The scope of steps in different embodiments is not limited; they may include optional steps, mandatory steps, or both. Unless otherwise specified or logically conflicting, the terminology and / or descriptions in different embodiments are consistent and can be mutually referenced. The order of steps in the embodiments of this application is not limited. The order of judgments of different conditions in the embodiments of this application is not limited. The terms "after" and "at" in this application are not strictly limited to specific time points.

[0086] The nouns and terms used in this application are merely examples and may be other names. This application does not impose any restrictions on their use.

[0087] Figure 4 shows a flowchart of a retrieval method provided in an embodiment of this application. As shown in Figure 4, the user inputs a user question into a first device, which then performs retrieval enhancement on the user question to obtain prompt words. The first device inputs the prompt words into a large language model to obtain the answer to the user question.

[0088] In one possible implementation, during the process of enhancing the retrieval of user questions, the first device categorizes the knowledge in its knowledge base according to type and configures appropriate retrieval methods for each type of knowledge, such as vector retrieval and keyword retrieval. Upon receiving a user question, the first device retrieves different types of knowledge from the knowledge base based on different retrieval methods, and selects the knowledge most relevant to the user question from the retrieved knowledge. Then, based on the knowledge type most relevant to the user question, the first device selects corresponding prompt word templates and / or prompt word examples, and fills the prompt word templates based on the retrieved most relevant knowledge, the user question, and / or prompt word examples, thus obtaining prompt words for the user question. The first device inputs the prompt words for the user question into a large language model to obtain the answer to the user question.

[0089] Optionally, knowledge in the knowledge base can be categorized according to the following types: FAQ, tables, images, and text. Different types of knowledge can be further divided into different fields. For example, FAQ-type knowledge can be divided into question and answer fields. Table-type knowledge can be divided into table text fields, semantic description fields, and Markdown fields. Image-type knowledge can be divided into image fields, image content fields, and image context information fields. Text-type knowledge can be divided into title fields and content fields.

[0090] Optionally, different search methods can be configured for different field types. Specifically, the search method for the question field in question-and-answer type knowledge, the semantic description field in table type knowledge, the image context information field in image type knowledge, and the title and content fields in text type knowledge is vector search. The search method for the answer field in question-and-answer type knowledge, the table text field in table type knowledge, the image content field in image type knowledge, and the title and content fields in text type knowledge is keyword search. When the first device retrieves relevant knowledge about a user's question from the knowledge base, it can retrieve the corresponding knowledge fields based on the configured search method.

[0091] Optionally, in this embodiment, an offline prompt word template library and / or prompt word sample library are pre-configured. The prompt word template library contains prompt word templates corresponding to various types of knowledge. For example, FAQ-type prompt word templates for FAQ-type knowledge; image-type prompt word templates for image-type knowledge; table-type prompt word templates for table-type knowledge; and text-type prompt word templates for text-type knowledge. The prompt word sample library contains prompt word samples corresponding to various types of knowledge. For example, FAQ-type prompt word samples for FAQ-type knowledge; image-type prompt word samples for image-type knowledge; table-type prompt word samples for table-type knowledge; and text-type prompt word samples for text-type knowledge. During the enhancement process, the first device selects a prompt word template of the corresponding type from the prompt word template library and / or a prompt word sample of the corresponding type from the prompt word sample library based on the type of knowledge most relevant to the user's question.

[0092] The relevant functions of the first device in this application embodiment can be implemented by the retrieval device 510 in FIG. 5. FIG. 5 shows a schematic diagram of a possible retrieval device. It is understood that the retrieval device 510 includes means of the necessary form, such as modules, units, elements, circuits, or interfaces, to be appropriately configured together to perform this solution. The retrieval device 510 can be the first device or a component (e.g., a chip) in these devices to implement the methods described in the following method embodiments. The retrieval device 510 includes one or more processors 511. The processor 511 can be a general-purpose processor or a special-purpose processor, etc. For example, it can be a baseband processor or a central processing unit. The baseband processor can be used to process communication protocols and communication data, and the central processing unit can be used to control the retrieval device (e.g., a controller, training server, or chip, etc.), execute software programs, and process data of the software programs.

[0093] Optionally, in one design, the processor 511 may include a program 513 (sometimes also referred to as code or instructions), which may be executed on the processor 511 to cause the retrieval device 510 to perform the methods described in the embodiments below. In yet another possible design, the retrieval device 510 includes circuitry (not shown in FIG5) for implementing the communication functions in the embodiments below.

[0094] Optionally, the retrieval device 510 may include one or more memories 512 storing a program 514 (sometimes referred to as code or instructions), which may be run on the processor 511 to cause the retrieval device 510 to perform the methods described in the following method embodiments.

[0095] Optionally, the processor 511 and / or memory 512 may include artificial intelligence (AI) modules 517 and 518, which are used to implement AI-related functions. AI modules 517 or 518 can be implemented through software, hardware, or a combination of both. For example, AI modules 517 or 518 may include a radio intelligent controller (RIC) module. For example, AI modules 517 or 518 can be near real-time RICs or non-real-time RICs. Furthermore, AI modules 517 or 518 can be used as modules employing large models.

[0096] Optionally, data may also be stored in the processor 511 and / or the memory 512. The processor and memory may be configured separately or integrated together.

[0097] Optionally, the retrieval device 510 may also include a transceiver 515 and / or an antenna 516. The processor 511, sometimes referred to as a processing unit, controls the retrieval device (e.g., a training server or controller). The transceiver 515, sometimes referred to as a transceiver unit, transceiver, transceiver circuit, or transceiver, is used to implement the transmission and reception functions of the retrieval device via the antenna 516.

[0098] The retrieval method provided in the embodiments of this application will be described in detail below with reference to Figure 5.

[0099] In the following embodiments of this application, the message names between devices, the names of parameters, or the names of information are merely examples. Other names may exist in other embodiments, and the methods provided in this application are not specifically limited in this regard. It is understood that in the embodiments of this application, each device may execute some or all of the steps in the embodiments of this application. These steps or operations are examples, and the embodiments of this application may also execute other operations or variations thereof. Furthermore, the steps may be executed in different orders as presented in the embodiments of this application, and it is not necessary to execute all the operations in the embodiments of this application.

[0100] Figure 6 illustrates an example of a retrieval method provided in this application. Taking a first device executing this retrieval method as an example, after receiving a user's input question, the first device retrieves different types of knowledge from the knowledge base based on different retrieval methods, constructs prompt words based on the retrieval results, and obtains the answer to the user's question based on the prompt words. For example, as shown in Figure 6, the retrieval method includes the following steps:

[0101] Step 601: The first device receives the user's input question.

[0102] In some embodiments, a user inputs a question on a first device, and the first device detects the user's action and determines the user's question based on the action. The user's action can be input through an input device, or it can be performed through voice, gestures, or other instructions; this application does not limit this. As an example, the user inputs the question, "What is the ID of the SSL Secure Sockets Layer feature?".

[0103] Optionally, after receiving the user's input question, the first device can also perform multi-task semantic understanding on the user's question, that is, construct the user's question into multiple semantic understanding tasks, and execute the retrieval method provided in the embodiments of this application for each semantic understanding task to generate a corresponding answer for each semantic understanding task.

[0104] As an example, after the first device receives the user's input question, it performs multi-task semantic understanding on the user question. This includes tasks such as intent understanding, entity recognition, and question rewriting. For each task, the task content is retrieved as the user question to obtain the corresponding answer. Optionally, the intent understanding task is to parse the user's true purpose or need from the input question. The entity recognition task is to identify entities (such as people, place names, times, specific names, etc.) represented by words with specific meanings from the input question. The question rewriting task is to rewrite the user question based on its semantics to obtain a question with the same (or similar) semantics but a different expression.

[0105] As a concrete example, a user could enter the question, "What is the ID of the SSL Secure Sockets Layer feature?".

[0106] Step 602: In response to the user's question, the first device displays the answer corresponding to the user's question.

[0107] The answer is obtained based on a first prompt word; the first prompt word is a prompt word constructed based on at least one first piece of knowledge. At least one piece of first knowledge includes at least one of the following: knowledge related to the user's question retrieved from a first type of knowledge in the knowledge base based on a first retrieval method, or knowledge related to the user's question retrieved from a second type of knowledge in the knowledge base based on a second retrieval method; the first retrieval method and the second retrieval method are different retrieval methods.

[0108] As one implementation method, after the first device receives the user's input question, it first searches the user's question, then performs content enhancement based on the search results, and finally inputs the enhanced content into the large language model to obtain the output result of the large language model. The output result of the large language model is then displayed as the answer to the user's question.

[0109] As an example, during the process of the first device retrieving user questions, in order to improve the accuracy of the retrieval results, the first device uses different retrieval methods for different types of knowledge. The first device retrieves knowledge related to the user question from the first type of knowledge in the knowledge base based on the first retrieval method, and / or, the first device retrieves knowledge related to the user question from the second type of knowledge in the knowledge base based on the second retrieval method. All the knowledge retrieved by the first device is collectively referred to as first knowledge. After retrieving at least one type of first knowledge, the type of knowledge with the highest relevance to the user question (denoted as second knowledge) is determined, and the prompt word template and / or prompt word sample corresponding to the knowledge type of the second knowledge are obtained. The first device fills the prompt word template based on the user question, the first knowledge, and / or the prompt word sample to obtain the first prompt word. The first device inputs the first prompt word into a large language model to obtain the output result of the large language model, and displays the output result of the large language model as the answer corresponding to the user question.

[0110] Based on the specific example in step 601 above, the output of the large language model is "The IDs of the SSL Secure Sockets Layer features are LBFD-004003 and TDLBFD-004003. These two IDs correspond to the SSL Secure Sockets Layer features used in LTE FDD and LTE TDD networks, respectively".

[0111] In this embodiment, when the first device searches for a user's question, it searches for different types of knowledge using different search methods. This allows the first device to use a search method more suitable for the knowledge type, resulting in more accurate search results. Subsequently, the first device constructs prompts based on the retrieved knowledge and determines the answer based on the prompts, which can also make the determined answer more accurate.

[0112] In some embodiments, the first retrieval method is vector retrieval; the second retrieval method is keyword retrieval. In other words, the first device uses vector retrieval to retrieve first type of knowledge and keyword retrieval to retrieve second type of knowledge. In this way, the first device can perform vector retrieval for knowledge suitable for vector retrieval and keyword retrieval for knowledge suitable for keyword retrieval, thereby improving the relevance of the retrieved knowledge to the user's question.

[0113] In some other embodiments, the first type of knowledge and the second type of knowledge include knowledge related to at least one of text, images, and tables. In other words, the first device can use different retrieval methods to retrieve different types of knowledge, such as text-type knowledge, image-type knowledge, and table-type knowledge, thereby improving the adaptability of retrieval methods to knowledge types.

[0114] In some embodiments, the first type of knowledge includes at least one of the following: a question field in knowledge of question-and-answer type, a semantic description field in knowledge of table type, an image context information field in knowledge of image type, and a title field and content field in knowledge of text type. In other words, the first device uses vector retrieval to search the knowledge base for: question fields in knowledge of question-and-answer type, semantic description fields in knowledge of table type, image context information fields in knowledge of image type, and title and content fields in knowledge of text type, to obtain knowledge related to the user's question from the aforementioned types of knowledge. Based on this, since the question field, semantic description field, image context information field, title field, and content field are all fields with complete semantics, the first device can retrieve knowledge with higher semantic similarity to the user's question by performing vector retrieval on the aforementioned knowledge, thereby improving the relevance of the retrieved knowledge to the user's question.

[0115] In some embodiments, the second type of knowledge includes at least one of the following: answer fields in knowledge of question-and-answer type, table text fields in knowledge of table type, image content fields in knowledge of image type, and title and content fields in knowledge of text type. In other words, the first device retrieves knowledge related to the user's question from the knowledge base using keyword retrieval: answer fields in knowledge of question-and-answer type, table text fields in knowledge of table type, image content fields in knowledge of image type, and title and content fields in knowledge of text type. Based on this, since the answer fields, title fields, and content fields are all pre-summarized high-quality and high-precision knowledge, the first device can perform keyword retrieval on these types of knowledge to ensure that the retrieved results are more relevant to the user's question and that the quality and accuracy of the retrieved knowledge are higher. However, the table text fields and image content fields are usually semantically incomplete and cannot be retrieved using vector retrieval; therefore, keyword retrieval can be used to retrieve knowledge of these types.

[0116] It is permissible that the first and second search methods can be other search methods besides the vector search and keyword search methods mentioned above. The first type of knowledge and the second type of knowledge can also include knowledge other than the knowledge listed above. This application does not limit this.

[0117] In some possible implementations, prior to step 602, the first device first retrieves knowledge associated with the user question from the knowledge base based on the user question. Referring to Figure 6, as shown in Figure 7, the process by which the first device retrieves knowledge associated with the user question from the knowledge base based on the user question includes steps 701 and / or 702.

[0118] Step 701: The first device retrieves knowledge related to the user's question from the first type of knowledge in the knowledge base based on the first retrieval method.

[0119] Step 702: The first device retrieves knowledge related to the user's question from the second type of knowledge in the knowledge base based on the second retrieval method.

[0120] In some embodiments, before performing a retrieval, a first device (or other device) pre-constructs a knowledge base (which may be an offline knowledge base). The knowledge in this knowledge base is categorized by type, and appropriate retrieval methods, such as vector retrieval and keyword retrieval, are configured for each type of knowledge. During the retrieval process, the first device converts the user question into a semantic vector and breaks it down into multiple keywords. Based on the semantic vector of the user question, the first device retrieves knowledge using vector retrieval to obtain knowledge that meets the semantic similarity condition to the user question; and based on the keywords of the user question, the first device retrieves knowledge using keyword retrieval to obtain knowledge that matches the number of keywords in the user question.

[0121] As an example, to facilitate knowledge retrieval in the knowledge base, the knowledge base is further divided into a vector index and a keyword index. The vector index includes at least one of the following: question fields in question-and-answer type knowledge, semantic description fields in table type knowledge, image context information fields in image type knowledge, and title and content fields in text type knowledge. The keyword index includes at least one of the following: answer fields in question-and-answer type knowledge, table text fields in table type knowledge, image content fields in image type knowledge, and title and content fields in text type knowledge. During the retrieval process, the first device retrieves knowledge related to the user's question from the vector index using vector retrieval, and retrieves knowledge related to the user's question from the keyword index using keyword retrieval.

[0122] Since different fragments of the same knowledge may employ different retrieval strategies during the retrieval process, different retrieval strategies may return the same knowledge. In this case, the first device merges and deduplicates the retrieved knowledge fragments, and then sorts the knowledge based on its relevance to the user's question to obtain the retrieval results for the user's question.

[0123] In the process of retrieving user questions, semantic understanding can be performed to determine multiple tasks corresponding to the user questions, such as intent understanding task, entity recognition task, and question rewriting task. For these tasks, the first device performs retrieval, enhancement, and generation respectively to determine the answer for each task.

[0124] In some other possible implementations, after the first device retrieves knowledge related to the user's question from the knowledge base, it can also construct prompt words based on the retrieved knowledge. As shown in Figure 7, the process by which the first device constructs a first prompt word based on at least one piece of first knowledge includes the following steps 703 and 704:

[0125] Step 703: The first device determines the second knowledge that is most relevant to the user's problem from at least one first knowledge.

[0126] In some possible embodiments, in steps 701 and / or 702 above, the first device retrieves at least one piece of first knowledge related to the user's question from the knowledge base based on a first retrieval method and / or a second retrieval method. These retrieved pieces of first knowledge have different relevance to the user's question. The first device sorts the first knowledge in descending order of relevance and selects the first knowledge with the highest ranking (i.e., the highest relevance) as the second knowledge.

[0127] Step 704: The first device constructs prompt words for the user's question based on the second knowledge.

[0128] In some embodiments, the first device selects a prompt word template and / or a prompt word example based on second knowledge, and then fills in the prompt word template based on the user question, the second knowledge and / or the prompt word example to obtain the prompt word for the user question.

[0129] Optionally, to ensure the compatibility of the prompt words with the second knowledge, the first device may select a prompt word template and / or a prompt word sample based on the type of the second knowledge. As shown in Figure 8, the process by which the second device constructs prompt words for the user's question based on the second knowledge includes the following steps 801 to 803.

[0130] Step 801: The first device determines the type of the second knowledge.

[0131] In one possible implementation, the type of the second knowledge can be any one of FAQ-type knowledge, table-type knowledge, image-type knowledge, or text-type knowledge. Optionally, in this embodiment, knowledge that is not of the FAQ, table, or image type can be considered as text-type knowledge.

[0132] Step 802: The first device selects a prompt word template and / or at least one first prompt word example that matches the type of the second knowledge.

[0133] In one possible implementation, a first device (or other device) pre-constructs a prompt word template library and / or a prompt word sample library. The prompt word template library stores prompt word templates corresponding to knowledge types, and the prompt word sample library stores prompt word samples corresponding to knowledge types. After determining the type of the second knowledge, the first device selects a prompt word template corresponding to the type of the second knowledge from the prompt word template library, and / or selects at least one prompt word sample corresponding to the type of the second knowledge from the prompt word sample library.

[0134] In the prompt word template library, one knowledge type can correspond to one or more prompt word templates, and one prompt word template can also correspond to one or more knowledge types; in the prompt word sample library, one knowledge type can correspond to one or more prompt word samples, and one prompt word sample can also correspond to one or more knowledge types. This application does not impose any limitations on this.

[0135] As an example, Table 1 below illustrates the fields, stored content, and strategies for constructing prompt templates and prompt samples. Table 1 shows the fields, stored content, and strategies for constructing prompt templates and prompt samples as follows:

[0136] Table 1

[0137] The prompt word examples can provide a combination of "search results, user question, and generated answer," instructing the large language model how to generate a satisfactory answer based on the search results and user question for that type of knowledge. For example, in image-type examples, to make the answer more visually appealing, an image can be inserted to improve readability. Optionally, the prompt word example library can be pre-built offline. The first device converts the user question text into a semantic vector, then uses vector indexing to select prompt word examples from the library that are semantically close to the user question, thereby improving the accuracy of the large language model's answer generation. Optionally, if the prompt word example is only used to insert an image into the answer generated by the large language model, then only image-type prompt word examples need to be selected from the prompt word example library, without considering the semantic similarity between the prompt word example and the question.

[0138] Step 803: The first device determines the prompt word for the user question by filling in the prompt word template based on the user question, the second knowledge, and / or at least one first prompt word sample.

[0139] In some embodiments, the first device may further select the prompt word example that best matches the user question from at least one first prompt word example, and then populate the prompt word template. In other words, the first device selects the second prompt word example with the highest relevance to the user question from at least one first prompt word example. The first device populates the prompt word template with the user question, the second knowledge, and the second prompt word example to obtain the prompt word for the user question. As an example, the first device calculates the similarity between the semantic vector of the user question and each prompt word example in the at least one first prompt word example, and selects the first prompt word example with the highest semantic vector similarity to the user question as the second prompt word example.

[0140] Optionally, after the first device selects the prompt word sample that best matches the user question from at least one first prompt word sample, it constructs contextual information in the prompt word based on the second knowledge type, the content of the second knowledge, and the user question; the first device fills the prompt word template with the user question, the second knowledge, the second prompt word sample, and the contextual information of the prompt word to obtain the prompt word for the user question.

[0141] As an example, when constructing the context information for the prompt, if the second knowledge is FAQ-type knowledge, the first device selects the answer field from the second knowledge as the context information for the prompt. Alternatively, if the second knowledge is table-type knowledge, the first device selects the markdown field from the second knowledge as the context information for the prompt. Or, if the second knowledge is image-type knowledge, the first device selects the image context information field from the second knowledge as the context information for the prompt. Or, if the second knowledge is text-type knowledge, the first device selects the content field from the second knowledge as the context information for the prompt.

[0142] In some other possible implementations, after the first device constructs the prompt words, it inputs the prompt words into a large language model to determine the answer to the user's question. As shown in Figure 7, the process by which the first device determines the answer based on the first prompt words includes the following steps 705 and 706:

[0143] Step 705: The first device inputs the prompt word into the large language model and obtains the output result of the large language model.

[0144] In some embodiments, the first device inputs prompt words into a large language model. The large language model determines the type of answer to be output (e.g., answer type is text, image, or table) based on the prompt word examples. It then summarizes and generates content based on the question and secondary knowledge provided in the prompt words to determine the answer to the user's question. Afterward, the large language model outputs the generated answer to the user's question based on the answer type.

[0145] As an example, when the prompt word suggests the answer type is text, the large language model can output a plain text answer. When the prompt word suggests the answer type is image, the large language model can output a text answer while adding image placeholders to the answer and then inserting the image into the placeholders, thus outputting a text-and-image answer. When the prompt word suggests the answer type is table, the large language model can output a table-type answer, for example, directly outputting the answer in Markdown format and rendering it as a table.

[0146] Step 706: The first device determines the answer to the user's question based on the output of the large language model.

[0147] Combining the retrieval phase (steps 701-702), enhancement phase (steps 703-704), and generation phase (steps 705-706) shown in Figure 7, different types of knowledge have different focuses in the retrieval and generation phases. Therefore, the processing methods used at the retrieval and generation nodes are also different. As shown in Table 2 below, the focuses of the retrieval and generation phases are as follows:

[0148] Table 2

[0149] In one possible implementation, before the first device retrieves the user's question, the first device can pre-build an offline knowledge base, classify the knowledge in the knowledge base, and construct corresponding indexes for each type of knowledge, as shown in Figure 9. The process of the first device pre-building the offline knowledge base may include:

[0150] Step 901: The first device acquires knowledge data.

[0151] The knowledge data acquired by the first device can be pre-organized, high-quality, and high-precision knowledge data. This knowledge data is used by the first device during the retrieval process of the retrieval enhancement generation process. The first device can retrieve knowledge related to the user's question from this knowledge data.

[0152] Step 902: The first device determines the type of each piece of knowledge data.

[0153] In some embodiments, the types of knowledge include, but are not limited to: FAQ-type knowledge, table-type knowledge, image-type knowledge, and text-type knowledge.

[0154] As one implementation method, after the first device acquires the knowledge data, it categorizes the knowledge data based on knowledge classification rules. For example, the knowledge can be divided into FAQ-type knowledge, table-type knowledge, image-type knowledge, and text-type knowledge.

[0155] Step 903: The first device performs structured processing on various types of knowledge data and constructs multiple types of fields.

[0156] In some embodiments, after classifying the knowledge, the first device further performs structured processing on the knowledge, dividing it into different knowledge fields. The field types differ depending on the type of knowledge being classified.

[0157] Optionally, the first device performs structured processing on the FAQ-type knowledge data to obtain a question field and an answer field. The question field represents the knowledge related to the question in the FAQ-type knowledge data; the answer field represents the knowledge related to the answer to the question in the FAQ-type knowledge data.

[0158] The first device performs structured processing on the table-type knowledge data to obtain semantic description fields, table original text fields, and / or markdown fields. The semantic description fields represent the complete semantics of each row (or column) of data in the table, generated by combining the table header data; the table original text fields represent the knowledge content in the original table text; and the markdown fields represent the content after the table has been converted to markdown format. Optionally, the markdown fields are the language text of the markdown tags corresponding to the table, where the symbol '|' indicates cell separation, '---' indicates header separation, and '\n' indicates row separation.

[0159] The first device performs structured processing on the image type knowledge to obtain an image context information field and an image content field. The image context information field is the content field obtained after supplementing the image content with context following OCR recognition. The image content field is the image content obtained after OCR recognition. Optionally, the first device can also determine the image's binary code or storage URI as a field for image type knowledge.

[0160] The first device performs structured processing on the text type knowledge, resulting in a title field and a content field. The title field represents the title of the text type knowledge, and the content field represents the content of the text type knowledge.

[0161] Step 904: The first device configures the corresponding indexing strategy for each type of field.

[0162] As an example, for fields in FAQ-type knowledge, since FAQ-type knowledge is usually high-quality, high-precision knowledge summarized by humans, it is divided into question fields and answer fields. The question field needs to match the user's question and has a higher semantic similarity to the user's question; however, this requirement is not required for keywords. Therefore, the first device configures a vector indexing strategy for the question field. The answer field focuses more on the accuracy of the answer during retrieval; therefore, the first device configures a keyword indexing strategy for the answer field.

[0163] For fields in table-type knowledge, the semantics of the original table fields may be incomplete without context. For example, the semantics of a cell lacking a header description may be incomplete. In such cases, vector-based retrieval may not find accurate content, and keyword retrieval can be used. Therefore, the first device configures a keyword indexing strategy for the original table fields. Semantic description fields in table-type knowledge are more semantically accurate and complete; therefore, the first device can configure a vector indexing strategy for semantic description fields.

[0164] Regarding the fields in image type knowledge, the image content field may be an incomplete semantic field; therefore, the first device configures a keyword indexing strategy for the image content field. The image context information field can accurately represent the true semantics of the image; therefore, the first device configures a vector indexing strategy for the image context information field.

[0165] For fields in text-type knowledge, since both the title field and the content field can represent complete semantics and have complete keywords, the first device configures both the title field and the content field in text-type knowledge with keyword indexing strategy and vector indexing strategy.

[0166] In the embodiments of this application, no indexing may be performed on the markdown fields and the original images, and this application does not limit this.

[0167] As an example, the first device configures the corresponding processing and indexing strategies for each type of field as shown in Table 3 below.

[0168] Table 3

[0169] In some embodiments, after the first device determines the indexing strategy for each type of field, it stores the knowledge in the database and builds corresponding indexes for each field of the knowledge based on the indexing strategy for each type of field.

[0170] As an example, taking table-type knowledge as an example, the method of segmenting and configuring indexing strategies is shown in Figure 10. The structure of this table-type knowledge in the HTML page is the table shown in the upper half of Figure 10. After parsing the table-type knowledge, the first device determines the table's origin field. Then, combining the origin field and the table header, it converts the content in the table into semantic description information, i.e., the semantic description field. Finally, it converts the table into Markdown fields. In this way, the table knowledge content in Figure 10 can be converted into the origin field, semantic description field, and Markdown field shown in the lower half of Figure 10. The processing strategy and indexing strategy for each field are shown in Table 3 above.

[0171] After the first device retrieves the user's question, the process of constructing the prompt words is shown in Figure 11. Taking the user question "What is the ID of the SSL Secure Sockets Layer feature?" as an example, the knowledge most relevant to this user question retrieved by the first device is the fields after parsing the table, as shown in the lower half of Figure 10. Specifically, this includes the original table text fields, the table semantic description fields, and the table's markdown fields. For this table type knowledge, the first device selects a table-type prompt word template from the prompt word template library (as shown in Figure 11). The first device fills the markdown field values ​​of the retrieved table type knowledge into the "content" position of the prompt word template and fills the user's question into the "query" position of the prompt word template, thus obtaining the prompt words.

[0172] Referring to Figure 11, as shown in Figure 12, after obtaining the prompt word shown in Figure 11, the first device inputs the prompt word into the large language model. The input content is shown in the upper part of Figure 12. The output content of the large language model after summarizing and generating based on the input content is shown in the lower part of Figure 12.

[0173] As another example, taking the knowledge retrieved by the first device that has the highest relevance to the user's input question as image-type knowledge as an example, as shown in Figure 13, the first device obtains image-type prompt word templates from the prompt word template library and image-type prompt word examples from the prompt word example library. The prompt word templates and / or prompt word examples include instructions to the large language model to generate image-type answers, and instructions to use the context field for generating the text portion of the answer, and the image field for appropriately embedding images during the answer generation process.

[0174] As another example, Figure 14 shows a comparison between the answer obtained by retrieval enhancement generation based on existing technology and the answer obtained by the retrieval method provided in this application embodiment. As can be seen from Figure 14, when retrieval enhancement generation is performed based on existing technology, the context information provided to the large language model during the retrieval enhancement generation process only includes content related to the original text of the table, lacking the structural information of the table. Therefore, the large language model cannot perceive the structural information of the table, resulting in its inability to generate an accurate answer based on the table's structural information. In contrast, when retrieval enhancement generation is performed using the retrieval method provided in this application embodiment, both the retrieved knowledge and the prompts during the enhancement process are specifically processed for table type knowledge. Therefore, the answer generated by the large language model is more accurate, reliable, and concise.

[0175] The above description primarily focuses on the implementation of the solutions provided in the embodiments of this application by corresponding devices. Accordingly, the embodiments of this application also provide a retrieval device for implementing the various methods described above. This retrieval device can be the first device in the above method embodiments, or a device including the first device, or a component usable in the first device. It is understood that, in order to achieve the above functions, the retrieval device includes hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein, this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed in hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0176] This application embodiment can divide the retrieval device into functional modules according to the above method embodiment. For example, each function can be divided into its own functional module, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module. It should be understood that the module division in this application embodiment is illustrative and represents a logical functional division; in actual implementation, there may be other division methods.

[0177] Figure 15 shows a schematic diagram of a retrieval device 150. The retrieval device 150 includes a processing module 1501, a transceiver module 1502, and a display module 1503. The transceiver module 1502, also known as a transceiver unit, is used to implement transceiver functions, and may be a transceiver circuit, transceiver, transceiver device, or communication interface.

[0178] The transceiver module 1502 is used to receive user questions input by the user.

[0179] The processing module 1501 is configured to respond to a user question by instructing the display module 1503 to display the answer corresponding to the user question; wherein the answer is obtained based on a first prompt word; the first prompt word is a prompt word constructed based on at least one first knowledge; at least one first knowledge includes at least one of the following: knowledge related to the user question retrieved from a first type of knowledge in a knowledge base based on a first retrieval method, or knowledge related to the user question retrieved from a second type of knowledge in a knowledge base based on a second retrieval method; the first retrieval method and the second retrieval method are different retrieval methods.

[0180] In one possible implementation, the first retrieval method is vector retrieval; the second retrieval method is keyword retrieval.

[0181] In one possible implementation, the first type of knowledge and the second type of knowledge include knowledge related to at least one of the text, image, and table.

[0182] In one possible implementation, the first type of knowledge includes at least one of the following: a question field in knowledge of question-and-answer type, a semantic description field in knowledge of table type, an image context information field in knowledge of image type, and a title field and a content field in knowledge of text type.

[0183] In one possible implementation, the second type of knowledge includes at least one of the following: an answer field in knowledge of question-and-answer type, a table text field in knowledge of table type, an image content field in knowledge of image type, and a title field and a content field in knowledge of text type.

[0184] In one possible implementation, the processing module 1501 is further configured to: determine the second knowledge with the highest relevance to the user question from at least one first knowledge; and construct prompt words for the user question based on the second knowledge.

[0185] In one possible implementation, the processing module 1501 is further configured to: determine the type of the second knowledge; select a prompt word template and / or at least one first prompt word example that matches the type of the second knowledge; and fill the prompt word template to determine the prompt word for the user question based on the user question, the second knowledge, and / or at least one first prompt word example.

[0186] In one possible implementation, the processing module 1501 is further configured to: select a second prompt word example that is most relevant to the user question from at least one first prompt word example; and fill the prompt word template with the user question, the second knowledge, and the second prompt word example to obtain the prompt word for the user question.

[0187] In one possible implementation, the processing module 1501 is further configured to instruct the transceiver module 1502 to input the prompt word into the large language model, obtain the output result of the large language model, and determine the answer to the user's question based on the output result of the large language model.

[0188] All relevant content of each step involved in the above method embodiments can be referenced from the functional description of the corresponding functional module, and will not be repeated here.

[0189] In this embodiment, the first device is presented as an integrated unit divided into functional modules. Here, "module" can refer to a specific ASIC, circuitry, a processor and memory executing one or more software or firmware programs, integrated logic circuitry, and / or other devices that can provide the aforementioned functions. In a simplified embodiment, those skilled in the art will recognize that the first device can take the form of the retrieval device 510 shown in FIG. 5.

[0190] For example, the processor 511 in the retrieval device 510 shown in Figure 5 can call the computer execution instructions stored in the memory 512 to make the retrieval device 510 execute the model training method in the above method embodiment.

[0191] Specifically, the functions / implementation processes of the transceiver module 1502 and the processing module 1501 in Figure 15 can be implemented by the processor 511 in the retrieval device 510 shown in Figure 5 calling computer execution instructions stored in the memory 512. Alternatively, the functions / implementation processes of the processing module 1501 in Figure 15 can be implemented by the processor 511 in the retrieval device 510 shown in Figure 5 calling computer execution instructions stored in the memory 512, and the functions / implementation processes of the transceiver module 1502 in Figure 15 can be implemented by the transceiver 515 in the retrieval device 510 shown in Figure 5.

[0192] Since the retrieval device 150 provided in this application embodiment can execute the above-described model training method, the technical effects it can achieve can be referred to the above-described method embodiment, and will not be repeated here.

[0193] In one possible implementation, this application embodiment also provides a retrieval device (e.g., the retrieval device may be a chip or a chip system), which includes a processor for implementing the methods in any of the above method embodiments. In one possible design, the retrieval device further includes a memory. The memory is used to store necessary program instructions and data, and the processor can call the program code stored in the memory to instruct the retrieval device to execute the methods in any of the above method embodiments. Of course, the memory may not be included in the retrieval device. When the retrieval device is a chip system, it may be composed of chips or may include chips and other discrete devices; this application embodiment does not specifically limit this.

[0194] In one possible implementation, this application also provides a computer-readable storage medium storing a computer program or instructions that, when run on a retrieval device, enable the retrieval device to execute the methods of any of the above-described method embodiments or any implementation thereof.

[0195] In one possible implementation, this application embodiment also provides a model training method, which includes the method of any of the above-described method embodiments or any implementation thereof.

[0196] In one possible implementation, this application embodiment also provides a retrieval system, which includes the first device described in the above method embodiment.

[0197] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0198] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0199] Although this application has been described in conjunction with specific features and embodiments, it is apparent that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are exemplary illustrations of this application as defined by the appended claims and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. A retrieval method characterized by, include: Receive user input for user questions; In response to the user's question, an answer corresponding to the user's question is displayed; wherein the answer is obtained based on a first prompt word; The first prompt word is a prompt word constructed based on at least one first knowledge; the at least one first knowledge includes at least one of the following: knowledge related to the user's question retrieved from a first type of knowledge in the knowledge base based on a first retrieval method, or knowledge related to the user's question retrieved from a second type of knowledge in the knowledge base based on a second retrieval method; the first retrieval method and the second retrieval method are different retrieval methods.

2. The method according to claim 1, characterized in that, The first retrieval method is vector retrieval; The second search method is keyword search.

3. The method according to claim 1 or 2, characterized in that, The first type of knowledge and the second type of knowledge include knowledge related to at least one of the following: text, image, and table.

4. The method according to any one of claims 1-3, characterized in that, The first type of knowledge includes at least one of the following: a question field in knowledge of the question-and-answer type, a semantic description field in knowledge of the table type, an image context information field in knowledge of the image type, and a title field and a content field in knowledge of the text type.

5. The method according to any one of claims 1 to 4, characterized in that, The second type of knowledge includes at least one of the following: answer fields in knowledge of question-and-answer type, table text fields in knowledge of table type, image content fields in knowledge of image type, and title and content fields in knowledge of text type.

6. The method according to any one of claims 1 to 5, characterized in that, The process of constructing the first prompt word based on at least one piece of first knowledge includes: Determine the second piece of knowledge that is most relevant to the user's question from the at least one piece of first knowledge; Based on the second piece of knowledge, prompt words for the user's question are constructed.

7. The method of claim 6, wherein, The step of constructing prompt words for the user's question based on the second knowledge includes: Determine the type of the second knowledge; Select a prompt word template and / or at least one first prompt word example that match the type of the second knowledge; Based on the user question, the second knowledge, and / or the at least one first prompt word example, the prompt word template is filled to determine the prompt word for the user question.

8. The method of claim 7, wherein, The step of filling the prompt word template with the prompt word template based on the user question, the second knowledge, and / or the at least one first prompt word example to determine the prompt word for the user question includes: Select the second prompt word example that is most relevant to the user's question from the at least one first prompt word example; The prompt word template is filled with the user question, the second knowledge, and the second prompt word sample to obtain the prompt word for the user question.

9. The method according to any one of claims 1 to 8, characterized in that, The process of determining the answer based on the first clue word includes: Input the prompt words into the large language model and obtain the output of the large language model; Based on the output of the large language model, the answer to the user's question is determined.

10. A retrieval device characterized by, include: A functional unit for performing the method as described in any one of claims 1-9; wherein the action performed by the functional unit is implemented by hardware or by hardware executing corresponding software.

11. A retrieval device, characterized in that include: processor; The processor is connected to a memory for storing computer execution instructions. The processor executes the computer execution instructions stored in the memory to enable the retrieval device to implement the method as described in any one of claims 1-9.

12. A computer-readable storage medium, characterized in that, Includes instructions that, when executed on a computer, cause the computer to perform the method as described in any one of claims 1-9.

13. A chip, characterized by The chip includes a processor; the processor is connected to a memory for storing computer execution instructions, and the processor executes the computer execution instructions stored in the memory to enable the retrieval device to implement the method as described in any one of claims 1-9.

14. A computer program product comprising instructions, characterized in that, When it is operated on a retrieval device, it causes the retrieval device to perform the method as described in any one of claims 1-9.

Images