Method and apparatus for retrieval enhancement generation

Abstract

A method for retrieval enhancement generation includes determining a retrieval routing factor for a first retrieval based at least in part on an input including a first query, where the retrieval routing factor indicates a ratio between numbers of retrieval results in a plurality of retrieval sources; retrieving a result in at least one of the plurality of retrieval sources based at least in part on a retrieval routing factor and the input; and obtaining a first response based at least in part on the retrieved result and the input.

Description

Technical Field

[0001] This application relates to artificial intelligence technology, and more specifically, to methods and apparatus for retrieving augmented generation (RAG). Background Technology

[0002] Language models (LMs) (e.g., large language models (LLMs)) can output responses based on user input (e.g., queries). However, LMs such as LLMs can suffer from the "hallucination" problem and may produce inaccurate responses that do not reflect reality. The RAG process can mitigate the "hallucination" problem by providing data outside the LM, allowing relevant data to be retrieved based on, for example, the user query, and taking the retrieved data into account when the LM generates its response.

[0003] However, existing LM systems using RAG typically employ inflexible and unadaptive retrieval strategies. For example, regardless of the user's query, the LM system retrieves data based on the query and generates a response based on the retrieved data. However, for some user queries (e.g., "write an essay about summer vacation"), this retrieval process may be time-consuming, and the retrieved data may not be very helpful for the LM to generate a response. Another example is that RAG-based LM systems retrieve data from a pre-built knowledge base. However, on the one hand, the knowledge base may not contain any information helpful in answering the query; on the other hand, maintaining and updating the knowledge base usually consumes a lot of resources and time, resulting in the knowledge base possibly not containing the latest information. In this case, the LM may not be able to generate a satisfactory response based on the data retrieved from the knowledge base.

[0004] Therefore, an improved RAG-based LM system is desired, which can flexibly determine retrieval strategies so that the LM can generate more accurate responses more efficiently based on user input. Summary of the Invention

[0005] The following brief introduction is provided to present some of the selected concepts in a simplified manner, which will be further described in the detailed description that follows. This brief introduction is not intended to highlight the key or essential features of the claimed subject matter, nor is it intended to limit the scope of the claimed subject matter.

[0006] To address the aforementioned problems, this application provides a novel computer implementation method for RAG, and correspondingly, a novel architecture for a neural network (NN) model for RAG. By employing the methods of the various embodiments of this application, the accuracy of the neural network model (e.g., LM) in generating responses to queries can be improved, and flexible retrieval strategies and improved efficiency can be provided.

[0007] According to one aspect of this application, a method for RAG is provided, comprising: determining a retrieval routing factor for a first retrieval based at least in part on input including a first query, wherein the retrieval routing factor indicates a ratio between the number of retrieval results in a plurality of retrieval sources; retrieving results in at least one of the plurality of retrieval sources based at least in part on the retrieval routing factor and the input; and generating a first response based at least in part on the retrieved results and the input.

[0008] According to one aspect of this application, a method for training an NN model for RAG is provided, comprising: determining, at least in part based on a first query, a first predicted value for a retrieval routing factor via a retrieval routing module in the NN model, wherein the retrieval routing factor indicates a proportion between the number of retrieval results in a plurality of retrieval sources; and updating learnable parameters of the NN model, at least in part based on the first predicted value and a label value for the retrieval routing factor.

[0009] According to one aspect of this application, an apparatus for RAG is provided, comprising: a retrieval routing module that determines a retrieval routing factor for a first retrieval based at least in part on an input including a first query, wherein the retrieval routing factor indicates a ratio between the number of retrieval results in a plurality of retrieval sources; a retrieval module that retrieves results in at least one of the plurality of retrieval sources based at least in part on the retrieval routing factor and the input; and a response module that obtains a first response based at least in part on the retrieved results and the input.

[0010] According to one aspect of this application, an apparatus for RAG is provided, comprising: one or more processors; and one or more memories storing computer-executable instructions that, when executed by the one or more processors, perform operations for RAG according to embodiments of this application.

[0011] According to one aspect of this application, a processing apparatus is provided, comprising: one or more processors; and one or more memories storing computer-executable instructions that, when executed by the one or more processors, perform operations for RAG according to embodiments of this application.

[0012] According to one aspect of this application, a machine-readable storage medium is provided that stores executable instructions that, when executed, cause one or more processors to perform operations for RAG according to embodiments of this application.

[0013] According to one aspect of this application, a computer program product is provided, which includes executable instructions that, when executed, cause one or more processors to perform operations for RAG according to embodiments of this application.

[0014] According to various aspects of this disclosure, the technical solution for RAG provided by this disclosure includes a determination module and a retrieval routing module, wherein the determination module determines whether to perform a retrieval based at least in part on the input, and the retrieval routing module determines the ratio between the number of retrieval results in multiple retrieval sources. By employing this technique, it is ensured that the system performs a retrieval only when required, and retrieves an appropriate amount of data from retrieval sources appropriate to the user's query, thereby improving the execution efficiency of RAG and the accuracy of LM's generated response. Other advantages of various aspects of this disclosure will be described below. Attached Figure Description

[0015] A further understanding of the nature and advantages of this application can be achieved by referring to the accompanying drawings. In the drawings, similar components or features may have the same reference numerals.

[0016] Figure 1 A block diagram of an apparatus for RAG according to one embodiment is shown.

[0017] Figure 2 A schematic diagram of a determining module according to one embodiment is shown.

[0018] Figure 3A A schematic diagram of a retrieval routing module according to one embodiment is shown.

[0019] Figure 3B A schematic diagram for training a retrieval routing module is shown according to one embodiment.

[0020] Figure 4 A schematic diagram of a retrieval module according to one embodiment is shown.

[0021] Figure 5 A schematic diagram of a query generation module according to one embodiment is shown.

[0022] Figure 6 A schematic diagram of a response module according to one embodiment is shown.

[0023] Figure 7 A flowchart of a method for RAG according to one embodiment is shown.

[0024] Figure 8 A flowchart of a method for training a neural network model for RAG is shown according to one embodiment.

[0025] Figure 9A block diagram of an apparatus for RAG according to one embodiment is shown.

[0026] Figure 10 A block diagram of a processing apparatus according to one embodiment is shown. Detailed Implementation

[0027] The subject matter described herein will now be discussed with reference to exemplary embodiments. It should be understood that these embodiments are discussed merely to enable those skilled in the art to better understand and implement the subject matter described herein, and are not intended to limit the scope, applicability, or examples set forth in the claims. The function and arrangement of the elements discussed may be changed without departing from the scope of this application. Various processes or components may be omitted, substituted, or added as needed in the various examples. For example, the described methods may be performed in a different order than described, and steps may be added, omitted, or combined. Furthermore, features described in some examples may be combined in other examples.

[0028] As used herein, the term "comprising" and its variations are open terms meaning "including but not limited to". The term "based on" means "at least partially based on". The terms "one embodiment" and "an embodiment" mean "at least one embodiment". The term "another embodiment" means "at least one other embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other definitions, whether explicit or implicit, may be included below. Unless explicitly indicated by the context, the definition of a term remains consistent throughout the specification.

[0029] Figure 1 A block diagram of an apparatus for RAG according to one embodiment is shown. Figure 1 In this example, device 100 includes an input module 110, a determination module 120, a retrieval routing module 130, a retrieval module 140, and a response module 150. It is understood that device 100 may include other modules. Figure 1 Only modules relevant to embodiments of this disclosure are shown.

[0030] exist Figure 1In the example, input module 110 receives input I 105 that includes a first query. For example, input I 105 may be from a user or another upstream module (not shown) of device 100. The first query may include a domain-specific query (e.g., a query for the domain "manufacturing engineering" such as "How can I optimize the shape of a part to reduce material waste and processing costs?"), or it may include a general query that is not domain-specific (e.g., "How do I create a new blank document?" or "Write an essay about summer vacation"). In the example where the first query includes a domain-specific query, input I 105 may also include a domain associated with the first query (e.g., "manufacturing engineering", "automotive parts manufacturing engineering", "vehicle exterior design", "fuel cells", "semiconductors", etc.). In some embodiments, input I 105 may also include feedback or a request associated with a response. For example, device 100 may have provided a more general previous response to a previous input; in this case, input I 105 may include feedback on the previous response (e.g., "The response should be more specific"); or input I 105 may include a request for a response to the current first query (e.g., "Please provide the most specific answer possible").

[0031] Based at least in part on input I 105, input module 110 determines a feature representation (e.g., vector features) R 115 corresponding to input I 105. For example, input module 110 may perform text embedding on input I 105 to obtain feature representation R 115. A suitable embedding model (e.g., BERT) may be used to transform input I 105 into feature representation R 115. Input module 110 may output the determined feature representation R 115 to other modules of device 100 (e.g., determination module 120 and response module 150).

[0032] The determining module 120 can determine, at least in part, whether to perform a first retrieval based on input I 105. Figure 1In the example, determining module 120 may determine whether to perform a first retrieval based at least in part on a feature representation R 115 from input module 110, which is based on input I 105. For example, determining module 120 may include a language model (LM) (e.g., a large language model (LLM)). In one embodiment, for an input (e.g., a query) where the LM lacks the required knowledge, determining module 120 may determine to perform a first retrieval for the query; while for an input where the LM has sufficient relevant knowledge, determining module 120 may determine not to perform a retrieval, thereby avoiding the reduced efficiency caused by performing unnecessary retrievals. For example, for a first query in input I 105, “What are the causes of small-scale fading in wireless communication channels?”, if input I 105 does not include the domain associated with the first query, determining module 120 may determine that the LM lacks relevant knowledge and determine to perform a first retrieval. For example, input I 105 may include a first query "What are the causes of small-scale fading in wireless communication channels?", the domain "wireless communication", and a requirement associated with the response "a response as detailed as possible". Based on this input I 105, determining module 120 can obtain the response "multipath effect, Doppler effect, and signal transmission bandwidth" through LM, but determines that the response does not meet the above requirements, and thus determines to perform the first search. As another example, for the first query in input I 105 "Please write an essay about your hobbies", determining module 120 can determine that LM does not lack relevant knowledge and meets the response requirements, and therefore determines not to perform the first search.

[0033] In one embodiment, in response to determining module 120's decision to perform a first retrieval, determining module 120 may output input I 105 or feature representation R 115 to other modules of device 100 (e.g., retrieval routing module 130 or retrieval module 140) for device 100 to further determine a retrieval strategy for RAG or to perform a retrieval. Alternatively or additionally, determining module 120 may, in response to determining to perform the first retrieval, output a second response OT 125 obtained based on input I 105 or feature representation R 115 to other modules of device 100 (e.g., retrieval routing module 130 or retrieval module 140). For example, in an example where input I 105 includes a first query “What are the causes of small-scale fading in wireless communication channels?”, the domain “wireless communication”, and the associated requirement “a response as detailed as possible”, determining module 120 may output a second response OT 125 including at least one term “multipath effect”, “Doppler effect”, and “signal transmission bandwidth” to retrieval module 140 to retrieve data about these terms, thereby obtaining a more detailed response to meet the response requirements. In one embodiment, the terms in the second response OT125 may have the form of feature representations (e.g., represented by feature vectors). In one embodiment, in response to the determination module 120 determining not to perform the first retrieval, the determination module 120 may output (e.g., via LM in the determination module 120) the second response obtained based on input I 105.

[0034] The device 100 for RAG in this application can be associated with multiple search sources. For example, the device 100 for RAG can be associated with multiple search sources including at least two of the following: a knowledge base, a customer service system, an e-commerce system, or a search engine. A knowledge base can store knowledge documents, which in some embodiments may be offline or not publicly available. A customer service system can be a specialized customer service system for a specific product or field (e.g., a training system for training manufacturing engineering professionals) or a general customer service system (e.g., a chatbot). An e-commerce system can be associated with e-commerce data, such as product-related specifications, prices, or sales volume. A search engine can be associated with search functions of various applications or platforms, such as web search or search functions within applications. Those skilled in the art will understand that the above search sources are merely examples, and the device 100 can also be associated with various other search sources. By associating with multiple search sources, the RAG of the device 100 can be associated with more comprehensive data and has advantages related to the characteristics of each search source. For example, by linking to a knowledge base, RAG can be associated with knowledge or experience in a professional field; by linking to a search engine, newer information can be introduced to RAG without frequently updating the knowledge base.

[0035] The retrieval routing module 130 may determine a retrieval routing factor α135 in response to determining that a first retrieval is to be performed, which indicates the proportion between the number of retrieval results in multiple retrieval sources. For example, the retrieval routing module 130 may determine the retrieval routing factor α135 in response to receiving input I 105 or feature representation R 115 or a second response OT 125.

[0036] The retrieval routing module 130 can determine a retrieval routing factor α135 for a first retrieval, at least in part, based on the input I 105 or feature representation R 115, which includes a first query. For example, device 100 can be associated with a customer service system and a search engine for software A. Based on the first query "How to create a new blank document in software A?" in input I 105, the retrieval routing module 130 can determine that the query is more suitable for retrieving results in the customer service system of software A, and determine that the retrieval routing factor α135 = number of search results in the customer service system / number of search results in the search engine = 4:1. In some embodiments, the retrieval routing factor α135 can be normalized. Continuing with the above example, the retrieval routing factor α135 can be 0.8. In some embodiments, as described above, the determining module 120 can output a second response OT 125 obtained based on input I 105 or feature representation R 115 to the retrieval routing module 130, and accordingly, the retrieval routing module 130 can determine the retrieval routing factor α135 based on the second response OT 125. For example, device 100 can be associated with a knowledge base and an e-commerce system. Based on the term "Doppler effect" in the second response OT125, retrieval routing module 130 can determine that it is more appropriate to retrieve results in the knowledge base and determine the retrieval routing factor α135 = number of retrieval results in the knowledge base / number of retrieval results in the e-commerce system = 1:0. Alternatively, the retrieval routing factor α135 can be normalized and equal to 1.

[0037] The retrieval routing module 130 can output the retrieval routing factor α135 to other modules of the device 100 (e.g., retrieval module 140).

[0038] The retrieval module 140 can retrieve results D 145 in at least one of the plurality of retrieval sources, at least in part, based on the retrieval routing factor α135 and the input I 105. In some embodiments, as described above, the retrieval module 140 can receive a feature representation R 115 based on a first query and / or a second response OT 125, and the retrieval module 140 can determine the ratio of the number of retrieval sources to be searched and the corresponding number of retrieval results based on the retrieval routing factor α135, and perform a search in the determined retrieval sources at least based on the feature representation R 115 and / or the second response OT 125 to obtain a corresponding number of retrieval results D 145 that satisfy the retrieval routing factor α135. For example, based on the (normalized) retrieval routing factor α135 indicating that the ratio of the number of search results in the professional knowledge base to the number of search results in the search engine is 0.8 (corresponding to 4:1), and based on the feature representation R 115 of the first query "How to optimize the shape of a part to reduce material waste and processing costs?", the retrieval module 140 can retrieve results in the professional knowledge base and the search engine respectively, based on the feature representation R 115, and the ratio of the first number of search results obtained by the retrieval module 140 in the professional knowledge base to the second number of search results obtained in the search engine is 4:1.

[0039] The retrieval module 140 can combine the retrieval results D 145 obtained from various retrieval sources and output them to other modules of the device 100 (e.g., response module 150).

[0040] The response module 150 may obtain a first response O 155 based at least in part on the retrieved result D 145 and the input I 105. In some embodiments, the response module 150 may obtain the first response O 155 based at least in part on the retrieved result D 145 and the feature representation R 115 based on the input I 105.

[0041] For example, response module 150 may include an LM (e.g., an LLM), which may be the same as or different from the LM described herein with respect to other embodiments. For example, the LM in determination module 120 may be the same LM as the LM in response module 150, or it may be a different LM. Based at least in part on the retrieved result D 145 and input I 105, the LM in response module 150 may generate a first response O 155. For example, since the first response O 155 is based on the result D 145 retrieved in a suitable proportion from a suitable search source, response module 150 is able to provide a first response O 155 with rich or comprehensive content more accurately, thereby better meeting the user's needs.

[0042] Figure 2 A schematic diagram of a determining module 120 according to one embodiment is shown. Figure 2 In the example, the determining module 120 may include an LM module 210 and a threshold comparison module 220. Optionally, the determining module 120 may also include a requirement comparison module 230 and a decision module 240. It is understood that the determining module 120 may include other modules. Figure 2 Only modules relevant to embodiments of this disclosure are shown.

[0043] The LM module 210 can obtain a second response OT 215, at least in part, based on the input, wherein the second response OT 215 includes at least one lexical and a probability associated with each of the at least one lexical. Those skilled in the art will understand that the LM module 210 can be implemented using any suitable language model, trained or pre-trained, such as ChatGPT or LlaMA. ​​For example, the probability associated with a lexical can indicate the confidence level of the LM module 210 for that lexical. For example, based at least in part on input including the first query “What are the classifications of small-scale fading in wireless communication channels?” and the domain “wireless communication”, LM module 210 can obtain a second response OT 215 including at least one term, “flat fading, frequency-selective fading, fast fading, and shadowing fading”, wherein the at least one term includes four terms T1-T4: {T1: “flat fading”, T2: “frequency-selective fading”, T3: “fast fading”, T4: “shadowing fading”}, and the second response OT 215 also includes probabilities P1-P4 in the probability set P corresponding to the terms T1-T4 in the at least one term. In some embodiments, LM module 210 can receive at least a feature representation R 205 based at least in part on the input to obtain the second response OT 215 including at least one term and the probability associated with each of the at least one term. LM module 210 can output the second response OT 215 together with the input or the input-based feature representation R 205 to threshold comparison module 220.

[0044] The threshold comparison module 220 can determine whether to perform the first retrieval or output the at least one word in the second response based on a comparison between the probability associated with a word in at least one word and a first threshold. The first threshold may be a pre-set threshold.

[0045] Continuing the example above, the threshold comparison module 220 can compare the probabilities P1-P4 corresponding to each term in the second response OT 215 (terms T1-T5) with a first threshold. In one example, the threshold comparison module 220 can determine that probabilities P1-P3 among probabilities P1-P4 are greater than the first threshold, and probability P4 is less than the first threshold. In this example, probabilities P1-P3 greater than the first threshold can indicate that the LM module 210 has high confidence in terms T1-T3, while probabilities P4 less than the first threshold can indicate that the LM module 210 has insufficient confidence in terms T4. Based on the comparison, the threshold comparison module 220 can determine that there are terms in the second response OT 215 whose associated probabilities are below the threshold, and therefore determines to perform a first retrieval. In one embodiment, based on determining to perform a first retrieval, the determining module 120 can output the second response OT 215 or the input or the input-based feature representation R 205 to other modules (e.g., retrieval routing module 130 and retrieval module 140). In embodiments where the determining module 120 also includes a request comparison module 230 and a decision module 240, based on the determination to perform a first retrieval, the threshold comparison module 220 may output a second response OT 215 and an input or input-based feature representation R 205 to the decision module 240 to instruct the decision module 240 to perform the first retrieval. Those skilled in the art will understand that the threshold comparison module 220 may transmit any appropriate information to the decision module 240 to instruct the performance of the first retrieval.

[0046] In another example, threshold comparison module 220 may determine that all probabilities associated with all terms in the second response OT 215 satisfy (e.g., greater than or equal to) a first threshold. In this example, the probability that all probabilities satisfy the first threshold may indicate that LM module 210 has a sufficiently high confidence level for all terms in the second response OT 215. In one embodiment, based on the comparison, threshold comparison module 220 may determine not to perform the first retrieval and output at least one term included in the second response OT 215 as a response to the input. In an embodiment where determination module 120 also includes request comparison module 230 and decision module 240, based on the determination not to perform the first retrieval, threshold comparison module 220 may output the second response OT 215 to decision module 240 to indicate to decision module 240 not to perform the first retrieval. Those skilled in the art will understand that threshold comparison module 220 may transmit any appropriate information to decision module 240 to indicate not to perform the first retrieval.

[0047] Optionally, the request comparison module 230 may determine whether to perform the first retrieval based on feedback or requests associated with the response in the second response OT 215 and the input or input-based feature representation R 205. For example, the request comparison module 230 may determine the feedback or request associated with the response in the input, or its corresponding feature representation in the input-based feature representation R 205. For example, conditions corresponding to the feedback or request associated with the response may be preset. For example, feedback or requests with semantics such as “accurate” or “factual” may correspond to the condition that the probability associated with all terms in the second response OT 215 should be greater than a request probability threshold, wherein the request probability threshold may be a higher threshold than the first threshold used in the threshold comparison module 210. As another example, feedback or requests with semantics such as “specific” or “detailed” may correspond to the condition that the number of words in the second response OT 215 should not be less than a request word count threshold. Those skilled in the art will understand that the above are merely examples, and appropriate conditions corresponding to any feedback or request may be preset.

[0048] Based on the determined feedback or requirement, the requirement comparison module 230 can determine whether the second response OT 215 meets the recognition conditions. For example, based on the requirement that the input includes "accurate", the requirement comparison module 230 can use a requirement probability threshold, as described above for a similar threshold comparison module 220, to determine whether the second response OT 215 meets the requirement probability threshold. In one embodiment, based on the probability that the probability associated with all terms in the second response OT 215 meets the requirement probability threshold, the requirement comparison module 230 can determine not to perform the first retrieval. Based on determining not to perform the first retrieval, the threshold comparison module 220 can output the second response OT 235 to the decision module 240. In another embodiment, based on the probability that the probability associated with at least one term in the second response OT 215 does not meet the requirement probability threshold, the requirement comparison module 230 can determine to perform the first retrieval. Based on determining to perform the first retrieval, the threshold comparison module 220 can output the second response OT 215 and the input or the feature representation R 205 based on the input to the decision module 240. Those skilled in the art will understand that the comparison module 230 may transmit any appropriate information to the decision module 240 to indicate whether or not to perform the first search.

[0049] The determination module 240 can determine whether to perform a first search based on inputs from the threshold comparison module 220 (and optionally the requirement comparison module 230). For example, if the input from either the threshold comparison module 220 or the requirement comparison module 230 indicates that the first search should be performed, the determination module 240 can determine that the first search should be performed and output a second response OT 215 and the input or input-based feature representation R 205. For example, if the inputs from both the threshold comparison module 220 and the requirement comparison module 230 indicate that the first search should not be performed, the determination module 240 can determine that the first search should not be performed and output a second response OT 215 as a response to the input.

[0050] Figure 3A A schematic diagram of a retrieval routing module 130 according to one embodiment is shown. Figure 3A In this example, the retrieval routing module 130 may include a first neural network module 310 and / or a factor determination module 320. It is understood that the retrieval routing module 130 may include other modules, in... Figure 3A Only modules relevant to embodiments of this disclosure are shown.

[0051] In one embodiment, the retrieval routing module 130 may include a first neural network module 310. The first neural network module 310 may determine a retrieval routing factor α325 for a first retrieval based on a first query or a feature representation R 305 of the first query, wherein the retrieval routing factor α325 indicates the proportion between the number of retrieval results in multiple retrieval sources. For example, the first neural network module 310 may determine the retrieval routing factor α325 based on the first query in the input, or the representation in the feature representation R 305 corresponding to the first query. For example, the first neural network module 310 may determine the retrieval routing factor α325 based on the input containing the first query, or the feature representation R 305 corresponding to the input. Those skilled in the art will understand that the first neural network module 310 may employ any neural network structure capable of implementing the above-described functions; for example, the first neural network module 310 may include an LM (e.g., an LLM), which may be the same as or different from the LM described herein with respect to other graphs. In this embodiment, the first neural network module 310 may be based on, for example, references herein. Figure 3B The described examples are used for training.

[0052] In another embodiment, the retrieval routing module 130 may include a first neural network module 310 and a factor determination module 320. In this embodiment, the first neural network module 310 may include any suitable neural network (e.g., an LM) that has been trained or pre-trained, generating a further feature representation Q 315 based on a first query or a feature representation R 305 based on the first query. The first neural network module 310 may be the same as or different from the LM described herein with respect to other graphs. The factor determination module 320 may be a second neural network used to generate retrieval routing factors α325 based on the feature representation Q 315 output by the first neural network module 310. In this embodiment, the factor determination module 320 may be based on, for example, references herein. Figure 3B The described embodiments are used for training. Using the factor determination module 320, the function of obtaining the retrieval routing factor α325 based on the input or feature representation R 305 can be achieved by training only a smaller neural network (factor determination module 320) without retraining the potentially larger first neural network module 310, without having to retrain it.

[0053] In another embodiment, the retrieval routing module 130 may include a first neural network module 310 and a factor determination module 320. In this embodiment, the first neural network module 310 may include any suitable neural network (e.g., an LM) that has been trained or pre-trained, determines a keyword W 335 corresponding to the first query or its feature representation R 305 based on the first query, and outputs the determined keyword W 335 to the factor determination module 320. The determined keyword W 335 may include keywords identified from the first query or its feature representation R 305, and / or may include synonyms (or their feature representations) that are semantically similar to words in the first query or its feature representation R 305. Those skilled in the art will understand that the first neural network module 310 may employ any neural network structure capable of achieving the above-described functions. For example, the first neural network module 310 may include an LM (e.g., an LLM), which may be the same as or different from the LMs described herein with respect to other graphs.

[0054] The factor determination module 320 can determine the retrieval routing factor α325 based on the keyword W 335. For example, the factor determination module 320 may include a third neural network for generating the retrieval routing factor α325 based on the keyword W 335 output by the first neural network module 310. Those skilled in the art will understand that the third neural network can employ any neural network structure capable of achieving the above functionality. The factor determination module 320 may be based on, for example, referenced herein. Figure 3B The described examples are used for training.

[0055] For example, the factor determination module 320 may include calculation logic for determining retrieval routing factors based on keywords. For instance, this calculation logic may include mappings from keywords to retrieval sources. Optionally, the calculation logic may also include weight information associated with each mapping. For example, for the keyword "size" in the first query, the calculation logic may include a mapping from "size" to "e-commerce system" (with a weight of 0.8) and a mapping from "size" to "customer service system" (with a weight of 0.2). For the keyword "benchmark interest rate," the calculation logic may include a mapping from "benchmark interest rate" to "search engine" (with a weight of 0.5) and a mapping from "benchmark interest rate" to "knowledge base" (with a weight of 0.5). The factor determination module 320 may calculate the retrieval routing factor α325 based on the mappings (and associated weights) from at least a portion of the keywords in keyword W 335 to retrieval sources. For example, the retrieval routing factor α325 may be based on a weighted average of the weights to each retrieval source, or it may be based on the number of mappings to different retrieval sources.

[0056] In another embodiment, the retrieval routing module 130 may include only the factor determination module 320, wherein the factor determination module 320 may include computational logic. For example, the computational logic may include a keyword library, and the factor determination module 320 may therefore identify keywords or key feature representations based on the keyword library in the first query or the feature representation R 305 based on the first query. The computational logic may also include a mapping from keywords to retrieval sources, and the factor determination module 320 may determine the retrieval routing factor α325 based on the determined keywords or feature representations, for example, in a manner similar to that described above.

[0057] Figure 3B A schematic diagram of a retrieval routing module 130 for training according to one embodiment is shown. During the training phase, the retrieval routing module 130 may include a loss function module 330, a first neural network module 310, and optionally a factor determination module 320. The loss function module 330 may include any suitable loss function, including the L1Loss function, the MSELoss function, or the cross-entropy function, etc., and the first neural network module 310 and the factor determination module 320 may implement [the following information about loss function]. Figure 3A The described function. During the training phase, training data can be provided to the retrieval routing module 130, which includes query or query-based feature representations R 345 and corresponding labeled values ​​L 365 for retrieval routing factors.

[0058] In an embodiment where the retrieval routing module 130 includes a first neural network module 310 for determining retrieval routing factors, the first neural network module 310 may determine a first predicted value P 355 for the retrieval routing factors based on training data (e.g., queries in the training data) or based on a feature representation R 345 of the input (e.g., the query). The loss function module 330 may update the learnable parameters of the first neural network module 310 at least in part based on the first predicted value P 355 and a labeled value L 365 related to the retrieval routing factors. For example, the loss function module 330 may update the learnable parameters of the first neural network module 310 based on the difference between the first predicted value P 355 and the labeled value L 365. For example, the loss function module 330 may determine a loss value based on the first predicted value P 355 and the labeled value L 365, and update the learnable parameters of the first neural network module 310 based on the loss value. It is understood that any suitable optimization method may be employed to update the learnable parameters of the neural network module.

[0059] In an embodiment where the retrieval routing module 130 includes a first neural network module 310 and a factor determination module 320, wherein the first neural network module 310 is used to generate a further feature representation Q 375, and the factor determination module 320 is a second neural network used to determine retrieval routing factors based on the further feature representation Q 375, the first neural network module 310 may generate the further feature representation Q 375 based on a query in the training data or based on a query feature representation R 345, and the factor determination module 320 may determine a first predicted value P 355 for the retrieval routing factors based on the further feature representation Q 375. The loss function module 330 may update the learnable parameters of the factor determination module 320 at least in part based on the first predicted value P 355 and a labeled value L 365 for the retrieval routing factors. For example, the loss function module 330 may update the learnable parameters of the factor determination module 320 based on the difference between the first predicted value P 355 and the labeled value L 365. For example, the loss function module 330 can determine the loss value based on the first predicted value P 355 and the label value L 365, and update the learnable parameters of the first neural network module 310 based on the loss value.

[0060] In an embodiment where the retrieval routing module 130 includes a first neural network module 310 and a factor determination module 320, wherein the first neural network module 310 is used to determine the keyword W 385, and the factor determination module 320 is a third neural network used to determine retrieval routing factors based on the keyword W 385, the first neural network module 310 may determine the keyword W 385 based on a query in the training data or based on a feature representation R 345 of the query, and the factor determination module 320 may determine a first predicted value P 355 for the retrieval routing factors based on the keyword W 385. The loss function module 330 may update the learnable parameters of the factor determination module 320 at least in part based on the first predicted value P 355 and a labeled value L 365 for the retrieval routing factors. For example, the loss function module 330 may update the learnable parameters of the factor determination module 320 based on the difference between the first predicted value P 355 and the labeled value L 365. For example, the loss function module 330 can determine the loss value based on the first predicted value P 355 and the label value L 365, and update the learnable parameters of the first neural network module 310 based on the loss value.

[0061] Figure 4 A schematic diagram of a retrieval module 140 according to one embodiment is shown. Figure 4 In this example, the retrieval module 140 includes a query generation module 410 and a retrieval execution module 420, wherein the retrieval execution module 420 can be associated with multiple retrieval sources. It is understood that the retrieval module 140 may include other modules. Figure 4 Only modules relevant to embodiments of this disclosure are shown.

[0062] The query generation module 410 can generate a second query Q 425 corresponding to the first search, at least in part, based on the retrieval routing factor α 405. The second query Q 425 includes a query for the first retrieval source among multiple retrieval sources and / or a query for the second retrieval source among multiple retrieval sources. In one embodiment, the query generation module 410 can generate queries for each retrieval source separately, at least in part, based on the different retrieval sources targeted by the retrieval routing factor α 405. For example, in an example where multiple retrieval sources include a professional knowledge base, a search engine, and an e-commerce system, if the retrieval routing factor α 405 indicates a 4:1:0 ratio of search results in the professional knowledge base, the search engine, and the e-commerce system, the query generation module 410 can generate queries specifically for the professional knowledge base and the search engine. The corresponding queries and retrieval routing factors α 405 in the second query Q 425 generated by the query generation module 410 can be output to the corresponding retrieval sources in the retrieval execution module 420.

[0063] The retrieval execution module 420 can obtain a corresponding number of results in the respective retrieval sources, at least in part, based on the second query Q 425 and the retrieval routing factor α 405. For example, the retrieval execution module 420 can obtain a first number of results retrieved in the first retrieval source, at least in part, based on the query for the first retrieval source in the second query Q 425, and / or obtain a second number of results retrieved in the second retrieval source, at least in part, based on the query for the second retrieval source in the second query Q 425, wherein the ratio between the first number and the second number satisfies the retrieval routing factor α 405. In one example, the retrieval execution module 420 can call a function for the retrieval source (e.g., call an application programming interface (API) for the retrieval source, or call a utility function for the retrieval source) to obtain a corresponding number of results in the retrieval source based on a query for that retrieval source.

[0064] In one example, the total number of search results K can be preset, and the search execution module 420 can obtain a corresponding number of results from the respective search sources based on the total number K and the search routing factor α405. In another example, the search execution module 420 can determine the number of search results obtained from different search sources based on the search routing factor α405 and the minimum or maximum number of search results in different search sources, so as to satisfy the search routing factor α405. For example, based on the search routing factor α405 indicating that the ratio of the number of search results in the knowledge base and the search engine is 4:1, and based on the search execution module 420 using the second query Q 425 for the knowledge base and the search engine, retrieving 7 results in the knowledge base and 20 results in the search engine, the search execution module 420 can determine to obtain 4 results from the knowledge base and 1 result from the search engine. In some examples, the ratio between the number of search results in the respective search sources satisfying the search routing factor α405 can include: the ratio between the number of search results in the respective search sources being within the threshold range of the search routing factor α405. For example, continuing the example above, if the retrieval routing factor α405 indicates a ratio of 4:1 between the number of search results in the knowledge base and the search engine, the threshold range for the retrieval routing factor can be determined to be between 4:1 and (4+1):(1+1) = 5:2. If the retrieval execution module 420 retrieves 7 results from the knowledge base and 20 results from the search engine, the retrieval execution module 420 can determine that it obtained 7 results from the knowledge base and 2 results from the search engine, because 7:2 is within the threshold range of the retrieval routing factor α405.

[0065] The retrieval execution module 420 can combine the retrieval results obtained from the corresponding retrieval source to obtain retrieval result D435. In some embodiments, the retrieval execution module 420 can concatenate the retrieval results obtained from the corresponding retrieval source to obtain retrieval result D435. Those skilled in the art will understand that any appropriate combination of retrieval results can also be used to obtain retrieval result D435. The retrieval execution module 420 can output retrieval result D435 to other modules (e.g., response module 150).

[0066] Figure 5 A schematic diagram of a query generation module 410 according to one embodiment is shown. Figure 5 In the example, the query generation module 410 includes an optional routing threshold comparison module 530, and includes a first retrieval source query generation module 540 and a second retrieval source query generation module 550. It can be understood that the query generation module 410 may include other modules. Figure 5 Only modules relevant to embodiments of this disclosure are shown.

[0067] Optionally, the query generation module 410 may include a routing threshold comparison module 530. The routing threshold comparison module 530 may determine to generate a query for the first retrieval source in response to a first condition being met by comparing the retrieval routing factor α505 with a second threshold; determine to generate a query for the second retrieval source in response to a second condition being met by comparing the retrieval routing factor α505 with a third threshold; or determine to generate both a query for the first retrieval source and a query for the second retrieval source in response to the retrieval routing factor α505 being between the third and second thresholds. The second and third thresholds may be pre-set thresholds. For example, the retrieval routing factor α505 may be normalized and have a value of 0 ≤ α ≤ 1, and the values ​​of the second and third thresholds may be between 0 and 1.

[0068] In one example, the first retrieval source can be a knowledge base and the second retrieval source can be a search engine, and the retrieval routing factor α505 can indicate the ratio of the number of retrieval results in the knowledge base to the number of retrieval results in the search engine. For example, the second threshold can be equal to 0.8 and the third threshold can be equal to 0.2. In this example, the routing threshold comparison module 530 can determine to generate a query for the knowledge base in response to the retrieval routing factor α505 being greater than 0.8; determine to generate a query for the search engine in response to the retrieval routing factor α505 being less than 0.2; or determine to generate both a query for the knowledge base and a query for the search engine in response to 0.2≤α≤0.8. Based on the determination of generating a query for a specific retrieval source, the query generation module 410 can route the feature representation 515 or the second response OT 525 to the corresponding retrieval source query generation module (e.g., the first retrieval source query generation module 540 and / or the second retrieval source query generation module 550). The routing threshold comparison module 530 can also route the retrieval routing factor α505 to other modules (e.g., the retrieval execution module 420).

[0069] In one embodiment, the first retrieval source query generation module 540 can generate a query Q1 545 for a first retrieval source based on the input or based on the input feature representation R 515 and / or the second response OT 525. The first retrieval source can be a knowledge base. In another example, the first retrieval source can be any retrieval source in a customer service system, an e-commerce system, or a search engine, or other retrieval sources. The first retrieval source query generation module 540 can output the query Q1 545 for the first retrieval source to the first retrieval source associated with the retrieval execution module 420.

[0070] For example, the first retrieval source query generation module 540 may include a fourth neural network that can generate keywords (or feature representations thereof) corresponding to the input or input-based feature representation R 515 based on the first query in the input (and optionally the domain associated with the first query), or based on feature representations corresponding to the first query (and optionally the domain associated with it), and include the generated keywords (or feature representations thereof) in the query Q1 545 for the first retrieval source. The generated keywords may include keywords (or feature representations thereof) identified from the input or input-based feature representation R 515, and / or include synonyms (or feature representations thereof) that are semantically similar to words in the input or input-based feature representation R 515. Those skilled in the art will understand that the fourth neural network may employ any neural network structure capable of achieving the above-described functions; for example, the fourth neural network may include an LM (e.g., an LLM), which may be the same as or different from the LMs described herein with respect to other graphs. As another example, the first retrieval source query generation module 540 may include first computational logic. For example, the first calculation logic may include a keyword library, and the first retrieval source query generation module 540 may identify keywords or their feature representations based on the keyword library in the input or based on the feature representation R 515, and include the identified keywords (or their feature representations) in the query Q1 545 for the first retrieval source.

[0071] For example, the first retrieval source query generation module 540 can generate a query Q1 545 for the first retrieval source based at least in part on the second response OT 525. For example, the second response OT 525 can be input from the determining module 120 and includes at least one term and a probability associated with each of the at least one term. For example, the second response OT 525 can include four terms T1-T4: {T1: “flat fading”, T2: “frequency-selective fading”, T3: “fast fading”, T4: “shadow fading”}, and probabilities P1-P4 corresponding to T1-T4 respectively. The first retrieval source query generation module 540 can compare at least one term in the second response OT 525 with a fourth threshold and mask the term in the at least one term that is associated with a probability that satisfies the comparison with the fourth threshold. Continuing the above example, the first retrieval source query generation module 540 can determine that P1-P3 are all greater than the fourth threshold and P4 is less than the fourth threshold, and therefore mask the term T4 corresponding to P4. For example, the fourth threshold may be the same as or different from the first threshold described with respect to the threshold comparison module 220 or the required probability threshold described with respect to the required comparison module 230. The first retrieval source query generation module 540 may include at least one unmasked term in the query Q1 545 for the first retrieval source. Continuing the above example, the first retrieval source query generation module 540 may include terms T1-T3 in the query Q1 545 for the first retrieval source: {T1: "flat fading", T2: "frequency-selective fading", T3: "fast fading"}. In this example, the retrieval execution module 420 may retrieve information including terms T1-T3 in the first retrieval source based on query Q1 545: "Small-scale fading can be divided into flat fading and frequency-selective fading in frequency, and into slow fading and fast fading in time."

[0072] For example, the first retrieval source query generation module 540 can generate a query Q1 545 for the first retrieval source based at least in part on the second response OT 525 and the feedback or request associated with the response in the input, or based on the feature representation corresponding to the feedback or request associated with the response in the input. The first retrieval source query generation module 540 can be implemented by computational logic or a neural network. For example, based on the fact that the feedback or request associated with the response has semantics such as "accurate" or "consistent with the facts," the first retrieval source query generation module 540 can mask at least one term in the second response OT 525 that is associated with the probability of satisfying the comparison condition with the request probability threshold, and include at least one unmasked term in the query Q1 545 for the first retrieval source, similar to what is described above regarding the fourth threshold. Additionally, based on the semantics of the feedback or request associated with the response, such as "specific" or "detailed," the first retrieval source query generation module 540 may include at least one terminology from the second response OT 525 in the query Q1 545 for the first retrieval source. For example, in the example of the second response containing four terms T1-T4 described above, query Q1 545 includes all terms T1-T4 without masking the fourth terminology, whose probability is less than a threshold. Those skilled in the art will understand that the first retrieval source query generation module 540 may also appropriately generate a query Q1 545 for the first retrieval source based on any feedback or request associated with the response.

[0073] In one embodiment, the second search source query generation module 560 can generate a query Q2 555 for a second search source based on the input or based on the input feature representation R 515 and / or the second response OT 525. The second search source can be a search engine. In another example, the second search source can be any search source different from the first search source, such as a customer service system, an e-commerce system, or a knowledge base, or other search sources. The second search source query generation module 550 can output the query Q2 555 for the second search source to the second search source associated with the search execution module 420.

[0074] For example, the second retrieval source query generation module 540 may include a fifth neural network or a second computational logic to identify keyword elements corresponding to at least one word element in the second response OT 525. The identified keyword elements may include word elements identified from the at least one word element, and / or include synonyms with similar semantics. Those skilled in the art will understand that the fifth neural network module may employ any neural network structure capable of achieving the above functions, and the second computational logic may employ any computational logic capable of achieving the above functions.

[0075] The second retrieval source query generation module 550 can generate a query Q2 555 for the second retrieval source, at least in part, based on a pre-set format and the identified keyword elements. For example, the pre-set format may include a question template. The question template may correspond to a keyword element. For example, based on the identification of at least one word element containing a keyword element corresponding to such as "why," "how could," or "for what reason," the second retrieval source query generation module 550 can determine to generate query Q2 555 using a format such as "the cause of...". For instance, for at least one word element including {"why," "appearance," "meteor"}, the second retrieval source query generation module 540 can generate query Q2 555 for the second retrieval source: "the cause of meteor appearance"; for at least one word element including {"summer," "how could," "hail"}, the second retrieval source query generation module 550 can generate query Q2 555 for the second retrieval source: "the cause of hail in summer." Queries generated in this way can structure query statements, thereby improving the stability of the responses generated based on the queries while maintaining their flexibility.

[0076] Those skilled in the art will understand that the first search source query generation module 540 or the second search source query generation module 550 may also use any of the methods described herein with respect to other embodiments or any other suitable methods to generate queries for search sources, and the query generation module 410 may also include other search source query generation modules.

[0077] Figure 6 A schematic diagram of a response module 150 according to one embodiment is shown. Figure 6 In this example, the response module 150 may include an optimization module 610, a prompt generation module 620, and an LM module 630. It is understood that the response module 150 may include other modules. Figure 6 Only modules relevant to embodiments of this disclosure are shown.

[0078] The optimization module 610 may apply reordering and / or context compression to the retrieved result D 615 to obtain an optimized search result OD 625. For example, the optimization module 610 may reorder the retrieved result D 615 based on the input or the input-based feature representation R 655, placing results in D 615 that better match the intent of the input or the input-based feature representation R 655 at the top and less relevant results at the bottom, thereby improving the relevance and accuracy of the search results and better meeting user needs. As another example, the optimization module 610 may apply context compression to the retrieved result D 615 to filter and process the information in the retrieved result D 615, thereby extracting information that is more useful (e.g., more relevant) to the first query in the answer input. The optimization module 610 may output the optimized search result OD 625 and the input or the input-based feature representation R 655 to other modules (e.g., the suggestion generation module 620).

[0079] The prompt generation module 620 can generate a prompt C 635 based on the optimized search result OD 625 and the input or input-based feature representation R 655. For example, the prompt generation module 620 can concatenate the optimized search result OD 625 and the input or input-based feature representation R 655 to generate the prompt C 635. Those skilled in the art will understand that any suitable prompt generation method can also be used to generate the prompt C 635. The prompt generation module 620 can output the prompt C 635 to other modules (e.g., LM module 630). For example, continuing the above example, the prompt C 635 may include the following information: "What are the classifications of small-scale fading in wireless communication channels?"; field "wireless communication"; search result 1: "Small-scale fading can be divided into flat fading and frequency-selective fading in frequency, and into slow fading and fast fading in time", search result 2: ..., search result 3: ...

[0080] LM module 630 can generate a first response O 645 at least in part based on prompt C 635. LM module 630 can be the same as or different from the LM modules described in other embodiments herein. For example, LM module 630 can include an LLM module. Those skilled in the art will understand that LM module 630 can be implemented using any suitable language model, trained or pre-trained, such as ChatGPT or LlaMA. ​​In this application, the first response O 645 obtained based on prompt C 635 can be more efficient and accurate based on a flexible and determined retrieval strategy. For example, LM can receive prompt C 635 in the above example: First query "What are the classifications of small-scale fading in wireless communication channels?"; Domain "Wireless Communication"; Search result 1: "Small-scale fading can be divided into flat fading and frequency-selective fading in frequency, and into slow fading and fast fading in time", Search result 2: ..., Search result 3: ... Based on this prompt C 635, LM module 630 can generate a first response O 625: "The classification of small-scale fading in wireless communication channels includes flat fading, frequency-selective fading, slow fading, and fast fading."

[0081] By way of example only, according to embodiments of this disclosure, Tables 1 and 2 below show exemplary first responses obtained without search enhancement and with search enhancement for the exemplary first query “How to optimize the shape of a part to reduce material waste and processing costs?” Table 1 Table 2

[0082] Figure 7 A flowchart of a method 700 for RAG according to one embodiment is shown.

[0083] In step 710, a retrieval routing factor for the first retrieval is determined, at least in part based on the input including the first query, wherein the retrieval routing factor indicates the proportion between the number of retrieval results across multiple retrieval sources. It will be understood that in different embodiments, or at different stages of application, the input may include only the first query, or it may include the first query and other information.

[0084] In step 720, results are retrieved in at least one of a plurality of retrieval sources, based at least in part on the retrieval routing factor and the input.

[0085] In step 730, a first response is obtained, at least in part, based on the retrieved results and the input.

[0086] According to one embodiment, the method further includes: determining, at least partially based on the input, whether to perform a first retrieval, wherein determining a retrieval routing factor for the first retrieval, based at least partially on the input including a first query, is performed in response to determining to perform the first retrieval. According to one embodiment, determining whether to perform the first retrieval further includes: obtaining a second response, at least partially based on the input, wherein the second response includes at least one term and a probability associated with each of the at least one term; and determining whether to perform the first retrieval or output at least one term in the second response based on a comparison of the probabilities associated with the at least one term and a first threshold.

[0087] According to one embodiment, in step 710, a retrieval routing factor is determined, at least in part, based on the first query, via a first neural network or computational logic.

[0088] According to one embodiment, in step 720, a second query corresponding to the first search is generated, at least in part based on the search routing factor and the input, wherein the second query includes a query for the first search source among a plurality of search sources and / or a query for the second search source among a plurality of search sources. According to one embodiment, generating the second query, at least in part based on the search routing factor and the input, includes one of the following: generating a query for the first search source in response to a first condition being met by comparing the search routing factor with a second threshold; generating a query for the second search source in response to a second condition being met by comparing the search routing factor with a third threshold; or generating both a query for the first search source and a query for the second search source in response to the search routing factor being between the third threshold and the second threshold. In one embodiment, the first condition is that the search routing factor is greater than the second threshold, or the first condition is that the search routing factor is greater than or equal to the second threshold. In one embodiment, the second condition is that the search routing factor is less than the third threshold, or the second condition is that the search routing factor is less than or equal to the third threshold.

[0089] According to one embodiment, the first search source among a plurality of search sources includes a professional knowledge base. According to one embodiment, step 720 includes generating a query corresponding to the first search source for the first search source by: masking at least one terminology whose probability is associated with a condition satisfied by a comparison with a fourth threshold; and including at least one unmasked terminology in the query for the first search source. In one embodiment, the condition is that the probability is less than the fourth threshold, or the probability is less than or equal to the fourth threshold.

[0090] According to one embodiment, the second search source among multiple search sources includes a search engine. According to one embodiment, step 720 includes generating a query for the second search source corresponding to the first search by: identifying keyword elements corresponding to at least one terminology; and generating the query for the second search source based at least in part on a pre-set format and keyword elements.

[0091] According to one embodiment, in step 720, a first number of results retrieved in the first retrieval source are obtained, at least in part based on the query against the first retrieval source in the second query, and / or a second number of results retrieved in the second retrieval source are obtained, at least in part based on the query against the second retrieval source in the second query; wherein the ratio between the first number and the second number satisfies the retrieval routing factor.

[0092] According to one embodiment, in step 730, reordering and / or context compression are applied to the retrieved results to obtain optimized search results; and a first response is generated based at least in part on the optimized search results and the input. In one embodiment, in step 730, a suggestion is generated based on the optimized search results and the input; and a first response is obtained based at least in part on the suggestion.

[0093] According to one embodiment, the input also includes the domain associated with the first query and / or feedback or requirements associated with the response.

[0094] According to one embodiment, the multiple retrieval sources include at least two of the following: a knowledge base, a customer service system, an e-commerce system, or a search engine.

[0095] Figure 8 A flowchart of a method 800 for training a neural network model for RAG is shown according to one embodiment.

[0096] In step 810, a first predicted value for a retrieval routing factor is determined, at least in part based on the first query, by the retrieval routing module in the NN model, wherein the retrieval routing factor indicates the proportion between the number of retrieval results in multiple retrieval sources.

[0097] In step 820, the learnable parameters of the NN model are updated, at least in part, based on the first predicted value and the label value related to the retrieval routing factor.

[0098] Figure 9 A block diagram of an apparatus 900 for RAG according to one embodiment is shown.

[0099] The apparatus 900 includes a retrieval routing module 910, a retrieval module 920, and a response module 930. The retrieval routing module 910 determines a retrieval routing factor for a first retrieval, at least in part based on input including a first query, wherein the retrieval routing factor indicates a ratio between the number of retrieval results in a plurality of retrieval sources. The retrieval module 920 retrieves results in at least one of the plurality of retrieval sources, at least in part based on the retrieval routing factor and the input. The response module 930 obtains a first response, at least in part based on the retrieved results and the input.

[0100] According to one embodiment, the apparatus 900 further includes a determining module that determines, at least in part, whether to perform a first retrieval based on the input, wherein determining a retrieval routing factor for the first retrieval, at least in part based on the input including a first query, is performed in response to determining to perform the first retrieval. According to one embodiment, the determining module determines whether to perform the first retrieval by: obtaining a second response, at least in part based on the input, wherein the second response includes at least one term and a probability associated with each of the at least one term; and determining whether to perform the first retrieval or output at least one term in the second response based on a comparison of the probabilities associated with the at least one term and a first threshold.

[0101] According to one embodiment, the retrieval routing module 910 determines retrieval routing factors based at least in part on a first query by: determining the retrieval routing factors by means of a first neural network or computational logic based at least in part on the input.

[0102] According to one embodiment, the retrieval module 920 includes a query generation module to retrieve results in at least one of a plurality of retrieval sources based at least in part on a retrieval routing factor and the input. The query generation module generates a second query corresponding to the first retrieval, also based at least in part on the retrieval routing factor and the input. The second query includes a query for the first retrieval source and / or a query for the second retrieval source. According to another embodiment, the query generation module includes a routing threshold comparison module to generate the second query based at least in part on the retrieval routing factor and the input. The routing threshold comparison module generates a query for the first retrieval source in response to a first condition being met by comparing the retrieval routing factor with a second threshold; generates a query for the second retrieval source in response to a second condition being met by comparing the retrieval routing factor with a third threshold; or generates both a query for the first retrieval source and a query for the second retrieval source in response to the retrieval routing factor being between the third and second thresholds.

[0103] According to one embodiment, the first search source among a plurality of search sources includes a professional knowledge base. According to one embodiment, the search module 920 includes a first search source query generation module to generate a query corresponding to the first search source by: masking at least one terminology whose probability of satisfying a comparison condition with a fourth threshold is associated with the first search source; and including at least one unmasked terminology in the query for the first search source.

[0104] According to one embodiment, the second search source among multiple search sources includes a search engine. According to one embodiment, the search module 920 includes a second search source query generation module to generate a query for the second search source corresponding to the first search by: identifying keyword elements corresponding to at least one terminology; and generating the query for the second search source based at least in part on a pre-set format and keyword elements.

[0105] According to one embodiment, the retrieval module 920 obtains a first number of results retrieved in the first retrieval source based at least in part on the query for the first retrieval source in the second query, and / or obtains a second number of results retrieved in the second retrieval source based at least in part on the query for the second retrieval source in the second query; wherein the ratio between the first number and the second number satisfies a retrieval routing factor.

[0106] According to one embodiment, the response module 930 includes the following: an optimization module that applies reordering and / or context compression to the retrieved results to obtain optimized search results; a suggestion generation module that generates suggestions based on the optimized search results and the input; and an LM module that generates a first response based at least in part on the suggestions.

[0107] According to one embodiment, the input also includes the domain associated with the first query and / or feedback or requirements associated with the response.

[0108] According to one embodiment, the multiple retrieval sources include at least two of the following: a knowledge base, a customer service system, an e-commerce system, or a search engine.

[0109] Figure 10 A block diagram of a processing apparatus 1000 according to one embodiment is shown.

[0110] The processing apparatus or processing system 1000 includes one or more control units or processing units 1010 that execute one or more machine-readable instructions stored or encoded in a machine-readable storage medium (i.e., memory 1020). In one embodiment, the processing unit 1010 is configured to execute the instructions in conjunction with the above when executing program instructions. Figure 1-9 The description includes various operations and functions.

[0111] Although not in Figures 1 to 6 and Figures 9 to 10 As shown, but those skilled in the art will understand that devices 100, 900 or 1000 may also include various other components, such as various communication modules, bus modules and possible user interface modules.

[0112] According to one embodiment, a program product, such as a non-transitory machine-readable medium, is provided. This non-transitory machine-readable medium may have instructions that, when executed by the processing unit 1010, are capable of performing the above-described combinations of various embodiments of this application. Figures 1 to 9 The description includes various operations and functions.

[0113] According to one embodiment, a computer program product is provided. The computer program product includes machine-executable instructions that, when executed by a processing unit 1010, are capable of performing the above-described combinations of various embodiments of this application. Figures 1 to 9 The description includes various operations and functions.

[0114] The specific embodiments described above with reference to the accompanying drawings are exemplary embodiments, but do not represent all embodiments that can be implemented or fall within the scope of the claims. The term "example" as used throughout this specification means "serving as an example, instance, or illustration" and does not imply that it is "preferred" or "advantageous" compared to other embodiments. Specific details are included to provide an understanding of the described technology. However, these technologies can be practiced without these specific details. In some instances, well-known structures and apparatuses are shown in block diagram form to avoid obscuring the concepts of the described embodiments.

[0115] The foregoing description of this application is provided to enable any person skilled in the art to implement or use the application. Various modifications to the application will be apparent to those skilled in the art, and the general principles defined herein can be applied to other variations without departing from the scope of protection of this application. Therefore, this application is not limited to the examples and designs described herein, but is consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A method for retrieval enhancement generation, comprising: A retrieval routing factor for a first retrieval is determined, at least in part, based on input including a first query, wherein the retrieval routing factor indicates a ratio between the number of retrieval results in multiple retrieval sources; Retrieve results in at least one of the plurality of retrieval sources, based at least in part on the retrieval routing factor and the input; and A first response is generated, at least in part, based on the retrieved results and the input.

2. The method of claim 1, further comprising: The determination of whether to perform the first retrieval is based at least in part on the input, wherein the determination of the retrieval routing factor for the first retrieval based at least in part on the input including the first query is performed in response to the determination to perform the first retrieval.

3. The method as described in claim 2, wherein, Determining whether to perform the first search also includes: Based at least in part on the input, a second response is obtained, wherein the second response includes at least one lexical unit and a probability associated with each of the at least one lexical unit; and Based on a comparison of the probability associated with a word in the at least one word and a first threshold, it is determined whether to perform the first retrieval or output the at least one word in the second response.

4. The method of claim 1, wherein, Determining the retrieval routing factor, at least in part based on the input including the first query, further includes: The retrieval routing factor is determined, at least in part, based on the input, via a first neural network or computational logic.

5. The method of claim 1, wherein, Based at least in part on the retrieval routing factor and the input, the retrieval results in at least one of the plurality of retrieval sources further include: A second query corresponding to the first retrieval is generated, at least in part based on the retrieval routing factor and the input, wherein the second query includes a query for the first retrieval source among the plurality of retrieval sources and / or a query for the second retrieval source among the plurality of retrieval sources.

6. The method of claim 5, wherein, Generating the second query, based at least in part on the retrieval routing factor and the input, includes one of the following: In response to the comparison between the retrieval routing factor and the second threshold satisfying the first condition, a query for the first retrieval source is generated; In response to the comparison between the retrieval routing factor and the third threshold satisfying the second condition, a query for the second retrieval source is generated; or In response to the retrieval routing factor being between a third threshold and a second threshold, a query for the first retrieval source and a query for the second retrieval source are generated.

7. The method of claim 3, wherein, The first search source among the plurality of search sources includes a professional knowledge base, and wherein, based at least in part on the search routing factor and the input, the search results in at least one of the plurality of search sources include generating a query corresponding to the first search source by means of the following operations: Mask the lexical units in the at least one lexical unit whose probabilities are associated with the comparison with the fourth threshold; and The query targeting the first retrieval source includes at least one unmasked term from the at least one term.

8. The method of claim 3, wherein, The second search source among the plurality of search sources includes a search engine, and wherein, based at least in part on the search routing factor and the input, the search results in at least one of the plurality of search sources include generating a query for the second search source corresponding to the first search by: Identify the key word element corresponding to the at least one word element; and A query for the second retrieval source is generated, at least in part based on a pre-set format and the keyword elements.

9. The method of claim 5, wherein, Based at least in part on the retrieval routing factor and the input, the retrieval results in at least one of the plurality of retrieval sources further include: At least in part based on the query against the first retrieval source in the second query, a first number of results retrieved from the first retrieval source are obtained, and / or The second quantity of results retrieved from the second retrieval source is obtained, at least in part, based on the query against the second retrieval source in the second query. The ratio between the first quantity and the second quantity satisfies the retrieval routing factor.

10. The method of claim 1, wherein, Obtaining a first response, at least in part based on the retrieved results and the input, also includes: Reordering and / or context compression are applied to the retrieved results to obtain optimized search results; and The first response is generated based at least in part on the optimized search results and the input.

11. The method of claim 1, wherein, The input also includes the domain associated with the first query and / or feedback or requirements associated with the response.

12. The method of claim 1, wherein, The multiple search sources include at least two of the following: a knowledge base, a customer service system, an e-commerce system, or a search engine.

13. A method for training a neural network (NN) model for retrieving augmented generation, comprising: The retrieval routing module in the NN model determines, at least in part, a first predicted value for a retrieval routing factor based on a first query, wherein the retrieval routing factor indicates the proportion between the number of retrieval results across multiple retrieval sources; and The learnable parameters of the NN model are updated at least in part based on the first predicted value and the label value related to the retrieval routing factor.

14. An apparatus for retrieving enhanced generation, comprising: A retrieval routing module determines a retrieval routing factor for a first retrieval based at least in part on input including a first query, wherein the retrieval routing factor indicates a ratio between the number of retrieval results in multiple retrieval sources; A retrieval module that retrieves results from at least one of the plurality of retrieval sources, based at least in part on the retrieval routing factor and the input; and A response module that obtains a first response based at least in part on the retrieved results and the input.

15. A device for skill planning based on scene graphs, comprising: One or more processors; as well as One or more memories storing computer-executable instructions that, when executed by the one or more processors, perform the operation of any one of claims 1 to 13.

16. A machine-readable storage medium storing executable instructions that, when executed, cause one or more processors to perform the method as described in any one of claims 1 to 13.

17. A computer program product comprising executable instructions that, when executed, cause one or more processors to perform the method as described in any one of claims 1 to 13.

Images