Method and apparatus for optimizing adjustment of retrieval enhancement generation process, and electronic device

Abstract

The application provides a method and device for optimizing and adjusting a retrieval augmentation generation process, and an electronic device, the retrieval augmentation generation process comprising multiple stages; the method comprises: performing the retrieval augmentation generation process on a sample question to obtain state features of each stage; determining target probability distributions of different candidate answers of each stage according to multiple candidate answers corresponding to the sample question and the state features of each stage; determining information increments of a next stage relative to a previous stage according to the target probability distributions of different candidate answers of each stage; positioning a failure stage according to the information increments between two adjacent stages; and adjusting an execution strategy of the failure stage according to the positioned failure stage. The scheme provided by the application can quickly position the failure stage without manual stage-by-stage troubleshooting, significantly improving the optimization and adjustment speed of the RAG process and reducing the labor and time costs.

Description

Technical Field

[0001] This application relates to the fields of artificial intelligence and natural language processing technology, and in particular to an optimization and adjustment method and apparatus for the retrieval enhancement generation process, electronic equipment, computer-readable storage medium, and computer program product. Background Technology

[0002] With the development of large language models, Retrieval-Augmented Generation (RAG) technology has become the mainstream technology path by introducing external knowledge bases to improve the factuality and reliability of generated content.

[0003] Existing RAG systems mostly adopt a pipeline architecture with two stages of "retrieval-generation" or three stages of "retrieval-rearrangement-generation," treating each stage as a series of black-box components. The system evaluates its overall performance by the correctness of the final output answer.

[0004] The above process is unobservable. When the final answer is incorrect, it is impossible to determine whether the problem stems from a retrieval failure, a rearrangement error, contextual pollution, or a problem with the generation model itself. Debugging and optimization require manual analysis at each stage, which is time-consuming and labor-intensive. Summary of the Invention

[0005] This application provides a method, apparatus, and electronic device for optimizing and adjusting the retrieval enhancement generation process, which can quickly locate the problematic stage without requiring manual stage-by-stage analysis, thereby improving efficiency and reducing costs.

[0006] Figure 1 This application provides an embodiment of an optimization and adjustment method for a retrieval enhancement generation process, which includes multiple stages; the method includes: Perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage; Based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage, determine the target probability distribution of different candidate answers for each stage; Based on the target probability distribution of different candidate answers in each stage, determine the information increment of the next stage relative to the previous stage; The failure stage is located based on the information increment between each pair of adjacent stages; Based on the identified failure stage, the execution strategy for that failure stage is adjusted.

[0007] In one embodiment, determining the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers at each stage includes: The information increment of the next stage relative to the previous stage is obtained by calculating the KL divergence between the target probability distribution of different candidate answers in the previous stage and the target probability distribution in the next stage.

[0008] In one embodiment, the target probability distribution includes a correctness probability distribution and an evidence enforceability probability distribution. The step of obtaining the information increment of the later stage relative to the earlier stage by calculating the KL divergence between the target probability distributions of different candidate answers in the previous stage and the target probability distributions in the later stage includes: Calculate the KL divergence between the probability distribution of correctness of different candidate answers in the previous stage and the probability distribution of correctness in the next stage to obtain the first gain value; The second gain value is obtained by calculating the KL divergence between the evidence enforceability probability distributions of different candidate answers in the previous stage and the evidence enforceability probability distributions in the next stage. Based on the first gain value and the second gain value, the information increment of the later stage relative to the previous stage is obtained.

[0009] In one embodiment, obtaining the information increment of the later stage relative to the earlier stage based on the first gain value and the second gain value includes: Based on the first gain value and the second gain value, the minimum value is selected as the information increment of the next stage relative to the previous stage.

[0010] In one embodiment, obtaining the information increment of the later stage relative to the earlier stage based on the first gain value and the second gain value includes: Based on the first gain value and the second gain value, the product of the first gain value and the second gain value is used as the information increment of the subsequent stage relative to the previous stage.

[0011] In one embodiment, locating the failure stage based on the information increment between each pair of adjacent stages includes: If the information increment of the next stage relative to the previous stage is less than the threshold, then the next stage is a failure stage.

[0012] In one embodiment, adjusting the execution strategy for the failure stage based on the located failure stage includes: If the failure phase is the query construction phase, expand the constructed query terms; If the failure stage is the document retrieval stage, increase the number of documents recalled; If the failure stage is the document reordering stage, change the sorting model used. If the failure phase is the context building phase, remove documents whose relevance is below the threshold.

[0013] Secondly, embodiments of this application also provide an optimization and adjustment apparatus for a retrieval enhancement generation process, the retrieval enhancement generation process comprising multiple stages; the apparatus includes: The state acquisition module is used to perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage. The probability determination module is used to determine the target probability distribution of different candidate answers for each stage based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage. The incremental calculation module is used to determine the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers at each stage. The failure location module is used to locate the failure stage based on the information increment between two adjacent stages; The strategy adjustment module is used to adjust the execution strategy for the failure stage based on the identified failure stage.

[0014] Thirdly, embodiments of this application also provide an electronic device, including: Memory is used to store executable instructions for a computer; When the processor executes the computer-executable instructions stored in the memory, it implements the above-described optimization and adjustment method for the search enhancement generation process.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the above-described optimization and adjustment method for the search enhancement generation process.

[0016] Fourthly, embodiments of this application also provide a computer program product, which includes computer-executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, it implements the above-mentioned optimization and adjustment method for the search enhancement generation process.

[0017] The technical solution provided in the above embodiments divides the RAG process into process stages, obtains the state characteristics of each stage, predicts the target probability distribution of different candidate answers in each stage, and determines the information increment of each stage by analyzing the differences in the target probability distributions of adjacent stages. Based on the information increment, the contribution of each stage to the final result can be known. Stages that do not contribute can be considered as failed or abnormal stages, and thus, failed stages can be quickly adjusted. Since existing technologies only output erroneous results at the end, it is necessary to manually check each stage to find out which stage caused the erroneous result, which is time-consuming and laborious. However, the solution provided in this application can quickly locate the failed stage without manual checking of each stage, which significantly improves the optimization and adjustment speed of the RAG process. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating an optimization and adjustment method for a retrieval enhancement generation process provided in an embodiment of this application; Figure 2 This is a detailed calculation flowchart of the information increment provided in the embodiments of this application; Figure 3 This is a block diagram of an optimization and adjustment device for a retrieval enhancement generation process provided in an embodiment of this application; Figure 4 This is a block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0019] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this specification. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this specification as detailed in the appended claims.

[0020] The terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of this specification. The singular forms “a,” “the,” and “the” as used in this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any and all possible combinations of one or more of the associated listed items.

[0021] It should be understood that although the terms first, second, third, etc., may be used in this specification to describe various information, this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this specification, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0022] Figure 1 This is a flowchart illustrating an optimization method for the retrieval enhancement generation process provided in this application embodiment. The retrieval enhancement generation process includes multiple stages, which can be: query construction, document retrieval, document reordering, context construction, and answer generation. Specifically, the query construction stage involves transforming the user's question into searchable query terms; the document retrieval stage involves the search engine finding relevant documents; the document reordering stage involves sorting the retrieved documents according to relevance, placing the most relevant documents at the top; the context construction stage involves concatenating documents into prompts; and the answer generation stage involves outputting the answer using a large model. For example... Figure 1 As shown, the method includes: Step S110: Perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage.

[0023] Here, the sample question refers to a user question with known candidate answers (including the correct answer). The user question is processed sequentially through multiple stages of the aforementioned retrieval enhancement generation process. The intermediate results obtained at each stage can be used as state features. For example, the first stage, query construction, yields query terms, which can be converted into vectors and represented by Q; the second stage, document retrieval, yields a set of documents, represented by E; the third stage, document reordering, yields a reordering result, represented by P; the fourth stage, context construction, yields contextual information, represented by C; and the fifth stage, answer generation, yields a result, represented by M. Therefore, the state features of the first stage can be represented as X1={Q}; the second stage as X2={Q, E}; the third stage as X3={Q, E, P}; the fourth stage as X4={Q, E, P, C}; and the fifth stage as X5={Q, E, P, C, M}.

[0024] Step S120: Based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage, determine the target probability distribution of different candidate answers for each stage.

[0025] Specifically, the target probability distribution refers to the probability that each candidate answer becomes the final output result after each stage of the operation process is completed. For example, in the initial stage when nothing is done, because the answers are not focused, the probabilities of each candidate answer are not significantly different. Suppose there are 5 candidate answers (a, b, c, d, e), the probability distribution might be 20%, 20%, 20%, 20%, 20%. After the first stage, for example, after constructing the query terms "2024", "Nobel", "Peace Prize", and "winner", making the answers more focused, the probability distribution of the 5 candidate answers might become 35%, 15%, 15%, 15%, 20%. Thus, after each stage, the probability value of each candidate answer can be obtained. The probability values ​​of multiple candidate answers form a probability distribution, which is used for differentiation and is called the target probability distribution.

[0026] In one embodiment, the target probability distribution may include a correctness probability distribution; in another embodiment, the target probability distribution may also include a correctness probability distribution and an evidence enforceability probability distribution.

[0027] The correctness probability distribution refers to the probability distribution of multiple candidate answers belonging to the correct answer, which can be obtained through a pre-trained target prediction model. Specifically, for any given stage, multiple candidate answers and the state features of that stage are used as input to the target prediction model to obtain the probability value (i.e., the correctness probability distribution) of each candidate answer output by the target prediction model. The specific formula can be expressed as follows: ,in, Indicates that the state feature is When multiple candidate answers Y are given, the probability distribution of their correctness is as follows: This represents the target prediction model. Indicates multiple candidate answers. This represents the state features at stage t, where t can be 1, 2, 3, 4, or 5. The target prediction model can be implemented based on supervised learning models, neural network models, reward models, classification models, or large language model evaluators. For example, training samples for the target prediction model can be prepared in advance to train the model. Specifically, this can be achieved from a certain state feature... Starting from the beginning, the subsequent stages are executed 100 times. Based on the final generated answer, for example, answer a is generated 45 times, answer b is generated 35 times, and answer c is generated 20 times, then the state characteristics are... Under the given conditions, the probability distributions of candidate answers a, b, and c are 0.45, 0.35, and 0.2, respectively. From this, we can obtain the training samples (input is...). The candidate answers are a, b, and c; the output is a probability distribution of 0.45, 0.35, and 0.2. The target prediction model is trained.

[0028] The enforceability probability distribution refers to the probability distribution of multiple candidate answers being supported by evidence. It is used to measure the consistency between the generated results and the retrieved evidence, avoiding illusions, and can be obtained through a pre-trained evidence consistency discrimination model. Since the evidence set is obtained in the second stage (document retrieval), the evidence is ranked by relevance in the third stage (document reordering), and context information is generated by integrating evidence with relevance above a threshold in the fourth stage (context construction), it is evident that the evidence only changes in the second, third, and fourth stages. Therefore, the enforceability probability distribution is only calculated in the second, third, and fourth stages.

[0029] Specifically, for the second, third, or fourth stage mentioned above, multiple candidate answers and the results obtained in that stage are used as input to the evidence consistency discrimination model to obtain the probability value (i.e., the evidence enforceability probability distribution) of each candidate answer output by the model. The specific formula can be expressed as follows: ,in, Indicates that the evidence is When multiple candidate answers Y are given, the probability distribution of their correctness is as follows: This represents the evidence consistency discrimination model. Indicates multiple candidate answers. Let d represent the result (i.e., evidence) obtained in stage t, where t can be 2, 3, or 4. Evidence D2 in stage 2 can be the result E obtained in stage 2, evidence D3 in stage 3 can be the result P obtained in stage 3, and evidence D4 in stage 4 can be the result C obtained in stage 4. The evidence consistency discrimination model is implemented based on a Natural Language Inference (NLI) model, cross-encoder, fact consistency model, or semantic matching model. The method for obtaining training samples for the evidence consistency discrimination model can refer to the method for obtaining training samples for the target prediction model described above.

[0030] Step S130: Determine the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers in each stage.

[0031] The information increment is used to characterize how much new information is contributed at each stage. Specifically, this difference can be calculated using KL divergence by comparing the target probability distributions of two adjacent stages. The difference in target probability distributions between two adjacent stages represents the amount of new information contributed by the later stage, which is the information increment.

[0032] Specifically, step S130 includes: calculating the KL divergence between the target probability distribution of different candidate answers in the previous stage and the target probability distribution in the next stage to obtain the information increment of the next stage relative to the previous stage.

[0033] In one embodiment, such as Figure 2As shown, the calculation of the information increment specifically includes steps S131-S133. When the target probability distribution includes a correctness probability distribution, step S131 is executed: the KL divergence between the correctness probability distributions of different candidate answers in the previous stage and the correctness probability distributions in the next stage is calculated to obtain the first gain value of the next stage relative to the previous stage. The first gain value can be calculated using the following formula: .in, Let y represent the first gain value, and y represent one of the candidate answers. Indicates that the state feature is When, the probability value of candidate answer y; Indicates that the state feature is When the target probability distribution only includes the correctness probability distribution, the information increment can be the first gain value.

[0034] In one embodiment, when the target probability distribution includes a correctness probability distribution and an evidence enforceability probability distribution, steps S132 and S133 can be further performed. Step S132 involves calculating the KL divergence between the evidence enforceability probability distributions of different candidate answers in the previous stage and the evidence enforceability probability distributions in the next stage to obtain a second gain value.

[0035] The second gain value can be calculated using the following formula: .in, This represents the second gain value, and y represents one of the candidate answers. Indicates that the evidence is When, the probability value of candidate answer y; Indicates that the evidence is When, the probability value of candidate answer y, t represents the previous stage, and t represents the next stage.

[0036] Step S133: Based on the first gain value and the second gain value, obtain the information increment of the later stage relative to the previous stage.

[0037] In one embodiment, the minimum value between the first gain value and the second gain value can be selected as the information increment of the subsequent stage relative to the previous stage.

[0038] Specifically, the information increment can be expressed as It should be noted that, since the correctness probability distribution has first, second, third, fourth, and fifth stages, the first gain value includes the first gain value of the first stage relative to the initial state. The first gain value of the second stage relative to the first stage The first gain value of the third stage relative to the second stage The first gain value of the fourth stage relative to the third stage The first gain value of the fifth stage relative to the fourth stage .

[0039] Since the evidence enforceability probability distribution is only calculated in the second, third, and fourth stages, only the second gain value of the third stage relative to the second stage is available. The second gain value of the fourth stage relative to the third stage .

[0040] Therefore, the information increment in the first stage relative to the initial state You can directly use the first gain value. This indicates that the information increment in the second phase relative to the first phase... You can directly use the first gain value. This indicates the information increment in the third stage relative to the second stage. You can take the min( The information increment in the fourth stage relative to the third stage. You can take the min( The information increment in the fifth stage relative to the fourth stage. You can directly use the first gain value. express.

[0041] In another embodiment, the product of the first gain value and the second gain value can be used as the information increment of the subsequent stage relative to the previous stage, based on the first gain value and the second gain value.

[0042] Specifically, the information increment can be expressed as Similarly, since the evidence enforceability probability distribution is only calculated in the second, third, and fourth stages, only the second gain value of the third stage relative to the second stage is available. The second gain value of the fourth stage relative to the third stage .

[0043] Therefore, the information increment in the first stage relative to the initial state You can directly use the first gain value. This indicates that the information increment in the second phase relative to the first phase... You can directly use the first gain value. This indicates the information increment in the third stage relative to the second stage. Can be taken The information increment in the fourth stage relative to the third stage Can be taken The information increment in the fifth stage compared to the fourth stage You can directly use the first gain value. express.

[0044] Step S140: Locate the failure stage based on the information increment between each pair of adjacent stages.

[0045] A failure phase refers to a stage where the execution process is abnormal and fails to contribute any information to the result. Specifically, if the information increment of a subsequent stage relative to the previous stage is negative, then the subsequent stage is considered a failure phase. For example, the information increments of all stages can be arranged into a sequence. Under normal circumstances, the above sequences are all positive (indicating a contribution of information). If at a certain stage... If the value is less than the threshold, such as being negative, it indicates that stage t has failed (i.e., t is the failed stage). The information increment of each stage after stage t may also be negative due to the influence of stage t. We can first define the stage t that first shows a negative value as the failed stage.

[0046] Step S150: Adjust the execution strategy for the failure stage according to the identified failure stage.

[0047] It should be noted that different adjustment strategies are adopted for different failure stages. Specifically, if the failure stage is the query construction stage, the constructed query vocabulary can be expanded, for example, by adding synonyms for the query vocabulary; if the failure stage is the document retrieval stage, the number of recalled documents can be increased, for example, from 300 documents to 500 documents; if the failure stage is the document reordering stage, the ranking model used can be changed, for example, from the lightweight BM25 reorderer to a cross-encoder; if the failure stage is the context building stage, a lower threshold can be set to remove documents with relevance below the threshold. If the failure stage is the answer generation stage, the generation model used can be changed, the control parameters of the generation model can be changed, or constraint rules can be added. After adjusting the execution strategy for the failure stage, steps S110-S150 above can be returned to execute until the information increment of all stages is positive or the maximum number of iterations is reached, resulting in the optimized and adjusted RAG process.

[0048] The technical solution provided in the above embodiments divides the RAG process into process stages, obtains the state characteristics of each stage, predicts the target probability distribution of different candidate answers in each stage, and determines the information increment of each stage by analyzing the differences in the target probability distributions of adjacent stages. Based on the information increment, the contribution of each stage to the final result can be known. Stages that do not contribute can be considered as failed or abnormal stages, and thus, failed stages can be quickly adjusted. Since existing technologies only output erroneous results at the end, it is necessary to manually check each stage to find out which stage caused the erroneous result, which is time-consuming and laborious. However, the solution provided in this application can quickly locate the failed stage without manual checking of each stage, which significantly improves the optimization and adjustment speed of the RAG process.

[0049] To verify the effectiveness of the RAG algorithm optimized and adjusted according to the embodiments of this application, a comparative experiment was conducted with two other methods: (1) Baseline RAG (without control mechanism) and (2) RAG + single-point evaluation feedback. Four indicators were compared: Exact Match (EM), F1 score, Evidence Support Rate, and Hallucination Rate. The comparison results are shown in Table 1 below.

[0050] Table 1: Comparison of the performance of the retrieval enhancement generative models generated by this application and two other methods As can be seen from the table above, the method of this application is significantly better than the comparative method in both EM and F1 indicators; it shows a significant improvement in evidence support rate, indicating that the multi-objective coupling mechanism (the information increment is obtained by coupling the first gain value and the second gain value) is effective; the illusion rate is significantly reduced, verifying the effectiveness of the information theory increment contribution mechanism; compared with the single-point evaluation method, this application has higher stability on complex problems.

[0051] The above are merely preferred embodiments of this application and are not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

[0052] Figure 3 This is a block diagram of an optimization and adjustment device for a retrieval enhancement generation process provided in an embodiment of this application. The retrieval enhancement generation process includes multiple stages; such as... Figure 3 As shown, the device includes: a status acquisition module 310, a probability determination module 320, an incremental calculation module 330, a failure location module 340, and a strategy adjustment module 350.

[0053] The state acquisition module 310 is used to perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage. The probability determination module 320 is used to determine the target probability distribution of different candidate answers for each stage based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage. The incremental calculation module 330 is used to determine the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers in each stage. The failure location module 340 is used to locate the failure stage based on the information increment between two adjacent stages; The strategy adjustment module 350 is used to adjust the execution strategy of the failure stage according to the identified failure stage.

[0054] The specific implementation process of the functions and roles of each module in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0055] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. Each method step is implemented by a corresponding module, which will not be described in detail here. The device embodiments described above are merely illustrative. The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules, that is, they may be located in one place or distributed across multiple network modules. Some or all of the modules can be selected to achieve the purpose of the solution in this specification according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0056] Figure 4 This specification illustrates a hardware structure diagram of an electronic device containing an optimization and adjustment device for a retrieval enhancement generation process, based on an exemplary embodiment. Figure 4 As shown, the device may include: a processor 301, a memory 302, an input / output interface 303, a communication interface 304, and a bus 305. The processor 301, memory 302, input / output interface 303, and communication interface 304 are interconnected within the device via the bus 305.

[0057] The processor 301 can be implemented using a general-purpose CPU (Central Processing Unit), microprocessor, application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the steps of the optimization and adjustment method for the search enhancement generation process provided in the embodiments of this specification.

[0058] The memory 302 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage device, dynamic storage device, etc. The memory 302 can store the operating system and other application programs. When the technical solutions provided in the embodiments of this specification are implemented by software or firmware, the relevant program code is stored in the memory 302 and is called and executed by the processor 301.

[0059] Input / output interface 303 is used to connect input / output modules to realize information input and output. Input / output modules can be configured as components in the device (not shown in the figure) or externally connected to the device to provide corresponding functions. Input devices may include keyboards, mice, touch screens, microphones, various sensors, etc., and output devices may include displays, speakers, vibrators, indicator lights, etc.

[0060] Communication interface 304 is used to connect a communication module (not shown in the figure) to enable communication and interaction between this device and other devices. The communication module can communicate via wired means (such as USB, Ethernet cable, etc.) or wireless means (such as mobile network, WIFI, Bluetooth, etc.).

[0061] Bus 305 includes a pathway for transmitting information between various components of the device (e.g., processor 301, memory 302, input / output interface 303, and communication interface 304).

[0062] It should be noted that although the above-described device only shows the processor 301, memory 302, input / output interface 303, communication interface 304, and bus 305, in specific implementations, the device may also include other components necessary for normal operation. Furthermore, those skilled in the art will understand that the above-described device may only include the components necessary for implementing the embodiments of this specification, and not necessarily all the components shown in the figures.

[0063] This specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the optimization adjustment method for the retrieval enhancement generation process as described in any of the embodiments of this specification.

[0064] This specification also provides a computer program product, which includes computer-executable instructions stored in a computer-readable storage medium; wherein, when the processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, it implements the steps of the optimization and adjustment method for the retrieval enhancement generation process described in the above embodiments of this application.

[0065] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0066] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0067] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

Claims

1. A method for optimizing and adjusting the retrieval enhancement generation process, characterized in that, The retrieval enhancement generation process includes multiple stages; the method includes: Perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage; Based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage, determine the target probability distribution of different candidate answers for each stage; Based on the target probability distribution of different candidate answers in each stage, determine the information increment of the next stage relative to the previous stage; The failure stage is located based on the information increment between each pair of adjacent stages; Based on the identified failure stage, the execution strategy for that failure stage is adjusted.

2. The method according to claim 1, characterized in that, The step of determining the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers at each stage includes: The information increment of the next stage relative to the previous stage is obtained by calculating the KL divergence between the target probability distribution of different candidate answers in the previous stage and the target probability distribution in the next stage.

3. The method according to claim 2, characterized in that, The target probability distribution includes a correctness probability distribution and an evidence enforceability probability distribution. The step of calculating the KL divergence between the target probability distributions of different candidate answers in the previous stage and the target probability distributions in the next stage to obtain the information increment of the next stage relative to the previous stage includes: Calculate the KL divergence between the probability distribution of correctness of different candidate answers in the previous stage and the probability distribution of correctness in the next stage to obtain the first gain value; The second gain value is obtained by calculating the KL divergence between the evidence enforceability probability distributions of different candidate answers in the previous stage and the evidence enforceability probability distributions in the next stage. Based on the first gain value and the second gain value, the information increment of the later stage relative to the previous stage is obtained.

4. The method according to claim 3, characterized in that, The step of obtaining the information increment of the later stage relative to the earlier stage based on the first gain value and the second gain value includes: Based on the first gain value and the second gain value, the minimum value is selected as the information increment of the next stage relative to the previous stage.

5. The method according to claim 3, characterized in that, The step of obtaining the information increment of the later stage relative to the earlier stage based on the first gain value and the second gain value includes: Based on the first gain value and the second gain value, the product of the first gain value and the second gain value is used as the information increment of the subsequent stage relative to the previous stage.

6. The method according to claim 1, characterized in that, The step of locating the failure stage based on the information increment between each pair of adjacent stages includes: If the information increment of the next stage relative to the previous stage is less than the threshold, then the next stage is a failure stage.

7. The method according to claim 1, characterized in that, The adjustment of the execution strategy for the failure stage based on the identified failure stage includes: If the failure phase is the query construction phase, expand the constructed query terms; If the failure stage is the document retrieval stage, increase the number of documents recalled; If the failure stage is the document reordering stage, change the sorting model used. If the failure phase is the context building phase, remove documents whose relevance is below the threshold.

8. An optimization and adjustment device for a retrieval enhancement generation process, characterized in that, The retrieval enhancement generation process includes multiple stages; the device includes: The state acquisition module is used to perform a retrieval enhancement generation process on the sample question to obtain the state features of each stage. The probability determination module is used to determine the target probability distribution of different candidate answers for each stage based on the multiple candidate answers corresponding to the sample question and the state characteristics of each stage. The incremental calculation module is used to determine the information increment of the next stage relative to the previous stage based on the target probability distribution of different candidate answers at each stage. The failure location module is used to locate the failure stage based on the information increment between two adjacent stages; The strategy adjustment module is used to adjust the execution strategy for the failure stage based on the identified failure stage.

9. An electronic device, characterized in that, include: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the optimization and adjustment method for the retrieval enhancement generation process as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The device stores computer-executable instructions, which, when executed by a processor, implement the optimization and adjustment method for the retrieval enhancement generation process as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes computer-executable instructions stored in a computer-readable storage medium; When the processor of the electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, it implements the optimization and adjustment method for the retrieval enhancement generation process as described in any one of claims 1 to 7.

Images