Retrieval enhancement generation method, system and equipment based on dynamic Top-K and medium

Abstract

The invention provides a retrieval enhancement generation method, system and device based on dynamic Top-K and a medium, and belongs to the technical field of computers. The method comprises the following steps: obtaining a document set related to user query by using a retrieval model based on a retrieval addition technology, and obtaining a correlation score of each document; calculating an average value and a standard deviation of the correlation scores based on the correlation score of each document; based on a preset score upper limit value, a score lower limit value and an experience threshold value, evaluating the average value and the standard deviation of the correlation scores, and determining whether the distribution of the document set is centralized; if the distribution of the document set is concentrated, calculating a confidence interval by adopting a confidence interval method, and screening out a document from the document set as a Top-K document according to the confidence interval; and if the distribution of the document set is dispersed, calculating a screening threshold value by adopting a dynamic threshold value method, and screening out the document from the document set as the Top-K document according to the screening threshold value.

Description

Technical Field

[0001] This invention belongs to the field of computer technology, and more specifically relates to a method, system, device and medium for generating enhanced search based on dynamic Top-K. Background Technology

[0002] Retrieval-Augmented Generation (RAG) is a natural language processing technique that combines information retrieval and text generation. It acquires external knowledge relevant to the user's question from a retrieval system and uses this knowledge as input to a generative model to generate more accurate and richer answers. In RAG, the `topk` parameter determines how many relevant documents to retrieve from the retrieval system. However, traditional fixed-topk methods suffer from the following problems when faced with significant variations in the number of retrieved results: When the recall results are limited, a fixed Top-K may lead to insufficient information, affecting the completeness and accuracy of the answers.

[0003] When there are many recall results, a fixed Top-K may introduce too much noise, reducing the relevance and efficiency of the answers.

[0004] Therefore, a method is needed that can dynamically adjust the Top-K based on the number of recall results to improve the performance of RAG. Summary of the Invention

[0005] To address the above problems, the present invention aims to provide a method, system, device, and medium for enhanced retrieval generation based on dynamic Top-K. By dynamically adopting different filtering strategies according to the document relevance score distribution, the most relevant Top-K documents to the user's query are selected accurately and efficiently, thereby improving retrieval quality.

[0006] To achieve the above objectives, the present invention employs the following technical solution: In a first aspect, embodiments of this application provide a retrieval enhancement generation method based on dynamic Top-K, including: A retrieval model based on retrieval augmentation techniques is used to obtain a set of documents relevant to the user's query and to obtain a relevance score for each document; The mean and standard deviation of the relevance scores are calculated based on the relevance score of each document; Based on preset scoring upper limit, scoring lower limit and empirical threshold, the mean and standard deviation of the relevance scores are evaluated to determine whether the distribution of the document set is concentrated. If the document set is concentrated, the confidence interval method is used to calculate the confidence interval, and documents are selected from the document set as Top-K documents based on the confidence interval; If the document set is distributed sparsely, a dynamic threshold method is used to calculate the filtering threshold, and documents are selected from the document set as Top-K documents based on the filtering threshold.

[0007] In an optional implementation, the retrieval model using retrieval enhancement techniques to obtain a set of documents relevant to the user query and to obtain a relevance score for each document includes: A retrieval model based on retrieval enhancement techniques is used to generate a document set D related to the user's query. The document set D includes N documents, denoted as D={d1,d2,…,dN}; Based on the relevance score output by the retrieval model, obtain the relevance score si for each document and form a score list S={s1,s2,…,sN}.

[0008] In an optional implementation, calculating the mean and standard deviation of the relevance scores based on the relevance score for each document includes: Through formula Calculate the average value μ of the correlation score.

[0009] In an optional implementation, the calculation of the mean and standard deviation of the relevance scores based on the relevance scores of each document further includes: Based on the average μ of the relevance score, using the formula Calculate the standard deviation σ of the correlation score.

[0010] In an optional implementation, the step of evaluating the mean and standard deviation of the relevance scores based on preset scoring upper limits, scoring lower limits, and empirical thresholds to determine whether the distribution of the document set is concentrated includes: Obtain the preset upper limit value a, lower limit value b, and experience threshold j; If μ ≥ a and σ < j, then the distribution of the document set is concentrated; If μ ≤ b or σ > j, then the distribution of the document set is dispersed.

[0011] In an optional implementation, the step of calculating a confidence interval using the confidence interval method and selecting documents from the document set as Top-K documents based on the confidence interval includes: Set the confidence level and generate the corresponding critical value Z using statistical library functions; The confidence interval [L,H] is calculated using the following formula:

[0012]

[0013] In the score list S, filter out the relevance scores that belong to the confidence interval [L,H], and determine the corresponding documents as Top-K documents based on the filtering results.

[0014] In an optional implementation, the step of calculating a filtering threshold using a dynamic threshold method and selecting documents from the document set as Top-K documents based on the filtering threshold includes: Based on the mean μ and standard deviation σ of the relevance score, the screening threshold T is calculated using the formula T=μ+k×σ; where k is an adjustable parameter. In the score list S, filter out the relevance scores that are greater than the screening threshold T, and determine the corresponding documents as Top-K documents based on the screening results.

[0015] Secondly, embodiments of this application also provide a retrieval enhancement generation system based on dynamic Top-K, including: The search results acquisition module is used to obtain a set of documents related to the user's query using a search model based on search enhancement techniques, and to obtain a relevance score for each document; The calculation module is used to calculate the mean and standard deviation of the relevance score based on the relevance score of each document; The distribution determination module is used to evaluate the average value and standard deviation of the relevance scores based on preset scoring upper limit, scoring lower limit and experience threshold, and to determine whether the distribution of the document set is concentrated. The first strategy execution module is used to calculate a confidence interval using the confidence interval method if the document set is concentrated, and to select documents from the document set as Top-K documents based on the confidence interval; The second strategy execution module is used to calculate a filtering threshold using a dynamic threshold method if the document set is distributed sparsely, and to filter documents from the document set as Top-K documents based on the filtering threshold.

[0016] Thirdly, embodiments of this application also provide an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the dynamic Top-K-based retrieval enhancement generation method as described in any of the above.

[0017] Fourthly, embodiments of this application also provide a storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the dynamic Top-K-based retrieval enhancement generation method as described in any of the above claims.

[0018] As can be seen from the above technical solutions, the present invention has the following advantages: The retrieval enhancement generation method based on dynamic Top-K provided in this application obtains a set of relevant documents and scores by using a retrieval model, calculates the mean and standard deviation, and evaluates the concentration or dispersion of document distribution based on preset parameters. It then uses the confidence interval method and the dynamic threshold method to filter Top-K documents. Based on the distribution characteristics of document relevance scores, it can dynamically, flexibly and accurately filter out the documents most relevant to the user's query, effectively improving the quality and relevance of retrieval results and meeting the needs of document filtering in different scenarios.

[0019] This application can automatically adjust the Top-K parameters based on the difference in the number of recalled results. Compared with the traditional fixed Top-K method, it can ensure sufficient information and improve the completeness and accuracy of the answer when the number of recalled results is small; it can reduce noise and improve the relevance and efficiency of the answer when the number of recalled results is large; it can also adapt to different application scenarios and data characteristics, improve the robustness and generalization ability of the system, and solve the problem of answer quality degradation caused by fixed Top-K values ​​in existing RAG technology when the number of recalled results varies greatly.

[0020] This application realizes the automatic adjustment of Top-K parameters based on the relevance score distribution (mean and standard deviation) of recall results, breaking through the limitations of traditional fixed Top-K and achieving adaptive parameter optimization.

[0021] This application enables a dynamic adjustment to increase the scope of effective information acquisition when the recall results are limited, avoiding insufficient information due to a fixed Top-K, and significantly improving the completeness and accuracy of the answers.

[0022] This application enables the narrowing of the Top-K range based on relevance distribution when there are many recall results, filtering low-relevance documents to reduce noise interference, and improving the relevance of answers and generation efficiency.

[0023] This application adopts a dynamic strategy to adapt to different application scenarios and data characteristics, effectively addressing situations where the number of recall results varies greatly, enhancing the robustness and generalization ability of the system, and optimizing the performance of RAG technology in multiple scenarios. Attached Figure Description

[0024] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the description will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 A flowchart illustrating the dynamic Top-K-based retrieval enhancement generation method provided in this application.

[0026] Figure 2 This is a schematic diagram of the structure of the dynamic Top-K-based retrieval enhancement generation system provided in this application.

[0027] Figure 3 A schematic diagram of the structure of the electronic device provided in this application. Detailed Implementation

[0028] The various embodiments of this disclosure will be described more fully in the detailed steps of the dynamic Top-K based retrieval enhancement generation method described below. This disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of this disclosure to the specific embodiments disclosed herein, but rather this disclosure should be understood to cover all adjustments, equivalents, and / or alternatives falling within the spirit and scope of the various embodiments of this disclosure.

[0029] In the following, the terms “comprising” or “may include”, which may be used in various embodiments of this disclosure, indicate the presence of the disclosed functions, operations, or elements, and do not limit the addition of one or more functions, operations, or elements. Furthermore, as used in various embodiments of this disclosure, the terms “comprising,” “having,” and their cognates are intended only to indicate a particular feature, number, step, operation, element, component, or combination of the foregoing, and should not be construed as primarily excluding the presence of one or more other features, numbers, steps, operations, elements, components, or combinations of the foregoing, or the possibility of adding one or more combinations of the foregoing.

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Please see Figure 1 The diagram shows a flowchart of a retrieval enhancement generation method based on dynamic Top-K in a specific embodiment. The method includes: S1: Use a retrieval model based on retrieval augmentation techniques to obtain a set of documents related to the user's query and obtain a relevance score for each document.

[0032] In a specific implementation, a retrieval model is used to obtain a set of documents related to the user's query, and a relevance score is obtained for each document. Specifically, firstly, a retrieval model based on retrieval enhancement techniques is used to generate a set of documents D related to the user's query; the document set D includes N documents, denoted as D={d1,d2,…,dN}. Simultaneously, based on the relevance scores output by the retrieval model, a relevance score si is obtained for each document, forming a score list S={s1,s2,…,sN}.

[0033] S2: Calculate the mean and standard deviation of the relevance scores based on the relevance scores of each document.

[0034] In a specific implementation, the mean μ and standard deviation σ of the correlation score are calculated using the following formulas:

[0035]

[0036] The central tendency and dispersion of the correlation score can be quantified through calculation.

[0037] S3: Evaluate the mean and standard deviation of the relevance scores to determine whether the distribution of the document set is concentrated. If yes, proceed to step S4; otherwise, proceed to step S5.

[0038] In a specific implementation, the average and standard deviation of the relevance scores are evaluated based on preset upper and lower score limits and an empirical threshold to determine whether the distribution of the document set is concentrated. Specifically, preset upper score limit *a*, lower score limit *b*, and empirical threshold *j* are obtained. If μ ≥ a and σ < j, the document set is concentrated, indicating that most documents are highly relevant and have small differences, and the Top-K distribution can be increased. If μ ≤ b or σ > j, the document set is dispersed, indicating that there are few relevant documents or large differences, and the Top-K distribution needs to be decreased to avoid noise.

[0039] S4: Calculate the confidence interval using the confidence interval method, and select documents from the document set as Top-K documents based on the confidence interval.

[0040] In a specific implementation, a confidence level is first set, and the corresponding critical value Z is generated using statistical library functions. Specifically, Python's stats.norm.ppf can be used as the statistical library function to obtain the accurate Z value, avoiding table lookup errors.

[0041] Then, the confidence interval [L,H] is calculated using the following formula:

[0042]

[0043] Finally, the relevance scores belonging to the confidence interval [L,H] are selected from the score list S, and the corresponding documents are determined as Top-K documents based on the selection results.

[0044] S5: Calculate the filtering threshold using the dynamic threshold method, and select documents from the document set as Top-K documents based on the filtering threshold.

[0045] In a specific implementation, the screening threshold T is first calculated based on the average μ and standard deviation σ of the relevance scores using the formula T = μ + k × σ; where k is an adjustable parameter used to control the tightness of the threshold. Then, relevance scores greater than the screening threshold T are selected from the score list S, and the corresponding documents are determined as Top-K documents based on the screening results.

[0046] In this embodiment, the proposed dynamic Top-K retrieval enhancement framework provides a systematic solution for optimizing retrieval results by constructing an intelligent distribution-aware and adaptive decision-making mechanism. Its core benefits lie in the comprehensive improvement of algorithm robustness, result accuracy, and system scalability. This framework innovatively introduces a dynamic distribution evaluation model, constructing an adaptive screening benchmark for document relevance scores based on a mean-standard deviation dual-factor model. When the score distribution is concentrated, a confidence interval method is used, automatically generating dynamic screening intervals through statistical confidence levels. This ensures high coverage of core relevant documents while intelligently filtering noisy data through interval constraints. When the score distribution is discrete, a dynamic threshold method is employed, using adjustable parameters to achieve stepless adjustment of screening strictness, effectively balancing retrieval recall and result redundancy. This dual-mode intelligent switching mechanism enables the system to have real-time awareness of data distribution characteristics, automatically selecting the optimal screening strategy based on the score distribution pattern, fundamentally solving the performance degradation problem of traditional fixed threshold methods when handling complex query scenarios. More importantly, this method constructs an interpretable control interface for the screening strategy through parameter decoupling design. Business users can dynamically adjust the confidence level or sensitivity parameters according to actual needs, achieving targeted optimization of retrieval accuracy while maintaining algorithm stability. This automated decision-making system based on statistical science not only eliminates the reliance on experience from manual rules, but also enables the system to adapt to data fluctuations within different time windows by continuously learning the changing patterns of score distribution. This provides technical support for the continuous optimization of the retrieval enhancement generation system in dynamic environments, significantly improving the intelligence level of information retrieval and generation tasks.

[0047] To illustrate more specifically the dynamic Top-K-based retrieval enhancement generation method disclosed in this invention, the implementation process of the method is described in detail below through specific examples.

[0048] First, obtain the relevance scores of the document set related to the user's query and generate a score list S={0.1,0.2, 0.3, 0.5, 0.6, 0.7, 0.8, 0.85, 0.9, 0.95}.

[0049] Then, the mean μ = 0.65 and the standard deviation σ = 0.28 were calculated according to the relevant formulas.

[0050] At this point, based on the preset upper limit, lower limit, and empirical threshold for scoring, the average and standard deviation of the relevance scores are evaluated to determine whether the distribution of the document set is concentrated.

[0051] If the document collection is centrally located, the following process is executed: 1. Setting the confidence level to 95%, Z=1.96 is generated using statistical library functions. 2. The lower limit of the confidence interval is: 0.65 - 1.96 * (0.28 / sqrt(10)) ≈ 0.48 3. The upper limit of the confidence interval is: 0.65 + 1.96 * (0.28 / sqrt(10)) ≈ 0.82 4. Select documents with relevance scores between [0.48, 0.82], i.e., the Top-K documents are the document set corresponding to set {0.5, 0.6, 0.7, 0.8, 0.85}.

[0052] If the document collection is scattered, the following process is executed: 1. Set the threshold value T = μ + σ = 0.93 2. Select documents with a relevance score higher than 0.93, i.e., Top-K documents are those with a relevance score of 0.95.

[0053] It should be noted that the parameters Z and k in the above method need to be adjusted according to the actual situation to obtain the best Top-K selection result.

[0054] like Figure 2 As shown, the following are embodiments of the retrieval enhancement generation system based on dynamic Top-K provided in this disclosure. This system and the retrieval enhancement generation method based on dynamic Top-K in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the retrieval enhancement generation system based on dynamic Top-K, please refer to the embodiments of the retrieval enhancement generation method based on dynamic Top-K described above.

[0055] A retrieval enhancement generation system based on dynamic Top-K includes: a retrieval result acquisition module, a calculation module, a distribution determination module, a first strategy execution module, and a second strategy execution module.

[0056] The search results acquisition module is used to obtain a set of documents related to the user's query using a search model based on search enhancement techniques, and to obtain a relevance score for each document.

[0057] The calculation module is used to calculate the mean and standard deviation of the relevance score based on the relevance score of each document.

[0058] The distribution determination module is used to evaluate the average value and standard deviation of the relevance scores based on preset scoring upper limit, scoring lower limit and experience threshold, and to determine whether the distribution of the document set is concentrated.

[0059] The first strategy execution module is used to calculate a confidence interval using the confidence interval method if the document set is concentrated, and to select documents from the document set as Top-K documents based on the confidence interval.

[0060] The second strategy execution module is used to calculate a filtering threshold using a dynamic threshold method if the document set is distributed sparsely, and to filter documents from the document set as Top-K documents based on the filtering threshold.

[0061] The dynamic Top-K based search enhancement generation system provided in this embodiment comprehensively considers the distribution characteristics of document relevance scores and flexibly uses the confidence interval method and dynamic threshold method to filter Top-K documents. It can dynamically adjust the filtering strategy according to the actual data situation, avoid the limitations of fixed filtering methods, accurately locate and select the most relevant and high-quality documents to the user's query, effectively improve the accuracy and effectiveness of search results, enhance the effect of search enhancement generation, and provide users with better search services.

[0062] Figure 3 A schematic diagram of the hardware structure of an electronic device for implementing various embodiments of the present invention.

[0063] The dynamic Top-K based retrieval enhancement generation method provided in this application can be applied to electronic devices. Those skilled in the art will understand that the electronic device structure involved in the embodiments of this invention does not constitute a limitation on the electronic device. An electronic device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. In the embodiments of this invention, electronic devices include, but are not limited to, laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the embodiments of this application described and / or claimed herein.

[0064] Electronic devices may include processors, external memory interfaces, internal memory, universal serial bus (USB) interfaces, charging management modules, power management modules, batteries, wireless communication modules, audio modules, speakers, microphones, sensor modules, buttons, cameras, displays, and SIM card interfaces, etc.

[0065] A processor may include one or more processing units, such as: a central processing unit (CPU), an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). Different processing units may be independent devices or integrated into one or more processors.

[0066] The processor can serve as the nerve center and command center of an electronic device. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0067] The processor may also include memory for storing instructions and data. In some embodiments, the memory in the processor is a cache memory. This memory can store instructions or data that the processor has just used or that are used repeatedly. If the processor needs to use the instruction or data again, it can retrieve it directly from this memory. This avoids repeated accesses, reduces processor latency, and thus improves system efficiency.

[0068] An external storage interface (ESI) can be used to connect external memory cards, such as microSD cards, to expand the storage capacity of electronic devices. The external memory card communicates with the processor through the ESI to perform data storage functions, such as saving music and video files on the external memory card.

[0069] Internal memory can be used to store computer executable program code, which includes instructions. The processor executes various functional applications and data processing of electronic devices by running the instructions stored in internal memory. Internal memory can include a program storage area and a data storage area. Internal memory can include high-speed random access memory, and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0070] Wireless communication functionality in electronic devices can be achieved through antennas, wireless communication modules, modem processors, and baseband processors.

[0071] Wireless communication modules can provide solutions for wireless communication applications in electronic devices, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0072] Electronic devices can implement audio functions through audio modules, speakers, receivers, microphones, headphone jacks, and application processors.

[0073] Electronic devices can achieve shooting functions through ISPs, cameras, video codecs, GPUs, displays, and application processors.

[0074] Electronic devices can achieve display functions through GPUs, displays, and application processors.

[0075] A GPU is a microprocessor for image processing, connected to the display screen and application processor. GPUs are used to perform mathematical and geometric calculations for graphics rendering. A processor may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] A display screen is used to display images, videos, etc. A display screen includes a display panel.

[0077] The aforementioned electronic device realizes the dynamic Top-K-based retrieval enhancement generation method of this application by constructing a dynamic distribution evaluation model based on mean-standard deviation and a dual-mode adaptive screening mechanism, and integrating a stepless switching design of confidence interval method and dynamic threshold method. At the same time, it combines a controllable adjustment interface with parameter decoupling and a statistically driven automated decision-making system to achieve intelligent perception and accurate adaptation of the algorithm to complex data distribution. It achieves a dynamic balance between noise filtering and high recall rate of core content in centralized and decentralized distribution scenarios, and supports business users to adjust retrieval accuracy and result diversity as needed. Ultimately, it eliminates the dependence on human experience and continuously adapts to changes in the dynamic data environment.

[0078] The storage medium provided in this application stores a program product capable of implementing a dynamic Top-K-based retrieval enhancement generation method.

[0079] Dynamic Top-K based retrieval enhancement generation methods include: A retrieval model based on retrieval augmentation techniques is used to obtain a set of documents relevant to the user's query and to obtain a relevance score for each document; The mean and standard deviation of the relevance scores are calculated based on the relevance score of each document; Based on preset scoring upper limit, scoring lower limit and empirical threshold, the mean and standard deviation of the relevance scores are evaluated to determine whether the distribution of the document set is concentrated. If the document set is concentrated, the confidence interval method is used to calculate the confidence interval, and documents are selected from the document set as Top-K documents based on the confidence interval; If the document set is distributed sparsely, a dynamic threshold method is used to calculate the filtering threshold, and documents are selected from the document set as Top-K documents based on the filtering threshold.

[0080] In some possible implementations, the dynamic Top-K based retrieval enhancement generation method of this disclosure can be implemented as a program product including program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0081] The storage medium disclosed herein may be any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof.

[0082] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A retrieval enhancement generation method based on dynamic Top-K, characterized in that, include: A retrieval model based on retrieval augmentation techniques is used to obtain a set of documents relevant to the user's query and to obtain a relevance score for each document; The mean and standard deviation of the relevance scores are calculated based on the relevance score of each document; Based on preset scoring upper limit, scoring lower limit and empirical threshold, the mean and standard deviation of the relevance scores are evaluated to determine whether the distribution of the document set is concentrated. If the document set is concentrated, the confidence interval method is used to calculate the confidence interval, and documents are selected from the document set as Top-K documents based on the confidence interval; If the document set is distributed sparsely, a dynamic threshold method is used to calculate the filtering threshold, and documents are selected from the document set as Top-K documents based on the filtering threshold.

2. The retrieval enhancement generation method based on dynamic Top-K according to claim 1, characterized in that, The retrieval model using retrieval enhancement techniques acquires a set of documents relevant to the user's query and obtains a relevance score for each document, including: A retrieval model based on retrieval enhancement techniques is used to generate a document set D related to the user's query. The document set D includes N documents, denoted as D={d1,d2,…,dN}; Based on the relevance score output by the retrieval model, obtain the relevance score si for each document and form a score list S={s1,s2,…,sN}.

3. The retrieval enhancement generation method based on dynamic Top-K according to claim 2, characterized in that, The calculation of the mean and standard deviation of the relevance score based on the relevance score of each document includes: Through formula Calculate the average value μ of the correlation score.

4. The retrieval enhancement generation method based on dynamic Top-K according to claim 3, characterized in that, The calculation of the mean and standard deviation of the relevance score based on the relevance score for each document also includes: Based on the average μ of the relevance score, using the formula Calculate the standard deviation σ of the correlation score.

5. The retrieval enhancement generation method based on dynamic Top-K according to claim 4, characterized in that, The evaluation of the average and standard deviation of the relevance scores based on preset scoring upper limits, scoring lower limits, and empirical thresholds to determine whether the distribution of the document set is concentrated includes: Obtain the preset upper limit value a, lower limit value b, and experience threshold j; If μ ≥ a and σ < j, then the distribution of the document set is concentrated; If μ ≤ b or σ > j, then the distribution of the document set is dispersed.

6. The retrieval enhancement generation method based on dynamic Top-K according to claim 5, characterized in that, The step of calculating confidence intervals using the confidence interval method and selecting documents from the document set as Top-K documents based on the confidence intervals includes: Set the confidence level and generate the corresponding critical value Z using statistical library functions; The confidence interval [L,H] is calculated using the following formula: In the score list S, filter out the relevance scores that belong to the confidence interval [L,H], and determine the corresponding documents as Top-K documents based on the filtering results.

7. The retrieval enhancement generation method based on dynamic Top-K according to claim 6, characterized in that, The step of calculating a filtering threshold using a dynamic threshold method and selecting documents from the document set as Top-K documents based on the filtering threshold includes: Based on the mean μ and standard deviation σ of the relevance score, the screening threshold T is calculated using the formula T=μ+k×σ; where k is an adjustable parameter. In the score list S, filter out the relevance scores that are greater than the screening threshold T, and determine the corresponding documents as Top-K documents based on the screening results.

8. A retrieval enhancement generation system based on dynamic Top-K, characterized in that, The system employs the dynamic Top-K-based retrieval enhancement generation method as described in any one of claims 1 to 7; The system includes: The search results acquisition module is used to obtain a set of documents related to the user's query using a search model based on search enhancement techniques, and to obtain a relevance score for each document; The calculation module is used to calculate the mean and standard deviation of the relevance score based on the relevance score of each document; The distribution determination module is used to evaluate the average value and standard deviation of the relevance scores based on preset scoring upper limit, scoring lower limit and experience threshold, and to determine whether the distribution of the document set is concentrated. The first strategy execution module is used to calculate a confidence interval using the confidence interval method if the document set is concentrated, and to select documents from the document set as Top-K documents based on the confidence interval; The second strategy execution module is used to calculate a filtering threshold using a dynamic threshold method if the document set is distributed sparsely, and to filter documents from the document set as Top-K documents based on the filtering threshold.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the dynamic Top-K based retrieval enhancement generation method as described in any one of claims 1 to 7.

10. A storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the dynamic Top-K-based retrieval enhancement generation method as described in any one of claims 1 to 7.

Images