Adaptive semantic reorganization and noise reduction method for large language model retrieval enhancement

Abstract

The application relates to a self-adaptive semantic reorganization and noise reduction method for large language model retrieval enhancement. The method comprises the following steps: acquiring a user query and a fragment set of a candidate document. Fine-grained semantic dimension labeling is performed on the candidate fragments in the fragment set, a semantic coverage vector is generated, and the candidate fragments are dynamically routed to the corresponding semantic combination pool. In each semantic combination pool, the context mutual information outlying degree of each candidate fragment is calculated. According to the generated entropy detection result, the filtering weight of the candidate fragment is dynamically adjusted to obtain a noise-reduced context set. The contexts in the noise-reduced context set are subjected to semantic logic reorganization and structured splicing to generate a structured prompt word context input to a large language model. By using the method, accurate, coherent and illusion-free generated context reasoning results can be obtained under the condition of incomplete information retrieval.

Description

Technical Field

[0001] This application relates to the field of contextual retrieval technology based on large model reasoning, and in particular to an adaptive semantic recombination and noise reduction method for enhancing retrieval using large language models. Background Technology

[0002] With the rapid development of large-scale language models, retrieval-enhanced generation technology has become a key means to solve the problems of model knowledge lag and illusion. In modern retrieval-enhanced generation systems, when a user inputs a complex query, the system first uses vector databases or search engines to retrieve multiple relevant document fragments from a massive knowledge base, and then uses these fragments as context input into a large model to generate an answer.

[0003] Currently, the mainstream retrieval enhancement generation technology in the industry focuses on two directions: one is to improve the recall rate of retrieval by combining keywords and semantic vectors through hybrid retrieval; the other is to optimize the context window by using large models to process longer inputs with long texts. However, in practical applications, the retrieved document fragments are often of inconsistent quality, exhibiting semantic fragmentation (only arguments without points) or containing highly relevant noise (such as similar but incorrect outdated information). Faced with this "incomplete" or "contaminated" context information, existing processing technologies are still in their infancy, struggling to effectively identify and clean it, severely impacting the accuracy of the final generation.

[0004] Currently, existing technologies have the following main drawbacks when handling incomplete contextual information in retrieval enhancement generation tasks:

[0005] First, the crude truncation strategy leads to semantic loss. Traditional methods typically select the top K document fragments with the highest retrieval scores directly as input to the model. This selection method based on a single similarity metric ignores the "semantic complementarity" between document fragments. For example, a fragment may have a slightly lower similarity score to the query, but it may contain key background or preconditions needed to understand the high-scoring fragment. Direct truncation breaks the contextual logic chain of the input model, causing the model to make inference errors due to a lack of complete information.

[0006] Second, there is a lack of effective suppression of noisy documents. Search results often contain a large number of seemingly plausible but actually misleading documents. While these documents may contain the query keywords, their semantic content may be completely contrary to the user's intent or outdated. Current technology lacks a mechanism for deep semantic cleaning of the context before generation, making large models susceptible to being misled by this highly relevant noise, resulting in a severe "model illusion."

[0007] Third, static prompt templates cannot adapt to dynamic contexts. Regardless of the quality of the retrieved information, existing systems typically use the same set of fixed prompt templates to guide model generation. When the search results are severely lacking in key information (such as a lack of specific code examples or data support), the model, forced by fixed instructions, often fabricates false content to "make up the numbers," lacking an adaptive mechanism that can automatically adjust the generation strategy based on the current information completeness (such as automatically switching to a conservative answer mode). Summary of the Invention

[0008] Therefore, it is necessary to address the aforementioned technical problems by providing an adaptive semantic recombination and denoising method for retrieval enhancement of large language models that can improve the robustness of retrieval enhancement generation tasks in task model processing under incomplete information environments and the quality of generated contextual reasoning information.

[0009] An adaptive semantic reconstruction and denoising method for enhancing retrieval in large language models, the method comprising:

[0010] Retrieve a collection of fragments of the user query and candidate documents.

[0011] Fine-grained semantic dimension annotation is performed on candidate segments in the segment set to generate semantic coverage vectors, and candidate segments are dynamically routed to the corresponding semantic combination pool.

[0012] Within each semantic combination pool, the contextual mutual information outlier of each candidate fragment is calculated, where the outlier is used to measure the degree of deviation of the candidate fragment in terms of query relevance and contextual semantic consistency.

[0013] Based on the entropy detection results obtained by generating and probing the current context using a small-parameter, large-language model, the filtering weights for candidate segments are dynamically adjusted, and the segment set is adaptively filtered to obtain the denoised context set.

[0014] The semantic logic of the context in the denoised context set is reorganized and structured to generate a structured prompt word context that is input into the large language model.

[0015] An adaptive semantic reconstruction and denoising device for enhancing retrieval of large language models, the device comprising:

[0016] The fragment set acquisition module is used to obtain the fragment set of user queries and candidate documents.

[0017] The semantic annotation and routing module is used to perform fine-grained semantic dimension annotation on candidate segments in the segment set, generate semantic coverage vectors, and dynamically route candidate segments to the corresponding semantic combination pool.

[0018] The outlier calculation module is used to calculate the contextual mutual information outlier of each candidate fragment within each semantic combination pool. The outlier is used to measure the degree of deviation of the candidate fragment in terms of query relevance and contextual semantic consistency.

[0019] The adaptive filtering module is used to dynamically adjust the filtering weights of candidate segments based on the entropy detection results obtained by generating the current context using a small-parameter, large-language model, and to adaptively filter the segment set to obtain a denoised context set.

[0020] The recombination and suppression module is used to perform semantic and logical recombination and structured splicing of the context in the denoised context set to generate structured prompt word context for input into the large language model.

[0021] The aforementioned adaptive semantic recombination and denoising method for enhancing retrieval in large language models first transforms chaotic text streams into structured "semantic coverage vectors" through fine-grained semantic dimension annotation and dynamic routing. Instead of simply discarding low-scoring segments, it accurately identifies what is "missing" in the current context (e.g., missing background or code), preserving key information with semantic complementarity, fundamentally solving the problem of broken logical chains caused by crude truncation. Based on this, an outlier calculation mechanism based on contextual mutual information is constructed. Introducing the dual constraint of "contextual mutual information," by calculating the deviation of segments from the mainstream semantics within the same pool, it can accurately identify deep noise that "seems relevant but conflicts with mainstream facts" (e.g., outdated information or counterexamples). This overcomes the shortcomings of traditional methods in dealing with highly relevant noise, suppressing the possibility of the model being misled and generating illusions from the source. Then, a generative entropy-driven adaptive filtering mechanism is designed. By detecting the "confusion level" (i.e., generative entropy) of the small model when reading the current document, the system can perceive the degree of information confusion in real time and dynamically adjust the stringency level of filtering accordingly. When information becomes cluttered and conflicts escalate, the system automatically performs more aggressive cleansing, achieving a leap from "static instruction templates" to "dynamic risk decision-making." Finally, through semantic logic reorganization and structured assembly, the cleansed information is constructed into a logically clear context with knowledge tags, standardizing the model's generation path from the input level. Through a technical closed loop of "perception-measurement-decision-reconstruction," the fuzzy retrieval quality problem is transformed into a computable and interventionist engineering problem, significantly improving the robustness of the Retrieval Augmentation (RAG) system in incomplete information environments and the quality of generated contextual reasoning information. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating an adaptive semantic reconstruction and denoising method for enhancing retrieval of large language models in one embodiment.

[0023] Figure 2This is a block diagram of an adaptive semantic reconstruction and denoising device for enhancing retrieval of large language models in one embodiment. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0025] In one embodiment, such as Figure 1 As shown, an adaptive semantic reconstruction and denoising method for enhancing retrieval in large language models is provided, including the following steps:

[0026] Step 102: Obtain the fragment set of user query and candidate documents.

[0027] Specifically, the system first receives the user's query statement and a set of N candidate document fragments returned by the initial retrieval from the Vector Database.

[0028] Furthermore, the system performs two parallel retrieval paths: one path uses the BM25 algorithm to calculate the lexical matching degree between the query and fragments in the document database; the other path uses a Bi-Encoder model to map the query and documents into high-dimensional vectors and calculate the cosine similarity. The results from the two retrieval paths are then fused using Reciprocal Rank Fusion (RRF) to obtain an initial set of candidate fragments.

[0029] ;

[0030] in, For the initial set of candidate fragments, For the first Candidate document fragments, This represents the total number of candidate document fragments.

[0031] Step 104: Perform fine-grained semantic dimension annotation on the candidate segments in the segment set, generate semantic coverage vectors, and dynamically route the candidate segments to the corresponding semantic combination pool.

[0032] Specifically, a lightweight natural language processing model (Teacher-Student architecture, such as DistilBERT) that has undergone knowledge distillation is used to perform fast semantic scanning on each candidate document fragment in the collection. The system predefines a set of general knowledge dimension labels, covering full-dimensional knowledge features from basic concepts to application cases:

[0033] ;

[0034] The labels above correspond to: definition / concept, reason / background, method / code, result / data, case / application, and limitation / counterexample. The model outputs the confidence probability distribution of each fragment belonging to a specific label, denoted as . This distribution reflects the semantic tendencies of the document fragment across different knowledge dimensions.

[0035] Furthermore, based on the annotation results, the system generates a binary vector for the entire retrieval set, serving as a "knowledge fingerprint" of the current context. This vector intuitively reflects whether the current retrieval results cover all the knowledge elements required to answer the question. The value rules for the k-th dimension of the vector are defined as follows:

[0036] ;

[0037] in, For semantic coverage vectors, For the semantic coverage vector, the first... The values ​​of each dimension, Candidate segments Belongs to the Knowledge dimension tags The confidence probability distribution For the first One candidate segment, The total number of candidate segments, For indicator functions, This is set to a preset validity threshold (e.g., 0.7). This step transforms the ambiguous "retrieval quality" into a discrete state code that can be processed by a computer, clearly indicating which knowledge dimensions are present in the current context and which are missing.

[0038] Step 106: Within each semantic combination pool, calculate the contextual mutual information outlier of each candidate fragment, where the outlier is used to measure the degree of deviation of the candidate fragment in terms of query relevance and contextual semantic consistency.

[0039] Specifically, a semantic decoupling strategy is employed to physically isolate different types of knowledge. Based on the probability distribution generated in the first step, the system dynamically allocates K temporary "semantic combination pools" in memory. Each document fragment is soft-routed to the pool corresponding to its most probable dominant dimension. This routing mechanism ensures that similar information is compared on the same benchmark. The routing logic is as follows:

[0040] ;

[0041] in, As candidate segments, For index The dynamic memory pool allocated by the dimension, Candidate segments The index corresponding to the dominant semantic dimension, Candidate segments Belongs to the Knowledge dimension tags The confidence probability distribution To select the index that maximizes the probability across all knowledge dimensions.

[0042] Furthermore, within each non-empty combination pool, the system calculates the semantic center vector for all fragments within that pool. This center vector represents the "mainstream semantic direction" of that type of knowledge under the current query, eliminating the specific interference of individual fragments. The calculation formula uses a weighted average based on query relevance:

[0043] ;

[0044] in, For the first The weighted semantic center vector of each semantic combination pool For the first A semantic combination pool, Candidate fragments in the semantic combination pool E ( ) is an embedding vector mapping function for documents or queries. Sim The cosine similarity function is used. Q For user queries. The center vector. This will serve as a reference benchmark for subsequent judgment of outlier noise.

[0045] Furthermore, within each semantic combination pool, this invention needs to accurately identify which fragments are "useful supplementary information" and which are "interference noise." To this end, this invention introduces contextual mutual information outlier as a quantitative indicator. For any fragment within the pool, a comprehensive score is calculated, encompassing both query relevance and contextual consistency.

[0046] The query relevance score utilizes a high-precision cross-encoder model to calculate the deep semantic interaction score between the fragment and the user query. The cross-encoder can capture subtle logical relationships between the query and the document, achieving higher accuracy than the dual-tower model. The output is then normalized using a sigmoid function to obtain the score.

[0047] ;

[0048] in, Candidate segments Query relevance score, The Sigmoid normalization function, For Cross-Encoder models to handle user queries With candidate fragments The matching score output after deep semantic interaction. For user queries, These are candidate segments.

[0049] Contextual Mutual Information Score: Calculates the semantic consistency between a fragment and other high-confidence fragments in the pool (i.e., the Top-M anchor fragment set). This reflects whether the fragment contradicts the mainstream view. The calculation formula uses a variant of Point Mutual Information (PMI) and introduces Kullback-Leibler (KL) divergence as a penalty term to punish significant deviations in the distribution.

[0050] ;

[0051] in, Candidate segments For the set of anchor points Contextual mutual information scoring, This is the set of high-confidence anchor fragments in the Top-M semantic composition pool. For the number of anchor fragments, For the first in the set of anchor fragments An anchor point segment, Candidate segments With anchor fragment Embedding similarity, For Gaussian distribution estimation based on local neighborhood, For Kullback-Leibler divergence, This is the penalty coefficient for distribution deviation. The higher the index, the more the segment blends into its surroundings; the lower the index, the more abrupt the segment appears.

[0052] Furthermore, combining the above two metrics, the final outlier score is calculated. To enhance sensitivity to noise, the reciprocal of the harmonic mean is used. A higher score indicates that the fragment neither matches the query nor conflicts with other information in the context, and is highly likely to be a noise source causing model illusion.

[0053] ;

[0054] in, Candidate segments Outlier score, Candidate segments Query relevance score, For balance coefficient, This is a smoothing term.

[0055] Step 108: Based on the entropy detection results obtained by generating and probing the current context using a small-parameter large-language model, dynamically adjust the filtering weights of candidate segments, adaptively filter the segment set, and obtain the denoised context set.

[0056] Specifically, an adaptive filtering algorithm based on generative entropy is designed, which can perceive the degree of disorder in the current information and automatically adjust the strictness of the filtering to achieve adaptive truncation under unsupervised conditions.

[0057] The system uses a language model with a small number of parameters to read the Top-M fragments in the current combinatorial pool as a prompt and attempts to generate the top T tokens of the answer. The system calculates the average Shannon entropy of the predicted distribution of these T tokens:

[0058] ;

[0059] in, For the generated draft, To test the number of tokens generated, For any candidate token in the vocabulary, For the model vocabulary, For the medium Step on the generated token sequence, To generate a token given a physician title prefix. The probability,

[0060] If entropy value A high value indicates that the model is still confused after reading these documents, suggesting that there are factual conflicts or serious noise interference between the documents. In this case, a more aggressive filtering strategy should be adopted.

[0061] Furthermore, the system calculates the final retention weight for each segment, introducing "generation entropy" as a penalty coefficient into the exponential weighting function, which drastically compresses the weights of segments with high outliers in a high-entropy environment:

[0062] ;

[0063] in, Candidate segments The final retained weight, The normalization constant is Based on the attenuation rate, The entropy sensitivity coefficient, Candidate segments Outlier score, This is the generated draft. The technical principle is that when a high generation entropy is detected (i.e., a high degree of information chaos), the negative exponent term in the formula will significantly increase the penalty for high outlier segments, thus adhering to the principle of "better to have none than to have bad ones".

[0064] Furthermore, the system retains a preset confidence level (e.g., 90%), calculates dynamic truncation boundaries and reassembles the context, and then injects control instructions based on the final semantic coverage state, thereby achieving closed-loop control of generation quality. In the weighted fragment distribution, the system uses kernel density estimation (KDE) to fit the weight distribution curve and finds a critical outlier value. This threshold satisfies the following condition: the sum of the weights of all fragments with outliers less than this value, as a proportion of the total weight, reaches a preset confidence level. :

[0065] ;

[0066] in, For dynamic truncation threshold, To satisfy the infimum of the aforementioned conditions, For candidate outlier threshold variables, This is the probability density function obtained after kernel density estimation and normalization of the weighted fragment outlier distribution. For outlier variables, Preserve confidence levels for the preset information.

[0067] Furthermore, set reorganization: the system performs hard filtering, directly removing all segments with outliers greater than a threshold from the context, resulting in a cleaned set. :

[0068] ;

[0069] in, This is the set of candidate fragments after cleaning. For the initial set of candidate fragments, This is a dynamic truncation threshold.

[0070] Step 110: Semantic logic reorganization and structured splicing are performed on the contexts in the denoised context set to generate structured prompt word contexts for input into the large language model.

[0071] Specifically, instruction injection: Semantic ordering is performed. The system, based on a predefined knowledge logic topology, performs... The fragments are reordered to establish a logically coherent context sequence. :

[0072] ;

[0073] in, This is the logically rearranged context sequence. For logical sorting functions, This is the set of filtered candidate segments. This is the semantic coverage vector.

[0074] Furthermore, structured assembly is performed. The system inserts explicit semantic delimiters or metadata tags between fragments of different semantic categories to form the final structured cue word context. This structured processing strengthens the model's attention boundaries for different knowledge modules at the data level, thereby guiding the model to generate data through the structural normativity of the context itself, without relying on natural language prompts.

[0075] ;

[0076] in, This represents a sequence concatenation operation. For the corresponding knowledge dimension, a structured identifier.

[0077] The aforementioned adaptive semantic recombination and denoising method for enhancing large language model retrieval first transforms the chaotic text stream into a structured "semantic coverage vector" through fine-grained semantic dimension annotation and dynamic routing. Instead of simply discarding low-scoring segments, it accurately identifies what is "missing" in the current context (e.g., missing background or code), preserving key information with semantic complementarity, fundamentally solving the problem of broken logical chains caused by crude truncation. Based on this, an outlier calculation mechanism based on contextual mutual information is constructed. Introducing the dual constraint of "contextual mutual information," by calculating the deviation of segments from the mainstream semantics within the same pool, it can accurately identify deep noise that "seems relevant but conflicts with mainstream facts" (e.g., outdated information or counterexamples). This overcomes the shortcomings of traditional methods in dealing with highly relevant noise, suppressing the possibility of the model being misled and generating illusions from the source. Then, a generative entropy-driven adaptive filtering mechanism is designed. By detecting the "confusion level" (i.e., generative entropy) of the small model when reading the current document, the system can perceive the degree of information confusion in real time and dynamically adjust the stringency level of filtering accordingly. When information becomes cluttered and conflicts escalate, the system automatically performs more aggressive cleansing, achieving a leap from "static instruction templates" to "dynamic risk decision-making." Finally, through semantic logic reorganization and structured assembly, the cleansed information is constructed into a logically clear context with knowledge tags, standardizing the model's generation path from the input level. Through a technical closed loop of "perception-measurement-decision-reconstruction," the fuzzy retrieval quality problem is transformed into a computable and interventionist engineering problem, significantly improving the robustness of the Retrieval Augmentation (RAG) system in incomplete information environments and the quality of generated contextual reasoning information.

[0078] In one embodiment, a lightweight natural language processing model is used to perform semantic tag classification on each candidate segment, outputting the confidence distribution of the classified candidate segments belonging to a predefined knowledge dimension. Based on whether the maximum confidence of the classified candidate segments in each dimension reaches a preset threshold, a binary semantic coverage vector is constructed to represent the coverage of the current search results in the knowledge dimension. According to the dominant semantic dimension of the classified candidate segments, the classified candidate segments are soft-routed to the corresponding semantic combination pool, completing the physical isolation and alignment of similar information.

[0079] In one embodiment, a cross-encoder model is used to calculate a deep semantic interaction score between each candidate segment and the user query, serving as a query relevance score. The semantic consistency between the candidate segment and high-confidence anchor segments within the semantic combination pool of the candidate segment is calculated, and a contextual mutual information score is obtained by combining point mutual information and a KL divergence penalty term. The query relevance score and the contextual mutual information score are then fused to obtain an outlier score for the candidate segment, identifying semantically conflicting or noisy segments.

[0080] In one embodiment, a small-parameter large language model is used to generate partial answers based on high-confidence candidate segments in the current semantic combination pool. The Shannon entropy of the predicted distribution of the partial answer output tokens is calculated as the entropy detection result. The decay coefficient of the exponential weighting function is dynamically adjusted based on the entropy detection result, and the retention weight of each candidate segment is calculated. Based on the distribution of the retention weights and a preset information retention confidence level, a dynamic truncation threshold is determined using kernel density estimation. Candidate segments with outliers exceeding the dynamic truncation threshold are filtered out, resulting in a denoised context set.

[0081] In one embodiment, the filtered candidate fragments are logically ordered according to the semantic coverage vector to generate a context sequence that conforms to the knowledge topology:

[0082] ;

[0083] ;

[0084] in, For context sequence, This is the set of filtered candidate segments. For semantic coverage vectors, For the semantic coverage vector, the first The values ​​of each dimension, For indicator functions, The preset validity threshold, This represents the confidence probability distribution of candidate segments belonging to a specific label. As candidate segments, For the first Each knowledge dimension tag This is a logical ranking function. A structured identifier corresponding to the knowledge dimension is inserted before each candidate fragment to form a structured context with metadata tags. The concatenated structured context serves as prompt words input to the large language model, guiding the reasoning and generation process.

[0085] ;

[0086] in, Provide a target structured context for the output. For context sequence, For the corresponding knowledge dimension, This is a sequence concatenation function. for The first in One candidate segment, These are categorical features.

[0087] In one embodiment, a set of candidate document fragments is obtained by using a hybrid retrieval and inverse ranking based on the query information input by the user.

[0088] In one embodiment, based on the dominant semantic dimension and query similarity of the classified candidate segments, a weighted semantic center vector is calculated for all classified candidate segments in the pool. This weighted semantic center vector is used as the semantic benchmark for the current classification knowledge. The semantic benchmark serves as the anchor point for mutual information calculation and as a distribution deviation penalty.

[0089] It should be understood that, although Figure 1 The steps in the flowchart are shown sequentially as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order in which these steps are executed, and they can be performed in other orders. Figure 1 At least some of the steps in the process may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least some of the sub-steps or stages of other steps.

[0090] In one embodiment, such as Figure 2As shown, an adaptive semantic recombination and denoising device for enhancing retrieval of large language models is provided, comprising: a fragment set acquisition module 202, a semantic annotation and routing module 204, an outlier calculation module 206, an adaptive filtering module 208, and a recombination and suppression module 210, wherein:

[0091] The fragment set acquisition module 202 is used to acquire the fragment set of user queries and candidate documents.

[0092] The semantic annotation and routing module 204 is used to perform fine-grained semantic dimension annotation on candidate segments in the segment set, generate semantic coverage vectors, and dynamically route candidate segments to the corresponding semantic combination pool.

[0093] The outlier calculation module 206 is used to calculate the contextual mutual information outlier of each candidate fragment within each semantic combination pool. The outlier is used to measure the degree of deviation of the candidate fragment in terms of query relevance and contextual semantic consistency.

[0094] The adaptive filtering module 208 is used to dynamically adjust the filtering weights of candidate segments based on the entropy detection results obtained by generating the current context using a small-parameter large-language model, and to adaptively filter the segment set to obtain a denoised context set.

[0095] The recombination and suppression module 210 is used to perform semantic and logical recombination and structured splicing of the context in the denoised context set to generate a structured prompt word context for input to the large language model.

[0096] Specific limitations regarding the adaptive semantic reconstruction and denoising device for large language model retrieval enhancement can be found in the limitations of the adaptive semantic reconstruction and denoising method for large language model retrieval enhancement described above, and will not be repeated here. Each module in the aforementioned adaptive semantic reconstruction and denoising device for large language model retrieval enhancement can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0097] Those skilled in the art will understand that Figure 2 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0098] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), Synchlink, DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0099] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0100] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. An adaptive semantic recombination and denoising method for enhancing retrieval in large language models, characterized in that, The method includes: Retrieve a collection of fragments from the user query and candidate documents; Fine-grained semantic dimension annotation is performed on the candidate segments in the fragment set to generate semantic coverage vectors. The candidate segments are dynamically routed to the corresponding semantic combination pool. A lightweight natural language processing model is used to classify the semantic labels of each candidate segment and output the confidence distribution of the classified candidate segments belonging to a predefined knowledge dimension. Based on whether the maximum confidence of the classified candidate fragments in each dimension reaches a preset threshold, a binary semantic coverage vector is constructed to represent the coverage of the current search results in the knowledge dimension. Based on the dominant semantic dimension of the classified candidate fragments, the classified candidate fragments are soft-routed to the corresponding semantic combination pool to complete the physical isolation and alignment of similar information; Within each semantic combination pool, the contextual mutual information outlier of each candidate fragment is calculated, wherein the outlier is used to measure the degree of deviation of the candidate fragment from the overall query relevance and contextual semantic consistency. Based on the entropy detection results obtained by generating detection of the current context using a small-parameter large-language model, the filtering weights of the candidate segments are dynamically adjusted, and the segment set is adaptively filtered to obtain a denoised context set. The specific steps are as follows: using a small-parameter large-language model to generate partial answers based on the high-confidence corresponding candidate segments in the current semantic combination pool, calculating the Shannon entropy of the token prediction distribution of the partial answer output as the entropy detection result; The decay coefficient of the exponential weighting function is dynamically adjusted based on the entropy detection results. The retention weight of each candidate segment is calculated. The retention confidence is determined based on the distribution of the retention weight and the preset information retention confidence. The dynamic truncation threshold is determined using kernel density estimation. The candidate segments corresponding to outliers exceeding the dynamic truncation threshold are filtered out to obtain the denoised context set. The contexts in the denoised context set are semantically and logically reorganized and structurally concatenated to generate a structured prompt word context that is input into the large language model.

2. The method according to claim 1, characterized in that, Within each semantic combination pool, the contextual mutual information outlier of each candidate fragment is calculated, including: The deep semantic interaction score between each candidate fragment and the user query is calculated using a cross-encoder model, and used as a query relevance score. Calculate the semantic consistency between the candidate segment and the high-confidence anchor segment in the semantic combination pool where the candidate segment is located, and obtain the context mutual information score by combining the point mutual information and the KL divergence penalty term; By combining the query relevance score and the context mutual information score, an outlier score is obtained for the candidate segment to identify semantically conflicting or noisy segments.

3. The method according to claim 1, characterized in that, The context set after noise reduction undergoes semantic logical reorganization and structured concatenation to generate a structured prompt word context input to the large language model, including: The denoised candidate segments are logically ordered based on the semantic coverage vector to generate a context sequence that conforms to the knowledge topology: in, For context sequence, This is the set of filtered candidate segments. For semantic coverage vectors, For the semantic coverage vector, the first The values ​​of each dimension, For indicator functions, The preset validity threshold, This represents the confidence probability distribution of candidate segments belonging to a specific label. As candidate segments, For the first Each knowledge dimension tag For logical sorting functions, The total number of candidate segments; A structured identifier corresponding to the knowledge dimension is inserted before each candidate fragment to form a structured context with metadata tags; The concatenated structured context serves as prompt words input into the large language model, guiding the reasoning and generation process. in, Provide a target structured context for the output. For context sequence, For the corresponding knowledge dimension, This is a sequence concatenation function. for The first in One candidate segment, These are categorical features.

4. The method according to claim 1, characterized in that, Retrieve a collection of fragments of the user query and candidate documents, including: Based on the query information entered by the user, a set of candidate document fragments is obtained by using a hybrid retrieval and reciprocal ranking.

5. The method according to claim 1, characterized in that, Based on the dominant semantic dimension of the classified candidate fragments, the classified candidate fragments are soft-routed to the corresponding semantic combination pool to complete the physical isolation and alignment of similar information, including: Based on the dominant semantic dimension and query similarity of the classified candidate segments, the weighted semantic center vector of all classified candidate segments in the pool is calculated, and the weighted semantic center vector is used as the semantic benchmark of the current classification knowledge. The semantic benchmark serves as the anchor point for mutual information calculation and as a penalty for distribution deviation.

Images