Teacher-query guide type compression optimization training method and system and query method and system based on open domain questions and answers

Abstract

The invention provides a teacher-query guide type compression optimization training method and system based on open domain questions and answers and a query method and system. The compression optimization training method comprises the steps that candidate paragraphs are obtained according to user queries, each query is matched with the corresponding candidate paragraphs, the candidate paragraphs are segmented into sentence sets, and training units are formed; the teacher model performs data distillation on the training unit based on a preset requirement to generate a distillation sample set; the method comprises the following steps: respectively coding queries and sentences in a distillation sample set into vector representations through a predefined coding function, measuring the similarity of the queries and sentences, and training a student model by taking the similarity as a supervision signal to obtain a trained student model; through a teacher model knowledge migration and query focusing mechanism, the accuracy and efficiency of a student model in an extraction type compression task are improved, redundant information interference is reduced, more compact and highly related context input is provided for a retrieval enhancement generation framework, and the method is suitable for an efficiency-sensitive open domain question and answer scene.

Description

Technical Field

[0001] This application relates to the field of text document compression, and in particular to a teacher-query guided compression optimization training method, system, and query method and system based on open-domain question answering. Background Technology

[0002] In recent years, with the widespread application of large-scale language models (LLMs) in Natural Language Processing (NLP), they have demonstrated unprecedented capabilities in Natural Language Understanding (NLU) and Natural Language Generation (NLG). Pre-trained language models based on the Transformer architecture, trained on large-scale corpora, possess strong language modeling and knowledge representation capabilities, enabling transfer learning across multiple downstream tasks. In open-domain question answering tasks, large-scale language models can not only handle diverse question types but also effectively extract core information relevant to the query through deep semantic modeling of lengthy contexts. Simultaneously, the models contain rich world knowledge and linguistic rules, supporting reasoning and supplementation of missing information when necessary, thereby generating natural language answers that conform to semantic logic and contextual consistency.

[0003] To improve the effectiveness and practicality of question-answering systems, current mainstream methods generally adopt the Retrieval Augmented Generation (RAG) architecture. RAG integrates information retrieval and text generation, typically comprising two main stages: the first stage is the retrieval stage, where several candidate paragraphs relevant to the user's query are retrieved from a large-scale unstructured corpus; the second stage is the generation stage, where the query and retrieved paragraphs are input into a large language model for comprehensive modeling and answer generation. This approach significantly improves the question-answering system's coverage of factual knowledge, avoiding the limitations of relying solely on the model's internal parameter memory. Experimental results on multiple open-domain question-answering datasets show that the RAG architecture outperforms traditional generative question-answering methods or retrieval-only extraction-based question-answering methods in terms of answer accuracy, coverage, and generation quality. However, existing RAG systems face several challenges when handling lengthy texts. For large language models, long text inputs lead to model inference delays, increased memory usage, and significantly higher computational costs. In addition, long texts often contain a lot of redundant information, which may cause the model to miss key information related to the query during processing, thus affecting the accuracy of the final answer.

[0004] In contrast, smaller models (such as BERT-based models) have advantages in resource consumption and inference efficiency, making them more suitable for practical deployment. However, due to their limited parameter size and knowledge capacity, they often struggle to capture the deep semantic relationships between complex queries and context, making it difficult to generate high-quality answers, especially when processing multiple complex text segments.

[0005] Existing Retrieval Augmentation (RAG) workflows still face two problems when handling efficiency-sensitive tasks: first, a large amount of redundant information leads to low efficiency in identifying key information; second, the accuracy of information extraction is insufficient. Specifically, the student model sorts and compresses sentences in a paragraph through similarity scoring, thereby achieving effective information compression and performance optimization.

[0006] To address the aforementioned issues, existing technologies attempt to tackle the challenges posed by long texts through contextual compression. Mainstream methods can be categorized into three types: token pruning, digest compression, and extractive compression. Token pruning methods reduce input length by removing unimportant tokens. For example, Selective-Context and LLMLingua utilize small models to evaluate token self-information for pruning, while LongLLMLingua introduces a combination of coarse and fine granularity and problem complexity evaluation to improve compression efficiency. LLMLingua-2 further investigates task-independent prompt compression to enhance generality and efficiency. Although these methods are efficient, they can easily disrupt syntactic structure at high compression rates, affecting model understanding and reasoning. Digest compression utilizes generative models to generate concise summaries; for example, RECOMP uses T5-Large to generate compressed text. While this method possesses a certain level of abstraction, its practical effectiveness and efficiency are often limited by issues such as model misgeneration, omission of key points, or strong dependence on prompts. Extractive compression, on the other hand, preserves semantic structure by selecting sentences or phrases relevant to the query, enhancing the understanding capabilities of downstream models. For example, zero-shot ranking is used to assess sentence importance, while RECOMP matches sentences based on the embedding similarity between the sentence and the query. Although it is more robust than the previous two methods, its compression effect still depends on the embedding quality and is susceptible to semantic ambiguity or insufficient training, leading to the omission or misselection of key information, which in turn affects question answering performance. Summary of the Invention

[0007] The purpose of this invention is to propose a teacher-query guided compression optimization training method based on open-domain question answering.

[0008] The purpose of this invention is to propose a teacher-query guided compression and optimization training system based on open-domain question answering.

[0009] The present invention also aims to propose a teacher-guided, query-compressed, and optimized query method based on open-domain question answering.

[0010] The purpose of this invention is to propose a teacher-query guided compressed and optimized query system based on open-domain question answering.

[0011] To achieve the above objectives, this invention proposes a teacher-query guided compression optimization training method based on open-domain question answering, comprising: Based on the candidate paragraphs obtained from the user query, each query is matched with the corresponding candidate paragraph, and the candidate paragraphs are segmented into a set of sentences to form a sentence-level training unit; The teacher model performs data distillation on the training units based on preset requirements to generate a distilled sample set. The query and the sentence in the distillation sample set are encoded into vector representations by a predefined encoding function, and their similarity is measured by their inner product. This similarity is then used as a supervision signal to train the student model, resulting in a well-trained student model.

[0012] According to the present invention, a teacher-query guided compression optimization training method based on open-domain question answering is proposed, wherein the candidate paragraphs are recalled from a large-scale corpus based on the query. According to the teacher-query guided compression optimization training method based on open-domain question answering proposed in this invention, the preset requirements include: extraction prompts constructed based on queries and candidate paragraphs.

[0013] According to the teacher-query guided compression optimization training method based on open-domain question answering proposed in this invention, the distillation sample set includes: Positive samples whose similarity score between the sentence and the query is greater than the first threshold; Candidate negative samples whose similarity score between the sentence and the query is less than or equal to a first threshold.

[0014] According to the teacher-query guided compression optimization training method based on open-domain question answering proposed in this invention, the distillation sample set further includes: When the number of candidate negative samples is less than a preset value, sentences not included in the candidate paragraphs are retrieved from the global corpus and used as external negative samples to supplement them.

[0015] According to the teacher-query guided compression optimization training method based on open-domain question answering proposed in this invention, a query guidance strategy is introduced when constructing the distillation sample set. The guidance input is generated based on the query and used to constrain the teacher model to extract content from candidate paragraphs.

[0016] According to the teacher-query guided compression optimization training method based on open-domain question answering proposed in this invention, a contrastive loss function is introduced during the training process of the student model to minimize the distance between positive sample pairs and maximize the distance with negative samples.

[0017] This invention also proposes a teacher-query guided compression optimization training system based on open-domain question answering, used to execute the aforementioned teacher-query guided compression optimization training method based on open-domain question answering, comprising: The query matching module is used to match each query with the corresponding candidate paragraphs; The candidate paragraph processing module is used to segment the candidate paragraphs into a set of sentences to form sentence-level training units; The sample set acquisition module is used by the teacher model to perform data distillation based on preset requirements and generate a distilled sample set. The model training module is used to encode each query and the sentence into vector representations using predefined encoding functions, measure similarity using their inner product, and use this as a supervision signal to train the student model, thereby obtaining a trained student model.

[0018] According to the teacher-query guided compression optimization training system based on open-domain question answering proposed in this invention, the sample set acquisition module is configured to: introduce a query guidance strategy when constructing the distilled sample set, generate guidance input based on the query, and use the guidance input to constrain the teacher model to extract content from candidate paragraphs.

[0019] According to the teacher-query guided compression optimization training system based on open-domain question answering proposed in this invention, the model training module is configured to: introduce a contrastive loss function during the training process of the student model to minimize the distance between positive sample pairs and maximize the distance with negative samples.

[0020] This invention also proposes a teacher-query guided compression and optimization query method based on open-domain question answering, implemented using a student model trained by a teacher-query guided compression and optimization training method based on open-domain question answering, including: The trained student model is applied to the retrieved candidate paragraphs, the similarity score between the query and the sentence is calculated, the sentences are sorted from high to low according to the similarity score, and the sentences are extracted as compressed context for the query generation stage according to a preset percentage to generate the initial answer. If the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context, and the supplementary information to generate the final answer.

[0021] This invention further proposes a teacher-query guided compressed and optimized query system based on open-domain question answering, used to execute the above-mentioned compressed and optimized query method, including: Candidate paragraph recall module: used to recall candidate paragraphs based on the retrieval. Similarity score calculation module: used to apply the trained student model to the retrieved candidate paragraphs and calculate the similarity score between the query and the sentence; Score sorting module: used to sort the similarity scores from high to low; Context generation module: Extracts sentences as compressed context for the query generation stage based on a preset percentage; Answer generation module: Used to generate initial answers; Answer enhancement module: When the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context and the supplementary information to generate the final answer.

[0022] The advantages of this application are: 1. By guiding the training of student models through teacher models, the performance of student models in context compression tasks was significantly improved, enhancing the accuracy and efficiency of compression, improving the compression performance of student models, effectively reducing redundant information, and increasing the practicality of the models.

[0023] 2. Furthermore, this invention transfers knowledge from a large language model (teacher model) to a small model (student model), guiding the student model to achieve more accurate sentence extraction. While ensuring computational efficiency, it effectively improves the compression performance of the small model, thereby achieving high-quality extractive compression, reducing redundant information interference and content misjudgment, providing more compact and relevant input content for downstream question-answering tasks, and improving the accuracy and processing efficiency of question-answering in the generation stage of the retrieval enhancement generation framework.

[0024] 3. This invention utilizes a query-focused extraction and compression training method to fully leverage the query-guided mechanism, thereby enhancing the relevance between compressed content and the query. This significantly improves the accuracy and reasoning efficiency of the question-answering system, ensuring the quality of generated answers and response speed. It is suitable for efficiency-sensitive open-domain question-answering scenarios. Attached Figure Description

[0025] Figure 1 The flowchart shows the teacher-query guided compression optimization training method based on open-domain question answering of the present invention. Figure 2 This is a schematic diagram of the structure of the teacher-query guided compression optimization training method based on open-domain question answering according to the present invention; Figure 3 This is the pseudocode for the training algorithm of this invention; Figure 4 This is the pseudocode for the compression algorithm of this invention. Detailed Implementation

[0026] The technical solutions provided in this application are further illustrated below with reference to the embodiments.

[0027] This invention proposes a teacher-query guided compression and optimization training method based on open-domain question answering. By utilizing the knowledge of the teacher model and the query guidance mechanism to construct high-quality samples, a lighter student model is trained, effectively alleviating two problems faced by the traditional RAG (Retrieval Augmentation) process when handling efficiency-sensitive tasks: firstly, the large amount of redundant information leads to low efficiency in identifying key information; secondly, the information extraction accuracy is insufficient. Specifically, the student model sorts and compresses sentences in a paragraph through similarity scoring, thereby achieving effective information compression and performance optimization. Considering that large language models (LLMs) may generate supplementary information that the student model fails to cover during the extraction process, our framework incorporates this information into the QA prompts, combining it with the original extracted information to achieve more information-density generative augmentation, improving the performance and inference efficiency of the end-to-end question answering system. A flowchart of a teacher-query guided compression method based on open-domain question answering is shown below. Figure 1 As shown.

[0028] Example 1 To achieve the above objectives, this invention proposes a teacher-query guided compression optimization training method based on open-domain question answering, comprising: Step 1: Construct a training sample set based on teacher guidance and query-guided extraction techniques; Step 1 specifically includes: Step 1-1: Construct a sentence-level training sample structure, defining the compression task as a relevance modeling task between queries and sentences. First, each query is matched with its corresponding paragraph, which is then segmented into a set of sentences, forming sentence-level training units. This task aims to learn the similarity between queries and individual sentences within a paragraph, thereby extracting key sentences that can answer the query.

[0029] Steps 1-2: Clarify compression standards to ensure high-quality training data. When generating training samples, the teacher model should follow the following standards: (1) Accuracy: The extracted sentences can directly answer the query; (2) Information content: While providing the answer, retain sufficient context; (3) Faithfulness: The content should truly reflect the original context and avoid introducing fictitious information.

[0030] Steps 1-3: Data distillation is performed using the teacher model to generate positive and negative sample sets. For each query and its corresponding paragraph, combined with manually constructed extraction prompts, the teacher model is guided to identify sentences highly relevant to the query from the paragraph as the positive sample set. The remaining unselected sentences in the paragraph are used as the candidate negative sample set. If the number of candidate negative samples is insufficient, to ensure the diversity and difficulty of the training samples, sentences semantically related to the query but not belonging to the current paragraph are retrieved from the global corpus as external negative samples. In this way, high-quality contrastive training samples are constructed, providing accurate guidance signals for the student model.

[0031] Steps 1-4: Combine the query guidance mechanism to further improve the targeting of sample construction.

[0032] By introducing a query guidance strategy, the teacher model is guided to focus on content areas that are strongly related to the query, thereby improving the quality of positive examples in the sample and enabling the compressed model to learn the mapping relationship between the query and key information more accurately.

[0033] Step 2: Train the context compression model based on extractive compression modeling technology; A compressed model is trained based on a distillation sample set to optimize query-sentence similarity learning.

[0034] Using positive and negative sample pairs constructed by the teacher model, we train the student model to learn the semantic similarity between queries and candidate sentences. We define an encoding function that encodes queries and sentences as vector representations, and measure similarity using their inner product. During training, a contrastive loss function is introduced to minimize the distance between positive sample pairs while maximizing the distance with negative samples, thereby improving the model's discriminative ability in sentence-level compression. This training strategy ensures that the student model can efficiently identify the key information that best answers the query.

[0035] In this embodiment, after training the compressed model, we apply it to the reordered paragraphs to extract the key sentences most relevant to the query, which are then used as the input context for the final large language model generation stage.

[0036] We selected the large language model Qwen2-72B-Instruct-GPTQ-Int4 as the teacher model in this embodiment, and a BERT-based model as the student model. In this embodiment, we chose bge-reranker-v2-m3 (approximately 568M) as the base model for the student model and performed 5 epochs of fine-tuning training. The training process adopted a distributed approach, deployed on three NVIDIA A100 GPUs with 80GB of VRAM, an initial learning rate of 5e-4, a batch size of 8 per GPU, and a gradient accumulation step of 4. During training, the optimal model was selected as the final compressor based on the performance on the validation set. The compressed context significantly improved the generation quality and response relevance of the question-answering system.

[0037] We model the context compression task as a sentence-level ranking problem to select the optimal subset of generated input. Specifically, for each query and its corresponding set of paragraph sentences, we construct query-sentence pairs and use a trained encoder to calculate a similarity score for each pair. These scores are used to rank the sentences, and the top t-th percentile of sentences selected constitutes the final compressed context subset used for generation. This approach ensures that the compressed context contains information-rich sentences that are highly relevant to the query, laying the foundation for accurate answers generated by large language models.

[0038] A generative enhancement mechanism is introduced to optimize the question-answering model's performance in scenarios with no answer. During the generation phase, the system first uses the query and a subset of compressed sentences as input, which are then fed into a large language model to generate an initial answer. When the model returns "no answer," the system automatically triggers the generative enhancement process. At this point, the large model uses its own knowledge base to generate supplementary information related to the query, and inputs this information along with the original compressed context into the generation module to form the enhanced input. The system provides two suggestion templates: standard suggestions are suitable for general situations, while extended suggestions, combined with supplementary information, enhance the question-answering ability in scenarios with missing information. This mechanism significantly improves the model's performance when dealing with difficult queries or questions with insufficient context.

[0039] This invention effectively improves the accuracy and efficiency of question-answering in the generation stage of the RAG framework through a series of modular steps. Particularly in efficiency-sensitive tasks, the proposed compression strategy significantly enhances the compression capabilities of small models, achieving high-quality sentence-level extraction, reducing redundant information and content misjudgments, and providing more compact and relevant input for downstream question answering. This teacher- and query-guided compression method shows promising application prospects in the field of retrieval-enhanced text compression.

[0040] Example 2 This invention proposes a teacher-query guided compression and optimization training method based on open-domain question answering. For details, please refer to [link / reference]. Figure 2 The main steps include: Step 1) Retrieval Phase Processing In the retrieval enhancement generation process, a retrieval engine first retrieves relevant paragraphs from the corpus based on the query. Then, a re-ranking engine (BAAI / bge-reranker-v2-m3 with a cross-encoder architecture) finely ranks the paragraphs to improve relevance. We integrate three retrieval methods: sparse retrieval based on keyword matching, such as BM25; embedding retrieval using a dual-encoder structure, such as BAAI / bge-m3; and dense retrieval based on DPR (Data Point Retrieval Rate) to enhance recall coverage. The ranked paragraphs are then fed into an extractive compressor to extract the most relevant sentences to the query, removing redundant information. These sentences serve as the final context input to the large language model to generate the answer, thereby improving the accuracy and efficiency of question answering.

[0041] Step 2) Context Compression Stage In the sample set construction phase, to train the extractive compressor, this invention combines a teacher model guidance and a negative sample expansion mechanism in the sample generation process. First, paired data between queries and corresponding paragraphs is constructed, dividing the paragraphs into sentence-level units. Given a query, a paragraph, and manually constructed extraction hints, the teacher model selects the most relevant sentences from the paragraphs as positive samples, and the remaining unselected sentences as candidate negative samples. If the number of candidate negative samples is insufficient, sentences related to the query but not appearing in the current paragraph are retrieved from the global corpus as supplementary samples, thus constructing a high-quality sample set containing both positive and negative samples. This process is as follows: Figure 3 As shown in Algorithm 1, the teacher model is systematically used to screen training samples and enhance the diversity and comparability of the data.

[0042] Subsequently, the student model maps queries and sentences into high-dimensional vectors using an encoding function. The similarity between these vectors is calculated to measure their relevance, and this similarity is used as a supervisory signal for training. During training, a contrastive loss function guides the student model to learn to distinguish between relevant and irrelevant sentences. The goal is to minimize the distance between positive samples and queries while maximizing the distance between negative samples and queries, thereby improving the compressor's discriminative ability and generalization performance in information filtering tasks.

[0043] After training, we used the student model to compress the retrieved paragraphs. Specifically, we structured the compression task as a sentence-level ranking problem. For each paragraph's sentence set, we constructed all query sentence pairs with the corresponding query, calculated similarity scores using the trained encoder, and formed a score set. Then, we sorted the sentences from highest to lowest score and selected the top-scoring sentences as the final compression result, forming the simplified context. This process is as follows: Figure 4 Algorithm 2 is shown in the figure.

[0044] In the subsequent question-answer generation stage, these key information sentences selected by the student model are input into the large language model along with the original query to generate an initial answer to the query. If the generated initial answer is empty or cannot be answered, the system will automatically trigger a generation enhancement mechanism, whereby the language model calls its internal knowledge base to generate supplementary information. Subsequently, the model combines the query, the compressed context, and the supplementary information to generate the final answer, thereby improving the completeness and coverage of the response. This process, as shown in Algorithm 2, constructs an efficient processing link from compression to generation, effectively improving the response quality and information integration capability of the question-answering system in practical applications.

[0045] Through the above steps, this invention provides an efficient and accurate compression optimization training method, which effectively solves the problems of low efficiency and insufficient accuracy in key information identification faced by traditional retrieval enhancement generation processes when dealing with large-scale redundant information in efficiency-sensitive tasks.

[0046] This invention calculates similarity by matching the query with sentences one by one within a paragraph, sorting and filtering the sentences to extract the most relevant content. During the filtering process, the knowledge guidance of a teacher model improves the accuracy of similarity judgment, thereby extracting concise sentences that cover the key information of the query while retaining necessary background and maintaining semantic coherence. The compressed text serves as input to the question-answering model, reducing input length and improving the relevance and accuracy of the generated results. Furthermore, when the compressed content is irrelevant to the task objective or lacks supporting information, the system identifies such cases and supplements them using a generation mechanism, thereby enhancing the robustness and versatility of the question-answering model in complex tasks.

[0047] Example 3 A teacher-query guided compression and optimization training system based on open-domain question answering, used for the aforementioned compression and optimization training method, includes: The query matching module is used to match each query with the corresponding candidate paragraphs; The candidate paragraph processing module is used to segment the candidate paragraphs into a set of sentences to form sentence-level training units; The sample set acquisition module is used by the teacher model to perform data distillation based on preset requirements and generate a distilled sample set. The model training module is used to encode each query and the sentence into vector representations using predefined encoding functions, measure similarity using their inner product, and use this as a supervision signal to train the student model, thereby obtaining a trained student model.

[0048] Example 4 This invention also proposes a teacher-query guided compression and optimization query method based on open-domain question answering, implemented using a student model trained by a teacher-query guided compression and optimization training method based on open-domain question answering, including: The trained student model is applied to the retrieved candidate paragraphs, the similarity score between the query and the sentence is calculated, the sentences are sorted from high to low according to the similarity score, and the sentences are extracted as compressed context for the query generation stage according to a preset percentage to generate the initial answer. If the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context, and the supplementary information to generate the final answer.

[0049] Example 5 This invention further proposes a teacher-query guided compressed and optimized query system based on open-domain question answering, used to execute the above-mentioned compressed and optimized query method, including: Candidate paragraph recall module: used to recall candidate paragraphs based on the retrieval. Similarity score calculation module: used to apply the trained student model to the retrieved candidate paragraphs and calculate the similarity score between the query and the sentence; Score sorting module: used to sort the similarity scores from high to low; Context generation module: Extracts sentences as compressed context for the query generation stage based on a preset percentage; Answer generation module: Used to generate initial answers; Answer enhancement module: When the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context and the supplementary information to generate the final answer.

[0050] Example 6 In the above embodiments of the present invention, an overall process of a question-answering system is constructed based on retrieval enhancement generation technology to illustrate the application of the context compression model trained in the present invention in actual question-answering scenarios; The specific steps involved in constructing a question-answering system include: Step 1: Build a retrieval component to retrieve relevant paragraphs from a large-scale corpus based on the query. Utilize multiple retrieval methods, including the sparse retrieval algorithm BM25, the vector similarity-based EmbeddingRetriever, and the dense retrieval tool DensePassageRetriever (DPR), to obtain a set of candidate paragraphs relevant to the user's query from the open-domain document collection. Different retrieval tools complement each other to improve the coverage and diversity of the retrieved paragraphs.

[0051] Step 2: Design a re-ranking module to refine the ranking of the initially recalled paragraphs. The trained re-ranking model is used to score the relevance of the recalled paragraphs, and the paragraph order is rearranged according to the scores to improve the match with the query and optimize the quality of subsequent compression and generated input.

[0052] Step 3: Integrate an extractive context compression model to filter information from the sorted paragraphs. Based on the query intent, the compression model extracts the most relevant sentences to the question from each paragraph, removing redundant content. This ensures that the compression result significantly reduces input length while maintaining semantic integrity, adapting to the input limitations of large language models.

[0053] Step 4: Concatenate the compressed context with the query and use it as input prompts to the large language model for answer generation. Ultimately, the compressed key information fragments and the original query together constitute the input prompts for the large language model, guiding it to generate accurate and concise answers, thus achieving a complete closed loop from information retrieval to natural language generation.

[0054] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to the embodiments, those skilled in the art should understand that modifications or equivalent substitutions to the technical solutions of the present invention do not depart from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A teacher-query guided compression and optimization training method based on open-domain question answering, comprising: Based on the candidate paragraphs obtained from the user query, each query is matched with the corresponding candidate paragraph, and the candidate paragraphs are segmented into a set of sentences to form a sentence-level training unit; The teacher model performs data distillation on the training units based on preset requirements to generate a distilled sample set. The query and the sentence in the distillation sample set are encoded into vector representations by a predefined encoding function, and their similarity is measured by their inner product. This similarity is then used as a supervision signal to train the student model, resulting in a well-trained student model.

2. The compression optimization training method according to claim 1, characterized in that, The candidate paragraphs are retrieved from a large-scale corpus based on the query.

3. The compression optimization training method according to claim 1, characterized in that, The preset requirements include: extraction suggestions constructed based on queries and candidate paragraphs.

4. The compression optimization training method according to claim 1, characterized in that, The distillation sample set includes: Positive samples whose similarity score between the sentence and the query is greater than the first threshold; Candidate negative samples whose similarity score between the sentence and the query is less than or equal to a first threshold.

5. The compression optimization training method according to claim 3, characterized in that, The distillation sample set also includes: When the number of candidate negative samples is less than a preset value, sentences not included in the candidate paragraphs are retrieved from the global corpus and used as external negative samples to supplement them.

6. The compression optimization training method according to claim 4, characterized in that, When constructing the distillation sample set, a query guidance strategy is introduced to generate guidance input based on the query, and the guidance input is used to constrain the teacher model to extract content from candidate paragraphs.

7. The compression optimization training method according to claim 1, characterized in that, During the training process of the student model, a contrastive loss function is introduced to minimize the distance between positive sample pairs and maximize the distance with negative samples.

8. A teacher-query guided compression and optimization training system based on open-domain question answering, used to execute the compression and optimization training method according to any one of claims 1-7, comprising: The query matching module is used to match each query with the corresponding candidate paragraphs; The candidate paragraph processing module is used to segment the candidate paragraphs into a set of sentences to form sentence-level training units; The sample set acquisition module is used by the teacher model to perform data distillation based on preset requirements and generate a distilled sample set. The model training module is used to encode each query and the sentence into vector representations using predefined encoding functions, measure similarity using their inner product, and use this as a supervision signal to train the student model, thereby obtaining a trained student model.

9. A teacher-query guided compression and optimization query method based on open-domain question answering, implemented using a student model trained by the compression and optimization training method described in claims 1-6, comprising: The trained student model is applied to the retrieved candidate paragraphs, the similarity score between the query and the sentence is calculated, the sentences are sorted from high to low according to the similarity score, and the sentences are extracted as compressed context for the query generation stage according to a preset percentage to generate the initial answer. If the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context, and the supplementary information to generate the final answer.

10. A teacher-guided compressed and optimized query system based on open-domain question answering, used to execute the compressed and optimized query method of claim 9, comprising: Candidate paragraph recall module: used to recall candidate paragraphs based on the retrieval. Similarity score calculation module: used to apply the trained student model to the retrieved candidate paragraphs and calculate the similarity score between the query and the sentence; Score sorting module: used to sort the similarity scores from high to low; Context generation module: Extracts sentences as compressed context for the query generation stage based on a preset percentage; Answer generation module: Used to generate initial answers; Answer enhancement module: When the initial answer is empty or cannot be answered, the large language model calls its internal knowledge base to generate supplementary information, and combines the query, the compressed context and the supplementary information to generate the final answer.

Images