A multi-stage semantic reordering fine-tuning method and device applied to vertical fields

Abstract

This invention discloses a multi-stage semantic re-ranking fine-tuning method and apparatus applied to vertical domains, comprising the following steps: S1, receiving original document input and segmenting document fragments; S2, designing a prompt template based on high-quality question-answer pairs annotated by domain experts to guide the large model in generating training data; S3, performing multi-stage hard negative sample mining using a vector model and a re-ranking model; S4, generating knowledge distillation signals based on the teacher model; S5, optimizing the re-ranking model using a multi-stage training strategy combining full-parameter fine-tuning and LoRA fine-tuning. This invention effectively improves the performance of the re-ranking model in vertical domains. Specifically, the two-stage fine-tuning strategy ensures that the model can fully learn global knowledge while specifically optimizing hard sample scenarios; while the knowledge distillation mechanism helps the model inherit the discriminative ability of the teacher model, achieving good results while maintaining the model's lightweight nature.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a multi-stage semantic reordering fine-tuning method and apparatus for application in vertical fields. Background Technology

[0002] With the rapid development of artificial intelligence technology, information retrieval and ranking systems are being applied more and more widely in various vertical fields. Traditional retrieval and ranking models usually adopt a single-stage approach, that is, directly ranking the search results all at once. However, in highly specialized vertical fields, such as medicine, law, and finance, how to effectively rank candidate answers or search results to ensure higher accuracy and relevance is a challenging technical problem.

[0003] Currently, research on re-ranking methods has made some progress, mainly based on single-stage fine-tuning methods using machine learning. Single-stage fine-tuning methods directly optimize ranking performance by training the model only once. However, this approach often has limitations when facing complex vertical domain tasks. Due to the specific characteristics of vertical domain data, a single training process cannot fully capture the diverse features within the domain, leading to poor model performance in applications.

[0004] In existing technologies, some patents propose model fine-tuning methods based on re-ranking. For example, some methods use contrastive learning or reinforcement learning for multi-stage optimization, but these methods often only focus on data-level optimization, neglecting the improvement of the model's capabilities at different task stages and their complementarity. Therefore, how to effectively combine the objectives and data features of each stage in multi-stage training to improve the re-ranking effect for vertical domain tasks remains an urgent problem to be solved. Summary of the Invention

[0005] To address the problems of existing technologies, current re-ranking models suffer from issues such as significant interference from out-of-domain data, low-quality negative samples, and poor generalization ability when applied to vertical domains. Specifically, traditional re-ranking models often use random sampling or simple similarity calculations to construct negative samples, which struggles to obtain challenging negative samples, leading to frequent misclassifications in practical applications. Furthermore, existing methods lack a deep understanding of domain-specific knowledge, making it difficult to accurately capture subtle semantic differences within the domain. In addition, traditional model training methods are computationally expensive, have low optimization efficiency, and are difficult to adapt quickly to the specific needs of vertical domains.

[0006] This invention provides a multi-stage semantic reordering fine-tuning method and apparatus for vertical domains. By optimizing the fine-tuning strategy at each stage, it fully utilizes data and task features to improve the application performance of the model in vertical domains. The specific scheme is as follows:

[0007] A multi-stage semantic reordering fine-tuning method applied to vertical domains includes the following steps:

[0008] S1 receives the original document input and performs segmentation;

[0009] S2, based on a small number of high-quality question-answer pairs annotated by domain experts, designs a prompt template to guide the generation of training data for a large model;

[0010] S3 utilizes vector models and reordering models for multi-stage hard negative sample mining;

[0011] S4, generating knowledge distillation signals based on the teacher model;

[0012] S5 employs a multi-stage training strategy that combines full parameter fine-tuning with LoRA fine-tuning to optimize the re-ranking model.

[0013] Preferably, step S2, which involves designing a prompt template to guide the generation of training data for a large model, includes:

[0014] S21, Domain experts select representative document segments from the original documents and annotate them with question-and-answer pairs;

[0015] S22, design a prompt template that includes domain background information, role settings, answer format requirements, quality control rules, and input / output examples;

[0016] S23, Input the prompt template and document fragments into the large model to generate question-answer pair data;

[0017] S24 evaluates the information content of document fragments and outputs "null" for fragments with insufficient information content.

[0018] Preferably, step S3, which involves multi-stage hard negative sample mining using a vector model and a reordering model, includes:

[0019] S31, Prepare training data containing the question and the corresponding positive sample, as well as a candidate sample pool;

[0020] S32, using a semantic embedding model to generate text embedding vectors;

[0021] S33, use FAISS to build an index and perform semantic retrieval to recall the TOP100 candidate samples;

[0022] S34, Use a re-ranking model to refine the ranking of candidate samples;

[0023] S35, filter negative samples from the ranking results and select the TOP K as training negative samples.

[0024] Preferably, step S35, which involves filtering negative samples from the sorting results, includes:

[0025] S351, use the re-ranking model trained in the previous stage to mine negative samples again.

[0026] S352, calculate the scores for positive samples and all negative samples;

[0027] S353: When the difference between the positive sample and the highest negative sample is less than a preset threshold, the sample is used as a difficult negative sample for the next stage of training.

[0028] Preferably, the step of generating the knowledge distillation signal in step S4 includes:

[0029] S41, Select a reordering model with a parameter size greater than 1B as the teacher model;

[0030] S42, scores the positive and negative samples in the training data;

[0031] S43, the scoring results are added to the training data as knowledge distillation signals.

[0032] Preferably, step S5 includes the following steps:

[0033] S51, the first stage is to fine-tune all parameters of the model and optimize the weighted sum of cross-entropy loss and knowledge distillation loss;

[0034] S52, the second stage mines hard negative samples based on the model of the first stage, uses LoRA technology to fine-tune the query, key, value and dense layers, and performs full parameter fine-tuning on the classifier layer.

[0035] Preferably, the knowledge distillation loss is calculated using the following formula:

[0036]

[0037] Where N is the batch size, M is the group size, and t ij p is the target output distribution of the teacher model. ij Output the distribution for the student model.

[0038] Preferably, the fine-tuning device based on the above-mentioned multi-stage semantic reordering fine-tuning method includes:

[0039] The data generation module is used to generate training data based on domain expert-annotated data;

[0040] The negative sample mining module is used for multi-stage hard negative sample mining.

[0041] The knowledge distillation module is used to generate knowledge distillation signals;

[0042] The model training module is used to perform multi-stage fine-tuning training.

[0043] Preferably, the data generation module includes:

[0044] The document processing unit is used to segment and process the input document;

[0045] The prompt design unit is used to design prompt templates to guide large models;

[0046] The data generation unit is used to generate training data using a large model.

[0047] The present invention also discloses a computer device, including a processor and a memory, wherein the memory stores a computer program, which, when executed by the processor, implements the steps of the multi-stage reordering fine-tuning method described above.

[0048] The beneficial effects of this invention are as follows:

[0049] First, by constructing high-quality question-answer pairs and employing a difficult negative sample mining strategy, the model's discriminative ability in vertical domains was significantly improved. In particular, the multi-stage iterative negative sample mining mechanism enabled the model to gradually learn finer-grained semantic discrimination capabilities, effectively reducing the impact of external interference.

[0050] Secondly, the introduction of the knowledge distillation mechanism enables the model to better learn the knowledge representations of the teacher model, significantly improving the model's generalization ability. Through knowledge transfer at the distribution level, the model not only learns sample-level discriminative ability but also masters more abstract domain knowledge representations, enabling it to better handle unseen query scenarios.

[0051] Third, the innovative multi-stage fine-tuning strategy effectively balances computational efficiency and model performance. Compared to traditional full-parameter fine-tuning methods, the solution of this invention significantly reduces computational resource consumption while maintaining similar performance. In particular, the LoRA technology used in the second stage enables the model to quickly adapt to the specific needs of the vertical domain, providing an efficient solution for iterative optimization of the model.

[0052] Fourth, the overall solution has strong scalability and can be flexibly applied to different vertical industry scenarios. By adjusting the prompt template, negative sample mining strategy, and fine-tuning parameters, the solution can quickly adapt to new domain requirements. Practice has shown that this method has been successfully applied in multiple professional fields such as medicine, law, and finance, demonstrating good adaptability and stability.

[0053] Furthermore, the method of this invention demonstrates excellent stability and reliability in practical applications, meeting the stability requirements of production environments. Simultaneously, the modular design of this solution facilitates maintenance and upgrades, providing sustainable technical support for intelligent retrieval systems in vertical industries. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the generation of question-answer pair data for training based on a large model, as described in an embodiment of the present invention.

[0056] Figure 2 This is a flowchart illustrating the hard negative sample mining based on vector models and rearrangement models in an embodiment of the present invention.

[0057] Figure 3 This is a flowchart illustrating the fine-tuning process of a multi-stage reordering model based on knowledge distillation, as described in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] As can be seen from the background technology, the following problems exist in the fine-tuning process of re-ranking models in vertical domains: First, it is difficult to obtain training data, and manual annotation is costly and inefficient; second, improper selection of negative samples may lead to a decline in model performance, and it is difficult to find effective difficult negative samples; third, there is a lack of effective knowledge distillation mechanism in the model training process, making it difficult to make full use of the knowledge of the teacher model to optimize the performance of the student model.

[0060] To address the aforementioned technical problems, this invention provides a multi-stage semantic re-ranking fine-tuning method and apparatus applicable to vertical domains. This method innovatively combines advanced techniques such as large language models, hard negative sample mining, and knowledge distillation. First, by designing a specific prompt template to guide the large language model, a large-scale question-answer pair training data is automatically constructed based on a small number of manually labeled high-quality samples, effectively reducing data acquisition costs. Second, a multi-stage iterative approach is used to mine hard negative samples. Pre-trained models are used for initial recall and re-ranking, selecting the most valuable negative samples. Multiple rounds of iterative optimization are then performed to continuously improve the quality of negative samples. Simultaneously, knowledge distillation technology is introduced, using a more powerful teacher model to guide the training process of the student model, enabling it to learn richer knowledge representations. Finally, by employing the LoRA fine-tuning strategy, only key parameters of the model are updated, ensuring improved model performance while significantly reducing computational resource consumption. Experimental results show that the method of this invention can significantly improve the adaptability and accuracy of the re-ranking model in vertical domains, providing an efficient and feasible solution for solving text matching problems in vertical domains.

[0061] like Figure 1 As shown in this embodiment, a method for generating question-answer pairs for training based on a large model is provided. This method aims to quickly generate high-quality question-answer pairs by combining a small amount of manually labeled data with the capabilities of a large model, for subsequent model fine-tuning and training. Specifically, the method includes the following steps:

[0062] S1 receives the input of the original document and segments the document into fragments.

[0063] The system loads the original input document, supporting various common document formats such as PDF, TXT, and WORD.

[0064] The document was parsed using the open-source tool MinerU. MinerU is a high-efficiency document parsing tool that supports extracting content from unstructured documents and has good multi-format compatibility. This tool can parse the document's body text, headings, and table content into structured data formats (such as JSON).

[0065] During the parsing process, text extraction and paragraph separation can be performed on PDF documents; for documents containing tables, the content of the tables and related title information can be extracted.

[0066] The parsed document content is segmented, and the segmentation granularity determines the size and content of each chunk.

[0067] The text is segmented using the semantic chunk method. Semantic chunk is a text segmentation technique based on semantics that ensures the semantic integrity of the segmented fragments, avoiding the impact on subsequent generation results due to fragments that are too short or semantically interrupted.

[0068] During segmentation, each chunk fragment must contain the following key information: Text content: The main information part of the chunk, containing specific semantic content; Main title: The main title information of the document, used to provide global context; Table title (if present): If the chunk fragment contains table data, the table title information must be extracted as a context reference.

[0069] The format is as follows:

[0070] {"text":"[text content]","main_title":"[document main title]","table_title":"[table title]"}

[0071] S2, based on high-quality question-answer pairs annotated by domain experts, designs a prompt template to guide the generation of training data for large models.

[0072] S21 involves domain experts annotating a small number of high-quality question-answer pairs. By leveraging their expertise and experience, these domain experts provide high-quality examples of question-answer pairs, offering directional guidance and quality standards for generating more question-answer pair data through a larger model.

[0073] Domain experts are individuals with specialized knowledge in their respective fields. Their responsibility is to extract key information from the original documents based on a thorough understanding of the document content, selecting representative and information-rich chunks for manual annotation of question-and-answer pairs. Priority is given to chunks containing important facts, statistical data, core business processes, or descriptions of key technologies to ensure that the annotated data is highly relevant and useful.

[0074] S22, based on the segmented chunks and high-quality data annotated by domain experts, designs a refined prompt template to guide large models in generating high-quality question-answer pair data.

[0075] The design of the prompt template includes the following key elements:

[0076] Domain context information: Provides domain-specific context information required to generate question-answer pairs;

[0077] Role definition: Clearly define the role of the large model, such as "RAG question-answer pair generation expert";

[0078] Answer format requirements: The generated question-and-answer pairs should be formatted in a standardized manner, including the correspondence between questions and answers;

[0079] Quality control rules: Establish rules for information screening, text comprehension, and handling of special cases;

[0080] Input and Generation Examples: A small amount of high-quality, manually labeled data is used as a reference example to provide a clear direction for the generation of large models.

[0081] One of the prompt designs is as follows:

[0082] #Role: RAG Question-Answer Pair Generation Expert

[0083] ##Profile

[0084] You are a RAG (Retrieval-Augmented Generation) sample generator, responsible for generating accurate and detailed questions based on provided document fragments. Your focus is on producing high-quality question-and-answer pairs from given document snippets. You need to generate the most appropriate question based on the provided text, main title, and table title (if any), and use the original text as the answer.

[0085] ##Knowledge

[0086] - Question Generation: Questions should be accurate and specific, fully covering the core information in the text fragment.

[0087] -Text understanding: Accurately understand the text content to ensure that the generated questions are highly relevant to the text content.

[0088] - Data processing: Capable of processing text containing HTML tags, as well as tabular data.

[0089] -Information screening: The ability to determine whether a text fragment contains enough information to generate meaningful questions.

[0090] ##Rules

[0091] 1. The input data format includes the following fields:

[0092] -text: Text content (required)

[0093] -main_title: Main title of the document (required)

[0094] -table_title: Table title (optional)

[0095] 2. Problem generation rules:

[0096] - You need to extract information from the three fields: text, main_title, and table_title, to generate a question that accurately reflects the key information in the document. The question should be as detailed and specific as possible.

[0097] - The question should include necessary contextual information (such as year, company name, etc.).

[0098] - For tabular data, generate questions targeting specific data items.

[0099] - For business data, generate questions that reflect the complete business situation.

[0100] - The generated questions should be helpful for further knowledge retrieval or text generation tasks, so the questions should lead to detailed answers.

[0101] 3. Handling of special circumstances:

[0102] - If the text fragment contains very little information (e.g., only a title), output "null".

[0103] - If the text contains HTML tags, the actual text content needs to be extracted before generating the question.

[0104] 4. Output format:

[0105] - The output is in JSON format and contains two fields: question and answer.

[0106] - If the text fragment contains very little information (e.g., only a title), output "null".

[0107] ##Example (Specific data content omitted here)

[0108] Input 1:

[0109] {"text":"[text content]","main_title":"[document main title]","table_title":"[table title]"}

[0110] Output 1:

[0111] {"Question":"[Manually Annotated Question]","Answer":"[Manually Annotated Excerpt Answer]"}

[0112] Input 2:

[0113] {"text":"[text content]","main_title":"[document main title]"}

[0114] Output 2:

[0115] {"Question":"[Manually Annotated Question]","Answer":"[Manually Annotated Excerpt Answer]"}

[0116] Input 3:

[0117] {"text":"[text content]","main_title":"[document main title]"}

[0118] Output 3:

[0119] "null"

[0120] ##Output Format

[0121] {"Question":"[Specific question generated from document content]","Answer":"[Original text fragment]"}

[0122] or

[0123] "null"

[0124] S23. Input the prompt template and document fragments into the large model to generate question-answer pair data. This involves inputting the completed prompt along with information from the chunk fragments (text, main_title, table_title) into the large model. The large model then generates corresponding question-answer pair data through understanding and reasoning about the input information.

[0125] S24. Before the large model generates question-answer pairs, it is necessary to evaluate the information content of the chunk fragments to determine whether they contain enough meaningful information.

[0126] For chunks with sufficient information, the large model generates question-answer pairs in a specified format. If a chunk has insufficient information, such as containing only a title or no content, it directly generates "null" as the output.

[0127] S3 utilizes vector models and reordering models for multi-stage hard negative sample mining. For example... Figure 2 As shown, Figure 2 This is a flowchart illustrating a method for mining hard negative samples according to an embodiment of the present invention. The method includes the following steps:

[0128] S31, Data Preparation: Prepare training data containing questions and corresponding positive samples, as well as a candidate sample pool. Specifically, prepare the input file `input_file` and the candidate sample pool `candidate_pool`. `input_file` is the training dataset, containing the query information for the questions and the position information for the corresponding positive samples. This file can be converted from the QA pairs in the format {"question":"[human-annotated question]","answer":"[human-annotated fragment answer]"} generated above. `candidate_pool` is an optional extended data source containing a possible set of negative sample candidates, providing basic data support for subsequent difficult negative sample mining.

[0129] The data format required by input_file is as follows:

[0130] {"query":str,"pos":List[str]}

[0131] The data format required by candidate_pool is:

[0132] {"text":"[text content]","main_title":"[document main title]","table_title":"[table title]"}

[0133] S32, using a semantic embedding model to generate text embedding vectors, specifically:

[0134] Model loading: Load a semantic embedding model, such as the bge-large-zh-v1.5 model, which is used to transform the input text data into a fixed-length vector representation, providing semantic embedding support for subsequent semantic retrieval and ranking tasks.

[0135] Embedding generation: The input data from the data preparation stage is processed to generate embedding vectors (p_vecs) for candidate texts and embedding vectors (q_vecs) for queries. The candidate text embeddings represent the semantic features of all texts in the sample pool, while the query embeddings represent the semantic features for each query.

[0136] S33, use FAISS to build an index and perform semantic retrieval to recall the top 100 candidate samples. Specifically:

[0137] FAISS Index Building and Search: A semantic retrieval index is built using the FAISS (Facebook AI Similarity Search) tool, and the query embedding vectors (q_vecs) are used to perform a fast semantic search on the embedding vectors (p_vecs) of the candidate sample pool to recall the TOP100 candidate samples that are semantically closest to each query, providing input data for the subsequent re-ranking steps.

[0138] S34 uses a re-ranking model to refine the ranking of candidate samples. Specifically:

[0139] Re-ranking involves loading a re-ranking model, such as the bge-reranker-large model, to further refine the ranking of candidate samples recalled by FAISS. Based on semantic relevance, a specified number of samples in the recalled samples are re-scored and ranked, and the top-ranked samples are selected from them.

[0140] S35, Filter negative samples from the ranking results and select the top K as training negative samples. Specifically:

[0141] Negative sample mining involves further filtering negative samples from reordered samples. It mainly includes the following two sub-steps: 1. Filtering negative samples: Remove positive sample information that is positively correlated with the query from the reordered samples and keep only negative sample data; 2. Select the top K negative samples from the filtered negative sample set as negative samples for training.

[0142] It should be noted that this invention relates to multi-stage hard negative sample mining, and the steps for screening negative samples from the ranking results include:

[0143] S351: The re-ranking model trained in the previous stage will be used to mine negative samples again, and a small number of difficult negative samples will be extracted for fine-tuning training. S352: The rule for selecting a small number of difficult negative samples is to simultaneously calculate the scores of positive samples and all negative samples. S353: If the difference between the score of a positive sample and the highest negative sample is less than a pre-set threshold, the sample is considered a difficult negative sample and will be used for training in the next stage.

[0144] Output results: The final results of negative sample mining will be output in file format to provide data support for subsequent model fine-tuning training.

[0145] The final output data format is

[0146] {"query":str,"pos":List[str],"neg":List[str]}

[0147] S4 generates knowledge distillation signals based on the teacher model.

[0148] After obtaining the positive and negative sample training data, this embodiment also needs to add a distillation signal to the training data, including the following steps: S41, select a reordering model with a model parameter size greater than 1B as the teacher model; S42, score the positive and negative samples in the training data; S43, add the scoring results as a knowledge distillation signal to the training data.

[0149] Specifically, input the training dataset. The training dataset is... Figure 2 The data obtained after execution includes information about the problem and positive and negative samples. Appropriate teacher and student models are selected. For example, the more parameterized and higher-performing bge-reranker-v2-minicpm-layerwise can be chosen as the teacher model, and the lighter bge-reranker-large as the student model. The teacher model is responsible for generating the distillation signal and is more capable of providing more useful information, while the student model gradually learns the knowledge of the teacher model through comparative learning and the distillation process. In this stage, the output of the teacher model serves as the learning objective of the student model, and the performance of the student model is improved through the distillation process. The final output data format is as follows:

[0150] {"query":str,"pos":List[str],"neg":List[str],"pos_scores":List[int],"neg_scores":List[int]}

[0151] Among them, pos_scores and neg_scores are the scores calculated by the teacher model.

[0152] S5 employs a multi-stage training strategy combining full parameter fine-tuning and LoRA fine-tuning to optimize the re-ranking model. For example... Figure 3 As shown, Figure 3 This is a flowchart illustrating the knowledge distillation-based reordering model fine-tuning method according to an embodiment of the present invention. The method includes the following steps:

[0153] Step 1) Construct the input dataset. The basic format of the input data is a dictionary structure containing "query" and "positive / negative sample word vector sequence", as follows:

[0154] {"query":str,"pos":List[str],"neg":List[str],"pos_scores":List[int],"neg_scores":List[int]}

[0155] Where query is the query text, pos is the set of positive samples related to the query text, neg is the set of negative samples related to the query text, and pos_scores and neg_scores are the matching scores of the positive and negative samples, respectively, used for knowledge distillation training.

[0156] Step 2) Sample sampling according to group size. This step groups the input positive and negative samples according to a preset group size. Each group contains a query and its corresponding positive and negative sample pair. Specifically, for each query, its positive sample list and negative sample list are merged, and then group size samples are randomly sampled to form a set of training data.

[0157] Step 3) Concatenate the input query with the positive / negative sample word vector sequence using a tokenizer to construct the following structure:

[0158] [CLS]query word vector sequence[SEP]positive / negative sample word vector sequence[SEP], where [CLS] is the classification marker and [SEP] is the separator marker.

[0159] Step 4) Re-ranking model encoding. A two-stage fine-tuning strategy is adopted. The first stage performs full parameter fine-tuning on all training data, and the second stage performs LoRA fine-tuning on a small number of selected hard-negative samples. Specifically, this includes:

[0160] In S51, the first stage fine-tunes all parameters of the model by optimizing the model through minimizing the weighted sum of cross-entropy loss and knowledge distillation loss. The knowledge distillation loss uses the teacher model's soft labels as a supervision signal to help the student model learn the knowledge from the teacher model.

[0161] S52, in the second stage, based on the model trained in the first stage, further negative samples are mined, and a small number of difficult negative samples are selected to construct new training data. The specific selection method is detailed in... Figure 2 Step B6 provides a detailed explanation. Then, LoRA fine-tuning is used to adjust some parameters of the model, only the query, key, value, and dense layers are fine-tuned using LoRA, while the classifier layer is left with all parameters fine-tuned and the remaining parameters frozen, thereby improving the model's ability to distinguish between hard-negative samples.

[0162] Step 5) Classifier Prediction. After two stages of fine-tuning, the re-ranking model predicts the relevance of the input query-document pairs and outputs a relevance score. The classifier performs binary classification based on the high-dimensional feature representation encoded by the model to determine whether the document is relevant to the query.

[0163] Step 6) Calculate the loss and optimize the model. During training, the cross-entropy loss and knowledge distillation loss are combined for joint optimization. The cross-entropy loss is used to supervise the model in learning correct relevance judgments, while the knowledge distillation loss guides the student model to imitate the prediction distribution of the teacher model and make full use of the knowledge in the teacher model.

[0164] Specifically, the knowledge distillation loss is achieved by minimizing the cross-entropy between the output distributions of the student model and the teacher model after softmax, and the mathematical expression is:

[0165]

[0166] Where N is the batch size, M is the group size, and t ij p is the target output distribution of the teacher model. ij Given the student model's output distribution, minimize this loss function so that the student model's output is as close as possible to the teacher model's output.

[0167] Through the above steps, the method of this invention can effectively improve the performance of the re-ranking model in vertical domains. In particular, the two-stage fine-tuning strategy ensures that the model can fully learn global knowledge and optimize hard sample scenarios in a targeted manner; while the knowledge distillation mechanism helps the model inherit the discriminative ability of the teacher model, achieving good results while maintaining the model's lightweight nature.

[0168] This invention significantly improves the model's discriminative ability in vertical domains through the construction of high-quality question-answer pairs and a difficult negative sample mining strategy. In particular, the multi-stage iterative negative sample mining mechanism enables the model to gradually learn finer-grained semantic discrimination capabilities, effectively reducing the impact of external interference.

[0169] Secondly, the introduction of the knowledge distillation mechanism enables the model to better learn the knowledge representations of the teacher model, significantly improving the model's generalization ability. Through knowledge transfer at the distribution level, the model not only learns sample-level discriminative ability but also masters more abstract domain knowledge representations, enabling it to better handle unseen query scenarios.

[0170] Third, the innovative multi-stage fine-tuning strategy effectively balances computational efficiency and model performance. Compared to traditional full-parameter fine-tuning methods, the solution of this invention significantly reduces computational resource consumption while maintaining similar performance. In particular, the LoRA technology used in the second stage enables the model to quickly adapt to the specific needs of the vertical domain, providing an efficient solution for iterative optimization of the model.

[0171] Fourth, the overall solution has strong scalability and can be flexibly applied to different vertical industry scenarios. By adjusting the prompt template, negative sample mining strategy, and fine-tuning parameters, the solution can quickly adapt to new domain requirements. Practice has shown that this method has been successfully applied in multiple professional fields such as medicine, law, and finance, demonstrating good adaptability and stability.

[0172] Furthermore, the method of this invention demonstrates excellent stability and reliability in practical applications, meeting the stability requirements of production environments. Simultaneously, the modular design of this solution facilitates maintenance and upgrades, providing sustainable technical support for intelligent retrieval systems in vertical industries.

[0173] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the appended claims. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to depart from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-stage semantic reordering fine-tuning method applied to vertical domains, characterized in that, Includes the following steps: S1 receives the original document input and segments the document into fragments; S2, based on high-quality question-answer pairs annotated by domain experts, designs a prompt template to guide the generation of training data for large models; S3 utilizes vector models and reordering models for multi-stage hard negative sample mining; specifically: S31, Prepare training data containing the question and the corresponding positive sample, as well as a candidate sample pool; S32, using a semantic embedding model to generate text embedding vectors; S33, use FAISS to build an index and perform semantic retrieval to recall the TOP100 candidate samples; S34, Use a re-ranking model to refine the ranking of candidate samples; S35, Filter negative samples from the ranking results and select the TOP K as training negative samples; S4, generating knowledge distillation signals based on the teacher model; S5 employs a multi-stage training strategy combining full parameter fine-tuning and LoRA fine-tuning to optimize the re-ranking model. Specifically: S51, the first stage is to fine-tune all parameters of the model and optimize the weighted sum of cross-entropy loss and knowledge distillation loss; S52, the second stage mines hard negative samples based on the model of the first stage, uses LoRA technology to fine-tune the query, key, value and dense layers, and performs full parameter fine-tuning on the classifier layer.

2. The method according to claim 1, characterized in that: Step S2, which involves designing a prompt template to guide the generation of training data for a large model, includes the following steps: S21, Domain experts select representative document segments from the original documents and annotate them with question-and-answer pairs; S22, design a prompt template that includes domain background information, role settings, answer format requirements, quality control rules, and input / output examples; S23, Input the prompt template and document fragments into the large model to generate question-answer pair data; S24 evaluates the information content of document fragments and outputs "null" for fragments with insufficient information content.

3. The method according to claim 1, characterized in that, Step S35, the step of filtering negative samples from the sorting results, includes: S351, use the re-ranking model trained in the previous stage to mine negative samples again. S352, calculate the scores for positive samples and all negative samples; S353: When the difference between the positive sample and the highest negative sample is less than a preset threshold, the sample is used as a difficult negative sample for the next stage of training.

4. The method according to claim 1, characterized in that, The steps in step S4 for generating the knowledge distillation signal include: S41, Select a reordering model with a parameter size greater than 1B as the teacher model; S42, scores the positive and negative samples in the training data; S43, the scoring results are added to the training data as knowledge distillation signals.

5. The method according to claim 1, characterized in that, The knowledge distillation loss is calculated using the following formula: Where N is the batch size, M is the group size, and t ij p is the target output distribution of the teacher model. ij Output the distribution for the student model.

6. A fine-tuning apparatus based on the multi-stage semantic reordering fine-tuning method according to any one of claims 1-5, characterized in that, include: The data generation module is used to generate training data based on domain expert-annotated data; The negative sample mining module is used for multi-stage hard negative sample mining. The knowledge distillation module is used to generate knowledge distillation signals; The model training module is used to perform multi-stage fine-tuning training.

7. The fine-tuning device as described in claim 6, characterized in that, The data generation module includes: The document processing unit is used to segment and process the input document; The prompt design unit is used to design prompt templates to guide large models; The data generation unit is used to generate training data using a large model.

8. A computer device comprising a processor and a memory, characterized in that, The memory stores a computer program, which, when executed by the processor, implements the steps of the multi-stage semantic reordering fine-tuning method for vertical domains as described in any one of claims 1 to 5.

Images