Two-stage parameter optimization method and system for vertical field large language model

Abstract

The present application relates to a two-stage parameter optimization method and system for vertical field large language models, and relates to the field of artificial intelligence. Basic text screening is performed based on a vertical field knowledge base to construct a multi-dimensional benchmark data set. The multi-dimensional benchmark data set is used to perform initial stage supervised fine-tuning on a large language base model, complete preliminary alignment, and obtain cold start model parameters. Based on the preliminary model parameters, samples containing multi-source context references in the multi-dimensional benchmark data set are used as input contexts to generate multiple candidate answer group internal sample sets for the same input question. A multi-dimensional reward function is used to quantify each answer, calculate the relative advantage value of each answer, update the model parameters, and optimize the large language model parameters to meet the vertical field question and answer goal. The technical problem of the present application is that the existing vertical field large language model professional annotation data and preference data are scarce, resulting in insufficient model knowledge attribution accuracy and inability to effectively suppress field knowledge hallucination.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and in particular to a two-stage parameter optimization method and system for large language models in vertical domains. Background Technology

[0002] With the rapid development of large language model technology, its application in various vertical fields such as agriculture, medicine, and industry continues to deepen. However, due to the lack of systematic training with domain-specific knowledge, general large language models generally suffer from problems such as frequent knowledge illusions, weak professional reasoning ability, and insufficient accuracy in decision support in professional scenarios, making it difficult to meet the high-precision and professional question-answering service needs of vertical fields. Current optimization solutions for large models in vertical fields have many limitations. Although full pre-training can achieve domain knowledge injection, it consumes huge amounts of computing power and is prone to destroying the general capabilities and instruction follow-up formed by the pre-training of the base model, leading to catastrophic forgetting problems. Traditional reinforcement learning solutions based on human feedback require additional training of a value model of the same scale, further increasing the computing power threshold. Moreover, professional labeled data and preference data in vertical fields are scarce, making it difficult to support large-scale reinforcement learning training. At the same time, existing model training is mostly based on pure standard answer samples, which does not simulate the complex environment of mixed multi-source information and intertwined correct and incorrect knowledge in real retrieval and reinforcement generation scenarios. As a result, after the model is applied, it is easily misled by the interference information returned by the retrieval, resulting in insufficient accuracy of knowledge attribution and inability to effectively suppress domain knowledge illusions. Summary of the Invention

[0003] This invention addresses the technical problem of insufficient accuracy in knowledge attribution and inability to effectively suppress domain knowledge illusion caused by the scarcity of professionally labeled and preference data in existing large-scale language models for vertical domains. It provides a two-stage parameter optimization method and system for large-scale language models for vertical domains to solve this problem.

[0004] The technical solution of the present invention to solve the above-mentioned technical problems is as follows: Firstly, this invention provides a two-stage parameter optimization method for large language models in vertical domains, including: Basic text filtering is performed based on a vertical domain knowledge base, and the filtered basic text is enhanced with retrieval distribution to construct a multidimensional benchmark dataset. The large language foundation model is then subjected to initial supervised fine-tuning using this multidimensional benchmark dataset to achieve preliminary alignment of the large language model with the vertical domain's basic knowledge question-answering paradigm and knowledge reference relationships, obtaining cold-start model parameters. The large language foundation model has been injected with a trainable low-rank factorization matrix, and the original weight parameters are frozen. Based on these preliminary model parameters, samples containing multi-source context references in the multidimensional benchmark dataset are used as input context to enter the reinforcement learning stage. Multiple candidate answer groups are generated for the same input question, and each answer is quantified using a multidimensional reward function. The relative advantage value of each answer within the group is calculated, and the model parameters are then pruned and updated in a second stage to optimize the large language model parameters and meet the vertical domain question-answering objectives.

[0005] Secondly, this invention provides a two-stage parameter optimization system for large language models in vertical domains, including: The filtering module is used to filter basic text based on a vertical domain knowledge base and enhance the retrieval distribution of the filtered basic text to construct a multidimensional benchmark dataset. The supervision module is used to perform initial supervised fine-tuning of the large language base model using the multidimensional benchmark dataset, complete the initial alignment of the large language model with the basic knowledge question-answering paradigm and knowledge reference relationships of the vertical domain, and obtain cold-start model parameters. The large language base model has been injected with a trainable low-rank factorization matrix and the original weight parameters are frozen. The update module is used to enter the reinforcement learning stage based on the initial model parameters, using samples containing multi-source context references in the multidimensional benchmark dataset as input context. For the same input question, multiple candidate answer group sample sets are generated, and each answer is quantified using a multidimensional reward function. By calculating the relative advantage value of each answer within the group, the model parameters are pruned and updated in a second stage to optimize the large language model parameters to meet the vertical domain question-answering objectives.

[0006] The beneficial effects of this invention are: By constructing a multidimensional benchmark dataset that integrates a retrieval distribution enhancement mechanism and multi-level adversarial negative samples, and combining a two-stage parameter optimization architecture that injects low-rank decomposition matrices, the system first achieves a cold start by completing the basic alignment of the domain knowledge system and question-answering paradigm through supervised fine-tuning. Then, it enhances the ability to identify complex information and make logical inferences by optimizing the group-relative strategy of the valueless model. Under the low-computing conditions of compressing the number of trainable parameters to one-thousandth of the original model and not requiring additional training of a large-scale value network, it simultaneously achieves deep injection of domain knowledge and improvement of complex reasoning ability. This effectively reduces the occurrence rate of knowledge illusion in vertical domains and improves the accuracy of knowledge attribution and anti-interference ability of the model in retrieval enhancement scenarios. Attached Figure Description

[0007] Figure 1 This is a flowchart illustrating the two-stage parameter optimization method for large language models in vertical domains provided by the present invention.

[0008] Figure 2 This is a schematic diagram of the structure of the two-stage parameter optimization system for large language models in vertical domains provided by the present invention.

[0009] Figure labeling: Filtering module 11, Supervision module 12, Update module 13. Detailed Implementation

[0010] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0011] In the description of this invention, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0012] In the description of this invention, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use the invention. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that the invention can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid obscuring the description of the invention with unnecessary detail. Therefore, the invention is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed herein.

[0013] Example 1: like Figure 1 As shown, this embodiment of the invention provides a two-stage parameter optimization method for large language models in vertical domains, including: Step S1: Perform basic text filtering based on the vertical domain knowledge base, and enhance the retrieval distribution of the filtered basic text to construct a multidimensional benchmark dataset.

[0014] Specifically, the vertical domain knowledge base is used as the initial input, and a vertical domain knowledge base refers to a structured knowledge set that integrates multi-source heterogeneous professional content such as academic literature, industrial standards, policy documents, monographs and classics, and practical experience in the corresponding industry; specifically, semantic analysis is first performed on unstructured long texts in the knowledge base to extract semantically complete text blocks. Such text blocks are continuous text fragments that have independent knowledge semantic units and contain complete logical premises, derivation processes and conclusion expressions. They neither lose core information due to truncation, nor mix multiple unrelated topics, can independently support the question-and-answer generation of a complete domain knowledge point, and are the smallest effective knowledge units for subsequent data synthesis. Meanwhile, all extracted semantically complete text blocks are subjected to semantic vectorization processing to construct a vector index. A vector index is a retrievable index structure established after converting text content into high-dimensional semantic vectors through an embedding model, which can quickly calculate the semantic similarity between different texts based on the cosine distance in vector space, and provide an efficient matching foundation for subsequent retrieval enhancement operations.

[0015] On this basis, based on the professional dimensions preset for vertical domain knowledge, multi-dimensional information density evaluation is carried out on all semantically complete text blocks. Specifically, the domain core keywords of the entire knowledge base are counted through the TF-IDF algorithm to construct a document-level keyword candidate list, the keyword coverage rate of each text block is calculated one by one, and text blocks with a coverage rate higher than a preset threshold are screened as candidate reference content. Specifically, a corpus preprocessing procedure is performed first: all text blocks are standardized and cleaned to remove punctuation marks, special characters, format marks and meaningless serial numbers, then Chinese word segmentation is completed through a professional word segmentation tool adapted to the vertical domain, and a customized vocabulary that integrates general stop words and domain stop words is loaded to filter out empty words, conjunctions and general expressions that do not carry exclusive knowledge, such as "of", "therefore", "related" and "technology". In addition, normalized mapping is performed on aliases, abbreviations and full names of the same domain concept to ensure that the same professional term corresponds to a unique statistical unit; corpus preprocessing is a pre-requisite basic procedure for word frequency statistics, and its function is to eliminate text format noise and ineffective interference of general vocabulary, so that the statistical weight is inclined to real domain professional terms.

[0016] On this basis, word frequency statistics are carried out, that is, the word frequency value of each word is calculated document by document. For each semantically complete text block, the occurrence frequency of each word in the text is counted first, and then it is divided by the total number of words after word segmentation of the text block to obtain the word frequency value of the word in the current document. Word frequency, whose full name is term frequency, reflects the local importance of a word in a single document. A higher occurrence frequency indicates that the word has higher centrality in the content of the current document. Since a single text block is used as an independent statistical unit, each document will generate a corresponding set of word frequency statistical results.

[0017] Subsequently, the inverse document frequency (IVF) across the entire corpus is calculated. This metric measures the prevalence and domain distinctiveness of a word within the entire knowledge base corpus. Specifically, the total number of documents in the entire knowledge base is first counted, followed by the number of documents containing a particular word. The IVF is then calculated using a logarithmic operation. The calculation logic is to take the logarithm of the ratio of the total number of documents to the number of documents containing the word, with an extra 1 added to the denominator to avoid division by zero when the word is not present. The higher the IVF value, the fewer documents a word appears in, indicating stronger domain specificity and higher distinctiveness. For example, specific pest terms like pear psyllid appear only in a small number of plant protection documents, resulting in a higher IVF value. Conversely, general terms like management appear in the vast majority of documents, leading to a lower IVF value. This metric effectively reduces the weight of general terms and accurately highlights core domain terms.

[0018] After completing the calculation of the two basic indicators, the TF-IDF weight value of each word is calculated for each document. This is done by multiplying the word frequency value in the current document by the inverse document frequency value that is consistent across the entire database to obtain the final TF-IDF value. This value takes into account both the local importance of the word in a single document and its global distinctiveness across the entire domain. The higher the value, the more likely the word is to be the core vocabulary of the current document.

[0019] Finally, the document-level keyword candidate table was constructed, which consisted of two parts: global filtering and document mapping. First, at the global level, the average TF-IDF values ​​of all words in the entire database were sorted in descending order. Based on the thresholds set by domain experts, words of the corresponding magnitude were selected as the core keyword set for the domain, covering professional terms in core areas such as crop varieties, cultivation techniques, pests and diseases, soil and fertilizer, and industrial policies. Then, for each semantically complete text block, the words with the highest TF-IDF values ​​in the document were extracted as candidate keywords for that document. These candidate keywords were matched with the global core keyword set for the domain, and the proportion of successfully matched keywords to the total number of candidate keywords in the document was counted, i.e., the document keyword coverage rate.

[0020] Finally, the document number, text content, corresponding candidate keyword list, and keyword coverage value of all text blocks are summarized to form a structured document-level keyword candidate table. Each row corresponds to a text block, fully recording its keyword composition and information density quantification indicators. The information density assessment is a quantitative screening mechanism for the knowledge content of the text. The higher the keyword coverage, the more domain-specific terms and core concepts the text contains, and the higher the professional concentration and reference value of the knowledge. Through this screening, low-value texts with sparse information, repetitive content, or only common-sense expressions can be eliminated, ensuring the professional quality of the subsequent synthesized training data.

[0021] After screening, standardized question-and-answer data is synthesized using the obtained candidate reference content as the reference context and a two-stage prompting strategy. The two-stage prompting strategy is a prompting engineering method that controls the generation quality step by step. The first stage is the question generation stage, in which only the candidate reference content is input into the large language model as context. The model is guided to generate vertical domain knowledge questions that are highly bound to the reference content and conform to the questioning habits of domain users with only the constraint prompts of generating questions, so as to avoid questions deviating from the reference content. The second stage is the answer generation stage, in which the same candidate reference content is paired with the questions generated in the first stage and input into the model. The model is instructed to generate professional and accurate answers strictly according to the expression of the reference context, ensuring that the knowledge source of the answer is completely anchored to the reference content.

[0022] After the basic question-answer pair synthesis is completed, retrieval distribution augmentation is performed using the candidate reference content on which the question-answer pair was generated as semantic anchors. The semantic anchors are the reference texts that serve as the retrieval benchmark, acting as the semantic reference standard for recalling relevant texts. The similarity and relevance of all recalled content are calculated based on this anchor text. Retrieval distribution augmentation is a data augmentation method that enhances the robustness of the model by expanding the contextual complexity of the question-answer samples. It simulates the environment of mixed multi-source information in real-world retrieval augmentation generation scenarios, allowing the model to learn the ability to locate core knowledge from redundant and interfering information during the training phase. Specifically, similarity retrieval is performed using vector indexing to recall multiple text blocks related to the semantic anchors from the full knowledge base. These include semantically highly matched positive sample text blocks that can supplement and corroborate the core knowledge points, as well as adversarial negative sample text blocks that share the same domain entities and thematic connections as the anchors but have different or even contradictory conclusions. The adversarial negative sample texts are constructed interfering training samples that share domain entities and some background information with the core question, but their conclusions differ from the correct answer. The model exhibits biases and can simulate complex scenarios such as homonyms and knowledge conflicts that occur in real-world retrieval, forcing the model to improve its knowledge identification and attribution judgment capabilities. In this process, all recalled text blocks are divided into three noise levels based on their semantic similarity to the semantic anchor. The first noise level consists of text blocks with semantic similarity higher than a first preset threshold, mainly used to provide richer contextual support for the core question, and is considered low-noise positive supplementary content. The second noise level consists of text blocks with semantic similarity between the first and second preset thresholds, whose content is partially related to the core topic but cannot directly answer the question, and is used to introduce appropriate interference information. The third noise level consists of text blocks belonging to the same vertical subdomain as the anchor text but with different research topics, and is considered high-noise interference content across topics. Adversarial negative samples are mainly selected from the second and third noise levels. Subsequently, texts from different noise levels are merged with the original reference context according to a preset dynamic ratio to construct complex question-answering samples containing redundant information and multi-source context. This dynamic ratio is asymmetrically configured during the training phase.

[0023] Finally, based on the knowledge structure system of the vertical domain, all complex question-answering samples are labeled and hierarchically divided in multiple dimensions. Hierarchical sampling is completed according to different sub-domains such as crop cultivation, pest and disease diagnosis, and soil and fertilizer. The samples are split into training and testing sets in a fixed ratio, ultimately forming a multi-dimensional benchmark dataset containing various data types such as traditional question answering, thought chain question answering, and context reference question answering. The multi-dimensional benchmark dataset is a comprehensive training dataset covering multiple question answering paradigms, multiple difficulty levels, and multiple knowledge sub-domains. It can simultaneously support the model's basic question answering ability alignment, reasoning ability improvement, and retrieval enhancement adaptation, among other multi-dimensional training objectives.

[0024] Step S2: Use the multidimensional benchmark dataset to perform initial supervised fine-tuning of the large language base model, complete the initial alignment of the large language model with the basic knowledge question-answering paradigm and knowledge reference relationship of the vertical domain, and obtain the cold start model parameters. The large language base model has been injected with a trainable low-rank decomposition matrix and the original weight parameters are frozen.

[0025] Specifically, the large language foundation model refers to a general pre-trained large language model based on the stacking of Transformer decoder units. Its units are composed of alternating stacks of multi-head self-attention mechanism layers and feedforward neural network layers. The multi-head self-attention layer realizes semantic dependency modeling within the sequence through weight mapping of query vector Q, key vector K, and value vector V, capturing the association information between context words. The feedforward neural network layer completes high-order transformation of features through two layers of linear transformation and non-linear activation functions such as GELU. Together, they support the model's language understanding and generation capabilities. All the original weight parameters of the model are obtained through pre-training on a large-scale general corpus, and it has mature general instruction following and natural language expression capabilities.

[0026] Subsequently, a trainable low-rank decomposition matrix is ​​injected into the linear weight matrices of all multi-head self-attention layers and feedforward neural network layers in the base model. Based on the principle of low-rank adaptation, this is applied to models with dimensions of... Instead of directly updating the original weights, the linear layer weight matrix is ​​modified by adding two low-dimensional matrices that are products of each other. and Where r is the low-rank number, whose value is much smaller than the dimensions d and k of the weight matrix, usually set to small values ​​such as 8, 16, and 32, thereby compressing the number of training parameters to one-thousandth of the original model. During matrix initialization, matrix A is initialized with Gaussian randomization, and matrix B is initialized as a matrix of all zeros to ensure that the output of the low-rank branch is 0 in the initial stage of training, without destroying the existing pre-training capability of the base model. The corresponding forward propagation calculation form is as follows: Where x is the input vector of the current linear layer and h is the output vector of the linear layer, which is equivalent to updating the original weights to... The accompanying freezing of original weight parameters means that the original weight matrix W of the base model remains at a fixed value throughout the entire supervised fine-tuning training process, does not participate in gradient calculation and parameter updates, and only the low-rank matrices A and B participate in optimization as trainable parameters.

[0027] Training uses a constructed multidimensional benchmark dataset as all training inputs. This dataset covers three core data formats: traditional question answering, thought chain question answering, and context reference question answering. It contains multi-source context samples injected with different levels of noise and adversarial negative samples, covering knowledge scenarios in multiple sub-directions within the vertical domain, such as crop cultivation, pest and disease diagnosis, and soil and fertilizer, providing multi-dimensional training materials for the model. When inputting data, the standard construction method of autoregressive language modeling is adopted. The instructions of each sample are concatenated with the reference context to form an input prompt sequence, and the standard answer is concatenated to form a target output sequence. Both are converted into a token sequence that the model can recognize through a token segmenter. During training, the loss is calculated only for the tokens of the target output sequence, and the input prompt sequence is not included in the loss calculation. This guides the model to learn the input-to-output mapping relationship that conforms to the domain norms.

[0028] Training employs a standard supervised fine-tuning paradigm, with the optimization objective of minimizing the cross-entropy loss function. The cross-entropy loss function is a quantitative metric that measures the difference between the token probability distribution predicted by the model and the actual standard answer token distribution; its calculation formula is as follows: ; Where x is the input prompt sequence. For the i-th token in the target sequence, The model predicts the probability of the i-th token based on the input and the preceding tokens. N is the total number of tokens in the target sequence. The smaller the loss value, the higher the overlap between the content generated by the model and the standard answer, and the smaller the semantic deviation. During training iterations, data is input into the model in batches to complete the forward propagation calculation to obtain the loss value. Then, the gradient of the loss with respect to the trainable parameters is calculated through the backpropagation algorithm. The gradient is only propagated along the low-rank matrix branch, and only the values ​​of matrices A and B are updated. The original weights remain unchanged throughout the process. During training, adaptive learning rate optimizers such as AdamW are used to update the parameters. In addition, strategies such as learning rate warm-up and cosine decay are used to ensure training stability. Gradient clipping is also set to prevent gradient explosion.

[0029] During training iterations, model performance is validated simultaneously from three core dimensions: First, basic language modeling capability, using perplexity metrics to verify the fluency, grammatical accuracy, and semantic coherence of the model-generated text, ensuring that the model's basic language expression capabilities do not deteriorate; second, knowledge reference alignment capability, quantifying the degree of knowledge matching between the model-generated answer and the corresponding reference context through entity matching rate and conclusion consistency metrics, judging whether the model can accurately anchor domain knowledge in the reference content, avoiding the generation of false content without basis, which is an initial evaluation dimension to constrain knowledge illusion; third, format standardization constraints, using metrics such as professional terminology accuracy and output structure compliance rate to control the consistency of professional terminology, standardization of expression structure, and adaptation to domain expression habits in the model's output, ensuring that the output conforms to the professional communication norms of the vertical domain.

[0030] Subsequently, when the cross-entropy loss on the validation set no longer decreases for several consecutive rounds and all evaluation metrics reach the preset thresholds, the training is considered to have converged, and the cold-start model parameters that have completed the initial domain adaptation can be obtained. Here, the cold-start model parameters refer to the model parameters that have been supervised and fine-tuned and have basic vertical domain question-answering capabilities and knowledge referencing awareness. They have not yet been strengthened with complex reasoning capabilities, but have completed the basic alignment of the domain question-answering paradigm and knowledge referencing logic, and can serve as a stable initial state for the second stage of reinforcement learning training.

[0031] Step S3: Based on the preliminary model parameters, using samples containing multi-source context references in the multi-dimensional benchmark dataset as input context, enter the reinforcement learning stage. Generate multiple candidate answer group sample sets for the same input question, quantify each answer using a multi-dimensional reward function, calculate the relative advantage value of each answer within the group, perform a two-stage pruning and updating of model parameters, and optimize the large language model parameters to meet the vertical domain question answering objectives.

[0032] Specifically, based on the model parameters of the supervised fine-tuning output, the training input is samples containing multi-source context references in a multi-dimensional benchmark dataset. These multi-source context reference samples refer to complex context data that has undergone retrieval distribution enhancement processing and incorporates semantically positive samples, interference information at different noise levels, and adversarial negative samples. After training begins, for the same input question and its associated multi-source context, different generation parameters such as sampling temperature and kernel sampling threshold are set to allow the current model to sample and generate G candidate answers with different expression styles and inference paths. These G candidate answers together constitute the within-group sample set, which consists of multiple output samples generated for the same question. The set is the foundation of group-based relative strategy optimization. It judges answer quality by comparing samples within the same group, eliminating the need for training a large-scale value network and reducing computational consumption during the reinforcement learning phase. After candidate answer generation, each candidate answer within the group is quantitatively scored using a pre-defined multi-dimensional reward function. This multi-dimensional reward function is a quantitative evaluation system designed from multiple core quality dimensions across various vertical domains, avoiding the one-sided optimization problem caused by a single reward dimension. It covers three core dimensions: knowledge attribution accuracy, domain logical coherence, and output format standardization. The final total reward value is obtained by weighted summation of each dimension. The corresponding calculation formula is as follows: ; in The total reward value for the i-th candidate answer. The weight coefficients for each dimension and satisfying The weighting can be adjusted according to the needs of the vertical industry. Specifically, the knowledge attribution accuracy reward... Calculated by comparing the entity matching rate and conclusion consistency between the answer content and the correct knowledge in the context, this measure is used to evaluate the model's ability to identify adversarial negative samples and anchor correct knowledge. It is the core reward for constraining domain knowledge illusion and is a reward for domain logical coherence. The evaluation criteria, including the completeness of the reasoning chain, the consistency of causal logic, and the fluency of professional semantics, are used to enhance the model's reasoning ability for complex problems, and to output a standardized format reward. The accuracy rate of professional terminology and the compliance rate of output structure in the statistical answers are then used to ensure that the output conforms to the professional expression standards of the vertical field. After obtaining the total reward value of all candidate answers in the group, a standardized calculation is performed within the group to obtain the relative advantage value of each answer. The relative advantage value is a standardized indicator that measures the relative superiority or inferiority of a single answer in the same sample group. Only the relative performance within the same group is used as the optimization basis. In the specific calculation, the mean of the reward values ​​of the G samples in the same group is first calculated: ; Compared with standard deviation: ; Then, the relative advantage value of the i-th answer is obtained through a standardized formula: ; in It is a very small constant used to avoid division-to-zero anomalies when the standard deviation is 0, allowing the model to judge the quality of its output simply by comparing it with its own generated results within the same group.

[0033] After calculating the relative advantage value, a two-stage pruning update of the model parameters is performed. Parameter optimization is carried out based on the pruned policy probability ratio, while KL divergence penalty is introduced to constrain the policy offset magnitude. Specifically, the probability of the current policy to be updated generating the i-th candidate answer is first calculated. The probability of generating this answer compared to the cold start reference strategy probability ratio: ; Where x represents the input question and multi-source context. For the i-th candidate answer, Given the currently trainable low-rank matrix parameters, to prevent policy collapse due to excessively large parameter updates in a single iteration, a pruning constraint is applied to the probability ratio, limiting it to a certain value. Within the interval, The clipping factor is denoted as , and the corresponding clipping operation is represented as . This pruning update mechanism can limit single-step strategy updates to a controllable range.

[0034] Based on this, the final policy loss function is constructed, which consists of the pruned policy objective term and the KL divergence penalty term, and the calculation formula is as follows: ; The first term in the formula represents the optimization objective of the pruned strategy. By taking the smaller value between the probability ratio and the pruned ratio, the maximum update reward is limited when the advantage is positive, and excessive penalty is limited when the advantage is negative, thus avoiding drastic fluctuations in strategy updates. The second term is the KL divergence penalty term. The penalty coefficient is... The KL divergence between the current strategy and the cold-start reference strategy is used to measure the degree of difference in their output distributions, constraining the model to not deviate excessively from the initial basic language capabilities and domain question-answering paradigm. During parameter updates, gradients are only backpropagated to the low-rank decomposition matrices A and B, and the original weights of the base model remain frozen throughout, continuing the efficient parameter training characteristics of the first stage. The low-rank matrix parameters are iteratively updated through an adaptive optimizer to gradually improve the overall question-answering quality of the model. After each round of reinforcement learning iteration, the improvement in the model's knowledge attribution consistency reward on the validation set is also calculated. When the improvement exceeds a preset high threshold, a difficulty increase operation is triggered to increase the proportion of noise injection and adversarial negative samples in subsequent training samples; otherwise, the difficulty is maintained or reduced. Through dynamic adjustment in a course-like manner, the pace of model capability advancement is matched, further improving training efficiency and final results.

[0035] Furthermore, in the method provided in the application embodiment, basic text filtering is performed based on a vertical domain knowledge base, and the filtered basic text is enhanced in terms of retrieval distribution to construct a multidimensional benchmark dataset. This includes: performing semantic analysis on the vertical domain knowledge base to extract semantically complete text blocks; performing semantic vectorization processing on the extracted semantically complete text blocks to construct a vector index; evaluating the multidimensional information density of the complete text blocks based on a preset dimension of vertical domain knowledge to filter candidate reference content that exceeds a preset threshold; using the candidate reference content as a reference context, performing search distribution reorganization enhancement based on the vector index to construct the multidimensional benchmark dataset.

[0036] Specifically, the model uses a vertical domain knowledge base as its input source. This knowledge base refers to a collection of heterogeneous professional content from multiple sources, including academic literature, industry standards, policy documents, monographs, and practical records. It possesses strong domain-specific and authoritative attributes and is the core source for the model to acquire domain knowledge. First, semantic analysis is performed on the original documents in the knowledge base to extract semantically complete text blocks. Text preprocessing is then performed to complete basic processing such as format markup cleaning, redundant content deduplication, and expression error correction. Next, a semantic segmentation algorithm that integrates text structure recognition and semantic similarity determination is used. Natural paragraph units are initially divided by locating logical connectives such as title level marking, paragraph boundary recognition, and words like "therefore," "in summary," and "firstly." Then, a pre-trained semantic coding model is used to calculate the semantic correlation between adjacent paragraph units. When the correlation is lower than a preset boundary threshold, it is determined as a semantic breakpoint. Continuous and highly semantically aggregated paragraph units are aggregated into a semantically complete text block. A semantically complete text block is an independent knowledge unit that revolves around a single domain knowledge point and has complete premise derivation and conclusion logic, which can support the generation of questions and answers for a complete domain knowledge point.

[0037] After extracting text blocks, semantic vectorization is performed on all semantically complete text blocks to construct vector indexes. This process uses a semantic embedding model adapted to the vertical domain as the encoding tool. First, the text content of each block is segmented and model-specific special markers are added. These are then converted into a token sequence recognizable by the model and input into the embedding model. The global semantic vector corresponding to the model's output layer is extracted as the semantic representation of that text block. Semantic vectors are dense vectors that map text semantic information to a high-dimensional numerical space, typically 768 or 1024 dimensions. They accurately capture the domain semantic connotations of the text. Semantically similar texts are closer in Euclidean distance and have higher cosine similarity in the vector space. The formula for calculating the cosine similarity between two text vectors is: ; The value ranges from [-1, 1], and the higher the value, the stronger the semantic relevance.

[0038] Subsequently, a vector index was constructed using efficient vector retrieval engines such as FAISS. The semantic vectors of all text blocks were imported into the index structure in batches, establishing a one-to-one mapping relationship between vector values ​​and corresponding text block IDs and original content. At the same time, the storage structure and retrieval algorithm of the index were optimized to achieve millisecond-level similarity retrieval of millions of text blocks. The vector index is the underlying retrieval structure that supports the rapid retrieval of related texts. It can reduce the complexity of full database traversal and comparison to the logarithmic level, ensuring the enhanced execution efficiency of retrieval distribution.

[0039] Based on this, a multi-dimensional information density assessment is conducted on all semantically complete text blocks using the professional dimensions pre-defined by vertical domain knowledge. Specifically, the TF-IDF algorithm is used to count the core keywords of the entire domain and construct a document-level keyword candidate table. The domain keyword coverage rate of each text block is calculated, and auxiliary indicators such as the number of professional entities and the density of logical relationship expressions within the text are also counted. The comprehensive information density score of each text block is obtained by weighted summation. Text blocks with scores higher than a preset threshold are selected as candidate reference content. The candidate reference content consists of high-quality text units with the highest knowledge concentration and the strongest professional attributes in the knowledge base, eliminating low-value texts with sparse information and containing only common-sense expressions.

[0040] After screening, the candidate reference content is used as the reference context. Based on the constructed vector index, the search distribution is reorganized and enhanced, i.e., the retrieval distribution is enhanced. Each candidate reference content is used as a semantic anchor. The semantic anchor is the text that serves as the retrieval benchmark and is the semantic reference standard for recalling relevant content. The vector index is used to perform a full database similarity search to recall multiple text blocks that are semantically related to the anchor. The recall results include both semantically highly matched positive samples that can supplement and corroborate the core knowledge, and adversarial negative samples that share domain entities with the anchor but have different or even contradictory core conclusions.

[0041] Subsequently, based on the semantic similarity scores between the recalled text and the anchor points, all recalled texts were divided into three noise levels. The first noise level consisted of high-similarity texts with semantic similarity scores above a first preset threshold, primarily used to provide supplementary background support and representing low-noise positive content. The second noise level consisted of medium-similarity texts with similarity scores between the first and second preset thresholds, whose content was partially related to the core topic but could not directly answer the question, and was used to introduce moderately distracting information. The third noise level consisted of low-similarity texts with similarity scores below the second preset threshold, belonging to the same sub-domain but with different themes, representing cross-topic high-noise distracting content. Adversarial negative samples were mainly selected from the second and third noise levels.

[0042] Next, the recalled texts with different noise levels are merged with the original candidate reference content according to a preset dynamic ratio. This ratio is dynamically adjusted with subsequent training stages. The proportion of negative samples is lower in the supervised fine-tuning stage to ensure alignment of basic capabilities, and the proportion of negative samples is higher in the reinforcement learning stage to increase the training difficulty. Finally, a complex question-answering context containing redundant information and multi-source context is constructed.

[0043] Based on this, a two-stage prompting strategy is adopted to synthesize standardized question-answering data. In the first stage, the model is guided to generate questions that fit the questioning habits of users in the domain, using only complex context as input. In the second stage, the questions are paired with complex context, guiding the model to generate professional answers based on correct knowledge. Finally, all question-answering samples are stratified and divided according to the subdomain classification of the vertical domain, and split into training and validation sets according to the proportion. This forms a multidimensional benchmark dataset covering various data types such as traditional question answering, thought chain question answering, and context reference question answering. The multidimensional benchmark dataset is a comprehensive training dataset covering multiple question answering paradigms, multiple difficulty levels, and multiple knowledge subcategories. It can simultaneously support the model's basic question answering ability alignment and advanced training of complex reasoning ability.

[0044] Furthermore, in the method provided in the application embodiment, the candidate reference content is used as a reference context, and the search distribution is reorganized and enhanced based on the vector index to construct the multidimensional benchmark dataset. This includes: using the candidate reference content as a reference context, synthesizing question-and-answer data through a two-stage prompting strategy; using the candidate reference content used in synthesizing the question-and-answer as a semantic anchor point, and performing retrieval distribution enhancement processing based on the vector index, wherein multiple semantically related text blocks are recalled from the full knowledge base through vector similarity retrieval, including semantically positive sample text blocks and adversarial negative sample text blocks that overlap with the anchor point but contradict the conclusion; the recalled multi-source text blocks are divided into different noise levels according to semantic similarity and merged according to a preset ratio to construct a complex question-and-answer sample containing redundant information and multi-source context; and the complex question-and-answer sample is multi-dimensionally labeled and hierarchically divided according to the knowledge structure of the vertical domain to obtain the multidimensional benchmark dataset.

[0045] Specifically, the first step involves using the selected candidate reference content as the reference context. Standardized question-and-answer data is synthesized through a two-stage prompting strategy. This two-stage prompting strategy is a prompting engineering method that controls the generation quality step by step. By breaking down question-and-answer generation into two independent stages—question generation and answer generation—it avoids the problems of insufficient relevance between questions and reference content and answers deviating from the knowledge basis that occur in single-stage synchronous generation. The first stage is the question generation stage, where a single candidate reference content is input into the large language model as the sole context. This is paired with restrictive prompting instructions that generate vertical domain questions based solely on the reference content, conforming to the questioning habits of domain users, without containing answers, and covering core knowledge points. This guides the model to generate 1 to 3 domain knowledge questions that are highly bound to the reference content and fit the production practice scenario. The second stage is the answer generation stage, where the same candidate reference content is paired with a single question generated in the first stage and input into the model. This is paired with instructions that strictly adhere to the reference context, prohibit the introduction of external information or fabricated knowledge, and conform to domain professional standards. This guides the model to generate professional standard answers anchored to the reference content, ensuring that the knowledge source of the question-and-answer pair is completely traceable and without any additional illusions. Each set of questions and corresponding standard answers together constitutes a basic question-and-answer sample.

[0046] After synthesizing basic question-answer samples, the candidate reference content upon which the question-answer pair is based is used as the semantic anchor. Retrieval distribution enhancement processing is then performed based on the constructed vector index. The semantic anchor serves as the core knowledge text that forms the retrieval benchmark and is the semantic reference standard for all recalled texts. Subsequent similarity calculations and relevance determinations are based on the semantic vector of this text. Retrieval distribution enhancement is a data augmentation method that expands the contextual complexity of question-answer samples and simulates the information-mixed environment of real-world retrieval scenarios. This allows the model to learn the ability to locate core knowledge from redundant and interfering information during the training phase. Specifically, the semantic vector corresponding to the semantic anchor is first extracted. Batch similarity retrieval is performed across the entire knowledge base using vector indexing, calculating the vector for each text block in the knowledge base one by one. The cosine similarity with the anchor vector is calculated using the following formula: ; The value ranges from [-1, 1]. The higher the value, the stronger the semantic relevance between the two texts. Based on the similarity results, a preset number of semantically related text blocks are recalled, which include two categories: semantically positive sample text blocks and adversarial negative sample text blocks. Semantically positive sample text blocks refer to texts that are consistent with the core conclusion of the anchor point, highly related to the topic, and can supplement and corroborate the core knowledge points and expand the background. They are positive knowledge supplements to the question and answer. Adversarial negative sample text blocks refer to texts that have the same domain entities as the anchor point, have a moderate degree of topic relevance, but have different or even contradictory core conclusions. These samples share domain entities and some background information, but the knowledge expression is biased. They can simulate the complex scenarios of homologous and heteronymous knowledge conflicts in real retrieval, forcing the model to improve its knowledge identification and attribution judgment capabilities.

[0047] After recall, all recalled text blocks are divided into three noise levels based on their semantic similarity to the anchor. The first noise level consists of text with semantic similarity higher than the first preset threshold, which is low-noise positive content and is mainly used to provide richer context and supporting information for the question. The second noise level consists of text with semantic similarity between the first and second preset thresholds, which is medium-noise interference content. The content is related to the core topic but cannot directly answer the question and is used to introduce moderate irrelevant information interference. The third noise level consists of text with semantic similarity lower than the second preset threshold, which belongs to the same subdomain as the anchor but has a different research topic, and is high-noise cross-topic interference content. Adversarial negative samples are mainly selected from the second and third noise levels.

[0048] Subsequently, the recalled texts with different noise levels are merged and spliced ​​with the original candidate reference content according to a preset dynamic ratio to construct a complex question-answering sample containing redundant information and multi-source context. This dynamic ratio is asymmetrically configured with subsequent training stages. In the supervised fine-tuning stage, a first ratio with a low proportion of negative samples is used to ensure that the model first masters the basic question-answering and knowledge reference logic. In the reinforcement learning stage, a second ratio with a high proportion of negative samples is used to gradually increase the training difficulty to match the progression of the model's capabilities. The merged complex context is bound to the original question-answering pair to form a question-answering sample with multi-source reference context.

[0049] After constructing all complex question-and-answer samples, multi-dimensional annotation and hierarchical classification are carried out on all samples according to the knowledge structure system of the vertical domain. Multi-dimensional annotation refers to labeling each sample with multi-dimensional attribute tags, including knowledge sub-domain tags, which indicate the vertical sub-domain to which the sample belongs, such as crop cultivation, pest and disease diagnosis, soil and fertilizer, and industrial policy, to support subsequent stratified sampling; question-and-answer paradigm tags, which indicate whether the sample belongs to traditional question-and-answer, thought chain question-and-answer, or question-and-answer with reference context, corresponding to different training objectives; difficulty level tags, which indicate the difficulty level of the sample according to the level of context noise and knowledge depth, to match the training rhythm of course learning; and knowledge attribution tags, which indicate the original text position and core conclusion corresponding to the correct knowledge in the sample, to provide the annotation basis for the subsequent calculation of the reward function.

[0050] After annotation, a stratified sampling method is used, with knowledge subdomains as the stratification dimension, to split all samples into training and validation sets according to a preset ratio. This ensures that the sample distribution of the training and validation sets is consistent across each subdomain, avoiding evaluation bias caused by data distribution shift. Ultimately, a multidimensional benchmark dataset covering multiple question-answering paradigms, multiple knowledge subdomains, and multiple difficulty levels is formed. A multidimensional benchmark dataset refers to a comprehensive training dataset that simultaneously possesses data type dimension, knowledge domain dimension, and difficulty level dimension.

[0051] Furthermore, in the method provided in the application embodiment, the two-stage prompting strategy includes: in the first stage, based on the reference context, the input large language model generates a vertical domain question closely related to the reference content; in the second stage, the instruction large language model generates a professional answer based on the reference context and the vertical domain question generated in the first stage.

[0052] Specifically, the two-stage prompting strategy breaks down the complete question-and-answer generation task into two independent and progressively advancing stages: question generation and answer generation. This achieves a strong binding between the question-and-answer pair and the reference knowledge, controlling the risk of knowledge illusion from the source of data synthesis. Specifically, the first stage generates vertical domain questions closely related to the reference content based on the large language model input from the reference context. The reference context refers to semantically complete and knowledge-concentration-compliant professional text units obtained from the vertical domain knowledge base through information density filtering; it is the sole knowledge basis for the entire question-and-answer pair generation process, without introducing any external knowledge. This stage first constructs structured prompt words, using the complete reference context text as the core information block of the prompt, coupled with multi-dimensional constraint instructions to clarify the generation rules. The model is required to generate only 1 to 2 questions that fit the domain scenario based solely on the reference content, and the rules must cover questions closely related to the domain. The process involves extracting core knowledge points from the reference text, ensuring the wording aligns with the actual questioning habits of practitioners in the field, prohibiting pre-set answers or expansion of information beyond the reference content, and outputting only the question text without additional explanations. After constructing a large language model using the input prompts as a data synthesis tool, the model generates question text token by token through an autoregressive generation mechanism. Upon completion, compliance checks are performed, verifying through keyword matching and semantic comparison that all core information of the question can be found in the reference context. Invalid questions that are out of scope, generalized, or deviate from the knowledge points are eliminated, ultimately resulting in vertical domain questions closely related to the reference content. These vertical domain questions refer to questions that fit real-world application scenarios in the corresponding industry, such as production, research, and management, use standardized professional terminology, and focus on specific sub-knowledge points, ensuring the adaptability of subsequent training data to real-world application scenarios.

[0053] The second stage then begins, where the instruction-based language model generates professional answers based on the reference context and the vertical domain questions generated in the first stage. This stage pairs the original reference context with the vertical domain questions generated in the first stage to construct new prompt inputs. The prompt words simultaneously contain the reference context text and the corresponding question. Strongly constrained instructions clarify the answer generation rules, requiring the model to strictly adhere to the expression in the reference context. All conclusions and knowledge points must have corresponding textual evidence; external knowledge from the model's own parameters cannot be used, and fabricated data or conclusions are prohibited. Furthermore, the answers must be logically coherent, use accurate professional terminology, and conform to the expression norms of the vertical domain. After the prompts are input into the model, the model uses the reference context as the knowledge boundary and generates a complete answer word by word through autoregression. After generation, knowledge attribution verification is performed. The knowledge fit of the answer is quantified by calculating the entity matching rate and conclusion consistency score between the answer and the reference context. The conclusion consistency score is calculated using textual entailment judgment logic to determine the logical fit between the core argument of the answer and the reference content. The calculation formula is as follows: ; in The number of core arguments that are consistent with the arguments in the reference content. The total number of core arguments contained in the answer is the threshold. Answers with scores below the preset threshold will be judged as unqualified and eliminated to ensure that the final generated answer is fully anchored to the reference content and without introducing any additional illusions.

[0054] Furthermore, the method provided in the application embodiment retrieves multiple semantically related text blocks from the full knowledge base through vector similarity retrieval, including: extracting a core entity set based on semantic anchors, and retrieving candidate text blocks from the full knowledge base that contain at least one entity from the core entity set; calculating the semantic similarity and conclusion consistency index between each candidate text block and the text block corresponding to the semantic anchor; selecting candidate text blocks with semantic similarity between a first preset similarity threshold and a second preset similarity threshold, and with a conclusion consistency index lower than a third preset threshold, as adversarial negative samples, wherein the first preset similarity threshold is greater than the second preset similarity threshold; and merging the adversarial negative samples and semantic positive samples into the reference context after mixing them according to a preset dynamic ratio, wherein the dynamic ratio changes with the training stage, the ratio of positive to negative samples is a first ratio in the supervised fine-tuning stage, and a second ratio in the reinforcement learning stage, wherein the proportion of negative samples in the second ratio is higher than the first ratio.

[0055] Specifically, the core entity set is first extracted based on the candidate reference text, which serves as the retrieval benchmark. The core entity set refers to the set of noun entities with domain-specific identifiers identified from the semantic anchor text, including core knowledge carriers such as crop varieties, pest and disease names, gene loci, pesticide components, and technical methods within the vertical domain. Specifically, the anchor text is extracted by adapting to the named entity recognition model of the vertical domain. After homonymous entity unification, abbreviation-full name mapping, and deduplication, a structured entity list is formed.

[0056] Then, a coarse recall is performed, which matches and recalls candidate text blocks containing at least one entity from the core entity set from the full knowledge base. The candidate text blocks are potentially relevant text units that have been filtered out by preliminary entity matching. This step filters out irrelevant texts that have no domain relevance through entity matching, compressing the search scope from the full database to less than a hundred.

[0057] Based on this, two core evaluation metrics, semantic similarity and conclusion consistency, are calculated for each candidate text block. Semantic similarity is used to quantify the degree of correlation between the candidate text and the semantic anchor in terms of overall semantic connotation. During the calculation, the pre-trained semantic vectors corresponding to the candidate text block and the semantic anchor are retrieved respectively. and The similarity score is calculated using the cosine similarity formula, which is: ; The value ranges from [-1, 1], and the higher the value, the stronger the correlation between the two overall semantic topics.

[0058] The conclusion consistency index is used to quantify the degree of fit between candidate texts and semantic anchors in terms of core conclusions and viewpoints. Unlike the semantic similarity that measures the overall theme association, it focuses on the core arguments and knowledge orientation of the text. In the specific calculation, the conclusion sentences of candidate texts and anchor texts are first extracted by the domain model after fine-tuning the instructions. Then, the logical relationship between the two is judged based on the text implication model. Contradictory relationships are assigned a low score, and consistent relationships are assigned a high score. Finally, the conclusion consistency index is normalized to obtain a conclusion consistency index in the range of 0 to 1. The lower the value, the higher the degree of contradiction between the core conclusions of the two.

[0059] After calculating the two metrics, adversarial negative samples are screened. Adversarial negative samples refer to interfering texts that are related to the anchor topic and share domain entities, but whose conclusions deviate from or even contradict the correct knowledge of the anchor. These are training materials used to train the model's knowledge discrimination ability and simulate real-world retrieval interference scenarios. The screening rules are set as a combination of two-level similarity thresholds and a single-level consistency threshold. The first preset similarity threshold is a higher similarity threshold, and the second preset similarity threshold is a lower similarity threshold. Furthermore, the first preset similarity threshold is greater than the second preset similarity threshold, and only texts with semantic similarity values ​​between the two are retained. The candidate texts are selected from those that are moderately related to the anchor topic and have some semantic connection but are not highly consistent. This avoids the problem that purely positive samples with extremely high similarity cannot form interference, while completely unrelated samples with extremely low similarity are too easy to identify. At the same time, the conclusion consistency index of the candidate texts is required to be lower than the third preset threshold, that is, the conclusions are significantly different from the anchor. Candidate texts that meet both conditions are judged as adversarial negative samples. By constructing samples with moderate similarity and low conclusion consistency, the typical interference of topic-related but knowledge-incorrect in retrieval is simulated to the greatest extent, making it difficult for the model to judge the correctness of knowledge based solely on topic relevance.

[0060] After screening, adversarial negative samples and semantic positive samples are mixed in a preset dynamic ratio and merged into the original reference context. Semantic positive samples refer to texts with semantic similarity higher than the first preset similarity threshold and a conclusion consistency index close to 1, used to provide supplementary evidence and background expansion for knowledge points; they are considered positive knowledge supplementation content. The dynamic ratio means that the mixing ratio of positive and negative samples will be dynamically adjusted according to different training stages, not a fixed ratio throughout. Specifically, the first ratio is used in the first stage of supervised fine-tuning training, where the proportion of adversarial negative samples is low and the proportion of semantic positive samples is high. At this time, the model is in the basic alignment stage between domain knowledge and question-answering paradigm. Low interference intensity ensures that the model first masters the basic knowledge reference logic and correct knowledge paradigm, avoiding premature introduction of strong interference that could lead to failure of basic alignment. In the second stage of reinforcement learning training, a second ratio is adopted, under which the proportion of adversarial negative samples is significantly higher than that of the first ratio. At this point, the model already has basic question answering capabilities. Higher interference intensity can force the model to improve its ability to identify multi-source information and attribute knowledge, matching the learning pace of the model's advanced capabilities. During mixing, a corresponding number of positive and negative sample texts are randomly selected according to the ratio and spliced ​​with the original candidate reference content in a random order to form a complete multi-source context, ultimately forming a complex question answering context sample containing correct knowledge, supplementary background, and interference information.

[0061] Furthermore, in the method provided in the application embodiment, the adversarial negative samples and semantic positive samples are mixed and merged according to a preset dynamic ratio, including: dividing the recalled text blocks into multiple noise levels according to their semantic similarity to the text blocks corresponding to the semantic anchor, wherein the first noise level consists of text blocks with semantic similarity higher than a first similarity threshold, providing contextual background support; the second noise level consists of text blocks with semantic similarity between the first and second similarity thresholds, introducing some relevant but not directly answering interference information; the third noise level consists of text blocks belonging to the same vertical subdomain as the semantic anchor text block but with different themes; the adversarial negative samples are obtained by screening from the second and third noise levels; and the text blocks of each noise level are merged according to a preset dynamic ratio, wherein the dynamic ratio is asymmetrically configured in the supervised fine-tuning stage and the reinforcement learning stage.

[0062] Specifically, all recalled text blocks are first divided into multiple noise levels based on their semantic similarity to the text blocks corresponding to the semantic anchors and their domain affiliation. These noise levels are a text grading system based on the semantic affinity between the text and the core knowledge anchors and the intensity of interference with the question-answering task. Different levels correspond to different relevance in real-world retrieval scenarios, used to construct a gradient-based interference training environment to avoid insufficient model generalization due to a single interference intensity. The core quantification basis for level division is semantic similarity, specifically the semantic affinity between the candidate text vector and the anchor text vector calculated using cosine similarity. The calculation formula is: ; in For the semantic vector of candidate text blocks, The semantic vector of the semantic anchor point ranges from -1 to 1, with higher values ​​indicating a higher degree of semantic relevance between the two. Specifically, it is divided into three distinct levels. The first noise level consists of text blocks with semantic similarity higher than the first similarity threshold, which is the critical value distinguishing between highly relevant and moderately relevant texts. Texts at this level highly match the core theme of the semantic anchor point, and their conclusions are largely consistent. They are mainly used to provide contextual support, supplement application scenarios for knowledge points, corroborating data, and extended explanations; they are low-noise positive supplementary content. The second noise level consists of text blocks with semantic similarity between the first and second similarity thresholds, where the second similarity threshold is the critical value distinguishing between moderately relevant and weakly relevant texts. The first level of noise consists of text blocks that have a similarity value below the first similarity threshold. These text blocks share some thematic connections and domain entities with the core question, but their discussion direction deviates from the core answer and cannot be directly used to answer the target question. They can introduce some relevant but non-directly answering interference information and belong to medium-noise interference content. Because they have a certain degree of thematic deception, they are the core source of adversarial negative samples. The third noise level consists of text blocks that belong to the same vertical subdomain as the semantic anchor text block but have different core research themes. Their semantic similarity is generally lower than the second similarity threshold. For example, they belong to the same subdomain of plant protection but discuss different disease control schemes. They belong to cross-topic high-noise interference content and are also an important source of adversarial negative samples.

[0063] After the hierarchical division is completed, adversarial negative samples are selected from the second and third noise levels. During the selection, the consistency index of conclusions is combined to select texts that are related in topic but whose core conclusions contradict the correct knowledge of the anchor point as adversarial samples. The first noise level is not included in the selection scope of adversarial negative samples because the text conclusions are highly consistent with the anchor point and have no interference value.

[0064] Subsequently, text blocks from each noise level are merged according to a preset dynamic ratio. The dynamic ratio refers to the proportion of texts from different noise levels in the final multi-source context, which is adjusted differently according to the capability objectives of subsequent training stages. The dynamic ratio is asymmetrically configured in the supervised fine-tuning stage and the reinforcement learning stage. The asymmetrical configuration means that the proportion of each level and the proportion of positive and negative samples in the two training stages are designed to match the core training objectives of different stages, forming a learning rhythm from easy to difficult. Specifically, in the first stage of supervised fine-tuning training, the first ratio adapted to the basic alignment objective is adopted. Under this ratio, the proportion of positive supplementary texts of the first noise level is the highest, usually accounting for 70% to 80% of all recalled supplementary texts, while the proportion of interference texts and adversarial negative samples of the second and third noise levels is only 20% to 30%. The core logic of this configuration is that the goal of the supervised fine-tuning stage is to enable the model to complete the basic alignment of domain knowledge and question-answering paradigm, establish the correct knowledge reference logic, and the lower interference intensity can avoid the model being misled by too much erroneous information before the basic capabilities are formed, ensuring the stability of training and the basic alignment effect.

[0065] In the second stage of reinforcement learning training, the second ratio of the adaptive capability advancement target is adopted. Under this ratio, the proportion of positive text in the first noise level is reduced to 40% to 50%, while the proportion of interference text and adversarial negative samples in the second and third noise levels is increased to 50% to 60%. The proportion of negative samples is significantly higher than in the supervised fine-tuning stage. This configuration is because the model already has basic domain question answering and knowledge referencing capabilities in the reinforcement learning stage. Higher interference intensity can force the model to improve its ability to identify multi-source information, judge knowledge attribution, and resist interference, matching the training needs of the model's capability advancement. After the ratio is determined, text blocks are randomly extracted from each noise level according to the corresponding ratio and randomly concatenated with the original semantic anchors, i.e., candidate reference content, while controlling the total length within a reasonable range of the model context window. Finally, a complex multi-source context sample containing correct knowledge, supplementary background, and gradient interference information is constructed.

[0066] Furthermore, in the method provided in the application embodiment, the initial stage of supervised fine-tuning of the large language foundation model is performed using the multidimensional benchmark dataset, including: injecting trainable low-rank decomposition matrices into the attention mechanism layer and feedforward neural network layer of the selected large language foundation model, and freezing the original weight parameters; using the multidimensional benchmark dataset, optimizing the low-rank matrix parameters of the large language foundation model with injected trainable low-rank decomposition matrices in an autoregressive language modeling manner by minimizing cross-entropy, wherein the model training comprehensively adopts the evaluation of basic language modeling ability, knowledge reference alignment ability, and format specification constraints, calculates the difference between the predicted output and the actual target, and uses the comprehensive difference results to guide the gradient update of the low-rank matrix parameters to achieve target alignment; wherein, the multidimensional benchmark dataset includes traditional question answering data, thought chain question answering data, and context reference question answering data, and the context reference question answering data includes multi-source reference context composed of a mixture of semantic positive samples and adversarial negative samples.

[0067] Specifically, firstly, trainable low-rank factorization matrices are injected into the attention mechanism layer and feedforward neural network layer of the selected Transformer decoder architecture large language foundation model. The large language foundation model has a core structure of alternating stacked multi-head self-attention layers and feedforward neural network layers. The attention mechanism layer calculates the semantic dependencies of words within a sequence through three sets of linear projections of query, key, and value, capturing contextual information. It is the core structure for the model to understand the semantics of long texts. The feedforward neural network layer completes high-order mapping and transformation of features through two layers of linear transformation and the GELU non-linear activation function, and is responsible for further processing the semantic features output by the attention layer. The weight matrices of both carry all the general language capabilities learned by the model during the pre-training stage of large-scale general corpus. The low-rank factorization matrix is ​​a lightweight trainable parameter structure designed based on the principle of low-rank adaptation. It only needs to simulate the weight update amount by multiplying two low-dimensional matrices. The linear layer weight matrix is ​​injected with dimension . The downlink projection matrix and dimension are The uplink projection matrix is ​​given by , where the rank r is a hyperparameter much smaller than the matrix dimensions d and k, typically taking values ​​of 8, 16, or 32. During matrix initialization, the downlink matrix A is initialized with Gaussian randomization, and the uplink matrix B is initialized to all zeros. This ensures that the output of the low-rank branch is zero at the initial training time, preventing disruption of the base model's pre-trained language capabilities. After injecting the low-rank decomposition matrix, the forward propagation calculation formula for the linear layer is updated to . Where x is the input feature vector of the linear layer and h is the output feature vector of the linear layer, it is equivalent to updating the original weight increment to This is accompanied by freezing the original weight parameters, which means fixing the values ​​of the original weight matrix W of the base model throughout the training process, not including it in the gradient calculation and parameter update scope, and only using the low-rank decomposition matrices A and B as trainable parameters.

[0068] Subsequently, a multidimensional benchmark dataset was constructed as the input for all training. This dataset includes three core sample types: traditional question-answering data, thought chain question-answering data, and context reference question-answering data. Among them, the context reference question-answering data contains multi-source reference contexts composed of a mixture of semantically positive samples and adversarial negative samples, allowing the model to be exposed to mixed information scenarios in the basic training stage. All samples were constructed according to the autoregressive language modeling paradigm. Autoregressive language modeling is the basic training paradigm for large language model generation tasks. The model predicts the probability distribution of the next word token by token based on the existing preceding text sequence, and generates complete output text by word-by-word recursion. In specific construction, the instructions and reference context of each sample are concatenated into an input prompt sequence, and the standard answer is concatenated into a target output sequence. After being converted into an ID sequence that the model can recognize by the model by a word segmenter, a causal attention mask is used to ensure that each position can only see the tokens before the current position during the prediction process, and cannot see the subsequent content. This fully fits the logic of real generation scenarios. During training, the loss is calculated only for the tokens of the target output sequence. The input prompt sequence does not participate in the loss calculation, thereby guiding the model to learn the mapping rules from domain questions to professional answers.

[0069] The training aims to minimize the cross-entropy loss function, which is a core quantitative indicator measuring the difference between the model's predicted token probability distribution and the actual standard answer token distribution. Essentially, it calculates the information difference between the model's predicted distribution and the true distribution, and its formula is as follows: ; Where x is the token sequence corresponding to the input prompt sequence. For the i-th token in the target output sequence, For all tokens preceding the i-th token in the target sequence, The model predicts the probability of the i-th token based on the input and the preceding tokens. N is the total number of tokens in the target output sequence. The smaller the loss value, the higher the overlap between the content generated by the model and the standard answer, and the smaller the semantic deviation. During the training process, batch data is input into the model to complete the forward propagation calculation, obtain the token prediction probability distribution at each position, and then calculate the cross-entropy loss value.

[0070] Subsequently, the gradient of the loss relative to the trainable parameters is calculated layer by layer using the backpropagation algorithm based on the chain rule. Since the original weights are frozen, the gradient is only propagated backward along the branch of the low-rank decomposition matrix and only acts on matrices A and B. The optimizer adopts the AdamW adaptive optimizer, which, together with the learning rate warm-up, cosine decay strategy and gradient clipping mechanism, updates the parameters of the low-rank matrix to avoid gradient explosion and training oscillation.

[0071] During the training iteration process, a comprehensive evaluation is conducted simultaneously from three dimensions. The results of these multiple dimensions jointly guide the parameter update rhythm and the determination of training termination. The first is the assessment of basic language modeling capability, which quantifies the language generation fluency of the model through the perplexity index. Perplexity is directly related to cross-entropy loss, and the calculation formula is as follows: ; The lower the numerical value, the stronger the model's language modeling ability and the more the generated text conforms to the laws of natural language. This is used to monitor whether the model's basic language ability has deteriorated. The second is the knowledge reference alignment ability assessment. By calculating the entity matching rate and conclusion consistency score between the model's generated answer and the reference context, the degree to which the answer fits the reference knowledge is quantified. This is used to judge whether the model has a tendency to fabricate knowledge outside the reference content. This is the core evaluation dimension for constraining the accuracy of domain knowledge. The third is the format specification constraint assessment. By statistically analyzing indicators such as the accuracy of professional terminology and the compliance rate of output structure, the model output is verified to ensure that it conforms to the professional expression specifications and format requirements of the vertical domain. The evaluation results of the three dimensions are combined to form the basis for judging the model quality. When the cross-entropy loss no longer decreases for several consecutive rounds on the validation set and all three evaluation indicators reach the preset threshold, the training is considered to have converged, the parameter update is stopped, and finally the cold start model parameters that have completed the initial adaptation to the domain are obtained.

[0072] Furthermore, in the method provided in the application embodiment, the step of generating multiple candidate answer group sample sets for the same input question, quantifying each answer using a multidimensional reward function, calculating the relative advantage value of each answer within the group, and performing a two-stage pruning update of model parameters to optimize the large language model parameters includes: sampling and generating G candidate answers for the same input question to form a group sample set, where G is a preset positive integer; scoring each of the G candidate answers using the multidimensional reward function to obtain a reward value for each candidate answer; performing group standardization based on the average and standard deviation of the reward values ​​of each candidate answer within the group, and calculating the relative advantage value of each candidate answer through group standardization; calculating the ratio of the probability of the current model generating each candidate answer to the probability of the previous model generating each candidate answer based on the relative advantage value, obtaining a probability ratio, and pruning the probability ratio to a preset interval; using the pruned probability ratio and the KL divergence penalty term to jointly constrain the policy update, iteratively optimizing the model parameters, wherein the KL divergence penalty term is used to constrain the deviation between the current policy and the policy corresponding to the cold start model parameters.

[0073] Specifically, for the same input problem bound to multiple source contexts, the randomness and diversity of the output are controlled by setting differentiated sampling temperatures, kernel sampling thresholds, and other hyperparameters. This allows the current model to sample and generate G candidate answers with different expressions, inference paths, and information selection, which together constitute the intra-group sample set. The intra-group sample set is a collection of multiple output samples generated by the same model under the same problem and context. It is the core basic unit for group relative strategy optimization. The quality of the answer is determined by the horizontal relative comparison of samples within the same group. Here, G is a preset positive integer number of intra-group samples, usually set to 4, 8, or 16. The larger the sample size, the stronger the relative evaluation stability, but the corresponding generation computing power overhead is also higher. It needs to be selected in a balanced way according to the actual computing power conditions.

[0074] Subsequently, a pre-defined multidimensional reward function is used to quantitatively score each of the G candidate answers within the group, obtaining the total reward value for each answer. The multidimensional reward function is a comprehensive quantitative evaluation system designed from multiple core quality dimensions across vertical domains. It includes three core evaluation dimensions: knowledge attribution accuracy, domain logical coherence, and output format standardization. The final total reward value is obtained by weighted summation of each dimension, and the corresponding calculation formula is as follows: ; in The total reward value for the i-th candidate answer. The weight coefficients for each dimension and satisfying The weighting can be adjusted according to the core needs of the vertical field. Specifically, the knowledge attribution accuracy reward... Calculated by comparing the entity matching rate and core conclusion consistency between the answer content and the correct knowledge fragments in the context, this measure is used to evaluate the model's ability to identify adversarial negative samples and anchor correct knowledge. It is the core reward for constraining domain knowledge illusion and is a reward for domain logical coherence. The evaluation criteria, including the completeness of the reasoning chain, the consistency of causal logic, and the fluency of professional semantics, are used to enhance the model's reasoning ability for complex problems, and to output a standardized format reward. This is calculated by statistically analyzing the accuracy of professional terminology in the responses and the compliance rate of the output structure, ensuring that the output conforms to the professional expression standards of the vertical field.

[0075] After obtaining the total reward value of all G candidate answers within the group, the relative advantage value of each candidate answer is calculated through in-group standardization. The relative advantage value is a standardized indicator that measures the relative superiority or inferiority of a single answer within the same sample group. Its core logic is to remove the scaling influence of the absolute value of the reward and use only the relative performance within the same group as the optimization basis, without relying on the absolute value judgment given by the external value model. In the specific calculation, the mean and standard deviation of the reward values ​​of the G samples in the same group are first calculated. The formula for calculating the mean is: ; The standard deviation represents the average reward level of the same sample group, and is calculated using the following formula: ; This represents the dispersion of reward values ​​among samples in the same group, and the relative advantage value of the i-th answer is then obtained through a standardization formula: ; in The constant term is extremely small and is used to avoid division-to-zero anomalies when the standard deviation is 0. After standardization, the relative advantage value can be positive or negative. A positive value means that the answer is better than the average level in the group, and the higher the value, the better the performance. A negative value means that the answer is worse than the average level in the group, and the lower the value, the worse the performance. This design allows the model to judge the quality of the output by simply comparing it with the self-generated results in the same group. From the architecture, it eliminates the pain points of high training cost and easy propagation of bias in traditional reinforcement learning value models.

[0076] After calculating the relative advantage value, the probability ratio for strategy updates is calculated and pruned. First, the probability ratio is calculated as the ratio of the probability of the current strategy to be updated generating the i-th candidate answer to the probability of the old strategy from the previous iteration generating that answer. The specific calculation formula is as follows: ; Where x represents the input question and multi-source context. For the i-th candidate answer, These are the trainable parameters that need to be updated. The parameters from the previous iteration, probability ratio This reflects the change in the probability of generating the same answer before and after the strategy update. A ratio closer to 1 indicates a smaller strategy change, while a ratio further away from 1 indicates a larger policy update magnitude. To prevent excessively large parameter updates in a single iteration from causing policy collapse and a precipitous drop in output quality, a pruning constraint is applied to the probability ratio, limiting it to a certain value. Within the preset range, The clipping factor is denoted as , and the corresponding clipping operation is represented as . .

[0077] Based on this, the final policy loss function is constructed, which consists of the pruned policy optimization objective term and the KL divergence penalty term, and the calculation formula is as follows: ; The first term in the formula represents the optimization objective of the pruned strategy. By taking the smaller value between the probability ratio and the pruned ratio, the maximum update reward is limited when the relative advantage is positive, and excessive penalty is limited when the relative advantage is negative, thus avoiding drastic fluctuations in strategy updates. The second term is the KL divergence penalty term. The penalty coefficient is... The KL divergence between the current strategy and the cold start reference strategy is used to measure the degree of difference in their output probability distributions. It constrains the model from deviating excessively from the initial basic language capabilities and domain question-answering paradigm. The cold start reference strategy corresponds to the model parameters after the first stage of supervised fine-tuning, and remains fixed throughout as a capability benchmark.

[0078] During parameter updates, the gradient of the loss is backpropagated only to the low-rank decomposition matrix injected into the model, continuing the efficient parameter training characteristic of the first stage. Here, the low-rank decomposition matrix refers to the low-rank fitting parameter structure injected into the attention mechanism layer and the feedforward neural network layer, which consists of two low-dimensional matrices that are products of each other. and The original weight parameters of the base model are kept frozen throughout the training process, and only the parameters of the low-rank matrix participate in gradient updates. This ensures training effectiveness while keeping memory usage and computing power consumption at a very low level. The training uses the AdamW adaptive optimizer to iteratively update the parameters of the low-rank matrix, and with gradient pruning and learning rate decay strategies, the overall question-answering quality of the model in complex scenarios is gradually improved.

[0079] Furthermore, in the method provided in the application embodiment, after each round of reinforcement learning iteration, the improvement of the attribution consistency reward value of the model on the validation set is calculated; when the improvement is higher than a preset high improvement threshold, a difficulty improvement operation is triggered, wherein the noise injection ratio and the adversarial negative sample sampling ratio in the retrieval distribution enhancement processing are increased; when the improvement is lower than a preset low improvement threshold, the current difficulty configuration is kept unchanged or the difficulty parameter is reduced.

[0080] Specifically, after each round of reinforcement learning parameter iteration and update, the improvement in the attribution consistency reward value of the model on the validation set is first calculated. The attribution consistency reward value is a quantitative indicator measuring the accuracy of the model's knowledge attribution. It is used to determine whether the core assertions in the model's generated answers are anchored to correct knowledge in the reference context and whether erroneous conclusions in adversarial negative samples are accepted. It is a core dimension reflecting the model's anti-interference ability and illusion suppression level. The calculation process first fixes the model parameters after the current iteration update and performs inference generation on a fixed-division validation set. The validation set consists of standardized multi-source contextual question-and-answer samples covering different noise levels, and each sample is labeled with the range of knowledge fragments that can be used as correct evidence. For a single sample, all core assertions in the model's generated answers are extracted first, and then each assertion is verified one by one by textual entailment judgment to see if it can be supported by correct knowledge fragments. The number of assertions that can be correctly attributed and the total number of assertions are statistically obtained using the formula: ; Calculate the attribution consistency reward value for a single sample, where The number of core arguments that can be supported by correct knowledge. To answer the question about the total number of core arguments included, the value ranges from [0,1]. A higher value indicates a more accurate knowledge attribution of the model and a lower degree of being misled by interfering information. Subsequently, the arithmetic mean of the attribution consistency reward values ​​of all samples in the validation set is taken to obtain the average attribution consistency reward value of the validation set in the current round. Then compare it with the average attribution consistency reward value of the validation set from the previous iteration. The relative difference is calculated to obtain the increase in the attribution consistency reward value. The calculation formula is as follows: ; This value, expressed as a percentage, reflects the rate of improvement in the model's knowledge attribution ability during a single iteration and is a core criterion for judging the suitability of the current training difficulty.

[0081] After calculating the improvement rate, the difficulty adjustment logic is executed based on different scenarios: when the improvement rate exceeds a preset high improvement threshold, a difficulty increase operation is triggered. This high improvement threshold is a pre-set critical value for the rate of progress. When the improvement rate in a single round exceeds this threshold, it indicates that the model has quickly mastered the sample features at the current difficulty level, the current training difficulty is too low, and the marginal benefit of continuing training decreases. At this point, the noise injection ratio and adversarial negative sample sampling ratio in the retrieval distribution augmentation process are increased. Specifically, when constructing the multi-source context of the next round of training samples, the proportion of interference text in the second and third noise levels is increased, the proportion of positive supplementary text in the first noise level is decreased, and the sampling ratio of adversarial negative samples in the interference text is increased, allowing more interference information that contradicts the core knowledge conclusions to be included. The following section discusses increasing the difficulty for the model to identify correct knowledge, thereby driving continuous advancement in the model's capabilities. When the improvement rate falls below a preset low improvement threshold, the current difficulty configuration is maintained or the difficulty parameters are reduced. The low improvement threshold is a pre-set critical value for stagnation. When the improvement rate in a single round falls below this threshold or even shows negative growth, it indicates that the model has reached a bottleneck at the current difficulty level. Continuing to maintain high-intensity training will not yield effective improvement and may even lead to policy degradation. In this case, the current difficulty configuration is maintained to allow the model to continue consolidating its capabilities at the current difficulty level. If the improvement rate is consistently below the threshold for multiple rounds, the noise injection ratio and the proportion of adversarial negative sample sampling are reduced to appropriately lower the training difficulty. This helps the model consolidate its basic capabilities before gradually advancing, avoiding policy collapse during the training process.

[0082] In summary, the two-stage parameter optimization method for large language models in vertical domains provided in this application has the following technical effects: By constructing a multidimensional benchmark dataset that integrates a retrieval distribution enhancement mechanism and multi-level adversarial negative samples, and combining a two-stage parameter optimization architecture that injects low-rank decomposition matrices, the system first achieves a cold start by completing the basic alignment of the domain knowledge system and question-answering paradigm through supervised fine-tuning. Then, it enhances the ability to identify complex information and make logical inferences by optimizing the group-relative strategy of the valueless model. Under the low-computing conditions of compressing the number of trainable parameters to one-thousandth of the original model and not requiring additional training of a large-scale value network, it simultaneously achieves deep injection of domain knowledge and improvement of complex reasoning ability. This effectively reduces the occurrence rate of knowledge illusion in vertical domains and improves the accuracy of knowledge attribution and anti-interference ability of the model in retrieval enhancement scenarios.

[0083] Example 2: like Figure 2 As shown, based on the same inventive concept as the two-stage parameter optimization method for large language models in vertical domains provided in Embodiment 1, this embodiment of the invention also provides a two-stage parameter optimization system for large language models in vertical domains, the system comprising: The filtering module 11 is used to filter basic text based on the vertical domain knowledge base and enhance the retrieval distribution of the filtered basic text to construct a multidimensional benchmark dataset. The supervision module 12 is used to perform initial-stage supervised fine-tuning of the large language base model using the multidimensional benchmark dataset, complete the initial alignment of the large language model with the vertical domain basic knowledge question-answering paradigm and knowledge reference relationship, and obtain cold-start model parameters. The large language base model has been injected with a trainable low-rank factorization matrix and the original weight parameters are frozen. The update module 13 is used to enter the reinforcement learning stage based on the initial model parameters, using samples containing multi-source context references in the multidimensional benchmark dataset as input context. It generates multiple candidate answer group sample sets for the same input question, quantifies each answer using a multidimensional reward function, calculates the relative advantage value of each answer within the group, performs a two-stage pruning update of model parameters, and optimizes the large language model parameters to meet the vertical domain question-answering objectives.

[0084] Furthermore, the filtering module 11 is also used to perform the following steps: performing semantic analysis on the vertical domain knowledge base, extracting semantically complete text blocks, performing semantic vectorization processing on the extracted semantically complete text blocks, and constructing a vector index; performing multi-dimensional information density evaluation on the complete text blocks based on the preset dimensions of vertical domain knowledge, and filtering candidate reference content that exceeds a preset threshold; using the candidate reference content as a reference context, performing search distribution reorganization and enhancement based on the vector index, and constructing the multi-dimensional benchmark dataset.

[0085] Furthermore, the filtering module 11 is also used to perform the following steps: using the candidate reference content as the reference context, synthesizing question-and-answer data through a two-stage prompting strategy; using the candidate reference content used in synthesizing the question-and-answer as the semantic anchor point, performing retrieval distribution enhancement processing based on the vector index, wherein multiple semantically related text blocks are recalled from the full knowledge base through vector similarity retrieval, including semantically positive sample text blocks and adversarial negative sample text blocks that overlap with the anchor point but contradict the conclusion; the recalled multi-source text blocks are divided into different noise levels according to semantic similarity and merged according to a preset ratio to construct a complex question-and-answer sample containing redundant information and multi-source context; according to the knowledge structure of the vertical domain, the complex question-and-answer sample is multi-dimensionally labeled and hierarchically divided to obtain the multi-dimensional benchmark dataset.

[0086] Furthermore, the filtering module 11 is also used to perform the following steps: the two-stage prompting strategy includes: in the first stage, based on the reference context, the input large language model generates a vertical domain question closely related to the reference content; in the second stage, the large language model is instructed to generate a professional answer based on the reference context and the vertical domain question generated in the first stage.

[0087] Furthermore, the filtering module 11 is also used to perform the following steps: extracting a core entity set based on semantic anchors, and recalling candidate text blocks containing at least one entity from the core entity set from the full knowledge base; calculating the semantic similarity and conclusion consistency index between each candidate text block and the text block corresponding to the semantic anchor; filtering candidate text blocks with semantic similarity between a first preset similarity threshold and a second preset similarity threshold, and with a conclusion consistency index lower than a third preset threshold, as adversarial negative samples, wherein the first preset similarity threshold is greater than the second preset similarity threshold; mixing the adversarial negative samples with semantic positive samples according to a preset dynamic ratio and merging them into the reference context, wherein the dynamic ratio changes with the training stage, the ratio of positive to negative samples is a first ratio in the supervised fine-tuning stage, and a second ratio in the reinforcement learning stage, wherein the proportion of negative samples in the second ratio is higher than the first ratio.

[0088] Furthermore, the filtering module 11 is also used to perform the following steps: dividing the recalled text blocks into multiple noise levels according to their semantic similarity to the text blocks corresponding to the semantic anchor, wherein the first noise level consists of text blocks with semantic similarity higher than a first similarity threshold, providing contextual background support; the second noise level consists of text blocks with semantic similarity between the first and second similarity thresholds, introducing some relevant but not directly answering interference information; the third noise level consists of text blocks belonging to the same vertical subdomain as the semantic anchor text block but with different topics; the adversarial negative samples are obtained from the second and third noise levels; and the text blocks of each noise level are merged according to a preset dynamic ratio, wherein the dynamic ratio is asymmetrically configured during the supervised fine-tuning stage and the reinforcement learning stage.

[0089] Furthermore, the supervision module 12 is also used to perform the following steps: injecting a trainable low-rank factorization matrix into the attention mechanism layer and feedforward neural network layer of the selected large language base model, and freezing the original weight parameters; using the multidimensional benchmark dataset, optimizing the low-rank matrix parameters of the large language base model with the injected trainable low-rank factorization matrix in an autoregressive language modeling manner by minimizing cross-entropy, wherein the model training comprehensively adopts the evaluation of basic language modeling ability, knowledge reference alignment ability, and format specification constraints, calculates the difference between the predicted output and the actual target, and guides the gradient update of the low-rank matrix parameters by comprehensively considering the difference results to achieve target alignment; wherein the multidimensional benchmark dataset includes traditional question answering data, thought chain question answering data, and context reference question answering data, and the context reference question answering data includes a multi-source reference context composed of a mixture of semantic positive samples and adversarial negative samples.

[0090] Furthermore, the update module 13 is also used to perform the following steps: sampling and generating G candidate answers for the same input question to form a sample set within the group, where G is a preset positive integer; scoring the G candidate answers using the multidimensional reward function to obtain the reward value of each candidate answer; performing within-group standardization based on the average and standard deviation of the reward values ​​of each candidate answer within the group, and calculating the relative advantage value of each candidate answer through within-group standardization; calculating the ratio of the probability of the current model generating each candidate answer to the probability of the previous model generating each candidate answer based on the relative advantage value, obtaining the probability ratio, and clipping the probability ratio to a preset interval; using the clipped probability ratio and the KL divergence penalty term to jointly constrain the policy update, iteratively optimizing the model parameters, wherein the KL divergence penalty term is used to constrain the deviation between the current policy and the policy corresponding to the cold start model parameters.

[0091] Furthermore, the update module 13 is also used to perform the following steps: after each round of reinforcement learning iteration, calculate the improvement of the attribution consistency reward value of the model on the validation set; when the improvement is higher than a preset high improvement threshold, trigger a difficulty improvement operation, wherein the noise injection ratio and adversarial negative sample sampling ratio in the retrieval distribution enhancement processing are increased; when the improvement is lower than a preset low improvement threshold, keep the current difficulty configuration unchanged or reduce the difficulty parameter.

[0092] Through the foregoing detailed description of the two-stage parameter optimization method for large language models in vertical domains, those skilled in the art can clearly understand the two-stage parameter optimization system for large language models in vertical domains in this embodiment. As the system disclosed in the embodiment corresponds to the method disclosed in the embodiment, the description is relatively simple, and relevant parts can be referred to the method section.

[0093] The above description of the disclosed embodiments enables those skilled in the art to make or use this application. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A two-stage parameter optimization method for large language models in vertical domains, characterized in that, include: Basic text filtering is performed based on a vertical domain knowledge base, and the retrieval distribution of the filtered basic text is enhanced to construct a multidimensional benchmark dataset. The large language base model is subjected to initial supervised fine-tuning using the multidimensional benchmark dataset to achieve the initial alignment of the large language model with the basic knowledge question-answering paradigm and knowledge reference relationship of the vertical domain, and to obtain the cold start model parameters. The large language base model has been injected with a trainable low-rank decomposition matrix and the original weight parameters are frozen. Based on the preliminary model parameters, the samples containing multi-source context references in the multi-dimensional benchmark dataset are used as input contexts to enter the reinforcement learning stage. Multiple candidate answer group sample sets are generated for the same input question. Each answer is quantified using a multi-dimensional reward function. The relative advantage value of each answer is calculated within the group. The model parameters are then pruned and updated in two stages to optimize the large language model parameters in order to meet the vertical domain question answering objectives.

2. The two-stage parameter optimization method for large language models in vertical domains as described in claim 1, characterized in that, Basic text filtering is performed based on a vertical domain knowledge base, and the retrieval distribution of the filtered basic text is enhanced to construct a multidimensional benchmark dataset, including: Semantic analysis is performed on the vertical domain knowledge base to extract semantically complete text blocks. The extracted semantically complete text blocks are then semantically vectorized to construct a vector index. Based on the preset dimensions of vertical domain knowledge, the information density of the complete text block is evaluated in multiple dimensions, and candidate reference content exceeding the preset threshold is selected. Using the candidate reference content as a reference context, the search distribution is reorganized and strengthened based on the vector index to construct the multidimensional benchmark dataset.

3. The two-stage parameter optimization method for large language models in vertical domains as described in claim 2, characterized in that, Using the candidate reference content as a reference context, the search distribution is reorganized and strengthened based on the vector index to construct the multidimensional benchmark dataset, including: Using the candidate reference content as a reference context, question-and-answer data is synthesized through a two-stage prompting strategy; The candidate reference content used in the synthesis of question answering is used as a semantic anchor point. Based on the vector index, the retrieval distribution enhancement process is performed. In this process, multiple semantically related text blocks are recalled from the full knowledge base through vector similarity retrieval. These include semantically positive sample text blocks and adversarial negative sample text blocks that have entity overlap with the anchor point but contradict the conclusion. The recalled multi-source text blocks are divided into different noise levels according to semantic similarity and merged according to a preset ratio to construct a complex question answering sample containing redundant information and multi-source context. Based on the knowledge structure of the vertical domain, the complex question-and-answer samples are labeled and hierarchically divided in multiple dimensions to obtain the multidimensional benchmark dataset.

4. The two-stage parameter optimization method for large language models in vertical domains as described in claim 3, characterized in that, The two-stage prompting strategy includes: in the first stage, based on the reference context, the input large language model generates a vertical domain question closely related to the reference content; in the second stage, the large language model is instructed to generate a professional answer based on the reference context and the vertical domain question generated in the first stage.

5. The two-stage parameter optimization method for large language models in vertical domains as described in claim 3, characterized in that, Multiple semantically related text blocks were retrieved from the full knowledge base using vector similarity retrieval, including: Extract the core entity set based on semantic anchors, and recall candidate text blocks containing at least one entity from the core entity set from the full knowledge base; Calculate the semantic similarity and conclusion consistency index between each candidate text block and the text block corresponding to the semantic anchor point; Based on semantic similarity and conclusion consistency index, candidate text blocks with semantic similarity between a first preset similarity threshold and a second preset similarity threshold, and candidate text blocks with conclusion consistency index lower than a third preset threshold are selected as adversarial negative samples, wherein the first preset similarity threshold is greater than the second preset similarity threshold. The adversarial negative samples and semantic positive samples are mixed and merged into the reference context according to a preset dynamic ratio. The dynamic ratio changes with the training stage. In the supervised fine-tuning stage, the ratio of positive to negative samples is a first ratio, and in the reinforcement learning stage, the ratio of positive to negative samples is a second ratio. In the second ratio, the proportion of negative samples is higher than that in the first ratio.

6. The two-stage parameter optimization method for large language models in vertical domains as described in claim 5, characterized in that, The adversarial negative samples and semantic positive samples are mixed and merged according to a preset dynamic ratio, including: The recalled text blocks are divided into multiple noise levels based on their semantic similarity to the text blocks corresponding to the semantic anchors. The first noise level consists of text blocks with semantic similarity higher than a first similarity threshold, providing contextual support. The second noise level consists of text blocks with semantic similarity between the first and second similarity thresholds, introducing some relevant but non-directly answering interference information. The third noise level consists of text blocks belonging to the same vertical subdomain as the semantic anchor text blocks but with different themes. The adversarial negative samples are obtained by screening from the second noise level and the third noise level; Text blocks at each noise level are merged according to a preset dynamic ratio, wherein the dynamic ratio is asymmetrically configured during the supervised fine-tuning stage and the reinforcement learning stage.

7. The two-stage parameter optimization method for large language models in vertical domains as described in claim 1, characterized in that, Initial-stage supervised fine-tuning of the large language foundation model was performed using the aforementioned multidimensional benchmark dataset, including: Trainable low-rank decomposition matrices are injected into the attention mechanism layer and feedforward neural network layer of the selected large language base model, and the original weight parameters are frozen. Using the aforementioned multidimensional benchmark dataset, a large language pedestal model with an injected trainable low-rank decomposition matrix is ​​used to optimize the low-rank matrix parameters by minimizing cross-entropy in an autoregressive language modeling manner. The model training comprehensively evaluates basic language modeling capabilities, knowledge reference alignment capabilities, and format specification constraints, calculates the difference between the predicted output and the actual target, and uses the combined difference results to guide the gradient update of the low-rank matrix parameters to achieve target alignment. The multidimensional benchmark dataset includes traditional question-answering data, thought chain question-answering data, and context reference question-answering data. The context reference question-answering data includes a multi-source reference context composed of a mixture of semantic positive samples and adversarial negative samples.

8. The two-stage parameter optimization method for large language models in vertical domains as described in claim 1, characterized in that, The process involves generating multiple candidate answer groups for the same input question, quantifying each answer using a multidimensional reward function, calculating the relative advantage value of each answer within the group, and performing a two-stage pruning and update of the model parameters to optimize the large language model parameters. This includes: For the same input question, sample and generate G candidate answers to form a sample set within the group, where G is a preset positive integer; The multidimensional reward function is used to score the G candidate answers respectively to obtain the reward value of each candidate answer; Within-group standardization is performed based on the mean and standard deviation of the reward values ​​of each candidate response within the group, and the relative advantage value of each candidate response is calculated through within-group standardization. Based on the relative advantage value, calculate the ratio of the probability of the current model generating each candidate answer to the probability of the previous model generating each candidate answer, obtain the probability ratio, and then trim the probability ratio to a preset range. The policy update is constrained by the pruned probability ratio and the KL divergence penalty term, and the model parameters are iteratively optimized. The KL divergence penalty term is used to constrain the deviation between the current policy and the policy corresponding to the cold start model parameters.

9. The two-stage parameter optimization method for large language models in vertical domains as described in claim 1, characterized in that, Also includes: After each round of reinforcement learning iteration, calculate the increase in the attribution consistency reward value of the model on the validation set; When the improvement is higher than the preset high improvement threshold, a difficulty improvement operation is triggered, which increases the noise injection ratio and the adversarial negative sample sampling ratio in the retrieval distribution enhancement process. When the improvement is lower than the preset low improvement threshold, the current difficulty configuration remains unchanged or the difficulty parameter is reduced.

10. A two-stage parameter optimization system for large language models in vertical domains, characterized in that, The system is used to implement the two-stage parameter optimization method for vertical domain large language models according to any one of claims 1-9, the system comprising: The filtering module is used to perform basic text filtering based on a vertical domain knowledge base, and to enhance the retrieval distribution of the filtered basic text to build a multidimensional benchmark dataset. The supervision module is used to perform initial supervision and fine-tuning of the large language base model using the multidimensional benchmark dataset, complete the initial alignment of the large language model with the basic knowledge question-answering paradigm and knowledge reference relationship of the vertical domain, and obtain the cold start model parameters. The large language base model has been injected with a trainable low-rank decomposition matrix and the original weight parameters are frozen. The update module is used to enter the reinforcement learning stage based on the preliminary model parameters, using samples containing multi-source context references in the multi-dimensional benchmark dataset as input context. It generates multiple candidate answer group sample sets for the same input question, quantifies each answer using a multi-dimensional reward function, calculates the relative advantage value of each answer within the group, performs a two-stage pruning update of model parameters, and optimizes the large language model parameters to meet the vertical domain question answering objectives.

Images