A method and device for avoiding dependence of large language model self parameter update

Abstract

The application relates to a method and device for avoiding self-parameter update dependence of a large language model, applied to the technical field of artificial intelligence, and comprising the following steps: through the adoption of a reinforcement learning mode, the output distribution of the large language model can be effectively changed through experience injection, the model can imitate the decision mode of high-quality experience, and problems such as generation illusion and output distribution deviation can be significantly reduced, and the reaction ability of the model to downstream task context changes and the professional semantic understanding ability of the model are improved; multi-dimensional optimization of an external experience injector guarantees the accuracy and diversity of experience screening, avoids the narrow reasoning view of the model caused by experience homogenization, and realizes adaptive optimization of reasoning behavior; meanwhile, the method does not need to modify model parameters, and fundamentally avoids the risk of catastrophic forgetting, and the general ability of the model is retained.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, and specifically to a method and apparatus for avoiding dependence on the parameter updates of large language models. Background Technology

[0002] Generative large language models have been widely used in general domains due to their excellent language understanding and generation capabilities. However, when faced with downstream tasks in specific professional fields, they still suffer from problems such as severe generation illusion, systematic deviation of content output distribution from standard answers, insufficient response to changes in task context, and lack of professional semantic understanding capabilities, which limit their reliable deployment in specific downstream tasks.

[0003] To address the aforementioned issues, existing technologies primarily employ three methods to achieve downstream task behavior calibration and scenario adaptation for large language models:

[0004] Full parameter update: By combining downstream task data, the model is retrained through self-supervised learning, supervised learning, reinforcement learning, etc., to update the parameters. However, large language models have tens or hundreds of billions of parameters, making this method computationally expensive, demanding high computing platform load, and posing a risk of catastrophic forgetting, which can easily impair the model's original general capabilities.

[0005] Local parameter fine-tuning: Optimizing only some parameters of the model or introducing additional parameters / neural network architectures (such as LoRa, Prompt Tuning) has a lower performance ceiling than full parameter updates, and additional modules will increase inference latency. It still suffers from high training costs and resource dependence.

[0006] Fixed prompt word engineering: Based on the zero-shot / few-shot generalization ability of large models, fixed prompt word templates (such as ReAct, Chain-of-Thought) are designed to enhance reasoning performance. However, this method lacks scalability and stability, relies on human experience, and different prompt words need to be designed for different downstream tasks. It lacks systematic, automated and robust support.

[0007] In summary, existing technologies cannot achieve efficient, scalable, and adaptable behavior calibration and adaptation of large language models for downstream subdivided tasks without relying on model parameter updates or controlling computational costs. A new technical solution is urgently needed to solve the above-mentioned technical problems. Summary of the Invention

[0008] In view of this, the purpose of this invention is to provide a method and apparatus for avoiding the dependence of large language models on their own parameter updates, aiming to solve the problems of high training cost, large dependence on computing resources, poor scalability of prompt word engineering, and high professional barriers in the downstream task adaptation of existing large language models.

[0009] According to a first aspect of the present invention, a method for avoiding dependencies on the update of parameters of a large language model is provided, the method comprising:

[0010] The user inputs a query text, and the large language model performs descriptive generation based on the query text to obtain descriptive text;

[0011] The descriptive text and historical decision cases from the pre-built experience pool are input into an externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the descriptive text and historical decision cases to obtain a high-dimensional semantic vector corresponding to each historical decision case.

[0012] The descriptive text and the high-dimensional semantic vector corresponding to each historical decision case are input into an externally pre-built experience screening module;

[0013] The experience screening module first obtains, based on vector similarity, the high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the descriptive text;

[0014] The experience screening module continues to select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision-making cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision-making cases.

[0015] The experience screening module then performs homogenization removal on the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

[0016] The selected F historical decision cases are combined with the user-input query text to form a reasoning context, which is then input into a large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream sub-tasks.

[0017] Preferably,

[0018] The construction of the experience pool includes:

[0019] An experience pool is constructed, and historical decision cases from multiple rounds of tasks are stored in the experience pool. Each historical decision case includes at least one of the following: context description, task structure, decision path, and risk warning. The experience pool supports dynamic expansion and human-initiated supplementation of experience content.

[0020] Preferably, it further includes:

[0021] An external experience injector is constructed, which includes a semantic embedding encoder, a quality controller, and an experience filtering module.

[0022] The experience screening module includes: a correlation sorter, a contrastive learner, and a diversity constraint;

[0023] The quality controller is used to constrain the deviation between the encoding result and the target value through a preset evaluation function, thereby calibrating the encoding quality and enhancing the semantic embedding encoder's understanding of downstream task attributes.

[0024] The relevant sorter is used to perform vector similarity-based operations to obtain high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the descriptive text.

[0025] The contrastive learner is used to perform semantic similarity-based operations to obtain the M high-dimensional semantic vectors corresponding to the historical decision cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision cases.

[0026] The diversity constraint is used to perform homogenization removal on the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, so as to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

[0027] Preferably,

[0028] The optimization training of the quality controller includes:

[0029] The evaluation value of the descriptive text is obtained based on the preset evaluation function. The deviation value is obtained based on the preset target score and the evaluation value of the descriptive text. The first sub-objective optimization function is constructed based on the deviation value.

[0030] Preferably,

[0031] The optimization training of the relevant sorter includes:

[0032] Based on expert labels, small samples are constructed in the experience pool. A relevant sorter is trained using these small samples to distinguish between positive and negative samples.

[0033] Positive and negative sample pairs are constructed based on the distinguished positive and negative samples, wherein positive samples are high-dimensional semantic vectors of historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the query text, and negative samples are high-dimensional semantic vectors of historical decision cases that are unrelated to the high-dimensional semantic vector representation of the query text.

[0034] The similarity between the high-dimensional semantic vector corresponding to the query text and the positive and negative sample pairs is obtained based on the preset vector similarity evaluation function. The relevance sorter is set to score 0 when it correctly distinguishes the positive sample between the positive and negative sample pairs, and negative when it does not. This constructs the second sub-objective optimization function.

[0035] Preferably,

[0036] The optimization training of the contrastive learner includes:

[0037] The positive examples associated with the high-dimensional semantic vector representation corresponding to the query text are brought closer in the representation space, while irrelevant negative examples are pushed away. A third sub-objective optimization function is constructed by measuring the advantage of positive examples over negative examples through a log-ratio structure.

[0038] Preferably,

[0039] The optimization training of the diversity constraint includes:

[0040] The semantic similarity between two high-dimensional semantic vectors is obtained by using a pre-defined semantic similarity measurement function. The partial similarity between the two vectors is used as a penalty term to construct a fourth sub-objective optimization function.

[0041] Preferably, it further includes:

[0042] Set the weight values ​​for the first sub-objective optimization function, the second sub-objective optimization function, the third sub-objective optimization function, and the fourth sub-objective optimization function;

[0043] The first sub-objective optimization function, the second sub-objective optimization function, the third sub-objective optimization function, and the fourth sub-objective optimization function are weighted and summed according to the set weight values ​​to obtain the total training objective function of the external experience injector.

[0044] Preferably,

[0045] The contrastive learner introduces a temperature coefficient into its log-ratio structure. This temperature coefficient is used to adjust the sensitivity of semantic similarity distribution. By normalizing the probability, the semantic scores of positive examples occupy a higher proportion, thereby achieving automatic dehomogenization of experience content.

[0046] According to a second aspect of the present invention, an apparatus is provided to avoid dependence on the update of parameters of a large language model itself, the apparatus comprising:

[0047] User input module: used to obtain the query text input by the user, and the large language model performs descriptive generation based on the query text to obtain the description text;

[0048] Encoding module: Used to input the description text and historical decision cases in the pre-built experience pool into an externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the description text and historical decision cases to obtain a high-dimensional semantic vector corresponding to the description text and each historical decision case.

[0049] External input module: used to input the description text and the high-dimensional semantic vector corresponding to each historical decision case into the pre-built external experience screening module;

[0050] Vector similarity filtering module: The experience filtering module first obtains high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the description text based on vector similarity;

[0051] Semantic similarity filtering module: used by the experience filtering module to further select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision cases;

[0052] Diversity filtering module: used by the experience filtering module to remove homogenization of the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, and obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

[0053] Output module: It is used to combine the selected F historical decision cases with the query text input by the user to form a reasoning context and input it into the large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream subdivision tasks.

[0054] The technical solutions provided by the embodiments of the present invention may include the following beneficial effects:

[0055] This application abandons the high computational resource requirements of traditional fine-tuning and reinforcement learning methods, avoiding the complex and costly model training process. It adopts a reinforcement learning-like approach, which can effectively change the output distribution of large language models through experience injection, allowing the model to imitate the decision-making paradigm of high-quality experience. This significantly reduces problems such as generation illusion and output distribution shift, and improves the model's responsiveness to changes in downstream task context and its professional semantic understanding ability. The multi-dimensional optimization of the adaptive external experience injector ensures the accuracy and diversity of experience selection, avoids the narrow inference vision of the model caused by experience homogenization, and achieves adaptive optimization of inference behavior. At the same time, this method does not require modification of model parameters, fundamentally avoiding the risk of catastrophic forgetting and preserving the model's generality.

[0056] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0057] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0058] Figure 1 This is a flowchart illustrating a method for avoiding dependencies on the update of parameters of a large language model itself, according to an exemplary embodiment.

[0059] Figure 2 This is a schematic diagram illustrating the differences between the reinforcement learning approach of this application and traditional reinforcement learning according to another exemplary embodiment;

[0060] Figure 3 This is a schematic diagram of the structure of an external experience injector according to another exemplary embodiment;

[0061] Figure 4 This is a schematic diagram illustrating a comparison of output distribution under no-experience prompts and with-experience prompts, according to another exemplary embodiment.

[0062] Figure 5 This is a system schematic diagram illustrating an apparatus for avoiding dependencies on the parameter updates of a large language model itself, according to another exemplary embodiment.

[0063] In the attached diagram: 1-User input module, 2-Encoding module, 3-External input module, 4-Vector similarity filtering module, 5-Semantic similarity filtering module, 6-Diversity filtering module, 7-Output module. Detailed Implementation

[0064] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Rather, they are merely examples of apparatuses and methods consistent with some aspects of the invention as detailed in the appended claims.

[0065] Example 1

[0066] Figure 1 This is a flowchart illustrating a method for avoiding dependencies on the update of parameters of a large language model, according to an exemplary embodiment. Figure 1 As shown, the method includes:

[0067] S1, Obtain the query text input by the user, and the large language model performs descriptive generation based on the query text to obtain the description text;

[0068] S2, the description text and the historical decision cases in the pre-built experience pool are input into the externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the description text and the historical decision cases to obtain the high-dimensional semantic vector corresponding to the description text and each historical decision case.

[0069] S3, input the description text and the high-dimensional semantic vector corresponding to each historical decision case into the externally pre-built experience screening module;

[0070] S4, the experience screening module first obtains the high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the description text based on vector similarity;

[0071] S5, the experience screening module continues to select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision-making cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision-making cases;

[0072] S6, the experience screening module finally removes homogenized high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

[0073] S7, combine the selected F historical decision cases with the user-input query text to form a reasoning context and input it into the large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream subdivided tasks.

[0074] It is understood that this embodiment proposes a reinforcement learning-like technical concept, similar to the optimization process of traditional reinforcement learning. However, unlike traditional reinforcement learning, this invention does not require retraining of large model parameters. Instead, it provides the large model with referable, learnable, and emulateable experiential content, enabling the model to automatically adapt to different task requirements. Considering the practical constraint of the limited context length of generative large models, this invention comprehensively proposes a method for automatically updating the inference context experiential content of generative large language models through various innovations such as external experience storage, semantic embedding encoding, risk control, ranking optimization, contrastive learning, and diversity constraints. Specifically, it includes:

[0075] As attached Figure 2As shown, traditional reinforcement learning relies on decision-making experience (states, actions, rewards, etc.) generated by environmental interactions and iteratively updates model parameters through gradient updates to shape policy behavior. In contrast, the reinforcement learning-like method proposed in this application requires no parameter updates. It leverages the general generative capabilities of large language models to achieve robust calibration and task adaptation of model behavior through external experience. Notably, the injected experience content is similar to that in traditional reinforcement learning, used to optimize parameters and guide the model to output gradient signals that yield higher rewards, thus guiding the generative large language model to complete downstream subdivision tasks with higher quality. At this point, only simple prompt word engineering design is needed to allow the large model to reference high-quality experience content and leverage its general capabilities before responding to the query. Considering the limited context length of the generative large language model, not all external experience can be injected into the context. Furthermore, the relevance and diversity of experience content with the query affect the quality of the final response from the generative large language model. Therefore, it is necessary to filter external experience. The following section details the multi-dimensional optimization and filtering of the adaptive external experience injector, including:

[0076] As attached Figure 3 As shown, for a user-input query, the large language model first performs a "descriptive generation" step to: clarify task requirements, reconstruct semantic structure, and extract potential constraints and background information. This description is regarded as the "semantic feature representation" of the task and is then written into a pre-built external experience pool.

[0077] Among them, the external experience pool stores high-quality historical decision-making cases from multiple rounds of tasks, including context descriptions, task structures, decision-making paths, risk warnings, etc. These contents do not contain explicit rewards, but they embody the "effective decision-making paradigm" at the semantic level.

[0078] Next, the description text and historical decision cases in the experience pool are input into the pre-built external adaptive experience injector. The external adaptive experience injector includes a semantic embedding encoder, a quality controller, and an experience filtering module. The experience filtering module includes a correlation sorter, a contrastive learner, and a diversity constraint.

[0079] The quality controller is used to improve the encoding quality and performance of the semantic embedding encoder. Simply put, the quality controller evaluates the degree of matching between the input and all experience bars (candidate experiences) in the experience pool. The main functions of the quality controller are: identifying the paradigm features of downstream sub-tasks from the description content and experience encoding; extracting and summarizing key decision rules; and feeding back the obtained structured information to the semantic embedding encoder and experience selection module to guide subsequent experience selection and injection strategies. Specifically, this includes:

[0080] Semantic embedding encoders are used to vectorize descriptive text and historical decision-making cases, taking a raw text as input. This will result in a high-dimensional semantic vector. , It's about asking questions. The semantic vector after encoding the description content;

[0081] During the process of encoding the input description by the semantic embedding encoder, the model not only needs to understand the semantic information but also needs to output a vector representation with a clear semantic direction. To ensure that the generated encoding can stably serve the strategy decision-making of downstream refinement tasks, this embodiment proposes an encoding quality control mechanism to constrain the representation quality, thereby avoiding task performance degradation caused by encoding bias and indirectly strengthening the semantic embedding encoder's understanding of task attributes. The optimization objective function during its training process is as follows:

[0082]

[0083] In the formula, It is the number of experience points extracted. It's about asking questions. The evaluation value of the description content, It is generally an evaluation function (determined by the specific task scenario, such as a scoring function or a risk measurement function). yes The target score is set independently. This loss term directly reflects the interpretability and performance effectiveness of the encoded vector by constraining the deviation between the encoding result and the target value. Its goal is to calibrate the encoding quality in the experience injection stage so that the model can generate semantic representations that are more in line with the task requirements in subsequent task execution.

[0084] The improved high-dimensional semantic vectors are input into the experience-based screening module to filter high-quality cases, specifically including:

[0085] Related sorters:

[0086] An adaptive external experience injector needs to promptly and accurately retrieve positively and negatively correlated historical experiences from the experience pool based on different queries, in order to distinguish between experiences that are "exemplary" and "irrelevant experiences." This invention enables the adaptive experience injector to learn to distinguish experiences relevant to the descriptive content of the query based on vector similarity. Regarding experience, the specific approach is: when the adaptive experience injector can accurately distinguish positive examples relevant to the query, The score is 0; if not, The score is then negative, and the optimization objective function of the relevant sorter during the training phase is:

[0087]

[0088] In the formula, It is a constant, and its typical range of values ​​is... , It is a constant, representing the total number of pairs. and These are semantic vectors. The positive and negative semantic vectors, functions Functions for evaluating the similarity between two semantic vectors, such as the Euclidean distance formula.

[0089] In this process, the positive and negative sample pairs are accumulated through continuous interaction with various queries by the large model and stored in an experience pool. However, at this initial stage, the samples stored in the experience pool are merely candidate positive and negative samples (in short, the positive and negative samples for a given query are not yet clear). At this point, expert labels can be provided to construct small samples from the experience pool, and a relevant ranking algorithm is trained based on these small samples. Similar to the paradigm of traditional reinforcement learning (value-based reinforcement learning algorithms continuously update parameters using experience points obtained from interactions), the reinforcement-like learning approach initially has an empty experience pool, but as interactions continue, the number of relevant samples increases, highlighting its advantages in the later stages. Furthermore, the reinforcement-like learning experience pool also supports proactive human input to supplement valuable experience content for storage. This means that this application can pre-construct an initial experience pool (collected from business experience), solving the problem of initial lack of experience.

[0090] Contrast learner:

[0091] This embodiment considers not only vector similarity but also semantic similarity, correctly distinguishing semantic differences to ensure that the adaptive external experience injector selects high-quality experience content that is logically relevant to the task. Specifically, it designs a contrastive learner for experience discriminability, bringing semantically relevant experiences (positive examples) closer in the representation space while pushing less relevant experiences (negative examples) further away. This mechanism creates clear boundaries between different experiences in the experience pool within the semantic space, allowing the sorter to more stably select "truly helpful experiences" and avoid injecting misleading experiences. This embodiment designs a log-ratio structure as an optimization form of the contrastive learner to measure the "advantage of positive experiences relative to all negative experiences." The lower the advantage, the greater the loss, forcing the contrastive learner to widen the gap between experiences, thereby improving discriminability. The optimization objective function of the contrastive learner during the training phase is:

[0092]

[0093] In the formula, the function It is a function that measures the semantic similarity between two vectors. This is the encoded representation of the current description content. For high-quality experiences relevant to the current content, A set of experiences that are irrelevant to or negatively related to the current content. It is a temperature coefficient used to adjust the sensitivity of the distribution. The more optimized the learner is, the more the experience injector will drive the semantic scores of positively related experiences to occupy a higher proportion in the normalized probability, thereby forming a prominent experience selection preference and realizing automatic dehomogenization of the selected historical experience content.

[0094] Diversity constraint:

[0095] To avoid excessive homogenization of experience and prevent the recommended experience from converging (a case of over-matching), and to improve the ability of the experience injector to cover the task space, if all the historical experience provided to a large model is on the same topic, the large model will fall into the situation of simply imitating, over-referencing existing information, and narrowing its vision. Therefore, this embodiment adds a diversity constraint component to the adaptive external experience injector, and its optimization objective function during the training phase is:

[0096]

[0097] Among them, the function It measures the semantic similarity between two vectors. It is a constant, representing Yes, this method essentially uses a portion of the similarity as a penalty to prevent the experience injector from selecting overly homogeneous historical experience content.

[0098] By concatenating the optimization functions of each sub-objective in a weighted manner, we obtain the overall optimization objective function of the entire adaptive external experience injector during the training phase, as shown below:

[0099]

[0100] In the formula, These are the weights of each sub-objective function.

[0101] A trained adaptive external experience injector can filter historical decision-making cases from the experience pool. The filtered experience content is then combined with the original query content and input into a large language model for inference generation. During the inference process, the model naturally "imitates" the structure, strategies, and thought patterns of high-quality experience cases, thereby outputting more robust, professional, and downstream task objectives.

[0102] To more intuitively understand the screening process of this application, this embodiment provides the following specific case based on a trained adaptive external experience injector:

[0103] This embodiment applies the method of this application to the downstream sub-task of contract review. Contract review requires large language models to accurately identify risk points, clause loopholes, and non-standard expressions in contracts, possessing strong professional semantic understanding and logical judgment capabilities. Traditional large language models are prone to generation illusions in this task, such as missing risk points and making incorrect judgments. This embodiment uses the method of this invention to calibrate the model's behavior. The specific steps are as follows:

[0104] Inquiry and description phase:

[0105] The system receives a user-input query: "Review a goods sales contract, identify legal risk points, and propose modification suggestions." The large language model performs descriptive generation processing on this query, clarifying the task requirement as the legal risk review and modification suggestion generation for a goods sales contract. The semantic structure is reconstructed as "Contract type: Goods sales contract; Task actions: Risk identification, modification suggestions; Core requirements: Legal compliance, clause completeness." Potential constraints are extracted as "Must comply with relevant provisions of the sales contract in the relevant laws, focusing on core clauses such as payment method, delivery period, and liability for breach of contract." The semantic feature representation of the query is obtained and written into the external experience pool.

[0106] Experience pool construction and management:

[0107] A dedicated experience pool for contract review will be constructed. Initially, this will be achieved by collecting lawyers' experience in contract review cases, typical sales contract risk assessment cases, and clause modification examples. The experience pool will contain high-quality historical decision-making cases, including:

[0108] Scenario description: such as "a steel purchase and sale contract with a total price of 1 million yuan, which does not stipulate the liability for breach of contract due to delayed delivery";

[0109] Task structure: such as "risk identification → clause analysis → legal basis → modification suggestions";

[0110] Decision-making path: such as "check payment terms → find no agreed payment milestones → propose a phased payment modification based on relevant legal provisions";

[0111] Risk warning: "Failure to agree on a quality inspection period may easily lead to disputes over the quality of goods."

[0112] Meanwhile, the experience pool also stores typical Q&A pairs from contract review, records of manual feedback from lawyers, and other content, supporting subsequent dynamic expansion.

[0113] Content-driven experience selection and injection:

[0114] An adaptive external experience injector is constructed, which adaptively filters based on the semantic feature representations mentioned above and historical experience in the contract review experience pool, specifically as follows:

[0115] Semantic Embedding Encoder: BERT is used as the semantic embedding encoder to vectorize the semantic feature representation "legal risk review and modification suggestion generation of goods sales contract" and the historical experience in the experience pool, respectively, to obtain a 768-dimensional high-dimensional semantic vector;

[0116] Coding quality control: The evaluation function of the quality controller adopts the risk identification accuracy scoring function, with a target score of 95 points and 20 experience points extracted. The deviation between the coding evaluation value corresponding to the 20 experience points and the target score of 95 points is constrained by the loss term, which strengthens the encoder's understanding of the task attribute of "contract review risk identification" and avoids the omission of risk points due to coding deviation.

[0117] Relevance Ranking: The relevance ranking uses cosine similarity to calculate semantic vector similarity, distinguishing between positive examples (experience related to the review of sales contracts for goods) and negative examples (experience related to the review of labor contracts and lease contracts). The ranking scoring rule is that 0 points are awarded when the difference is accurate and -1 points are awarded when the difference cannot be made. After ranking, 100 contract review experiences that are highly relevant to the current query are obtained.

[0118] Contrastive learner: The contrastive learner constructs a log ratio structure based on cosine similarity, with a temperature coefficient set to 0.1. It brings positive examples of goods sales contract review closer in the representation space and pushes negative examples such as labor contracts further away. By normalizing the probability, it increases the semantic score ratio of positive examples to over 90%, thereby achieving automatic de-homogenization of experience content and selecting 80 highly relevant and high-quality experiences from 100 experiences.

[0119] Diversity Constraint: The diversity constraint applies homogenization penalty based on Euclidean distance between vectors. Content with a similarity greater than 0.9 between experiences is judged as highly redundant and penalized. From 80 experiences, the final 10 high-quality experiences covering core dimensions such as payment terms, delivery time, liability for breach of contract, and quality inspection are selected to avoid experiences being concentrated on a single dimension.

[0120] Context injection and reasoning generation:

[0121] The 10 high-quality experience snippets selected through screening (such as "risk points and modification suggestions for steel sales contracts that do not stipulate liability for delayed delivery" and "compliance modification solutions for unclear payment terms in goods sales contracts") are combined with the original query to form a reasoning context.

[0122] The reasoning context is input into a generative large language model of 7B scale. The model imitates the risk identification logic, legal basis citation method and modification suggestion writing paradigm in the experience fragment to generate reasoning and output accurate risk point identification results and compliant modification suggestions.

[0123] As attached Figure 4 As shown, it can be observed that, under the same input query conditions, compared with the case without experience prompts, this application can effectively change the output distribution of the generative large language model. This means that the large model can combine the information of experience prompts to change the original reasoning behavior, which indirectly illustrates that the paradigm of reinforcement learning can be used to improve the adaptation and behavior calibration of the large model to downstream subdivided tasks.

[0124] Example 2

[0125] Figure 5 This is a system schematic diagram illustrating an apparatus for avoiding dependencies on the update of parameters of a large language model itself, according to another exemplary embodiment. The apparatus includes:

[0126] User input module 1: Used to obtain the query text input by the user. The large language model performs descriptive generation based on the query text to obtain the description text.

[0127] Encoding module 2: Used to input the description text and the historical decision cases in the pre-built experience pool into the externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the description text and the historical decision cases to obtain the high-dimensional semantic vector corresponding to the description text and each historical decision case.

[0128] External input module 3: used to input the description text and the high-dimensional semantic vector corresponding to each historical decision case into the pre-built external experience screening module;

[0129] Vector similarity filtering module 4: This module first obtains high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the descriptive text based on vector similarity.

[0130] Semantic similarity filtering module 5: is used by the experience filtering module to further select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision cases;

[0131] Diversity screening module 6: is used by the experience screening module to remove homogenization of the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, so as to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

[0132] Output module 7: It is used to combine the selected F historical decision cases with the user-input query text to form a reasoning context and input it into the large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream subdivided tasks.

[0133] It is understood that the same or similar parts in the above embodiments can be referred to each other, and the contents not described in detail in some embodiments can be referred to the same or similar contents in other embodiments.

[0134] It should be noted that in the description of this invention, the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. Furthermore, in the description of this invention, unless otherwise stated, "a plurality of" means at least two.

[0135] Any process or method description in the flowchart or otherwise herein can be understood as representing a module, segment, or portion of code comprising one or more executable instructions for implementing a particular logical function or process, and the scope of the preferred embodiments of the invention includes additional implementations in which functions may be performed not in the order shown or discussed, including substantially simultaneously or in reverse order depending on the functions involved, as will be understood by those skilled in the art to which embodiments of the invention pertain.

[0136] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented in software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0137] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0138] Furthermore, the functional units in the various embodiments of the present invention can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into a module. The integrated module can be implemented in hardware or as a software functional module. If the integrated module is implemented as a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium.

[0139] The storage media mentioned above can be read-only memory, disk, or optical disk, etc.

[0140] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0141] Although embodiments of the present invention have been shown and described above, it is understood that the above embodiments are exemplary and should not be construed as limiting the present invention. Those skilled in the art can make changes, modifications, substitutions and variations to the above embodiments within the scope of the present invention.

Claims

1. A method to avoid dependency on the parameter updates of a large language model, characterized in that, The method includes: The user inputs a query text, and the large language model performs descriptive generation based on the query text to obtain descriptive text; The descriptive text and historical decision cases from the pre-built experience pool are input into an externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the descriptive text and historical decision cases to obtain a high-dimensional semantic vector corresponding to each historical decision case. The descriptive text and the high-dimensional semantic vector corresponding to each historical decision case are input into an externally pre-built experience screening module; The experience screening module first obtains, based on vector similarity, the high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the descriptive text; The experience screening module continues to select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision-making cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision-making cases. The experience screening module then performs homogenization removal on the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases. The selected F historical decision cases are combined with the user-input query text to form a reasoning context, which is then input into a large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream sub-tasks.

2. The method according to claim 1, characterized in that, The construction of the experience pool includes: An experience pool is constructed, and historical decision cases from multiple rounds of tasks are stored in the experience pool. Each historical decision case includes at least one of the following: context description, task structure, decision path, and risk warning. The experience pool supports dynamic expansion and human-initiated supplementation of experience content.

3. The method according to claim 2, characterized in that, Also includes: An external experience injector is constructed, which includes a semantic embedding encoder, a quality controller, and an experience filtering module. The experience screening module includes: a correlation sorter, a contrastive learner, and a diversity constraint; The quality controller is used to constrain the deviation between the encoding result and the target value through a preset evaluation function, thereby calibrating the encoding quality and enhancing the semantic embedding encoder's understanding of downstream task attributes. The relevant sorter is used to perform vector similarity-based operations to obtain high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the descriptive text. The contrastive learner is used to perform semantic similarity-based operations to obtain the M high-dimensional semantic vectors corresponding to the historical decision cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision cases. The diversity constraint is used to perform homogenization removal on the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, so as to obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases.

4. The method according to claim 3, characterized in that, The optimization training of the quality controller includes: The evaluation value of the descriptive text is obtained based on the preset evaluation function. The deviation value is obtained based on the preset target score and the evaluation value of the descriptive text. The first sub-objective optimization function is constructed based on the deviation value.

5. The method according to claim 4, characterized in that, The optimization training of the relevant sorter includes: Based on expert labels, small samples are constructed in the experience pool. A relevant sorter is trained using these small samples to distinguish between positive and negative examples. Positive and negative sample pairs are constructed based on the distinguished positive and negative samples, wherein positive samples are high-dimensional semantic vectors of historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the query text, and negative samples are high-dimensional semantic vectors of historical decision cases that are unrelated to the high-dimensional semantic vector representation of the query text. The similarity between the high-dimensional semantic vector corresponding to the query text and the positive and negative sample pairs is obtained based on the preset vector similarity evaluation function. The relevance sorter is set to score 0 when it correctly distinguishes the positive sample between the positive and negative sample pairs, and negative when it does not. This constructs the second sub-objective optimization function.

6. The method according to claim 5, characterized in that, The optimization training of the contrastive learner includes: The positive examples associated with the high-dimensional semantic vector representation corresponding to the query text are brought closer in the representation space, while irrelevant negative examples are pushed away. A third sub-objective optimization function is constructed by measuring the advantage of positive examples over negative examples through a log-ratio structure.

7. The method according to claim 6, characterized in that, The optimization training of the diversity constraint includes: The semantic similarity between two high-dimensional semantic vectors is obtained by using a pre-defined semantic similarity measurement function. The partial similarity between the two vectors is used as a penalty term to construct a fourth sub-objective optimization function.

8. The method according to claim 7, characterized in that, Also includes: Set the weight values ​​for the first sub-objective optimization function, the second sub-objective optimization function, the third sub-objective optimization function, and the fourth sub-objective optimization function; The first sub-objective optimization function, the second sub-objective optimization function, the third sub-objective optimization function, and the fourth sub-objective optimization function are weighted and summed according to the set weight values ​​to obtain the total training objective function of the external experience injector.

9. The method according to claim 6, characterized in that, The contrastive learner introduces a temperature coefficient into its log-ratio structure. This temperature coefficient is used to adjust the sensitivity of semantic similarity distribution. By normalizing the probability, the semantic scores of positive examples occupy a higher proportion, thereby achieving automatic dehomogenization of experience content.

10. A device for avoiding dependencies on the parameter updates of a large language model, characterized in that, The device includes: User input module: used to obtain the query text input by the user, and the large language model performs descriptive generation based on the query text to obtain the description text; Encoding module: Used to input the description text and historical decision cases in the pre-built experience pool into an externally constructed semantic embedding encoder. The semantic embedding encoder vectorizes the description text and historical decision cases to obtain a high-dimensional semantic vector corresponding to the description text and each historical decision case. External input module: used to input the description text and the high-dimensional semantic vector corresponding to each historical decision case into the pre-built external experience screening module; Vector similarity filtering module: The experience filtering module first obtains high-dimensional semantic vectors corresponding to N historical decision cases that are positively correlated with the high-dimensional semantic vector representation of the description text based on vector similarity; Semantic similarity filtering module: used by the experience filtering module to further select, based on semantic similarity, the M high-dimensional semantic vectors corresponding to the historical decision cases that are most semantically related to the high-dimensional semantic vectors corresponding to the description text from the high-dimensional semantic vectors corresponding to the N historical decision cases; Diversity filtering module: used by the experience filtering module to remove homogenization of the high-dimensional semantic vectors corresponding to the M historical decision cases based on semantic similarity, and obtain the final F high-dimensional semantic vectors corresponding to the historical decision cases. Output module: It is used to combine the selected F historical decision cases with the query text input by the user to form a reasoning context and input it into the large language model. The large language model imitates the decision paradigm in the F historical decision cases to generate reasoning and outputs results adapted to downstream subdivision tasks.

Images