Large language model constraint instruction following method and device based on adaptive contrast decoding

Abstract

The application provides a large language model constraint instruction following method and device based on adaptive contrast decoding, which divides the prompt information into two parts of instructions and constraints, generates an initial output sequence according to a dynamic window by an expert model, introduces an auxiliary model to calculate the difference between the prediction probability distribution of each token under the constraint and non-constraint conditions, and distinguishes constraint-sensitive tokens and constraint-insensitive tokens based on a preset standard. For the window containing only constraint-insensitive tokens, the window is directly output to the target output sequence, and the window length is enlarged according to a first set proportion; for the window containing constraint-sensitive tokens, the tokens before the first constraint-sensitive token are directly input to the target output sequence, the first constraint-sensitive token is corrected based on contrast decoding, and the window length is reduced according to a second proportion. The application can reduce the interference of constraint conditions on model generation, avoid redundant calculation of non-sensitive tokens, significantly improve the inference efficiency, and ensure the constraint following quality.

Description

Technical Field

[0001] This invention relates to the field of model output control technology, and in particular to a method and apparatus for following large language model constraint instructions based on adaptive contrastive decoding. Background Technology

[0002] In recent years, large language models have achieved widespread success in the field of natural language processing, demonstrating powerful instruction-following capabilities. Instruction-following refers to the ability of large language models to understand and execute various tasks described by users in natural language. This capability enables LLMs to demonstrate great potential in numerous practical application scenarios.

[0003] However, existing instruction-following methods show significant shortcomings when faced with complex constraints. First, when constraints are tightly coupled with task instructions, the model generation process is significantly affected by the constraints, limiting the model's ability to freely explore a wider and more suitable solution space, and easily leading to generated results that deviate from the user's actual needs. Furthermore, existing methods, such as inter-layer self-comparative decoding and speculative comparison decoding, often require a large number of additional computational and verification steps during the inference process for handling complex constraints, greatly reducing inference efficiency and making it difficult to meet the real-time requirements of practical applications.

[0004] Meanwhile, to enforce complex constraints, models often require strict output constraint control. Such forced constraints can disrupt the model's natural reasoning process, weakening its deep understanding of the task and domain knowledge, leading to decreased quality and accuracy of the generated results. Furthermore, while traditional contrastive decoding methods can improve output quality to some extent, they typically employ static or single contrastive strategies, failing to dynamically adapt to the varying sensitivities of different tokens to constraints. This results in wasted computational resources on non-constraint-sensitive tokens, severely impacting generation efficiency. Therefore, a new constraint instruction compliance scheme for large language models is urgently needed. Summary of the Invention

[0005] In view of this, embodiments of the present invention provide a method and apparatus for following constraint instructions in a large language model based on adaptive contrastive decoding, so as to eliminate or improve one or more defects existing in the prior art and solve the problems of low inference efficiency and poor quality of generated results in large language models under complex constraints at the present stage.

[0006] One aspect of the present invention provides a method for constraining instruction compliance in a large language model based on adaptive contrastive decoding, the method comprising the following steps:

[0007] The original prompt words are decoupled to obtain the task instruction part and the constraint condition part;

[0008] An expert model generates an initial output sequence based on the task instructions under unconstrained conditions, according to a window of a preset initial length. For each round of the initial output sequence, the following operations are performed:

[0009] The auxiliary model calculates the predicted probability distribution of the words in the initial output sequence under constrained and unconstrained conditions in parallel, and calculates the difference between the predicted probability distribution of the words under constrained and unconstrained conditions. Words whose difference is greater than a set standard are marked as constrained sensitive words, and words whose difference is less than or equal to the set standard are marked as constrained insensitive words. The parameter scale of the auxiliary model is smaller than that of the expert model.

[0010] If the initial output sequence contains only the constraint-insensitive word, then the word in the initial output sequence is output to the target output sequence, and the window length is expanded according to a first predetermined ratio; if the initial output sequence contains the constraint-sensitive word, then the word before the first constraint-sensitive word is output to the target output sequence, the first constraint-sensitive word is corrected based on the expert model and the auxiliary model using comparative decoding, and then output to the target output sequence, and the window length is reduced according to a second predetermined ratio.

[0011] The window, whose length changes dynamically, performs multiple rounds of operations to obtain and output the complete target output sequence.

[0012] In some embodiments, calculating the difference in the predicted probability distribution of the lexical under constrained and unconstrained conditions, marking lexicals with differences greater than a set standard as constrained sensitive lexicals, and marking lexicals with differences less than or equal to the set standard as constrained insensitive lexicals, includes:

[0013] Calculate the JS divergence of the predicted probability distribution of the lexical under constrained and unconstrained conditions, mark lexicals with JS divergence greater than a preset value as constrained sensitive lexicals, and mark lexicals with JS divergence less than or equal to the preset value as constrained insensitive lexicals.

[0014] In some embodiments, the JS divergence is calculated as follows:

[0015]

[0016] Where P represents the predicted probability distribution of the word element under constrained conditions, and Q represents the predicted probability distribution of the word element under unconstrained conditions; D KL This represents the Kullback-Leibler divergence.

[0017] In some embodiments, the first constraint-sensitive word unit is corrected based on the expert model and the auxiliary model using a contrastive decoding method, including:

[0018] Based on the expert model and the auxiliary model, multiple candidate word units are compared and scored at each step, and the calculation formula is as follows:

[0019] S imp (x i |x ＜i )=(1+α)S e (x i |x ＜i )-αS a (x i |x ＜i );

[0020] Among them, S e (x i |x ＜i ) represents the logits value of the expert model for the i-th candidate word; S a (x i |x ＜i ) represents the logits value of the auxiliary model for the i-th candidate word; α represents the hyperparameter, which ranges from 0.5 to 2;

[0021] The candidate word with the highest score is output and replaced with the first constrained sensitive word.

[0022] In some embodiments, the window length is expanded according to a first predetermined ratio, including:

[0023] Calculate the first density of the constrained sensitive word element within the window in the number of rounds prior to the current time, and expand the length of the window based on the first set ratio corresponding to the multiple preset first density intervals to perform the next round of operation;

[0024] Reducing the window length according to a second preset ratio includes:

[0025] Calculate the second density of the constrained sensitive word elements within the window in the current round, and reduce the length of the window based on the second set ratio corresponding to the preset multiple second density intervals for the next round of operation.

[0026] In some embodiments, the method further includes:

[0027] If the first constraint-sensitive word is repeatedly output when the expert model and the auxiliary model correct the first constraint-sensitive word based on the form of comparative decoding, then the external database is called to generate a substitute word for the first constraint-sensitive word based on retrieval enhancement.

[0028] In some embodiments, the method further includes: recording the original terminology, correction process, and final terminology for each constrained sensitive terminology, establishing a correction log, and adding an index for traceability.

[0029] On the other hand, the present invention also provides a large language model constraint instruction compliance device based on adaptive contrastive decoding, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0030] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0031] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0032] The beneficial effects of the present invention are at least as follows:

[0033] The method and apparatus for constraint instruction compliance in a large language model based on adaptive contrastive decoding, as described in this invention, divides the prompt information into two parts: instructions and constraints. An expert model generates an initial output sequence according to a dynamically changing window. An auxiliary model is introduced to calculate the difference in the predicted probability distribution of each word under constrained and unconstrained conditions, and distinguishes between constraint-sensitive and constraint-insensitive words based on a preset standard. For windows containing only constraint-insensitive words, the output is directly sent to the target output sequence, and the window length is increased according to a first preset ratio. For windows containing constraint-sensitive words, the words before the first constraint-sensitive word are directly input into the target output sequence, the first constraint-sensitive word is corrected based on contrastive decoding, and the window length is reduced according to a second ratio. This invention can effectively reduce the interference of constraints on the initial generation stage of the model, adaptively judge the constraint sensitivity, avoid redundant calculation of non-sensitive tokens, dynamically adjust the window size, significantly improve the inference efficiency of the model, and at the same time ensure the constraint compliance quality of the generated results.

[0034] Additional advantages, objects, and features of the invention will be set forth in part in the description which follows, and will also become apparent in part to those skilled in the art upon studying the description, or may be learned by practice of the invention. The objects and other advantages of the invention can be realized and obtained by means of the structures specifically pointed out in the description and drawings.

[0035] Those skilled in the art will understand that the objectives and advantages achievable with the present invention are not limited to those specifically described above, and that the above and other objectives achievable with the present invention will become clearer from the following detailed description. Attached Figure Description

[0036] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, are not intended to limit the scope of the invention. In the drawings:

[0037] Figure 1 This is a flowchart illustrating the method for constraining instruction compliance based on adaptive contrastive decoding of a large language model according to an embodiment of the present invention.

[0038] Figure 2 This is a logical schematic diagram of the adaptive comparison decoding method according to another embodiment of the present invention. Detailed Implementation

[0039] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the embodiments and accompanying drawings. Here, the illustrative embodiments and descriptions of this invention are used to explain the invention, but are not intended to limit the invention.

[0040] It should also be noted that, in order to avoid obscuring the invention with unnecessary details, only the structures and / or processing steps closely related to the solution according to the invention are shown in the accompanying drawings, while other details that are not closely related to the invention are omitted.

[0041] It should be emphasized that the term "including / comprises" as used herein refers to the presence of a feature, element, step, or component, but does not exclude the presence or addition of one or more other features, elements, steps, or components.

[0042] Supervised fine-tuning is a widely used technique to improve the instruction-following ability of large language models. This method involves manually constructing a large number of explicit instruction-output pairs and training the model in a supervised manner, enabling the model to generate the expected output based on instructions in natural language form. However, this requires a large amount of high-quality, diverse, manually labeled data, and the cost and difficulty of data construction are very high, making it difficult to guarantee sufficient data coverage. Secondly, supervised fine-tuning on large-scale models typically requires significant computational resources and power, significantly increasing the training threshold for practical applications. Furthermore, this method has limited generalization ability; when faced with changes in task scenarios or constraints, the model struggles to adapt quickly and cannot effectively handle complex, unseen constraints.

[0043] This invention aims to address the problems of insufficient instruction compliance, slow inference speed, and poor generalization in existing large language models under complex constraints. By decoupling instructions from constraints, dynamically identifying constraint-sensitive tokens, and employing a dynamic window mechanism, the invention significantly improves the model's inference efficiency and its ability to comply with complex constraint instructions while ensuring generation quality.

[0044] Specifically, this invention provides a method for constraining instruction compliance in a large language model based on adaptive contrastive decoding, such as... Figure 1 As shown, the method includes the following steps S101 to S103:

[0045] Step S101: Decouple the original prompt words to obtain the task instruction part and the constraint condition part.

[0046] Step S102: The expert model generates an initial output sequence based on the task instructions under unconstrained conditions, according to a window of a preset initial length; for each round of the initial output sequence, the following operations are performed:

[0047] Step S1021: The auxiliary model calculates the predicted probability distribution of the words in the initial output sequence under constrained and unconstrained conditions in parallel, and calculates the difference between the predicted probability distribution of the words under constrained and unconstrained conditions. Words with a difference greater than the set standard are marked as constrained sensitive words, and words with a difference less than or equal to the set standard are marked as constrained insensitive words. The parameter scale of the auxiliary model is smaller than that of the expert model.

[0048] Step S1022: If the words in the current initial output sequence contain only constraint-insensitive words, then output the words in the current initial output sequence to the target output sequence and expand the window length according to the first set ratio; if the words in the current initial output sequence contain constraint-sensitive words, then output the words before the first constraint-sensitive word to the target output sequence, correct the first constraint-sensitive word based on the expert model and the auxiliary model in the form of comparative decoding and output it to the target output sequence, and reduce the window length according to the second set ratio.

[0049] Step S103: Perform multiple rounds of operations based on the dynamically variable window of length to obtain the complete target output sequence and output it.

[0050] In step S101, instruction decoupling is the core preprocessing step that decomposes the original prompt words into two independent parts. The original input may contain both the task objective and the constraints, such as "write a popular science article about climate change, which should not exceed 300 words and should not use technical terms." Traditional language models may not be able to clearly distinguish between the two, causing the generated results to deviate from the constraints.

[0051] This invention identifies two types of information in the original prompt words through semantic analysis or a pre-trained classifier, dividing the original prompt words into a task instruction part and a constraint instruction part. The task instruction part describes the core objective the model needs to achieve, such as "write a popular science article about climate change." The constraint part specifies the restrictions that must be followed during the generation process, such as "no more than 300 words and no use of technical jargon." These two decoupled parts are used for subsequent generation and constraint verification, respectively. The accuracy of this step directly affects the alignment between subsequent generation and constraints.

[0052] In step S102, the expert model is required to generate preliminary predictions according to the task instructions and the window length, without any constraints. It is important to clarify that the expert model's preliminary predictions are not complete output predictions, but rather segmented. After generating the initial output sequence in each round, steps S1021 and S1022 are executed to mark whether each word in the current window is sensitive to constraints, and to decide how to introduce words into the target output sequence and how to adjust the window length based on whether the current window contains constraint-sensitive words.

[0053] In step S1021, the auxiliary model analyzes the initial output sequence generated by the expert model. The expert model generates the initial output sequence based on the task instructions, under unconstrained conditions and within a window of a preset initial length. The auxiliary model has a smaller parameter scale than the expert model, ensuring computational efficiency in analyzing whether a word is sensitive to constraints. It calculates the predicted probability distribution of each word in the initial output sequence under both constrained and unconstrained conditions in parallel. The predicted probability distribution is a probability estimate of the word's likelihood of occurrence under different conditions. By calculating the difference between the probability distributions under constrained and unconstrained conditions, it determines whether the word is significantly affected by constraints. If the difference is greater than a set standard, it is marked as a constraint-sensitive word, meaning that the probability of occurrence of this word changes significantly between constrained and unconstrained conditions, and it is a word affected by constraints; conversely, it is marked as a constraint-insensitive word, meaning it is less affected by constraints.

[0054] In some embodiments, calculating the difference between the predicted probability distributions of lexical units under constrained and unconstrained conditions, marking lexical units with differences greater than a set standard as constrained sensitive lexical units, and marking lexical units with differences less than or equal to the set standard as constrained insensitive lexical units, includes: calculating the JS divergence of the predicted probability distributions of lexical units under constrained and unconstrained conditions, marking lexical units with JS divergence greater than a preset value as constrained sensitive lexical units, and marking lexical units with JS divergence less than or equal to a preset value as constrained insensitive lexical units.

[0055] In some embodiments, the JS divergence is calculated as follows:

[0056]

[0057] Where P represents the predicted probability distribution of a word under constrained conditions, and Q represents the predicted probability distribution of a word under unconstrained conditions; D KL This represents the Kullback-Leibler divergence.

[0058] In step S1022, the word elements in the initial output sequence are assessed and appropriate actions are taken. If all words in the current initial output sequence are constraint-insensitive words, these words can be directly output to the target output sequence. Since the currently output words all meet the requirements, the window length is expanded according to a first predetermined ratio to more efficiently complete the output task, allowing for the processing of more meta-word content at once. However, if constraint-sensitive words exist, the words preceding the first constraint-sensitive word (those that meet the requirements) are output to the target output sequence. Then, for the first constraint-sensitive word, it is corrected using a comparative decoding method based on the expert model and the auxiliary model before being output to the target output sequence. This is done because the first constraint-sensitive word does not meet the constraint requirements, necessitating the combined power of two models for appropriate adjustment. After processing this sensitive word, the window length is reduced according to a second predetermined ratio to more precisely handle potentially problematic words in subsequent outputs, allowing for more accurate control over whether subsequent output content meets the constraint requirements.

[0059] One decoding strategy, based on contrastive decoding, simultaneously introduces an expert model and an auxiliary model. It compares and scores the output distributions of both models when generating each token, suppressing low-quality or untrustworthy candidate tokens, thereby improving the accuracy, coherence, and security of the generated content. This method requires no additional training and exhibits significant optimization effects in generation, but its inference efficiency is relatively low.

[0060] In some embodiments, the first constraint-sensitive word element is corrected based on a comparative decoding method using an expert model and an auxiliary model. This includes: comparing and scoring multiple candidate word elements at each step using the expert model and the auxiliary model, outputting the candidate word with the highest score, and replacing the first constraint-sensitive word element. The score is calculated as follows:

[0061] S imp (x i |x ＜i )=(1+α)S e (x i |x ＜i )-αS a (x i |x ＜i );

[0062] Among them, S e(x i |x ＜i S represents the logits value of the expert model for the i-th candidate word; a (x i |x ＜i ) represents the logits value of the auxiliary model for the i-th candidate word; α represents the hyperparameter, which ranges from 0.5 to 2.

[0063] The scoring function rewards tokens favored by the expert model and penalizes tokens preferred by the auxiliary model. Since auxiliary models tend to generate common, patterned, and low-quality outputs, this contrast mechanism effectively suppresses errors such as "illusions" and improves the credibility of the output.

[0064] In some embodiments, if the window contains only constraint-insensitive terms, the window length is doubled; if the window contains constraint-sensitive terms, the window length is halved.

[0065] In some embodiments, the first and second preset ratios can be dynamically set to achieve better inference efficiency and output quality.

[0066] The window length is expanded according to a first set ratio, including step S201: calculating the first density of constrained sensitive words in the window in a set number of rounds before the current time, and expanding the window length according to the first set ratio based on multiple preset first density intervals to perform the next round of operation.

[0067] The language model generation process involves multiple rounds of operations. At this point, it's necessary to review a predetermined number of previous rounds. In these rounds, the frequency of constraint-sensitive words appearing within the statistical window is calculated; that is, the proportion of constraint-sensitive words in the total number of words within these windows is calculated. This proportion is the first density.

[0068] The system pre-divides the data into multiple intervals based on the possible first density range. Each interval corresponds to a first predetermined ratio, which determines the extent to which the window length is expanded. For example, when the first density is in a lower interval, it indicates that the frequency of constraint-sensitive terms appeared in previous rounds, and the generated sequences perform well in terms of constraints. In this case, a larger first predetermined ratio can be used to expand the window length so that the model can generate richer content. Conversely, when the first density is in a higher interval, a smaller first predetermined ratio is used to moderately expand the window length to avoid potential constraint problems.

[0069] Assuming the preset first density ranges are [0~0.2], (0.2~0.5), and [0.5~1], it should be noted that these numerical ranges are examples, and the actual values ​​can be determined based on specific application scenarios and experimental results. The corresponding first set percentages are 20%, 10%, and 5%, respectively. The number of rounds is set to 3. There are three rounds before the current time, with a total of 100 words in the window across these three rounds, and the number of constrained sensitive words being 5, 10, and 15, respectively. Therefore, the constrained sensitive word densities for each of these three rounds are calculated to be 0.05, 0.1, and 0.15, respectively. The average of these three densities is (0.05+0.1+0.15) / 3≈0.1. According to the preset ranges, 0.1 belongs to the [0~0.2] range, corresponding to a first set percentage of 20%. If the current window length is 50, then the window length will be expanded to 50+(50×20%)=60 to allow for the next round of operation.

[0070] The window length is reduced according to a second set ratio, including step S202: calculating the second density of constrained sensitive words in the current round window, and reducing the window length according to the corresponding second set ratio based on multiple preset second density intervals for the next round of operation.

[0071] Within the current round's window, the frequency of occurrence of constraint-sensitive words is calculated, which is the ratio of the number of constraint-sensitive words to the total number of words within the window. This ratio is the second density. Similarly, the possible intervals for the second density are pre-defined, with each interval corresponding to a second set ratio. For example, when the second density is in a higher interval, it means that there are more constraint-sensitive words in the sequence generated in the current round, potentially indicating a greater risk of non-compliance with constraints. In this case, a larger second set ratio is used to shorten the window length, strengthening the constraints and control over the generated sequence. Conversely, when the second density is in a lower interval, a smaller second set ratio is used to moderately shorten the window length.

[0072] Assuming the preset second density intervals are [0~0.3], (0.3~0.6), and [0.6~1], the corresponding second set percentages are 5%, 10%, and 15%, respectively. In the current round, the total number of words in the window is 80, of which 30 are constrained sensitive words, so the second density is 0.375. According to the preset intervals, 0.375 belongs to the (0.3~0.6) interval, and the corresponding second set percentage is 10%. If the current window length is 80, the window length will be reduced to 80 - (80 × 10%) = 72 for the next round of operation.

[0073] In some embodiments, the method further includes step S301: If the first constraint-sensitive token is repeatedly output when correcting it in the form of contrastive decoding based on the expert model and the auxiliary model, then call an external database to generate an alternative token for the first constraint-sensitive token based on retrieval augmentation.

[0074] When correcting the first constraint-sensitive token in the form of contrastive decoding based on the expert model and the auxiliary model, if the situation of repeatedly outputting this constraint-sensitive token occurs, it means that the model has difficulty effectively avoiding the constraint-sensitive token in the current contrastive decoding process, or the model's correction strategy is not effective enough in this specific situation. Then, an external database can be called for processing based on retrieval augmentation.

[0075] The external database is a pre-established knowledge base that contains a large number of tokens and their related semantic information, usage scenarios, possible alternative tokens, etc. These alternative tokens are selected manually or through specific algorithms and can meet the requirement of replacing the original token under specific constraint conditions. For example, in the database, there may be stored alternative solutions for some colloquial or non-formal style tokens in the scenario of formal document writing. For example, replacing "and then" with "next", etc.

[0076] Retrieval-Augmented Generation (RAG) is a technology that combines information retrieval and language generation and is widely used in the field of natural language processing, especially when accurate and information-rich text needs to be generated. RAG combines the two processes of information retrieval and language generation. First, through the retrieval module, relevant document fragments are found from an external large-scale corpus, and then these fragments are input into the language generation model as additional context information to help the model generate more targeted and accurate outputs.

[0077] By retrieving the external document library, RAG can introduce a large amount of specific knowledge related to the task, which may not be covered in the model training stage, thus greatly expanding the knowledge scope of the model and enabling the model to generate text containing rich details and accurate information. Based on the reliable document content retrieved, the generated text is significantly improved in terms of factual accuracy and task relevance and can better meet the high-precision requirements in practical applications. It can alleviate the hallucination problem and make the generated text more credible.

[0078] In some embodiments, the method further includes: recording the original token, correction process, and final token of each constraint-sensitive token, establishing a correction log and adding an index for tracing.

[0079] In step S103, after multiple rounds of operation processing in the preceding steps, the content is continuously generated, judged, corrected, and output based on the length of the dynamically variable window. Meta sequences of words that meet the constraint requirements are continuously accumulated, and finally a complete target output sequence is formed and output. This target output sequence is the final result obtained after comprehensively considering the task instructions and constraints, and can better follow the given constraint instructions to complete the corresponding task.

[0080] On the other hand, the present invention also provides a large language model constraint instruction compliance device based on adaptive contrastive decoding, including a processor, a memory, and a computer program / instructions stored in the memory, wherein the processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the device implements the steps of the above method.

[0081] On the other hand, the present invention also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implement the steps of the above-described method.

[0082] On the other hand, the present invention also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps of the above-described method.

[0083] The present invention will now be described with reference to a specific embodiment:

[0084] This embodiment aims to address the problems of insufficient instruction compliance, slow inference speed, and poor generalization in existing large language models under complex constraints. Specifically, this embodiment proposes an adaptive contrastive decoding method that significantly improves the model's inference efficiency and its ability to comply with complex constraint instructions by decoupling instructions from constraints, dynamically identifying constraint-sensitive tokens, and using a dynamic window mechanism, while ensuring generation quality.

[0085] The technical solution of this embodiment mainly includes three core steps: instruction decoupling, constraint-sensitive token adaptive estimation, and dynamic window speculative comparison decoding mechanism. The steps are as follows:

[0086] First, the original prompt information is decomposed into two independent parts: instructions and constraints. The model generates preliminary prediction output based on the unconstrained instructions, thereby avoiding the interference of constraints on the initial generation stage.

[0087] Secondly, predicted tokens are generated using both constrained and unconstrained hints. The sensitivity of each token to constraints is dynamically identified by calculating the JS divergence between the two predicted distributions. When the JS divergence exceeds a predetermined threshold, the token is identified as a constraint-sensitive token and corrected using comparative decoding; otherwise, the initially predicted token is used directly.

[0088] Finally, a dynamic window speculative comparison decoding mechanism is designed. During inference, the model generates and verifies the constraint sensitivity of multiple tokens in parallel with dynamically adjusted window sizes. When no constraint-sensitive token is detected within the window, the window size is doubled in the next step to accelerate inference; if a constraint-sensitive token is detected, the first sensitive token is compared, decoded, and refined, and the window size for the next step is halved, thereby achieving precise allocation of computing resources and efficient inference.

[0089] This embodiment combines speculative decoding and comparative decoding and is optimized based on the characteristics of instruction-following tasks. The overall architecture of the method is as follows: Figure 2 As shown, it mainly includes three core modules: (1) instruction decoupling module; (2) constraint-sensitive token adaptive estimation module; and (3) dynamic window speculative comparison decoding module. The specific implementation details of each module will be described in detail below.

[0090] 1. Instruction decoupling

[0091] To improve the flexibility and effectiveness of large language models in handling complex constraint instructions, this embodiment proposes an instruction decoupling mechanism, which separates instructions from constraints to avoid interference from constraints in the early stages of generation. When constraints are directly embedded in instructions, the model is easily limited by the constraints during generation, preventing it from fully exploring the possible solution space and reducing the diversity and accuracy of the generated results.

[0092] Specifically, given an initial prompt message:

[0093] p = (i, c);

[0094] Where i represents the instruction for a specific task, and c represents the constraints related to the task.

[0095] During the generation phase, the model first generates an initial output based solely on the unconstrained instruction portion i:

[0096] y init =Generate(i);

[0097] Where Generate(i) means that the expert model generates the initial output sequence y based solely on instruction i. init The initial output result y initUnaffected by constraints, this effectively expands the solution space explored by the model, avoiding biases introduced by constraints and ensuring greater flexibility in the initial generation phase. It is important to emphasize that this process requires generation step-by-step according to a window length, with subsequent adaptive decoding, comparative decoding, and window length adjustment performed after each window is generated.

[0098] The key idea of ​​instruction decoupling is to first generate unconstrained initial outputs, and then process the constraints separately in subsequent stages, thereby effectively separating the exploration of the model from the constraint verification, and significantly improving the generation quality and efficiency of the model.

[0099] 2. Adaptive decoding of constraint-sensitive tokens

[0100] After completing instruction decoupling and obtaining preliminary prediction output, this embodiment further proposes an adaptive estimation method based on constraint-sensitive tokens to determine the sensitivity of each token to constraints. For tokens identified as constraint-sensitive, we will use comparative decoding for refinement; while for non-sensitive tokens, we will directly use the initially generated results.

[0101] Specifically, the core idea of ​​this embodiment is that constraint-sensitive tokens exhibit a greater difference in their predicted probability distribution when predicted using hints with and without constraints; conversely, constraint-insensitive tokens show a smaller difference in their predicted probability distributions in both cases. Therefore, we quantitatively measure the constraint sensitivity of each token by calculating the Jensen-Shannon (JS) divergence between the predicted probability distributions of tokens generated with and without constraints.

[0102] The formula for calculating JS divergence is:

[0103]

[0104] Where P represents the predicted probability distribution of a word under constrained conditions, and Q represents the predicted probability distribution of a word under unconstrained conditions; D KL This represents the Kullback-Leibler divergence.

[0105] For each generated token, let y be the denoted y. i Calculate the prediction probability distribution P under unconstrained cues. U (y i ) and the predicted probability distribution P with constraints C (y i JS divergence between )

[0106] Then, a predefined threshold τ is set, and the token's constraint sensitivity is determined based on this divergence value:

[0107]

[0108] The above formula shows that if the JS divergence exceeds the threshold τ, it indicates that the token is sensitive to constraints and needs to be compared, decoded and refined; otherwise, the preliminary prediction results without constraints can be used directly, thereby significantly improving computational efficiency and ensuring generation quality.

[0109] The constraint-sensitive token adaptive estimation module proposed in this invention achieves fine-grained allocation of computing resources by accurately judging the constraint sensitivity of each token, thereby effectively improving the inference efficiency and generation quality of the model in constraint instruction compliance tasks.

[0110] 3. Dynamic window speculation comparison decoding mechanism

[0111] To further improve the efficiency of model inference, this embodiment proposes a dynamic window speculative comparison decoding mechanism. Specifically, this module first generates multiple tokens (with a window size of K) at once using unconstrained hints, then performs parallel constraint sensitivity evaluation on these tokens using constrained hints, and finally dynamically adjusts the window size to optimize inference efficiency based on the distribution of sensitive tokens within the window.

[0112] The specific process is as follows:

[0113] First, the model predicts a sequence of tokens with an unconstrained cue window of size K:

[0114] y1,y2,...,y K =Generate(i);

[0115] Next, based on the above calculation results, the following processing will be performed:

[0116] If there are no constraint-sensitive tokens in the window (the JS divergence of all tokens is less than or equal to the threshold τ), then the current window generation results are considered valid, all generated tokens are directly retained, and the window size K is doubled to accelerate the subsequent generation process.

[0117] If a constraint-sensitive token exists within the window (at least one token's JS divergence exceeds the threshold τ), then all tokens preceding the first constraint-sensitive token are retained, and only the first sensitive token is refined using contrastive decoding. After refinement, the window size K is halved to more finely verify the constraint sensitivity of subsequent tokens.

[0118] The formula for dynamically adjusting the window size is as follows:

[0119]

[0120] The key idea behind the dynamic window mechanism is that constraint-sensitive tokens in the actual generated sequence are usually not uniformly distributed, but rather sparsely distributed. Therefore, by adaptively adjusting the window size, this embodiment can effectively reduce unnecessary computation on non-sensitive tokens and concentrate computational resources on sensitive regions, thereby significantly improving overall inference efficiency while ensuring that the generated results meet the constraints.

[0121] Through the aforementioned dynamic window speculative comparison decoding mechanism, this embodiment achieves dynamic optimization between inference efficiency and generation accuracy, effectively overcoming the inefficiency of traditional comparison decoding methods and greatly improving the performance of practical applications.

[0122] To further clarify the execution flow of this embodiment, the entire "adaptive comparison decoding" process is described in pseudocode. This algorithm comprehensively implements the decoupling of instructions and constraints, adaptive identification of constraint-sensitive tokens, and a dynamic window speculative comparison decoding mechanism, effectively improving the inference efficiency of the model under complex constraints while ensuring generation quality. The specific algorithm flow is as follows:

[0123]

[0124]

[0125] This embodiment decouples the decoding method of instructions and constraints, effectively reducing the interference of constraints on the initial generation stage of the model; the proposed method of dynamically identifying constraint-sensitive tokens based on JS divergence enables the model to adaptively judge the constraint sensitivity and avoid redundant calculation of non-sensitive tokens; the proposed dynamic window speculative comparison decoding mechanism can dynamically adjust the window size according to the actual generation situation, significantly improving the inference efficiency of the model, while ensuring the constraint compliance quality of the generated results.

[0126] Corresponding to the above method, the present invention also provides an apparatus / system including a computer device, the computer device including a processor and a memory, the memory storing computer instructions, the processor executing the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the apparatus / system performs the steps of the method as described above.

[0127] This invention also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, implements the steps of the aforementioned edge computing server deployment method. The computer-readable storage medium can be a tangible storage medium, such as random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, floppy disks, hard disks, removable storage disks, CD-ROMs, or any other form of storage medium known in the art.

[0128] In summary, the large language model constraint instruction compliance method and apparatus based on adaptive contrastive decoding of this invention decouples instructions and constraints in user queries using a large model. During inference, it focuses on the instruction part, avoiding interference from complex constraints in the initial stage of language model generation. The constraint part is implemented through post-processing in the decoding stage, improving the flexibility and accuracy of model generation. By evaluating the difference in the predicted probability distribution of each token under constrained and unconstrained conditions, it adaptively identifies constraint-sensitive tokens and performs targeted contrastive decoding. Based on a dynamic window speculative contrastive decoding mechanism, the model adaptively adjusts the window size according to the distribution of constraint-sensitive tokens in the generated sequence to achieve precise allocation of computational resources to sensitive regions, significantly improving the model's inference efficiency and generation quality.

[0129] Those skilled in the art will understand that the exemplary components, systems, and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software, or a combination of both. Whether implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention. When implemented in hardware, it can be, for example, electronic circuits, application-specific integrated circuits (ASICs), appropriate firmware, plug-ins, function cards, etc. When implemented in software, the elements of this invention are programs or code segments used to perform the desired tasks. The programs or code segments can be stored in a machine-readable medium or transmitted over a transmission medium or communication link via data signals carried in a carrier wave.

[0130] It should be clarified that the present invention is not limited to the specific configurations and processes described above and shown in the figures. For the sake of brevity, detailed descriptions of known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown. Those skilled in the art can make various changes, modifications, and additions, or change the order of steps, after understanding the spirit of the present invention.

[0131] In this invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with or in place of features of other embodiments.

[0132] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, various modifications and variations of the embodiments of the present invention are possible. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A large language model constraint instruction following method based on adaptive contrast decoding, characterized in that, The method comprises the following steps: The original prompt is decoupled to obtain a task instruction part and a constraint condition part; An initial output sequence is generated by an expert model based on the task instruction part under unconstrained conditions and according to a preset initial length window; the following operations are performed on the initial output sequence of each round: An auxiliary model is used to calculate the prediction probability distribution of the word units in the initial output sequence under constrained and unconstrained conditions, and the difference between the prediction probability distributions of the word units under constrained and unconstrained conditions is calculated; the word units with a difference greater than a set standard are marked as constraint-sensitive word units, and the word units with a difference less than or equal to the set standard are marked as constraint-insensitive word units; the parameter size of the auxiliary model is smaller than that of the expert model; If the word units of the current initial output sequence only contain constraint-insensitive word units, the word units of the current initial output sequence are output to a target output sequence, and the window length is expanded by a first set proportion; if the word units of the current initial output sequence contain constraint-sensitive word units, the word units before the first constraint-sensitive word unit are output to the target output sequence, the first constraint-sensitive word unit is modified based on the expert model and the auxiliary model based on a comparative decoding form and output to the target output sequence, and the window length is reduced by a second set proportion. The complete target output sequence is obtained by performing multiple rounds of operations based on the dynamically changing window length, and the target output sequence is output.

2. The method of claim 1, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction. The difference between the prediction probability distributions of the word units under constrained and unconstrained conditions is calculated, the word units with a difference greater than a set standard are marked as constraint-sensitive word units, and the word units with a difference less than or equal to the set standard are marked as constraint-insensitive word units, comprising: The JS divergence of the prediction probability distributions of the word units under constrained and unconstrained conditions is calculated, the word units with a JS divergence greater than a preset value are marked as constraint-sensitive word units, and the word units with a JS divergence less than or equal to a preset value are marked as constraint-insensitive word units.

3. The method of claim 2, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction, and wherein the method is characterized by: The calculation formula of the JS divergence is: where P denotes the predicted probability distribution of the wordpiece under constraints, Q denotes the predicted probability distribution of the wordpiece without constraints; D KL denotes the Kullback-Leibler divergence.

4. The method of claim 2, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction, and wherein the method is characterized by: The first constraint-sensitive word unit is modified based on the expert model and the auxiliary model based on a comparative decoding form, comprising: The expert model and the auxiliary model perform comparative scoring on multiple candidate word units at each step, and the calculation formula is: S imp (x i |x ＜i )=(1+α)S e (x i |x ＜i )-αS a (x i |x ＜i ); wherein S e (x i |x ＜i ) represents logits values of the i-th candidate word piece by the expert model; S a (x i |x ＜i ) represents logits values of the i-th candidate word piece by the auxiliary model; and a represents a hyperparameter, and the value range of a is 0.5-2. The candidate word with the highest score is output and replaces the first constraint-sensitive word unit.

5. The method of claim 1, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction, and wherein the method is characterized by: The window length is expanded by a first set proportion, comprising: A first density of constraint-sensitive word units in the window in a set number of rounds before the current time is calculated, and the corresponding first set proportion is configured based on a plurality of preset first density intervals to expand the length of the window for the next round of operation; The window length is reduced by a second set proportion, comprising: A second density of constraint-sensitive word units in the window in the current round is calculated, and the corresponding second set proportion is configured based on a plurality of preset second density intervals to reduce the length of the window for the next round of operation.

6. The method of claim 1, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction, and wherein the method is characterized by: The method further comprises: If the first constraint-sensitive token is modified based on the contrastive decoding form based on the expert model and the auxiliary model, and the first constraint-sensitive token is repeatedly output, an external database is called to generate a replacement token of the first constraint-sensitive token based on retrieval augmentation.

7. The method of claim 1, wherein the method is based on an adaptive contrast decoding of a large language model constraint following instruction, and wherein the method is characterized by, The method further comprises recording the original token, the modification process and the final token of each constraint-sensitive token, establishing a modification log and adding an index for tracing.

8. A large language model constraint instruction following device based on adaptive contrast decoding, comprising a processor, a memory and a computer program / instruction stored on the memory, characterized in that, The processor is configured to execute the computer program / instructions, and when the computer program / instructions are executed, the apparatus implements the steps of the method according to any one of claims 1 to 7.

9. A computer readable storage medium having stored thereon computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

10. A computer program product comprising computer programs / instructions, characterized in that, The computer program / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7. The computer program / instructions, when executed by the processor, implement the steps of the method according to any one of claims 1 to 7.

Images