An Improved Context-Aware Adaptive Contrast Decoding Method

Abstract

This invention discloses an improved context-aware adaptive contrastive decoding method, relating to the field of artificial intelligence technology; it includes: for relevant search content obtained by the retrieval enhancement generation RAG framework, using the DC-ACD method to perform adaptive contrastive decoding management of dynamic context weights on the relevant search content: Step 1: Establish a dynamic weight calculation model for contextual semantic relevance, ensuring that search content with higher semantic similarity is assigned higher weights; Step 2: Perform multi-document fusion; Step 3: Evaluate context quality; Step 4: Calculate token-level dynamic weights based on whether it relies on prior knowledge of the intelligent language model and / or context information; Step 5: Analyze the final decoding probability distribution: synthesize the weight ratios of each part to obtain the final decoding probability distribution.

Description

Technical Field

[0001] This invention discloses an improved context-aware adaptive contrastive decoding method, which relates to the field of artificial intelligence technology. Background Technology

[0002] Retrieval-Augmented Generation (RAG) technology has made significant progress in recent years, becoming an effective method to improve the performance of Large Language Models (LLMs). By incorporating external knowledge bases, RAGs enable models to access broader and more up-to-date information, thereby generating more accurate and relevant content. In practical applications, the retrieved contextual information often contains noise or irrelevant content, negatively impacting the quality of the generated text and hindering text optimization. Summary of the Invention

[0003] This invention addresses the problems of existing technologies by providing an improved context-aware adaptive contrastive decoding method that effectively balances the prior knowledge and contextual information of the text generation model, achieving a more accurate and flexible text generation process.

[0004] The specific solution proposed in this invention is as follows:

[0005] This invention provides an improved context-aware adaptive contrastive decoding method, comprising:

[0006] For the relevant search content obtained by the search enhancement generation RAG framework, the DC-ACD method is used to perform adaptive comparative decoding management of dynamic context weights on the relevant search content:

[0007] Step 1: Establish a dynamic weight calculation model for contextual semantic relevance. This model ensures that retrieved content with higher semantic similarity is assigned a higher weight in the context.

[0008] Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete contextual information.

[0009] Step 3: Evaluate Context Quality: Introduce a context quality evaluation network to assess the quality of the retrieved content's context, giving greater weight to higher-quality context.

[0010] Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model.

[0011] Step 5: Analyze the final decoding probability distribution: Combine the weight ratios of each part to obtain the final decoding probability distribution.

[0012] Furthermore, in step 1 of the improved context-aware adaptive contrastive decoding method, a dynamic weight calculation model for contextual semantic relevance is established, as shown in the following formula:

[0013] α DC-ACD (t)=λ·α ACD (t)+(1-λ)·sim(q,c)

[0014] Where: α DC-ACD (t) represents the dynamic weight at time t, λ·α ACD (t) represents the weights calculated by the original ACD method at time t, sim(q,c) represents the semantic similarity between question q and context c, and λ is the balancing factor tuned through the validation set.

[0015] Furthermore, in step 2 of the improved context-aware adaptive contrastive decoding method, the formula is used:

[0016]

[0017] Multi-document fusion is performed, including: It is the merged context logit, w i This represents the weight of the i-th document. It is a logit based on the i-th document.

[0018] Furthermore, in step 3 of the improved context-aware adaptive contrastive decoding method, the formula is used:

[0019] α DC-ACD (t)=α DC-ACD (t)·σ(S(c))

[0020] To evaluate the quality of the context, S(c) is a two-layer feedforward network: S(c) = W2·ReLU(W1·c+b1)+b2, where W1, W2, b1, and b2 are learnable parameters, the network input c is the average word vector of the context or a vector representation obtained using a pre-trained model, and the network output q... score =S(c) is mapped to the interval (0,1) by the sigmoid function σ.

[0021] Furthermore, in step 4 of the improved context-aware adaptive contrastive decoding method, the formula is utilized:

[0022] α token (t)=f(e t ,h t c)

[0023] Calculate the dynamic weight at the token level, where e tIt is the token embedding at time t, h t is the hidden state at time t, c is the context representation, and f is a small neural network.

[0024] Furthermore, in step 5 of the improved context-aware adaptive contrastive decoding method, the formula is used:

[0025]

[0026] Obtain the final decoding probability distribution, where z t This is the logit output of the original intelligent language model, representing the model's prediction based on parameter knowledge, α. token (t) is the dynamic weight at the token level, α DC-ACD (t) is the dynamically calculated context weight. It is the merged context logit. This represents the additional contribution of contextual information relative to the prior knowledge of the intelligent language model.

[0027] The present invention also provides an improved context-aware adaptive contrast decoding device, including a decoding management module.

[0028] The decoding management module uses the DC-ACD method to perform adaptive comparative decoding management of the relevant search content obtained from the RAG framework for retrieval enhancement, based on dynamic context weights.

[0029] Step 1: Establish a dynamic weight calculation model for contextual semantic relevance. This model ensures that retrieved content with higher semantic similarity is assigned a higher weight in the context.

[0030] Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete contextual information.

[0031] Step 3: Evaluate Context Quality: Introduce a context quality evaluation network to assess the quality of the retrieved content's context, giving greater weight to higher-quality context.

[0032] Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model.

[0033] Step 5: Analyze the final decoding probability distribution: Combine the weight ratios of each part to obtain the final decoding probability distribution.

[0034] The present invention also provides a computer-readable medium storing computer instructions that, when executed by a processor, cause the processor to perform the improved context-aware adaptive contrastive decoding method.

[0035] The advantages of this invention are:

[0036] Refined context utilization: By combining semantic similarity and dynamic token-level weights, more refined and accurate utilization of contextual information is achieved.

[0037] Enhanced robustness: Multi-document fusion and context quality assessment mechanisms improve the model's robustness to noise and irrelevant information.

[0038] Flexible adaptability: Dynamic weight calculation enables the model to flexibly respond to different types and qualities of context.

[0039] Improved information utilization efficiency: The multi-document fusion mechanism makes full use of the rich information provided by the retrieval system.

[0040] Enhanced interpretability: The interpretability of model decisions is improved through explicit context quality assessment and token-level weights.

[0041] The DC-ACD method in this invention, namely Dynamic Context-Weighted Adaptive Contrastive Decoding (DC-ACD), is applicable to various natural language processing tasks that require the integration of external knowledge, especially when facing potential noise or incomplete context, such as open-domain question answering, knowledge-intensive dialogue systems, fact checking and verification, and personalized content generation. Attached Figure Description

[0042] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0043] The present invention will be further described below with reference to the accompanying drawings and specific embodiments, so that those skilled in the art can better understand and implement the present invention. However, the embodiments described are not intended to limit the present invention.

[0044] Example 1

[0045] This invention provides an improved context-aware adaptive contrastive decoding method, comprising:

[0046] For the relevant search content obtained by the search enhancement generation RAG framework, the DC-ACD method is used to perform adaptive comparative decoding management of dynamic context weights on the relevant search content:

[0047] Step 1: Establish a dynamic weight calculation model for context semantic relevance, and ensure that the retrieval content context with higher semantic similarity is assigned a higher weight through the dynamic weight calculation model for context semantic relevance.

[0048] Among them, the formula for establishing the dynamic weight calculation model for context semantic relevance is as follows:

[0049] α DC-ACD α(t) = λ · α(t) + (1 - λ) · sim(q, c) ACD

[0050] Among them: α(t) represents the dynamic weight at time t, λ · α(t) is the weight calculated by the original ACD method at time t, sim(q, c) represents the semantic similarity between the question q and the context c, and λ is the balance factor tuned through the validation set. DC-ACD ACD

[0051] In specific implementation, the calculation of λ · α(t) follows the following method and is calculated based on entropy value. The entropy value calculation process of the ACD method is as follows: ACD

[0052] For the given input x and the previously generated sequence y<t, calculate the output probability distributions of the language model at time t in the cases with and without context c respectively: P(Y|x, y, c) and P(Y|x, y), where θ represents the parameters of the language model, Y represents the output token at time t, x is the input (usually the question in the question-answering task), y represents the sequence generated before time t, and c is the context information. θ (Y t |x,y <t ,c) and P θ (Y<00,semantic similarity between the question q and the context c, and λ is the balance factor tuned through the validation set. DC-ACD α(t) represents the dynamic weight at time t, λ · α(t) is the weight calculated by the original ACD method at time t, sim(q, c) represents the semantic similarity between the question q and the context c, and λ is the balance factor tuned through the validation set. ACD

[0051] In specific implementation, the calculation of λ · α(t) follows the following method and is calculated based on entropy value. The entropy value calculation process of the ACD method is as follows: s the semantic similarity between the question q and the context c, and λ is the balance factor tuned through the validation set. DC-ACD α(t) represents the dynamic weight at time t, λ · α(t) is the weight calculated by the original ACD method at time t, sim(q, c) represents the semantic similarity between the question q and the context c, and λ is the balance factor tuned through the validation set. ACD

[0051] In specific implementation, the calculation of λ · α(t) follows the following method and is calculated based on entropy value. The entropy value calculation process of the ACD method is as follows: ACD

[0052] For the given input x and the previously generated sequence y<t, calculate the output probability distributions of the language model at time t in the cases with and without context c respectively: P(Y|x, y, c) and P(Y|x, y), where θ represents the parameters of the language model, Y represents the output token at time t, x is the input (usually the question in the question-answering task), y represents the sequence generated before time t, and c is the context information. θ (Y t |x,y <t ,c) and P θ (Y t |x,y <t ),θ表示语言模型的参数,Y表示t时刻的输出token,x是输入(在问答任务中通常是问题),y表示t时刻之前已生成的序列,c是上下文信息。 t represents the output token at time t, x is the input (usually the question in the question-answering task), y represents the sequence generated before time t, and c is the context information. <t

[0053] Calculate the entropy of these two probability distributions: H(Y|c) = -∑P(Y|x, y, c) · logP(Y|x, y, c) and H(Y) = -∑P(Y|x, y) · logPθYt|x, y<t, where the summation is performed over all tokens in the vocabulary. t |c) = -∑P θ (Y t |x,y <t ,c) · logP θ (Y t |x,y <t ,c) and H(Y t ) = -∑P θ (Y t |x,y <t )·logPθYt|x,y<t, where the summation is performed over all tokens in the vocabulary.

[0054] Calculate the entropy difference: ΔH = H(Y) t )-H(Y t |c)

[0055] Using the sigmoid function to map the entropy difference to the (0,1) interval, we obtain α. ACD (t): This α ACD (t) represents the weights calculated at time t. It reflects the contribution of contextual information to reducing model uncertainty. When ΔH is large, it indicates that the context significantly reduces model uncertainty, and in this case, α ACD (t) is close to 1; otherwise, it is close to 0.

[0056] sim(q, c) can be calculated using pre-trained semantic matching models such as BERT and RoBERTa. The calculation steps are as follows:

[0057] 1. Input the question q and the context c into the pre-trained model to obtain their vector representations.

[0058] 2. Calculate the cosine similarity between the two vectors as the value of sim(q, c).

[0059] To better understand the effect of this formula, suppose there is a question, "Who invented the telephone?", and the retrieval system returns two pieces of context:

[0060] c1: "Alexander Graham Bell invented the telephone in 1876."

[0061] c2: "Thomas Edison invented many important electronic devices."

[0062] Using the semantic matching model, we calculated sim(q, c1) = 0.85 and sim(q, c2) = 0.60. Assuming λ = 0.7 and α... ACD If (t) = 0.5, then:

[0063] For c1, α DC-ACD (t)=0.7·0.5+0.3·0.85=0.605,

[0064] For c2, α DC-ACD (t)=0.7·0.5+0.3·0.60=0.530,

[0065] The above example demonstrates that even in two contexts, α ACD Although the values ​​of c(t) are the same, c1 will receive a greater weight because it has a higher semantic similarity to the question. This method ensures that more relevant information can play a greater role in the generation process.

[0066] Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete information context.

[0067] The formula used is:

[0068]

[0069] Multi-document fusion is performed, including: It is the merged context logit, w i This represents the weight of the i-th document. It is a logit based on the i-th document.

[0070] Specifically:

[0071] a) Document weight w i It can be determined based on the relevance score returned by the retrieval system, for example, by using the softmax function to normalize the relevance score: Where exp represents the softmax function, and score i This represents the relevance score of the i-th document. The relevance score can be calculated based on the calculation logic of traditional search engines.

[0072] b) For each document, calculate the ACD score.

[0073] c) Finally, the logits of all documents are weighted and summed to obtain the merged result.

[0074] For example, suppose the retrieval system returns three documents with relevance scores of 10, 8, and 6. First, calculate the weights:

[0075]

[0076] Then, assuming that at a certain time step t, the logit values ​​for these three documents are as follows: The merged logit is:

[0077]

[0078] This fusion method can comprehensively consider information from multiple related documents, improving the comprehensiveness and accuracy of information utilization, while enhancing the system's robustness to noise and incomplete information.

[0079] Step 3: Evaluate context quality: Introduce a context quality evaluation network to evaluate the context quality of the retrieved content, giving greater weight to higher quality contexts.

[0080] The formula used is:

[0081] α DC-ACD (t)=α DC-ACD (t)·σ(S(c))

[0082] To evaluate the quality of the context, S(c) is a two-layer feedforward network: S(c) = W2·ReLU(W1·c+b1)+b2, where W1, W2, b1, and b2 are learnable parameters, the network input c is the average word vector of the context or a vector representation obtained using a pre-trained model, and the network output q... score =S(c) is mapped to the interval (0,1) by the sigmoid function σ.

[0083] For example, suppose we consider the previous two contexts:

[0084] c1: "Alexander Graham Bell invented the telephone in 1876."

[0085] c2: "Thomas Edison invented many important electronic devices."

[0086] The outputs of the quality assessment network are: q score 1 = 0.85, q score If 2 = 0.60, then the final weight adjustment is:

[0087] For c1, α DC-ACD (t) = 0.605·sigmod(0.85) ≈ 0.605·0.701 ≈ 0.424; for c2, α DC-ACD (t)=0.530·sigmod(0.60)≈0.530·0.646≈0.342.

[0088] This indicates that the higher-quality context c1 receives greater weight, while the influence of the lower-quality context c2 is further reduced. In this way, the impact of high-quality information can be further highlighted, while the interference from low-quality or irrelevant information can be reduced.

[0089] Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model.

[0090] The formula used is:

[0091] α token (t)=f(e t ,h t c)

[0092] Calculate the dynamic weight at the token level, where et is the token embedding at time t, h t is the hidden state at time t, c is the context representation, and f is a small neural network.

[0093] The specific process is as follows:

[0094] a) f can be a multi-layer perceptron (MLP), for example:

[0095] f(e t , h t , c) = sigmod(W2·ReLU(W1·[e t ; h t ; c] + b1) + b2), where [e t ; h t ; c] means concatenating these three vectors,

[0096] b) The output of the network is mapped to the interval (0, 1) through the sigmoid function σ, representing the degree of dependence of the current token on the context.

[0097] c) In the training stage, the f network can be trained in an end-to-end manner to learn the degree of dependence of different tokens on the context. For example, assuming that when generating the sentence "Alexander Graham Bell invented the telephone":

[0098] For the token "Alexander", α token (t) may be close to 1 because it highly depends on context information.

[0099] For the common grammar word "了", α token (t) may be close to 0 because it mainly depends on the language knowledge of the model.

[0100] For the word "telephone", α token (t) may be around 0.5 because it depends on both context and the prior knowledge of the model. This fine-grained weight adjustment enables the model to more precisely utilize context information, thereby generating more accurate and coherent text.

[0101] Step 5: Analyze the final decoding probability distribution: Integrate the weight ratios of each part to obtain the final decoding probability distribution.

[0102] Among them, using the formula:

[0103]

[0104] Obtain the final decoding probability distribution, where z t is the logit output of the original intelligent language model, representing the prediction of the model based on parameter knowledge, αtoken (t) is the dynamic weight at the token level, α DC-ACD (t) is the dynamically calculated context weight. It is the merged context logit. This represents the additional contribution of contextual information relative to the prior knowledge of the intelligent language model.

[0105] In this way, the DC-ACD method can dynamically balance the prior knowledge and contextual information of the model at the token level, achieving a more accurate decoding process.

[0106] Example 2

[0107] The present invention also provides an improved context-aware adaptive contrast decoding device, including a decoding management module.

[0108] The decoding management module uses the DC-ACD method to perform adaptive comparative decoding management of the relevant search content obtained from the RAG framework for retrieval enhancement, based on dynamic context weights.

[0109] Step 1: Establish a dynamic weight calculation model for contextual semantic relevance. This model ensures that retrieved content with higher semantic similarity is assigned a higher weight in the context.

[0110] Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete contextual information.

[0111] Step 3: Evaluate Context Quality: Introduce a context quality evaluation network to assess the quality of the retrieved content's context, giving greater weight to higher-quality context.

[0112] Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model.

[0113] Step 5: Analyze the final decoding probability distribution: Combine the weight ratios of each part to obtain the final decoding probability distribution.

[0114] The information interaction and execution process between the modules in the above-mentioned device are based on the same concept as the method embodiment of the present invention, and the specific details can be found in the description in the method embodiment of the present invention, and will not be repeated here.

[0115] Similarly, the advantages of the device of the present invention are:

[0116] Refined context utilization: By combining semantic similarity and dynamic token-level weights, more refined and accurate utilization of contextual information is achieved.

[0117] Enhanced robustness: Multi-document fusion and context quality assessment mechanisms improve the model's robustness to noise and irrelevant information.

[0118] Flexible adaptability: Dynamic weight calculation enables the model to flexibly respond to different types and qualities of context.

[0119] Improved information utilization efficiency: The multi-document fusion mechanism makes full use of the rich information provided by the retrieval system.

[0120] Enhanced interpretability: The interpretability of model decisions is improved through explicit context quality assessment and token-level weights.

[0121] It should be noted that not all steps and modules in the above processes and device structures are mandatory; some steps or modules can be omitted as needed. The execution order of each step is not fixed and can be adjusted as required. The system structure described in the above embodiments can be a physical structure or a logical structure. That is, some modules may be implemented by the same physical entity, or some modules may be implemented by multiple physical entities, or they may be jointly implemented by certain components in multiple independent devices.

[0122] Example 3

[0123] The present invention also provides a computer-readable medium storing computer instructions, which, when executed by a processor, cause the processor to perform the improved context-aware adaptive contrastive decoding method. Specifically, a system or apparatus equipped with a storage medium storing software program code that implements the functions of any of the embodiments described above, and enabling the computer (or CPU or MPU) of the system or apparatus to read and execute the program code stored in the storage medium.

[0124] In this case, the program code read from the storage medium can itself implement the function of any of the above embodiments, and therefore the program code and the storage medium storing the program code constitute part of the present invention.

[0125] Examples of storage media used to provide program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, program code can be downloaded from a server computer via a communication network.

[0126] Furthermore, it should be clear that not only can the program code read by the computer be executed, but also the operating system or other components operating on the computer can be instructed based on the program code to perform some or all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0127] Furthermore, it is understood that the program code read from the storage medium is written to the memory set in the expansion board inserted into the computer or to the memory set in the expansion unit connected to the computer. Then, based on the instructions of the program code, the CPU or other components installed on the expansion board or expansion unit execute some and all of the actual operations, thereby realizing the function of any of the embodiments described above.

[0128] The above-described embodiments are merely preferred embodiments provided to fully illustrate the present invention, and the scope of protection of the present invention is not limited thereto. Equivalent substitutions or modifications made by those skilled in the art based on the present invention are all within the scope of protection of the present invention. The scope of protection of the present invention is defined by the claims.

Claims

1. An improved context-aware adaptive contrastive decoding method, characterized by: include: For the relevant search content obtained by the search enhancement generation RAG framework, the DC-ACD method is used to perform adaptive comparative decoding management of dynamic context weights on the relevant search content: Step 1: Establish a dynamic weight calculation model for contextual semantic relevance. This model ensures that retrieved content with higher semantic similarity is assigned higher weights based on its context. The formula for establishing this dynamic weight calculation model is as follows: a DC-ACD (t)=λ·a ACD (t)+(1-λ)·sim(q,c) Where: α DC-ACD (t) represents the dynamic weight at time t, λ·α ACD (t) represents the weights calculated by the original ACD method at time t, sim(q,c) represents the semantic similarity between question q and context c, and λ is the balance factor optimized through the validation set. Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete contextual information. Step 3: Evaluate Context Quality: Introduce a context quality evaluation network to assess the quality of the retrieved content's context, assigning greater weight to higher-quality contexts. This is achieved using the formula: a DC-ACD (t)=a DC-ACD (t)·σ(S(c)) To evaluate the context quality, S(c) is a two-layer feedforward network: S(c) = W2·ReLU(W1·c+b1)+b2, where W1, W2, b1, and b2 are... The learnable parameters are: the network input c is the average of the context word vectors or a vector representation obtained using a pre-trained model; and the network output q. score =S(c) is mapped to the interval (0,1) by the sigmoid function σ; Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model. Step 5: Analyze the final decoding probability distribution: By combining the weight ratios of each part, the final decoding probability distribution is obtained, using the formula: Obtain the final decoding probability distribution, where z t This is the logit output of the original intelligent language model, representing the model's prediction based on parameter knowledge, α. token (t) is the dynamic weight at the token level, α DC-ACD (t) is the dynamically calculated context weight. It is the merged context logit. This represents the additional contribution of contextual information relative to the prior knowledge of the intelligent language model.

2. The improved context-aware adaptive contrastive decoding method according to claim 1, characterized in that: Step 2 uses the formula: Multi-document fusion is performed, including: It is the merged context logit, w i This represents the weight of the i-th document. It is a logit based on the i-th document.

3. An improved context-aware adaptive contrastive decoding method according to claim 1, characterized in that: The formula used in step 4 is: α token (t)=f(e t ,h t ,c) Calculate the dynamic weight at the token level, where e t It is the token embedding at time t, h t is the hidden state at time t, c is the context representation, and f is a small neural network.

4. An improved context-aware adaptive contrastive decoding device, characterized in that: Including the decoding management module, The decoding management module uses the DC-ACD method to perform adaptive comparative decoding management of the relevant search content obtained from the RAG framework for retrieval enhancement, based on dynamic context weights. Step 1: Establish a dynamic weight calculation model for contextual semantic relevance. This model ensures that retrieved content with higher semantic similarity is assigned higher weights based on its context. The formula for establishing this dynamic weight calculation model is as follows: a DC-ACD (t)=λ·a ACD (t)+(1-λ)·sim(q,c) Where: α DC-ACD (t) represents the dynamic weight at time t, λ·α ACD (t) represents the weights calculated by the original ACD method at time t, sim(q,c) represents the semantic similarity between question q and context c, and λ is the balance factor optimized through the validation set. Step 2: Perform multi-document fusion: Utilize the top k documents of relevant search content to perform multi-document fusion based on relevance scores. Combining multi-document information can compensate for incomplete contextual information. Step 3: Evaluate Context Quality: Introduce a context quality evaluation network to assess the quality of the retrieved content's context, assigning greater weight to higher-quality contexts. This is achieved using the formula: a DC-ACD (t)=a DC-ACD (t)·σ(S(c)) To evaluate the context quality, S(c) is a two-layer feedforward network: S(c) = W2·ReLU(W1·c+b1)+b2, where W1, W2, b1, and b2 are learnable parameters, the network input c is the average of the context word vectors or a vector representation obtained using a pre-trained model, and the network output q... score =S(c) is mapped to the interval (0,1) by the sigmoid function σ; Step 4: Calculate the dynamic weights at the token level based on whether or not the system relies on prior knowledge and / or contextual information from the intelligent language model. Step 5: Analyze the final decoding probability distribution: By combining the weight ratios of each part, the final decoding probability distribution is obtained, using the formula: Obtain the final decoding probability distribution, where z t This is the logit output of the original intelligent language model, representing the model's prediction based on parameter knowledge, α. token (t) is the dynamic weight at the token level, α DC-ACD (t) is the dynamically calculated context weight. It is the merged context logit. This represents the additional contribution of contextual information relative to the prior knowledge of the intelligent language model.

5. A computer-readable medium, characterized in that... The computer-readable medium stores computer instructions that, when executed by a processor, cause the processor to perform an improved context-aware adaptive contrastive decoding method according to any one of claims 1 to 3.

Images