A training method, platform, device, medium, and product for diffusion language models.

Abstract

This specification provides a training method, platform, device, medium, and product for a diffusion language model. The solution includes: acquiring multiple intermediate prediction results generated by an initial diffusion language model for sample prompt words; determining a reward value for each intermediate prediction result; and adjusting the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model. This solution effectively avoids the drawback of a mismatch between the reward and the contribution corresponding to the intermediate prediction results, enabling refined hierarchical control of the model optimization process, thereby improving the overall training effect of the model and enhancing the inference performance of the trained diffusion language model.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of artificial intelligence technology, and in particular to a training method, platform, device, medium, and product for a diffusion language model. Background Technology

[0002] Diffusion language models, with their advantages of generating text with strong semantic coherence and content controllability, have been widely applied in diverse task scenarios such as content creation, code generation, multimodal document understanding, and complex logical reasoning. However, diffusion language models built using existing training methods still have room for improvement in their overall performance when adapting to various real-world tasks, making it difficult to meet the high performance standards required by various industries.

[0003] Therefore, an efficient model training method is needed to meet the high performance requirements of various industries. Summary of the Invention

[0004] In view of this, embodiments of this specification provide a training method for a diffusion language model. One or more embodiments of this specification also relate to a task processing method, a model training platform, a training device for a diffusion language model, a task processing device, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0005] According to a first aspect of the embodiments of this specification, a training method for a diffusion language model is provided, comprising:

[0006] Obtain multiple intermediate prediction results generated by the initial diffusion language model for sample prompt words; the intermediate prediction results are the prediction results generated by performing a denoising operation during the process of the initial diffusion language model generating the final prediction result.

[0007] For any one of the plurality of intermediate prediction results, determine the reward value of that intermediate prediction result;

[0008] The parameters of the initial diffusion language model are adjusted based on the reward value to obtain the trained diffusion language model.

[0009] According to a second aspect of the embodiments of this specification, a task processing method is provided, comprising:

[0010] Obtain tasks to be processed, including text understanding and generation tasks or multimodal semantic understanding tasks;

[0011] The task to be processed is input into the diffusion language model to obtain the processing result for the task to be processed. The diffusion language model is a model generated according to the training method for the diffusion language model provided in the first aspect above.

[0012] According to a third aspect of the embodiments of this specification, a method for processing a content generation task is provided, comprising:

[0013] Obtain content generation tasks to be processed; the content generation tasks to be processed include at least one of code completion tasks, text generation tasks, image generation tasks, audio generation tasks, and video generation tasks;

[0014] The task to be processed is input into the diffusion language model to obtain task generation result information; the diffusion language model is a model generated according to a training method for diffusion language models provided in the first aspect above.

[0015] According to a fourth aspect of the embodiments of this specification, a model training platform is provided, including a model training unit and a response unit;

[0016] The model training unit is used to train the initial diffusion language model according to the training method for diffusion language models provided in the first aspect above, so as to obtain the diffusion language model.

[0017] The response unit is used to output the diffusion language model.

[0018] According to a fifth aspect of the embodiments of this specification, a training apparatus for a diffusion language model is provided, comprising:

[0019] The prediction result acquisition module is used to acquire multiple intermediate prediction results generated by the initial diffusion language model for sample prompt words; the intermediate prediction results are the prediction results generated by performing a denoising operation during the process of the initial diffusion language model generating the final prediction result.

[0020] The reward value determination module is used to determine the reward value of any intermediate prediction result among the plurality of intermediate prediction results.

[0021] The parameter adjustment module is used to adjust the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model.

[0022] According to a sixth aspect of the embodiments of this specification, a task processing apparatus is provided, comprising:

[0023] The pending task acquisition module is used to acquire pending tasks, which include text understanding and generation tasks or multimodal semantic understanding tasks.

[0024] The task input module is used to input the task to be processed into the diffusion language model to obtain the processing result for the task to be processed. The diffusion language model is a model generated according to the training method for diffusion language models provided in the first aspect above.

[0025] According to a seventh aspect of the embodiments of this specification, a processing apparatus for a content generation task is provided, comprising:

[0026] The content generation task acquisition module is used to acquire content generation tasks to be processed; the content generation tasks to be processed include at least one of code completion tasks, text generation tasks, image generation tasks, audio generation tasks, and video generation tasks;

[0027] The content generation task input module is used to input the content generation task to be processed into the diffusion language model to obtain task generation result information; the diffusion language model is a model generated according to the training method for diffusion language models provided in the first aspect above.

[0028] According to an eighth aspect of the embodiments of this specification, a computing device is provided, comprising:

[0029] Memory and processor;

[0030] The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions. When the computer programs or instructions are executed by the processor, they implement the steps of the training method or task processing method for the diffusion language model described above.

[0031] According to a ninth aspect of the embodiments of this specification, an electronic device is provided, comprising:

[0032] The memory and processor are connected via a bus;

[0033] The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions. When the computer programs or instructions are executed by the processor, they implement the steps of the training method or task processing method for the diffusion language model described above.

[0034] According to a tenth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores a computer program or instructions which, when executed by a processor, implement the steps of the training method or task processing method for the diffusion language model described above.

[0035] According to the eleventh aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described training method or task processing method for a diffusion language model.

[0036] One embodiment of this specification implements a method for optimizing a model strategy based on each intermediate prediction result generated during the process of a diffusion language model generating the final prediction result. First, the reward value corresponding to the intermediate prediction result is calculated. Then, this reward value is compared with a preset threshold, and differentiated model strategy optimization operations are performed based on the comparison result. In this way, the generation probability of each intermediate prediction result can be accurately adjusted according to the actual situation of the model training process. This effectively avoids the drawback of a mismatch between the reward and the contribution corresponding to the intermediate prediction result, enabling refined, hierarchical control of the diffusion language model optimization process. This, in turn, improves the overall training effect of the diffusion language model and ultimately enhances the inference performance of the trained diffusion language model. Attached Figure Description

[0037] Figure 1 This is a schematic diagram illustrating an application scenario of a training method for a diffusion language model provided in one embodiment of this specification.

[0038] Figure 2 This is a flowchart illustrating a training method for a diffusion language model provided in one embodiment of this specification;

[0039] Figure 3 This is a schematic diagram of the generation and gradient update iteration process of a diffusion language model proposed in one embodiment of this specification;

[0040] Figure 4 This is a flowchart illustrating a task processing method provided in one embodiment of this specification;

[0041] Figure 5 This is a flowchart illustrating a content generation task processing method provided in one embodiment of this specification;

[0042] Figure 6 This is a flowchart illustrating a training method for a diffusion language model used for text generation, provided in one embodiment of this specification.

[0043] Figure 7 This is a schematic diagram of a model training platform provided in one embodiment of this specification;

[0044] Figure 8 This is a schematic diagram of the structure of a training device for a diffusion language model provided in one embodiment of this specification;

[0045] Figure 9 This is a schematic diagram of the structure of a task processing device provided in one embodiment of this specification;

[0046] Figure 10 This is a schematic diagram of the structure of a content generation task processing device provided in one embodiment of this specification;

[0047] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification;

[0048] Figure 12 This is a structural block diagram of an electronic device provided in one embodiment of this specification. Detailed Implementation

[0049] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0050] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0051] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0052] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in one or more embodiments of this specification are obtained through open-source datasets or public datasets that comply with their license agreements, or are obtained with full authorization from the relevant parties. Moreover, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0053] In one or more embodiments of this specification, a large model refers to a deep learning model with a large number of model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even tens of trillions of model parameters. A large model can also be called a foundation model. It is pre-trained using large-scale unlabeled corpora to produce a pre-trained model with hundreds of millions of parameters. Such models can adapt to a wide range of downstream tasks and have good generalization ability, such as Diffusion Large Language Models (DLLM) and multi-modal pre-training models.

[0054] In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before they can be applied to different tasks. Large models can be widely used in fields such as Natural Language Processing (NLP) and Computer Vision. Specifically, they can be applied to computer vision tasks such as Visual Question Answering (VQA), Image Captioning (IC), and Image Generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0055] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0056] The Diffusion Large Language Model (DLLM) is a non-autoregressive generative language model that achieves parallel text generation by iteratively denoising and remasking a fully masked sequence.

[0057] A lexical is the basic semantic unit when a diffusion language model processes text. It can correspond to a character, word, subword, or punctuation mark. It is the smallest unit for semantic encoding and decoding by the model. In practical applications, its English expression can be "token".

[0058] Entropy is a quantitative indicator that measures the degree of uncertainty of a random variable.

[0059] Lexical entropy refers to the uncertainty of a single lexical unit in its text sequence. The higher the entropy value, the lower the probability of the lexical unit appearing in the current sequence and the stronger the semantic randomness; the lower the entropy value, the higher the probability of the lexical unit appearing in the current sequence and the more stable the semantics.

[0060] Denoising refers to the optimization process by which a diffusion language model, based on contextual semantic association and the probability distribution of word generation, eliminates semantic noise, logical conflicts, or probabilistic uncertainties in the current word sequence.

[0061] Masking is a process that uses specific identifiers (such as the MASK symbol) to mask, hide, or mark certain words in the intermediate prediction results.

[0062] Fixed operation refers to the processing method of locking the semantic content and sequence position of target words in the intermediate prediction results. Its purpose is to prevent these words from being modified in subsequent iterations.

[0063] Currently, in the training process of diffusion language models, the reward value is usually calculated based on the final prediction result of the model, and this reward value is directly applied to all intermediate prediction results in the model generation process, so that all intermediate prediction results use this reward value as the guiding basis for optimization strategies.

[0064] When the reward value indicates that the probability of a predicted result should be increased, the generation probability of all intermediate predicted results will be increased simultaneously; conversely, if the reward value indicates that the probability of a predicted result should be decreased, the generation probability of all intermediate predicted results will be decreased simultaneously. This blanket reward allocation method makes it difficult to achieve fine-grained, layered control over the model optimization process, and will ultimately significantly weaken the inference performance of the post-trained diffusion language model.

[0065] To address the aforementioned technical problems, this specification provides a training method for a diffusion language model. This specification also relates to a task processing method, a model training platform, a training device for a diffusion language model, a task processing device, a computing device, an electronic device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0066] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0067] Figure 1 This is a schematic diagram illustrating an application scenario of a training method for a diffusion language model provided in one embodiment of this specification.

[0068] Considering the large number of parameters in diffusion language models and the limited computing resources of mobile terminals, the training method for diffusion language models provided in this application can be applied to, for example... Figure 1 The application scenarios shown are not limited to these. In, for example... Figure 1 In the application scenario shown, the diffusion language model can be deployed on server 10. Server 10 can connect to one or more client devices 20 via a local area network (LAN), wide area network (WAN), internet connection, or other types of data network. Client devices 20 may include, but are not limited to, smartphones, tablets, laptops, PDAs, personal computers, smart home devices, and in-vehicle devices. Client devices 20 can interact with users through a graphical user interface to train the diffusion language model, thereby implementing the method provided in the embodiments of this specification.

[0069] In one embodiment of this specification, the system comprising a client device and a server can perform the following steps: the client device receives sample prompts input by the user, and the server performs a training process for the diffusion language model based on the sample prompts. It should be noted that, provided the client device's operating resources can meet the deployment and operating conditions of the diffusion language model, this embodiment can be performed on the client device.

[0070] See Figure 2 , Figure 2 This is a flowchart of a training method for a diffusion language model provided in one embodiment of this specification. From a hardware perspective, the execution entity of this process can be a device used to train the diffusion language model, or from a software perspective, the execution entity of this process can be an application program installed on the device used to train the diffusion language model.

[0071] like Figure 2 As shown, the process may include the following steps.

[0072] Step 202: Obtain multiple intermediate prediction results generated by the initial diffusion language model for the sample prompt words.

[0073] The intermediate prediction results are generated during the denoising operation performed by the initial diffusion language model in the process of generating the final prediction results.

[0074] In one embodiment of this specification, the initial diffusion language model can be any type of model that requires pre-training, targeted fine-tuning training, or reinforcement learning training. A sample prompt word can refer to a training data point. The task corresponding to the training data can include text understanding and generation tasks or multimodal semantic understanding tasks. Text understanding tasks can refer to tasks that parse text semantics, extract key information, and determine text attributes. These tasks mainly involve interpreting and analyzing input text, such as semantic parsing of technical solutions, extraction of key information from contract terms, and analysis of user feedback sentiment. Text generation tasks can refer to tasks that create new text that meets requirements based on input instructions or understanding results, such as generating technical solution reports, intelligent code writing, and generating intelligent customer service dialogue scripts. Multimodal semantic understanding tasks can refer to tasks that allow the model to process two or more different modal input data simultaneously, understand the semantic relationships between different modal data, and ultimately complete a unified semantic level analysis or decision-making task. The different modal input data can include text, images, speech, tables, video frames, and other data. For example, combining experimental data tables and technical report texts to analyze model performance, integrating customer service voice recordings and text work order summaries to understand core user needs, and using medical images and text to assist in judging disease characteristics, etc.

[0075] In practical applications, when sample prompts are input into a diffusion language model, the model generates a final prediction result for each prompt through multiple rounds of iterative denoising operations. Each iteration of the denoising operation generates a corresponding intermediate prediction result based on the sample prompt. The model training platform can have data recording capabilities, enabling it to fully record the intermediate prediction results generated by the diffusion language model at each stage of the generation process, as well as the final prediction result output after iteration.

[0076] In one embodiment of this specification, multiple intermediate prediction results generated by the diffusion language model for sample prompt words can be extracted from the information recorded by the model training platform. By extracting the intermediate prediction results generated by the diffusion language model in the actual generation process, subsequent iterative optimization strategies can be accurately applied to the intermediate prediction results in the actual generation process, thereby effectively improving the effectiveness of strategy optimization and the accuracy of model tuning.

[0077] Step 204: For any one of the multiple intermediate prediction results, determine the reward value of that intermediate prediction result.

[0078] In one embodiment of this specification, for each of the multiple intermediate prediction results, a reward value can be determined, thus obtaining each reward value. The reward value refers to a numerical value obtained by quantifying and scoring the prediction result output at a certain stage in the iterative generation process of the diffusion language model based on a reward function adapted to the current task type, considering dimensions such as semantic quality, task suitability, and compliance. This value characterizes the quality of the prediction result and serves as the basis for subsequent model parameter adjustments.

[0079] In practical applications, a reward function tailored to the task type corresponding to the sample prompts can be invoked for each intermediate prediction result. For example, for a sentiment analysis task, the appropriate reward function is the classification consistency reward function. The core logic of the classification consistency reward function is to use the matching degree between the model's predicted sentiment label (positive / negative / neutral) and the true label as the core indicator. A successful match is rewarded, while a mismatch is penalized. Another example is a technical report summary generation task, where the appropriate reward function is the semantic condensation and coherence reward function. The core logic of the semantic condensation and coherence reward function is to evaluate whether the summary fully covers the core conclusions of the report, while also detecting the logical flow and fluency between sentences to avoid semantic breaks. Higher semantic condensation and stronger coherence are rewarded, while lower levels are penalized. For example, for a task type that combines technical schematic diagrams with text parsing, the appropriate reward function is the cross-modal feature alignment reward function. The core logic of the cross-modal feature alignment reward function is to evaluate the matching dimension between the visual features extracted from the image and the semantic features of the text description. The more comprehensive the matching dimension, the higher the reward value, and vice versa.

[0080] The intermediate prediction results are input into the invoked reward function, yielding a reward value generated by the function based on the accuracy metrics of these intermediate prediction results. These accuracy metrics can include semantic consistency, content completeness, semantic coherence, feature matching, and format compliance metrics. The reward value is a quantitative score output by the reward function, representing a numerical evaluation of whether the prediction results generated by the diffusion language model meet the task requirements. The reward value is typically a non-negative number; a higher value indicates that the corresponding prediction result better meets the task requirements.

[0081] In one embodiment of this specification, a reward value can be determined for each intermediate prediction result in the diffusion language model generation process. This reward value allows for dynamic adjustment of the probability of the model generating such intermediate prediction results, thereby effectively improving the completeness and comprehensiveness of the end-to-end model tuning. It is understood that a corresponding reward value can also be determined for some intermediate prediction results in the diffusion language model generation process.

[0082] Step 206: Adjust the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model.

[0083] In one embodiment of this specification, the strategy optimization direction for adjusting the parameters of the initial diffusion language model can be determined based on the magnitude of the reward value. As one implementation, the reward value can be compared with a preset threshold, and the strategy optimization direction for adjusting the parameters of the initial diffusion language model can be determined based on the comparison result. The preset threshold can be a value calculated from multiple reward values, or it can be a pre-defined value.

[0084] Optionally, if the reward value is greater than or equal to a preset threshold, the parameters of the initial diffusion language model are adjusted positively based on the reward value to obtain the trained diffusion language model; if the reward value is less than the preset threshold, the parameters of the initial diffusion language model are adjusted negatively based on the reward value to obtain the trained diffusion language model.

[0085] In one embodiment of this specification, for each intermediate prediction result, if the reward value of the intermediate prediction result is greater than or equal to a preset threshold, the model parameters are adjusted in a way that increases the probability of the model generating the intermediate prediction result, so as to improve the probability of the model generating the intermediate prediction result in subsequent training or application stages; if the reward value of the intermediate prediction result is less than the preset threshold, the model parameters are adjusted in a way that decreases the probability of the model generating the intermediate prediction result, so as to reduce the probability of the model generating the intermediate prediction result in subsequent training or application stages.

[0086] In practical applications, adjusting the model parameters to increase the probability of generating the intermediate prediction result can mean using the reward value of the intermediate prediction result as a positive weight and updating the parameters of the diffusion language model using a gradient ascent strategy, thus optimizing the parameters in a direction that increases the probability of generating the intermediate result. Conversely, adjusting the model parameters to decrease the probability of generating the intermediate prediction result can mean using the reward value of the intermediate prediction result as a negative weight and updating the parameters of the diffusion language model using a gradient descent strategy, thus optimizing the parameters in a direction that decreases the probability of generating the intermediate result.

[0087] It should be understood that the order of some steps in the methods described in one or more embodiments of this specification may be interchanged according to actual needs, or some steps may be omitted or deleted.

[0088] Figure 2The method described above adjusts the parameters of the diffusion language model based on the reward value of the intermediate prediction result. The reward value of the intermediate prediction result is calculated by the reward function for the intermediate prediction result itself. Therefore, the generation probability of each intermediate prediction result can be precisely adjusted according to its own reward value, so as to achieve fine-grained hierarchical control over the optimization process of the diffusion language model, thereby improving the overall training effect of the diffusion language model.

[0089] based on Figure 2 In addition to the method described herein, this specification also provides some specific implementation methods of the method, which will be described below.

[0090] To reduce the computational load during the training phase and save resources on the training platform, a portion of representative intermediate predictions can be selected from the multiple intermediate predictions generated by the diffusion language model, and the model can be optimized based on these intermediate predictions.

[0091] Optionally, determining the reward value of any intermediate prediction result among the plurality of intermediate prediction results may specifically include: selecting an intermediate prediction result to be optimized from the plurality of intermediate prediction results; and determining the reward value of any intermediate prediction result among the intermediate prediction results to be optimized.

[0092] In one embodiment of this specification, the intermediate prediction result to be optimized may refer to a prediction result with a relatively poor accuracy index. In practical applications, the intermediate prediction result to be optimized can be selected from multiple intermediate prediction results based on the accuracy index. As one implementation method, for each intermediate prediction result, if the accuracy index corresponding to the intermediate prediction result is less than a threshold, then the intermediate prediction result is determined as the intermediate prediction result to be optimized.

[0093] In one embodiment of this specification, a corresponding reward value is determined for each intermediate prediction result to be optimized obtained from multiple intermediate prediction results generated by the diffusion language model. This avoids calculating the reward value for each intermediate prediction result individually, thereby effectively reducing the amount of computation during model training and improving training efficiency.

[0094] Accuracy metrics can include the average entropy of lexical units corresponding to intermediate prediction results. Accuracy metrics can also include semantic consistency metrics, content completeness metrics, semantic coherence metrics, feature matching metrics, format compliance metrics, etc. In one embodiment of this specification, an example is proposed for selecting intermediate prediction results to be optimized from multiple intermediate prediction results based on the average entropy of lexical units.

[0095] Optionally, selecting the intermediate prediction results to be optimized from the plurality of intermediate prediction results may specifically include: calculating the average entropy of each intermediate prediction result to obtain the average entropy of each intermediate prediction result; sorting the intermediate prediction results in descending order of the average entropy of the word to obtain a first intermediate prediction result sequence; determining the first preset number of intermediate prediction results in the first intermediate prediction result sequence as the intermediate prediction results to be optimized; or, sorting the intermediate prediction results in ascending order of the average entropy of the word to obtain a second intermediate prediction result sequence; determining the last preset number of intermediate prediction results in the second intermediate prediction result sequence as the intermediate prediction results to be optimized.

[0096] In one embodiment of this specification, an intermediate prediction result may include several lexical units. The number of lexical units contained in different intermediate prediction results may be equal or unequal. The average lexical unit entropy can refer to the value obtained by calculating the arithmetic mean of the entropy values ​​of the lexical units contained in an intermediate prediction result; it is a quantitative indicator measuring the overall semantic uncertainty of the intermediate prediction result. The preset quantity can be understood as the number of lexical units corresponding to the average entropy, such as the average entropy of the first 5 lexical units or the average entropy of the last 3 lexical units in the lexical unit average entropy sequence; or the preset quantity can also be understood as the proportion corresponding to the average entropy of lexical units, such as the average entropy of the first 10% or the average entropy of the last 15% of lexical units in the lexical unit average entropy sequence.

[0097] In practical applications, during the process of progressively generating the final prediction result, the diffusion language model performs a fixing operation on some of the intermediate prediction results output at each step, while performing a masking operation on the remaining words. When generating the next intermediate prediction result in subsequent steps, the diffusion language model only re-predicts the words that have undergone the masking operation, and does not predict the words that have already undergone the fixing operation. Based on this, the average word entropy corresponding to the intermediate prediction results generated at each step can specifically refer to the average word entropy calculated for each word that actually participated in the prediction during the generation of the intermediate prediction result.

[0098] For each intermediate prediction result, the average word entropy of that intermediate prediction result can be calculated using a preset formula. The preset formula can be: , The average word entropy represents the intermediate prediction result generated in the t-th iteration. Let represent the set of masked terms contained in the data to be processed in the intermediate prediction results generated in the t-th iteration. This indicates the probability assigned to a specific word v in the vocabulary V by the mask position k in the data to be processed before the intermediate prediction result generated by the old strategy in the t-th iteration. t is a positive integer greater than or equal to 1. The old strategy refers to the generation strategy adopted by the diffusion language model when generating the intermediate prediction result in the t-th iteration. The vocabulary refers to the basic vocabulary on which the diffusion language model relies when generating the intermediate prediction result in the t-th iteration.

[0099] In one embodiment of this specification, the intermediate prediction results to be optimized are screened using the objective quantitative indicator of lexical average entropy. This avoids the problem of subjective bias caused by manual screening and ensures that the selected prediction results are all real prediction results that need to be optimized. This allows the strategy optimization to directly affect the generation process that needs improvement, thereby improving the efficiency and effectiveness of the overall tuning of the diffusion language model.

[0100] When filtering predictions to be optimized from multiple intermediate predictions of a diffusion language model, the final prediction can also be included in the filtering scope to determine whether the final prediction belongs to the predictions to be optimized, thereby improving the completeness and comprehensiveness of model strategy optimization.

[0101] Optionally, the method may further include: obtaining the final prediction result. The step of selecting the intermediate prediction result to be optimized from the plurality of intermediate prediction results may specifically include: selecting the prediction result to be optimized from the final prediction result and the plurality of intermediate prediction results.

[0102] In one embodiment of this specification, the final prediction result refers to the final result generated by the diffusion language model for the sample prompt words. The number of lexical units contained in the final prediction result may or may not be equal to the number of lexical units contained in the intermediate prediction results. In practical applications, the final prediction result can be extracted from the information recorded by the model training platform. Specifically, multiple intermediate prediction results generated by the diffusion language model for the sample prompt words and the final prediction result can be extracted simultaneously from the information recorded by the model training platform.

[0103] In one embodiment of this specification, a prediction result to be optimized can be selected from multiple intermediate prediction results and the final prediction result based on an accuracy index. For any prediction result among the multiple intermediate prediction results and the final prediction result, if the accuracy index corresponding to the prediction result is less than a threshold, then the prediction result is determined as a prediction result to be optimized. The accuracy index may include the average entropy of the lexical units corresponding to the prediction result, semantic consistency index, content completeness index, semantic coherence index, feature matching index, format compliance index, etc. When determining the prediction result to be optimized, the final prediction result is also included in the screening scope, thereby improving the comprehensiveness of the screening of prediction results to be optimized, and thus improving the training effect of the diffusion language model.

[0104] To improve the training effect of the diffusion language model, it is necessary to ensure the accuracy of the preset threshold used for comparison with the reward value. In one embodiment of this specification, the preset threshold can be calibrated according to the reward value corresponding to each intermediate prediction result, wherein the preset threshold is used to define the parameter adjustment method of the initial diffusion language model.

[0105] Optionally, after determining the reward value of any intermediate prediction result among the plurality of intermediate prediction results, the method may further include: determining the preset threshold based on each reward value of each intermediate prediction result, wherein the preset threshold includes at least the median or average value determined by each reward value.

[0106] In one embodiment of this specification, the median value of each reward value of each intermediate prediction result is determined, and this median value is used as the preset threshold. In practical applications, the reward values ​​are sorted in ascending or descending order to obtain a reward value sequence. The reward value in the middle of this reward value sequence is determined as the median value of each reward value. If the total number of reward values ​​is even, the average of the two reward values ​​in the middle of the reward value sequence is calculated, and this average is used as the median value of each reward value. The median value reflects the intermediate level of each reward value.

[0107] Alternatively, the average value of each reward value in each intermediate prediction result can be determined, and this average value can be used as the preset threshold. In practical applications, the reward values ​​are added together to obtain a sum, and the sum is divided by the total number of reward values ​​to obtain the average value of each reward value. The average value can reflect the overall average level of each reward value. The average value can include not only the arithmetic mean but also a weighted average or geometric mean, etc. In one embodiment of this specification, the preset threshold can be determined based on the reward value of the actual prediction results generated during the model generation process, thereby eliminating the bias of subjective human setting. Furthermore, the true quality level of the intermediate prediction results can be accurately determined based on this preset threshold, improving the accuracy of model training.

[0108] To further improve the training effect of the diffusion language model, it is also necessary to ensure that the determined preset threshold matches the model optimization requirements. In one embodiment of this specification, the preset threshold can be determined based on the reward value of each intermediate prediction result to be optimized selected from each intermediate prediction result.

[0109] Optionally, determining the preset threshold based on the reward values ​​of each of the intermediate prediction results may specifically include: selecting each intermediate prediction result to be optimized from the plurality of intermediate prediction results; and determining the preset threshold based on the reward values ​​of each intermediate prediction result to be optimized.

[0110] In one embodiment of this specification, the method for selecting the intermediate prediction result to be optimized from multiple intermediate prediction results can refer to the above content, and will not be repeated here.

[0111] In one embodiment of this specification, the median value of each reward value of each intermediate prediction result to be optimized is determined, and this median value is set as the preset threshold. The method for determining the median value of each reward value of each intermediate prediction result to be optimized can refer to the method for determining the median value of each reward value of each intermediate prediction result, and will not be repeated here.

[0112] Alternatively, the average value of each reward value of each intermediate prediction result to be optimized can be determined, and this average value can be used as the preset threshold. The method for determining the average value of each reward value of each intermediate prediction result to be optimized can refer to the method for determining the average value of each reward value of each intermediate prediction result, and will not be repeated here.

[0113] In one embodiment of this specification, the preset threshold can be determined based on the reward value of the intermediate prediction result to be optimized, so that the preset threshold can be highly adapted to the intermediate prediction result to be optimized, thereby providing a scientific basis for adjusting the parameters of the diffusion language model and improving the training effect of the diffusion language model.

[0114] The initial diffusion language model can predict one or more final prediction results for a single sample prompt word.

[0115] Optionally, the final prediction result may include multiple final prediction results, with each final prediction result corresponding to an intermediate prediction result set. The intermediate prediction result set includes multiple intermediate prediction results generated during the process of the initial diffusion language model generating a final prediction result for the sample prompt word.

[0116] In one embodiment of this specification, during the process of generating final prediction results for sample prompt words, the initial diffusion language model can simultaneously generate multiple final prediction results, such as six final prediction results or other numbers of prediction results, such as four, seven, or ten. During the generation of each final prediction result, the initial diffusion language model can iteratively generate a series of intermediate prediction results. Here, a series of intermediate prediction results corresponding to a final prediction result can be referred to as an intermediate prediction result set. An intermediate prediction result set can include multiple intermediate prediction results, such as 256 intermediate prediction results or other numbers of intermediate prediction results, such as 128 or 512. Different final prediction results can correspond to different intermediate prediction result sets, and the number of intermediate prediction results contained in different intermediate prediction result sets can be equal or unequal.

[0117] Optionally, selecting the intermediate prediction results to be optimized from the plurality of intermediate prediction results may specifically include: selecting the intermediate prediction results to be optimized from each set of intermediate prediction results to obtain each set of intermediate prediction results to be optimized, wherein each set of intermediate prediction results to be optimized contains intermediate prediction results to be optimized selected from the same set of intermediate prediction results.

[0118] In one embodiment of this specification, for any intermediate prediction result set in each intermediate prediction result set, each intermediate prediction result to be optimized can be selected from that intermediate prediction result set to obtain an intermediate prediction result set to be optimized. In this way, each intermediate prediction result set to be optimized can be selected from each intermediate prediction result set. As one implementation, each intermediate prediction result to be optimized selected from the same intermediate prediction result set can be assigned to an intermediate prediction result set to be optimized. Each intermediate prediction result to be optimized in an intermediate prediction result set to be optimized can be selected from the same intermediate prediction result set. The number of intermediate prediction results to be optimized contained in different intermediate prediction result sets to be optimized can be equal or unequal.

[0119] For example, suppose that for a sample prompt word A, the initial diffusion language model generates three final prediction results, corresponding to three intermediate prediction result sets, namely intermediate prediction result set a, intermediate prediction result set b, and intermediate prediction result set c; the intermediate prediction results to be optimized selected from intermediate prediction result set a are intermediate prediction results a1, a3, and a6, then the intermediate prediction result set x to be optimized selected from intermediate prediction result set a may include intermediate prediction results a1, a3, and a6; the intermediate prediction results to be optimized selected from intermediate prediction result set b are intermediate prediction results a1, a3, and a6; the intermediate prediction results to be optimized selected from intermediate prediction result set b are intermediate prediction results a1, a3, and a6; the intermediate prediction results a1, a3, and a6 are intermediate prediction results a1, a3, and a6, and the intermediate prediction results a1, a3, and a6 are intermediate prediction results a1, a3, and a6, respectively. If the intermediate prediction results are b7, b10, b20, b23, and b30 respectively, then the intermediate prediction result set y to be optimized selected from the intermediate prediction result set b may include intermediate prediction results b7, b10, b20, b23, and b30; if the intermediate prediction results to be optimized selected from the intermediate prediction result set c are c5, c13, c25, c32, and c36 respectively, then the intermediate prediction result set z to be optimized selected from the intermediate prediction result set c may include intermediate prediction results c5, c13, c25, c32, and c36.

[0120] Optionally, determining the preset threshold based on the reward values ​​of each intermediate prediction result to be optimized may include: determining the preset threshold based on the reward values ​​of the intermediate prediction results to be optimized in the set of intermediate prediction results to be optimized.

[0121] In one embodiment of this specification, the method for determining the preset threshold may include a variety of methods. For example, the reward value of each intermediate prediction result in a set of intermediate prediction results to be optimized can be used to determine a sub-preset threshold, and then the preset threshold can be determined based on each sub-preset threshold; or, at least some intermediate prediction results to be optimized can be selected from each set of intermediate prediction results to be optimized, and then the preset threshold can be determined based on the reward value of each selected intermediate prediction result to be optimized.

[0122] In one embodiment of this specification, the parameters of the initial diffusion language model can be adjusted based on the reward value of each intermediate prediction result in each set of intermediate prediction results to be optimized. The number of times the parameters of the initial diffusion language model are adjusted can be determined based on the number of intermediate prediction results to be optimized.

[0123] For example, continuing with the intermediate prediction result set x, y, and z to be optimized from the previous example. If intermediate prediction result set x contains 3 intermediate prediction results, intermediate prediction result set y contains 5 intermediate prediction results, and intermediate prediction result set z contains 5 intermediate prediction results, then during the training of the initial diffusion language model using sample prompt word A, the parameters of the initial diffusion language model can be adjusted 13 times.

[0124] In one embodiment of this specification, based on an intermediate prediction result to be optimized, the parameters of the initial diffusion language model can be adjusted once. For the intermediate prediction result to be optimized, the parameters of the initial diffusion language model need to be adjusted according to the comparison between the reward value of the intermediate prediction result to be optimized and the corresponding preset threshold. Therefore, for each intermediate prediction result to be optimized, a preset threshold corresponding to that intermediate prediction result needs to be determined.

[0125] In one embodiment of this specification, the preset threshold used to compare the reward value with different intermediate prediction results to be optimized can be the same, in order to simplify the process of determining the preset threshold. As another implementation, to further reflect the differences in optimizing the initial diffusion language model using various intermediate prediction results to be optimized, a separate preset threshold can also be determined for each different intermediate prediction result to be optimized.

[0126] Optionally, determining the preset threshold based on the reward values ​​of the intermediate prediction results to be optimized in each set of intermediate prediction results to be optimized may specifically include: for any intermediate prediction result to be optimized in any set of intermediate prediction results to be optimized, determining a target intermediate prediction result from other intermediate prediction result sets; the other intermediate prediction result sets are intermediate prediction result sets in each set of intermediate prediction results other than the intermediate prediction result set containing any intermediate prediction result to be optimized; the target intermediate prediction result represents an intermediate prediction result that uses the same number of denoising steps as any intermediate prediction result to be optimized; and determining the preset threshold based on the reward values ​​of each target intermediate prediction result and the reward value of any intermediate prediction result to be optimized.

[0127] In one embodiment of this specification, for any intermediate prediction result to be optimized, the "any intermediate prediction result to be optimized" can be any intermediate prediction result in any set of intermediate prediction results to be optimized. In practical applications, the target intermediate prediction result set containing the "any intermediate prediction result to be optimized" can be queried based on the record information of the model training platform. The number of denoising operations corresponding to any intermediate prediction result to be optimized refers to the number of times the initial diffusion language model has performed denoising when it iteratively generating the "any intermediate prediction result to be optimized" during the process of generating the final prediction result corresponding to the target intermediate prediction result set. In other words, the number of denoising operations can refer to the number of iterations corresponding to the initial diffusion language model when iteratively generating the "any intermediate prediction result to be optimized".

[0128] To facilitate understanding of this solution by those skilled in the art, a specific example is provided in one embodiment of this specification to explain the method for generating a preset threshold.

[0129] For example, continuing with the previous example, suppose the initial diffusion language model generates three final prediction results for the sample prompt word A, namely final prediction result L, final prediction result M, and final prediction result N, where final prediction result L corresponds to intermediate prediction result set a, final prediction result M corresponds to intermediate prediction result set b, and final prediction result N corresponds to intermediate prediction result set c.

[0130] For the intermediate prediction result a6 to be optimized in the intermediate prediction result set x, determine the intermediate prediction result set a containing the intermediate prediction result a6; based on the recorded information of the model training platform, query the number of times the initial diffusion language model iterated to the intermediate prediction result a6 during the process of generating the final prediction result L. For example, if the initial diffusion language model performed a total of six denoising operations when it iterated to the intermediate prediction result a6, then the number of denoising operations corresponding to the intermediate prediction result a6 is six; determine the intermediate prediction result b6 that iterated six times during the process of generating the final prediction result M from the intermediate prediction result set y, and determine the intermediate prediction result c6 that iterated six times during the process of generating the final prediction result N from the intermediate prediction result set z; determine the median or average value of the reward value of the intermediate prediction result a6, the reward value of the intermediate prediction result b6, and the reward value of the intermediate prediction result c6, and obtain the preset threshold corresponding to the intermediate prediction result a6.

[0131] The preset threshold corresponding to the intermediate prediction result a6 to be optimized refers to the preset threshold used to compare the reward value of the intermediate prediction result a6 to be optimized when adjusting the parameters of the initial diffusion language model based on the intermediate prediction result a6 to be optimized. In practical applications, different intermediate prediction results to be optimized correspond to different preset thresholds. As one implementation method, if the number of denoising operations corresponding to different intermediate prediction results to be optimized is the same, then the preset thresholds corresponding to different intermediate prediction results to be optimized can be the same. For example, in the example above, if intermediate prediction results b6 and c6 are also intermediate prediction results to be optimized, then the preset thresholds corresponding to intermediate prediction results a6, b6, and c6 are the same.

[0132] In practical applications, for different sets of intermediate prediction results, the intermediate prediction results with the same number of denoising steps can all belong to the intermediate prediction results to be optimized, some can belong to the intermediate prediction results to be optimized, or none of them can belong to the intermediate prediction results to be optimized.

[0133] In one embodiment of this specification, for any intermediate prediction result to be optimized, a preset threshold corresponding to the intermediate prediction result to be optimized can be determined based on the reward values ​​of each intermediate prediction result in the intermediate prediction result set to which the intermediate prediction result to be optimized belongs. Alternatively, the preset threshold corresponding to the intermediate prediction result to be optimized can also be determined based on the reward values ​​of each intermediate prediction result in the intermediate prediction result set to which the intermediate prediction result to be optimized belongs. Alternatively, the preset threshold corresponding to the intermediate prediction result to be optimized can also be determined based on the reward values ​​of each intermediate prediction result in each intermediate prediction result set. Alternatively, the preset threshold corresponding to the intermediate prediction result to be optimized can also be determined based on the reward values ​​of each intermediate prediction result in each intermediate prediction result set.

[0134] To further improve the training effect of the diffusion language model, the final prediction result can be included in the selection range of prediction results to be optimized, and then the preset threshold can be determined based on the reward value of the selected prediction results to be optimized.

[0135] Optionally, the method may further include: obtaining the final prediction result; and determining the reward value of the final prediction result. Specifically, determining the preset threshold based on the reward values ​​of each of the intermediate prediction results may include: selecting each prediction result to be optimized from the final prediction result and the plurality of intermediate prediction results; and determining the preset threshold based on the reward values ​​of each prediction result to be optimized.

[0136] In one embodiment of this specification, the final prediction result can be extracted from the information recorded by the model training platform. In practical applications, the obtained final prediction result may include multiple final prediction results. For each final prediction result, a reward function adapted to the task type corresponding to the sample prompt word can be called to determine the reward value of the final prediction result.

[0137] In one embodiment of this specification, a prediction result to be optimized can be selected from multiple intermediate prediction results and a final prediction result based on an accuracy index. In practical applications, a final prediction result can correspond to a set of intermediate prediction results, and a final prediction result and its corresponding set of intermediate prediction results can form a prediction result set. For a prediction result set, a prediction result to be optimized can be selected from the prediction result set to obtain a prediction result set to be optimized corresponding to the prediction result set. In this way, various prediction result sets to be optimized can also be obtained from other prediction result sets.

[0138] In practical applications, the set of prediction results to be optimized can include any one of the following: intermediate prediction results, final prediction results, intermediate prediction results, and final prediction results.

[0139] In one embodiment of this specification, the method for determining the preset threshold corresponding to any test result to be optimized in any set of prediction results to be optimized can refer to the method for determining the preset threshold corresponding to any intermediate test result to be optimized described above, and will not be repeated here.

[0140] In one embodiment of this specification, for any prediction result to be optimized, a preset threshold corresponding to the prediction result to be optimized can be determined based on the reward values ​​of each prediction result in the prediction result set to which the prediction result to be optimized belongs. Alternatively, the preset threshold corresponding to the prediction result to be optimized can also be determined based on the reward values ​​of each prediction result in the prediction result set to which the prediction result to be optimized belongs. Alternatively, the preset threshold corresponding to the prediction result to be optimized can also be determined based on the reward values ​​of each prediction result in each prediction result set. Alternatively, the preset threshold corresponding to the prediction result to be optimized can also be determined based on the reward values ​​of each prediction result in each prediction result set.

[0141] In one embodiment of this specification, the final prediction result is included in the range of prediction results to be optimized. A preset threshold is then determined based on the reward value of the selected prediction results to be optimized. This ensures that the calculation data for the preset threshold covers the prediction results at each stage of the entire generation process of the diffusion language model, thereby improving the rationality of the preset threshold. Furthermore, the preset threshold is determined based on the reward value of the prediction results to be optimized selected from the entire generation process, ensuring that the determined preset threshold is more closely matched to the prediction results that need optimization. This provides a more scientific basis for adjusting the parameters of the diffusion language model, thereby improving the training effect of the diffusion language model.

[0142] In related technologies, during the training phase of a diffusion language model, a manual random masking operation is typically performed on the final prediction results generated by the model to construct the data to be processed using the artificial mask. The model then predicts the data to be processed using this artificial mask to generate corresponding intermediate prediction results, and model strategy optimization is performed based on these intermediate prediction results. However, the data to be processed constructed using the artificial mask is not the data to be processed generated by the diffusion language model in the actual generation process. This means that subsequent optimization strategies cannot be applied to the intermediate prediction results in the actual generation process, resulting in poor model training performance. In one embodiment of this specification, the data to be processed corresponding to the intermediate prediction results obtained is the state in the actual generation process of the model, so that subsequent optimization strategies can be applied to the intermediate prediction results in the actual generation process of the model, thereby improving the model training performance.

[0143] Optionally, the plurality of intermediate prediction results include a first intermediate prediction result and a second intermediate prediction result. Specifically, obtaining the plurality of intermediate prediction results generated by the initial diffusion language model for the sample prompt word can include: inputting the sample prompt word into the initial diffusion language model; wherein the initial diffusion language model can determine, based on the sample prompt word, a first set of data to be processed containing a preset number of word positions to be filled; performing a denoising operation on the first set of data to be processed to generate the first intermediate prediction result; converting the first intermediate prediction result into second data to be processed by fixing some word positions to be filled and masking other word positions to be filled; performing a denoising operation on the masked word positions in the second data to be processed to obtain the second intermediate prediction result; and obtaining the first intermediate prediction result and the second intermediate prediction result.

[0144] In one embodiment of this specification, multiple intermediate prediction results may include a first intermediate prediction result, a second intermediate prediction result, and other intermediate prediction results. The data to be processed may refer to the state data before the diffusion language model generates the intermediate prediction results, and this data to be processed may include lexical positions to be filled marked by a masking operation. Furthermore, if the data to be processed corresponds to state data outside the initial iteration process, the data to be processed may also include lexical units locked by a fixing operation. A lexical position to be filled refers to an empty position used to fill lexical units.

[0145] In one embodiment of this specification, the initial diffusion language model can predict and generate multiple final prediction results for a sample prompt word. One final prediction result can correspond to one intermediate prediction result set. Multiple intermediate prediction results can all belong to the same intermediate prediction result set, or some of the intermediate prediction results can belong to the same intermediate prediction result set, while other intermediate prediction results belong to other intermediate prediction result sets.

[0146] Figure 3 This is a schematic diagram of the generation and gradient update iteration process of a diffusion language model proposed in one embodiment of this specification.

[0147] In one embodiment of this specification, such as Figure 3 As shown, the initial diffusion language model generates a final prediction result as an example. After receiving a sample prompt word, the initial diffusion language model first plans the number of lexical units in the prediction result corresponding to that sample prompt word based on its semantic content and the requirements of the generation task. Specifically, this can include 128 lexical units, 256 lexical units, 512 lexical units, or other custom numbers. Assuming the initial diffusion language model plans to include 128 lexical units in the prediction result (not all are shown in the figure),... Figure 3The initial diffusion language model further determines the first set of data to be processed based on the number of lexical units obtained from the planning. Subsequently, the initial diffusion language model executes a prediction process on the first set of data, generating a first intermediate prediction result by performing denoising operations on all masked lexical units in the first set of data. After obtaining the first intermediate prediction result, the initial diffusion language model converts the first intermediate prediction result into the second set of data to be processed according to the rule of performing a fixing operation on some lexical units and a masking operation on the remaining lexical units. The fixing operation refers to locking a selected portion of the lexical units in the first intermediate prediction result; its core function is to ensure that the semantics and position of the fixed lexical units remain unchanged during subsequent iterative prediction processes. The masking operation refers to resetting the corresponding lexical units of the remaining lexical units that were not fixed to a masked state. After determining the second set of data to be processed, the model will predict the word bits in the second set of data to be processed that have been masked, fill in the corresponding semantic words, and generate the second intermediate prediction result. Referring to the process of generating the first intermediate prediction result and the second intermediate prediction result, the initial diffusion language model can iteratively generate other intermediate prediction results. Thus, the initial diffusion language model can complete a complete task reasoning process and generate the final prediction result.

[0148] In one embodiment of this specification, the initial diffuse language module can be iteratively trained based on the intermediate prediction results generated during the inference process of the initial diffuse language model described above. In practical applications, during the gradient update iteration process of the diffuse language model, the model training platform can extract each intermediate prediction result during the generation process of the initial diffuse language model, and can also extract the final prediction result. Then, the average word entropy of each prediction result is calculated, and based on the average word entropy of each prediction result, the prediction results that need to be optimized are selected, such as... Figure 3 As shown, assume that the prediction results to be optimized selected from the generation process include the first intermediate prediction result and the second intermediate prediction result. To facilitate tracing the data to be processed corresponding to the prediction results to be optimized, the data to be processed corresponding to the prediction results to be optimized can also be selected simultaneously after the prediction results to be optimized are selected. Then, for the selected prediction results to be optimized, a reward value is determined, and the parameters of the initial diffusion language model are adjusted based on this reward value to obtain the trained diffusion language model.

[0149] In one embodiment of this specification, the first intermediate prediction result and the second intermediate prediction result obtained by the model training platform are both prediction results generated by the initial diffusion language model based on the data to be processed in its real generation process. This allows subsequent optimization strategies to be accurately applied to the intermediate prediction results in the real generation process of the model, thereby improving the training effect of the model.

[0150] To improve the accuracy of the final prediction results output by the diffusion language model, in each iteration of the model, words with high generation probability can be selected for fixed processing.

[0151] Optionally, the step of fixing the word elements at the positions to be filled in the first intermediate prediction result may specifically include: obtaining the generation probability value of each word element in the first intermediate prediction result; sorting each word element in descending order of the generation probability value to obtain a first word element sequence; fixing the first preset number of word elements in the first word element sequence; or, sorting each word element in ascending order of the generation probability value to obtain a second word element sequence; and fixing the last preset number of word elements in the first word element sequence.

[0152] In one embodiment of this specification, the generation probability value refers to the confidence quantification index of a certain word element output by the model during the iterative prediction process. The value range can be [0,1]. The higher the generation probability value, the higher the fit between the word element and the current text context and the stronger the semantic accuracy; the lower the probability value, the stronger the semantic randomness of the word element.

[0153] In practical applications, when the diffusion language model performs prediction operations on each word element in the first intermediate prediction result, it calculates the generation probability value of each candidate word in the vocabulary as that word element; and based on the above generation probability values, it determines the target word to be used to fill that word element. As a specific implementation method, the candidate word with the highest generation probability value can be selected as that word element.

[0154] During the process of the diffusion language model predicting the first intermediate prediction result, the model training platform can synchronously record the generation probability value of each word in the first intermediate prediction result, so that in subsequent iterations, the generation probability value of each word in the intermediate prediction result can be directly obtained from the recorded information of the model training platform.

[0155] In practical applications, the "pre-preset number of lexical units" can be understood as a fixed number of pre-selected lexical units, such as the first 5 lexical units; it can also be understood as a percentage of pre-selected lexical units, such as the first 30% of lexical units. Similarly, the "post-preset number of lexical units" can be understood as a fixed number of post-selected lexical units, such as the last 3 lexical units; it can also be understood as a percentage of post-selected lexical units, such as the last 20% of lexical units.

[0156] In practical applications, when performing fixed operations on certain words in each intermediate prediction result, the number of words whose fixed operations are performed can remain the same or differ for different intermediate prediction results.

[0157] In one embodiment of this specification, when performing a fixing operation on some word characters at positions to be filled in the first intermediate prediction result, the implementation may further include randomly selecting some word characters from the first intermediate prediction result by the diffusion language model and fixing the selected word characters. For example, randomly selecting the nth and (n+2)th word characters in the first intermediate prediction result and fixing them.

[0158] Figure 4 This is a flowchart illustrating a task processing method according to one embodiment of this specification. From a hardware perspective, the entity executing this process can be a device used for processing tasks; or, from a software perspective, the entity executing this process can be an application program installed on the device used for processing tasks.

[0159] like Figure 4 As shown, the process may include the following steps.

[0160] Step 402: Obtain the task to be processed, which includes text understanding and generation tasks or multimodal semantic understanding tasks.

[0161] In one embodiment of this specification, the task to be processed may refer to a task proposed by the user that requires processing by the diffusion language model. Text understanding and generation tasks may include text understanding tasks and text generation tasks. Text understanding tasks refer to tasks such as semantic parsing, information extraction, and logical judgment of input text, such as semantic understanding of text content, extraction of technical features from technical documents, and semantic similarity comparison of technical solutions. Text generation tasks refer to generating text that conforms to semantic logic and format requirements based on input prompts, such as assisting in the writing of technical solutions, automatic generation of technical reports, and continuation of dialogue content. Multimodal semantic understanding tasks refer to tasks where the input or output contains two or more modal types. Modal types may include text, images, speech, tables, etc., such as tasks that generate corresponding technical solutions based on technical schematics and text descriptions.

[0162] In one embodiment of this specification, obtaining a task to be processed can refer to the operation of a task processing platform or a diffusion language model receiving a task to be processed. The implementation method can include passive reception or active retrieval. Passive reception refers to receiving task instructions directly input by the user; active retrieval refers to the task processing platform or diffusion language model extracting tasks to be processed from the task queue.

[0163] Step 404: Input the task to be processed into the diffusion language model to obtain the processing result for the task to be processed. The diffusion language model is a model trained and generated according to the method in at least one of the above embodiments.

[0164] In one embodiment of this specification, inputting a task to be processed into a diffusion language model refers to the operation of sending the task instructions and input data corresponding to the task to be processed to the processing interface of the diffusion language model. The input method may include direct input or batch input. Direct input means sending the task to be processed to the diffusion language model through a user interface; batch input means importing the tasks to be processed in the task queue into the diffusion language model in batches through a task scheduling system.

[0165] Optionally, before inputting the task to be processed into the diffusion language model, the process may further include preprocessing the task according to its type to ensure that the data format of the task meets the model's input requirements. In practical applications, if the task to be processed is a text understanding and generation task, the input text is lexicalized to remove invalid characters, and truncated or padded according to the model's text length input requirements to generate a standard text input sequence that meets the model's input requirements; if the task to be processed is a multimodal semantic understanding task, features are extracted from the non-text modal data, the extracted features are converted into vector forms that the model can recognize, and then modally aligned with the text data to generate a cross-modal input sequence.

[0166] In one embodiment of this specification, the model used to process the task to be processed can be a diffusion language model trained using the training method described in at least one of the above embodiments. During training, the optimization strategy of this model does not apply to the prediction results corresponding to the manually masked data to be processed, but rather precisely applies to the intermediate or final prediction results generated by the data to be processed in the actual generation process of the model. This effectively avoids the optimization failure problem caused by the disconnect between the manually masked data and the actual generation state, thereby improving the optimization effect of the optimization strategy on the model. Secondly, in the model parameter adjustment stage, the intermediate prediction results to be optimized, used to drive parameter optimization, optimize the model parameters based on the reward value of each intermediate prediction result itself, rather than uniformly using the reward value of the final prediction result as the optimization basis for each intermediate prediction result to be optimized. This allows for differentiated parameter adjustment strategies to be implemented for intermediate prediction results of different quality levels, avoiding the drawbacks of indiscriminate parameter adjustment, and further enhancing the training effect of the model. Therefore, adopting... Figure 2 When the diffusion language model trained by the training method is executed, it can give full play to the optimization advantages of the training stage, thereby effectively improving the accuracy, rationality and compliance of the processing results of the task.

[0167] The following is combined Figure 5 Taking the application of the task processing method provided in this specification in a content generation task scenario as an example, the above task processing method will be further explained. Among them, Figure 5This is a flowchart illustrating a content generation task processing method provided in one embodiment of this specification. Figure 5 This may include the following steps.

[0168] Step 502: Obtain the content generation task to be processed; the content generation task to be processed includes at least one of the following: code completion task, text generation task, image generation task, audio generation task, and video generation task.

[0169] Step 504: Input the task to be processed into the diffusion language model to obtain task generation result information; the diffusion language model is a model trained according to the above-mentioned training method for diffusion language models.

[0170] It should be noted that code completion tasks refer to tasks that complete incomplete code. The result information for code completion tasks refers to the complete and formatted code output by the diffusion language model after inputting the prompt words for code completion requests. Text generation tasks refer to tasks that generate text content that meets user requirements. The result information for text generation tasks refers to the text content that meets user requirements after inputting the prompt words for text generation requests. Image generation tasks refer to tasks that generate images that meet user requirements. The result information for image generation tasks refers to the image information that meets user requirements after inputting the prompt words for image generation requests. Audio generation tasks refer to tasks that generate audio that meets user requirements. The result information for audio generation tasks refers to the audio information that meets user requirements after inputting the prompt words for audio generation requests. A video generation task refers to the task of generating a video that meets the user's requirements. The task generation result information corresponding to the video generation task refers to the video information that meets the user's requirements when the prompt words for requesting video generation are input into the diffusion language model.

[0171] In practical applications, the implementation methods of steps 502 and 504 are the same as those of steps 402 and 404, and will not be repeated here.

[0172] To facilitate understanding of this solution by those skilled in the art, the training method for the diffusion language model will now be further explained using the application of the diffusion language model to a text generation scenario.

[0173] Figure 6 This is a flowchart of a training method for a diffusion language model for text generation, provided in one embodiment of this specification, including the following steps.

[0174] Suppose our desired generated text is "Training process of diffusion language model". We set up an initial diffusion language model to generate this final prediction result through three iterative steps. In each iteration, the initial diffusion language model selects two words from the current intermediate prediction result and performs a fixing operation, while masking the remaining words. During the prediction process, the masked words need to be re-predicted, while the fixed words remain unchanged. Assume the initial diffusion language model generates a final prediction result based on the sample prompt words. The specific training process is as follows.

[0175] Step 602: Input the sample prompt words into the initial diffusion language model to obtain the first intermediate prediction result generated by the initial diffusion language model.

[0176] Specifically, sample prompt words can be input into the initial diffusion language model. The initial diffusion language model plans the first data to be processed based on the sample prompt words. The first data to be processed can include 6 word positions to be filled [X1, X2, X3, X4, X5, X6], where X1 corresponds to "diffusion", X2 corresponds to "language", X3 corresponds to "model", X4 corresponds to "of", X5 corresponds to "training", and X6 corresponds to "process".

[0177] The initial diffusion language model, based on the first data to be processed, predicts and generates the first intermediate prediction result, such as [diffusion, large, model, of, pre-training, result].

[0178] Step 604: Obtain the second intermediate prediction result generated by the iterative generation of the initial diffusion language model.

[0179] Specifically, the initial diffusion language model fixes the two words with relatively high generation probability values ​​in the first intermediate prediction result, while masking other words. For example, it fixes the words

diffusion, model

big, of, pre-training, result

diffusion, X2, model, X4, X5, X6

[0180] The initial diffusion language model re-predicts the masked words in the second set of data to be processed, generating a second intermediate prediction result [diffusion, language, model, of, reinforcement, result].

[0181] Step 606: Obtain the third intermediate prediction result generated by the iterative generation of the initial diffusion language model.

[0182] Specifically, the initial diffusion language model fixes the two words with relatively high generation probabilities in the predicted words in the second intermediate prediction results, and masks the other unfixed words. For example, it fixes the words

language, of

strengthening, result

diffusion, language, model, of, X5, X6

[0183] The initial diffusion language model re-predicts the masked words in the third set of data to be processed, generating a third intermediate prediction result [diffusion, language, model, of, training, process].

[0184] The model training platform can record the first, second, and third intermediate prediction results during the entire generation process of the diffusion language model. Furthermore, to track the pending data corresponding to each intermediate prediction result, the model training platform can also record the first, second, and third pending data.

[0185] Step 608: Calculate the first word average entropy of the first intermediate prediction result, the second word average entropy of the second intermediate prediction result, and the third word average entropy of the third intermediate prediction result.

[0186] Step 610: Sort the first intermediate prediction result, the second intermediate prediction result, and the third intermediate prediction result in descending order of the first word average entropy, the second word average entropy, and the third word average entropy to obtain an intermediate prediction result sequence.

[0187] Step 612: The preset number of intermediate prediction results preceding the intermediate prediction result sequence are determined as intermediate prediction results to be optimized.

[0188] Step 614: Determine the reward value for each intermediate prediction result to be optimized.

[0189] Step 616: Determine the average value of each reward value of each intermediate prediction result to be optimized.

[0190] Step 618: For any intermediate prediction result to be optimized, if the reward value of any intermediate prediction result to be optimized is greater than or equal to the average value, then the parameters of the initial diffusion language model are positively adjusted to obtain the trained diffusion language model.

[0191] Step 620: If the reward value of any intermediate prediction result to be optimized is less than the average value, then the parameters of the initial diffusion language model are adjusted in reverse to obtain the trained diffusion language model.

[0192] To facilitate understanding by those skilled in the art of the performance improvement of the diffusion language model obtained by using the training methods described in this specification, the inference accuracy of the diffusion language model in various tasks to be processed is explained below.

[0193] Table 1 is a comparison of the accuracy between LLaDA-8B-Instruct and the diffusion language model obtained based on the training method in this manual.

[0194] Table 1 Accuracy Comparison Table

[0195]

[0196] As shown in Table 1, the Large Language Diffusion with mAsking-8B-Instruct (LLaDA-8B-Instruct) represents the baseline model, referring to a language diffusion model with 8 billion parameters and finely tuned instructions. The Grade School Math 8K (GSM8K) dataset is a question bank of basic elementary school math word problems, used to test the model's basic logic and computational abilities. The Mathematics Dataset 50-item Subset (MATH50) is a question bank of high school or university-level math problems, used to test the model's complex multi-step reasoning abilities. The Countdown Number Game Task (Countdown) is a number reasoning problem where given numbers and operators are used to form a target number, used to test the model's arithmetic combination and fast computation abilities. The Sudoku Logic Puzzle Task (Sudoku) is a logic puzzle, used to test the model's logical reasoning and global planning abilities.

[0197] Table 1 shows that as the input sequence length increases from 128 to 512, the performance of the LLaDA-8B-Instruct model significantly decreases. Figure 2 The training method yields a smaller performance degradation in the post-trained diffusion language model under long sequences, indicating that the performance of this post-trained diffusion language model is relatively stable when handling long inputs and complex reasoning. Furthermore, the post-trained diffusion language model achieves good results on various tasks and sequence lengths listed in Table 1. For example, on the Countdown-256 task, based on… Figure 2 The model obtained by this method achieved an accuracy of 65.6%, a 46.1% improvement over the baseline model; on the Countdown-512 task, based on Figure 2The model obtained by this method achieved an accuracy of 65.2%, a 49.2% improvement over the baseline model; on the Sudoku-512 task, based on Figure 2 The model obtained by this method achieved an accuracy of 20.2%, which is 14.7% higher than the baseline model.

[0198] Corresponding to the above method embodiments, this specification also provides an embodiment of a model training platform. Figure 7 This is a schematic diagram of a model training platform provided in one embodiment of this specification. Figure 7 As shown, the model training platform may include a model training unit 702 and a response unit 704.

[0199] The model training unit 702 is used to train the initial diffusion language model to obtain the diffusion language model.

[0200] The response unit 704 is used to output the diffusion language model.

[0201] The processing methods for the model training unit and the response unit are the same as those for the initial diffusion language model described above. For details, please refer to the description in the aforementioned embodiments, which will not be repeated here.

[0202] Optionally, the model training platform may also include a data receiving unit.

[0203] The data receiving unit is used to receive sample prompt words input by the user and send the sample prompt words to the model training unit.

[0204] The model training unit is also used to train the initial language model based on the sample prompt words to obtain the diffusion language model.

[0205] Optionally, the model training platform may further include a model library, wherein the model library stores multiple machine learning models.

[0206] The model training unit is also used to determine the initial diffusion language model from the model library.

[0207] In practical applications, users can select or add model attribute information for the initial diffusion language model to be used, such as model name, model version, model type, task adaptation type, model source, parameter magnitude, fine-tuning method, etc., based on their actual needs. The model training unit can then determine the initial diffusion language model to be trained from the model library based on the model attribute information provided by the user.

[0208] In one embodiment of this specification, the model training platform can adapt to user needs to train models, realizing personalized model training services and providing users with an efficient, flexible and easy-to-use model training service platform, thereby improving user experience.

[0209] The model training platform can execute the model training methods described above, or it can provide the diffusion language model used in the task processing methods described above, or train the initial diffusion language model.

[0210] Corresponding to the above method embodiments, this specification also provides an embodiment of a training device for a diffusion language model. Figure 8 This is a schematic diagram of a training device for a diffusion language model provided in one embodiment of this specification. Figure 8 As shown, the device may include:

[0211] The prediction result acquisition module 802 is used to acquire multiple intermediate prediction results generated by the initial diffusion language model for sample prompt words; the intermediate prediction results are the prediction results generated by performing a denoising operation during the process of the initial diffusion language model generating the final prediction result.

[0212] The reward value determination module 804 is used to determine the reward value of any intermediate prediction result among the plurality of intermediate prediction results.

[0213] The parameter adjustment module 806 is used to adjust the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model.

[0214] Optionally, the reward value determination module 804 may specifically include:

[0215] The first intermediate prediction result selection unit is used to select the intermediate prediction result to be optimized from the plurality of intermediate prediction results.

[0216] The first reward value determination unit is used to determine the reward value of any intermediate prediction result among the intermediate prediction results to be optimized.

[0217] Optionally, the first intermediate prediction result selection unit to be optimized may specifically include:

[0218] The average entropy calculation subunit is used to calculate the average entropy of each of the intermediate prediction results to obtain the word average entropy of each of the intermediate prediction results.

[0219] The first sorting subunit is used to sort the intermediate prediction results in descending order of the average entropy of the word units to obtain the first intermediate prediction result sequence.

[0220] The first intermediate prediction result determination subunit is used to determine the first preset number of intermediate prediction results in the first intermediate prediction result sequence as intermediate prediction results to be optimized.

[0221] The second sorting subunit is used to sort the intermediate prediction results in ascending order of the average entropy of the word units to obtain a second intermediate prediction result sequence.

[0222] The second intermediate prediction result determination subunit is used to determine the next preset number of intermediate prediction results in the second intermediate prediction result sequence as intermediate prediction results to be optimized.

[0223] Optionally, the parameter adjustment module 806 may include:

[0224] A positive adjustment unit is used to positively adjust the parameters of the initial diffusion language model based on the reward value if the reward value is greater than or equal to a preset threshold, so as to obtain the trained diffusion language model.

[0225] The reverse adjustment unit is used to adjust the parameters of the initial diffusion language model in reverse based on the reward value if the reward value is less than the preset threshold, so as to obtain the trained diffusion language model.

[0226] Optionally, the device may further include:

[0227] A preset threshold determination module is used to determine the preset threshold based on each reward value of each of the intermediate prediction results. The preset threshold includes at least the median or average value determined by each reward value.

[0228] Optionally, the preset threshold determination module may specifically include:

[0229] The second intermediate prediction result selection unit is used to select each intermediate prediction result to be optimized from the plurality of intermediate prediction results.

[0230] The first preset threshold determination unit is used to determine the preset threshold based on the reward values ​​of each intermediate prediction result to be optimized.

[0231] Optionally, the final prediction result includes multiple final prediction results, and each final prediction result corresponds to an intermediate prediction result set. The intermediate prediction result set contains multiple intermediate prediction results generated by the initial diffusion language model in the process of generating the final prediction result for the sample prompt words.

[0232] Optionally, the second intermediate prediction result selection unit to be optimized may specifically include:

[0233] The intermediate prediction result selection subunit is used to select the intermediate prediction results to be optimized from each of the intermediate prediction result sets to obtain each intermediate prediction result set to be optimized. Each intermediate prediction result set to be optimized contains intermediate prediction results selected from the same intermediate prediction result set.

[0234] The first preset threshold determination unit may specifically include:

[0235] The preset threshold determination subunit is used to determine the preset threshold based on the reward value of the intermediate prediction results to be optimized in each set of intermediate prediction results to be optimized.

[0236] The preset threshold determination subunit is specifically used to determine a target intermediate prediction result from other intermediate prediction result sets for any intermediate prediction result to be optimized in any intermediate prediction result set to be optimized in any intermediate prediction result set to be optimized; the other intermediate prediction result sets are intermediate prediction result sets in each intermediate prediction result set other than the intermediate prediction result set containing any intermediate prediction result to be optimized; the target intermediate prediction result represents an intermediate prediction result that has the same number of denoising steps as any intermediate prediction result to be optimized.

[0237] The preset threshold determination subunit is used to determine the preset threshold based on the reward value of each of the target intermediate prediction results and the reward value of any intermediate prediction result to be optimized.

[0238] Optionally, the device may further include:

[0239] The final prediction result acquisition module is used to acquire the final prediction result;

[0240] The reward value determination module is used to determine the reward value of the final prediction result;

[0241] The preset threshold determination module may specifically include:

[0242] The prediction result selection unit is used to select each prediction result to be optimized from the final prediction result and the plurality of intermediate prediction results;

[0243] The second preset threshold determination unit is used to determine the preset threshold based on each reward value of each prediction result to be optimized.

[0244] Optionally, the plurality of intermediate prediction results include a first intermediate prediction result and a second intermediate prediction result.

[0245] Optionally, the prediction result acquisition module 802 may specifically include:

[0246] A sample prompt word input unit is used to input the sample prompt word into the initial diffusion language model; wherein, the initial diffusion language model can determine, based on the sample prompt word, a first unprocessed data containing a preset number of unfilled word positions, perform a denoising operation on the first unprocessed data to generate a first intermediate prediction result, and convert the first intermediate prediction result into second unprocessed data by fixing some word positions in the first intermediate prediction result and masking word positions in other unfilled word positions; and perform a denoising operation on the masked word positions in the second unprocessed data to obtain the second intermediate prediction result;

[0247] An intermediate prediction result acquisition unit is used to acquire the first intermediate prediction result and the second intermediate prediction result.

[0248] Optionally, the step of fixing the word elements at the positions to be filled in the first intermediate prediction result may specifically include: obtaining the generation probability value of each word element in the first intermediate prediction result; sorting each word element in descending order of the generation probability value to obtain a first word element sequence; fixing the first preset number of word elements in the first word element sequence; or, sorting each word element in ascending order of the generation probability value to obtain a second word element sequence; and fixing the last preset number of word elements in the first word element sequence.

[0249] The above is an illustrative scheme of a training device for a diffusion language model according to this embodiment. It should be noted that the technical solution of this training device for a diffusion language model and the technical solution of the training method for a diffusion language model described above belong to the same concept. For details not described in detail in the technical solution of the training device for a diffusion language model, please refer to the description of the technical solution of the training method for a diffusion language model described above.

[0250] Corresponding to the above method embodiments, this specification also provides a task processing device embodiment. Figure 9 This is a schematic diagram of the structure of a task processing device provided in one embodiment of this specification. Figure 9 As shown, the device may include:

[0251] The task acquisition module 902 is used to acquire tasks to be processed, including text understanding and generation tasks or multimodal semantic understanding tasks.

[0252] The task input module 904 is used to input the task to be processed into the diffusion language model to obtain the processing result for the task to be processed. The diffusion language model is a model trained according to the method in at least one of the above embodiments.

[0253] The above is an illustrative scheme of a task processing device according to this embodiment. It should be noted that the technical solution of this task processing device and the technical solution of the task processing method described above belong to the same concept. For details not described in detail in the technical solution of the task processing device, please refer to the description of the technical solution of the task processing method described above.

[0254] Corresponding to the above method embodiments, this specification also provides an embodiment of a processing apparatus for content generation tasks. Figure 10 This is a schematic diagram of the structure of a content generation task processing device provided in one embodiment of this specification. Figure 10 As shown, the device may include: a content generation task acquisition module 1002, used to acquire content generation tasks to be processed; the content generation tasks to be processed include at least one of code completion tasks, text generation tasks, image generation tasks, audio generation tasks, and video generation tasks; a content generation task input module 1004, used to input the content generation tasks to be processed into a diffusion language model to obtain task generation result information; the diffusion language model is a model trained according to the method in at least one of the above embodiments.

[0255] Figure 11 This is a structural block diagram of a computing device provided in one embodiment of this specification.

[0256] The computing device 1100 includes: a memory 1110 and a processor 1120;

[0257] The memory 1110 is used to store computer programs or instructions, and the processor 1120 is used to execute the computer programs or instructions. When the computer programs or instructions are executed by the processor 1120, they implement the steps of the above-described training method or task processing method for the diffusion language model.

[0258] In one or more embodiments of this specification, the computing device can be understood as an integrated smart terminal, including but not limited to a server, desktop computer, PC (Personal Computer), all-in-one model machine, mobile phone, tablet computer or other portable smart terminal, etc., and the computing device may have the model described in the above embodiments of this application pre-installed.

[0259] Specifically, this computing device can pre-install various types of models, including but not limited to models in natural language processing, visual processing, speech processing, code processing, and multimodal task processing, thus providing diverse model selection. In different product forms, this computing device can support one or more model usage methods, including but not limited to model training, model invocation, model fine-tuning, model deployment, model inference, and application. In some product forms, this computing device also supports model management, including but not limited to multi-type model management (supporting the management of discriminative, generative, and other model types), model version control (supporting the control of different model versions), and model evaluation (evaluating model performance and effectiveness based on model evaluation tools). In other product forms, this computing device can also create applications based on models, providing API (Application Programming Interface) calling capabilities. Users can call models into created applications through the API interface, and application management tools are also provided to manage and monitor the applications.

[0260] Furthermore, the computing device can also include data management (supporting the creation and management of model tuning datasets), a training center (providing abundant training resources to help users learn and master AI (Artificial Intelligence) technology), and basic control capabilities (providing enterprise-level basic control capabilities to ensure the security and efficient operation of the system). Through the above functions, it provides a comprehensive and integrated device for AI development, training, deployment, and application.

[0261] Figure 12 This is a structural block diagram of an electronic device provided in one embodiment of this specification.

[0262] The memory 1210 and the processor 1220 are connected via a bus 1230;

[0263] The memory 1210 is used to store computer programs or instructions, and the processor 1220 is used to execute the computer programs or instructions. When the computer programs or instructions are executed by the processor 1220, they implement the steps of the above-described training method or task processing method for the diffusion language model.

[0264] Specifically, the components of the electronic device 1200 include, but are not limited to, a memory 1210 and a processor 1220. The processor 1220 is connected to the memory 1210 via a bus 1230, and the database 1250 is used to store data.

[0265] Electronic device 1200 also includes access device 1240, which enables electronic device 1200 to communicate via one or more networks 1260. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. Access device 1240 may include one or more of any type of wired or wireless network interface (e.g., network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and so on.

[0266] In one embodiment of this specification, the above-described components of the electronic device 1200 and Figure 12 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 12 The block diagram of the electronic device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0267] Electronic device 1200 can be any type of stationary or mobile electronic device, including mobile computers or mobile electronic devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable electronic devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary electronic devices such as desktop computers or personal computers (PCs). Electronic device 1200 can also be a mobile or stationary server.

[0268] The above is an illustrative scheme of an electronic device according to this embodiment. It should be noted that the technical solution of this electronic device belongs to the same concept as the technical solution of the above-described training method or task processing method for diffusion language models. For details not described in detail in the technical solution of the electronic device, please refer to the description of the technical solution of the above-described training method or task processing method for diffusion language models.

[0269] An embodiment of this specification also provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, implement the steps of the training method or task processing method for the diffusion language model described above.

[0270] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the above-described training method or task processing method for a diffusion language model. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the above-described training method or task processing method for a diffusion language model.

[0271] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described training method or task processing method for a diffusion language model.

[0272] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the above-described training method or task processing method for diffusion language models. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above-described training method or task processing method for diffusion language models.

[0273] The foregoing has described specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in a different order than that shown in the embodiments and may still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require the specific or sequential order shown to achieve the desired result. In some embodiments, multitasking and parallel processing are possible or may be advantageous.

[0274] The computer program or instructions include computer program code, which may be in the form of source code, object code, executable file, or certain intermediate forms. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording media, USB flash drive, portable hard drive, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0275] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments in this specification are not limited to the described order of actions, because according to the embodiments in this specification, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments in this specification.

[0276] In the above embodiments, the descriptions of each embodiment have different focuses. For parts not described in detail in a certain embodiment, please refer to the relevant descriptions of other embodiments.

[0277] The preferred embodiments disclosed above are merely illustrative of this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific implementations described. Clearly, many modifications and variations can be made based on the embodiments described herein. These embodiments are selected and specifically described in this specification to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A training method for a diffusion language model, characterized in that, include: Obtain multiple intermediate prediction results generated by the initial diffusion language model for the sample prompt words; The intermediate prediction results are the prediction results generated by performing a denoising operation during the process of generating the final prediction result from the initial diffusion language model; the multiple intermediate prediction results include a first intermediate prediction result, a second intermediate prediction result, and a third intermediate prediction result. For any one of the plurality of intermediate prediction results, determine the reward value of that intermediate prediction result; The parameters of the initial diffusion language model are adjusted based on the reward value to obtain the trained diffusion language model; The step of adjusting the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model includes: If the reward value of any intermediate prediction result is greater than or equal to the average value, the parameters of the initial diffusion language model are positively adjusted to obtain the trained diffusion language model; the average value is generated based on the reward value of the first intermediate prediction result, the reward value of the second intermediate prediction result, and the reward value of the third intermediate prediction result. If the reward value of any intermediate prediction result is less than the average value, the parameters of the initial diffusion language model are adjusted in reverse to obtain the trained diffusion language model.

2. The method according to claim 1, characterized in that, Determining the reward value for any intermediate prediction result among the plurality of intermediate prediction results includes: Select the intermediate prediction result to be optimized from the plurality of intermediate prediction results; For any intermediate prediction result among the intermediate prediction results to be optimized, determine the reward value of any intermediate prediction result.

3. The method according to claim 2, characterized in that, The step of selecting the intermediate prediction result to be optimized from the plurality of intermediate prediction results includes: The average entropy of each intermediate prediction result is calculated to obtain the word average entropy of each intermediate prediction result. The intermediate prediction results are sorted in descending order of the average entropy of the word units to obtain a first intermediate prediction result sequence. The first preset number of intermediate prediction results in the first intermediate prediction result sequence are identified as intermediate prediction results to be optimized. Alternatively, the intermediate prediction results are sorted in ascending order of the average entropy of the word units to obtain a second intermediate prediction result sequence. The last preset number of intermediate prediction results in the second intermediate prediction result sequence are identified as intermediate prediction results to be optimized.

4. The method according to claim 1, characterized in that, The step of adjusting the parameters of the initial diffusion language model based on the reward value to obtain the trained diffusion language model includes: If the reward value is greater than or equal to a preset threshold, the parameters of the initial diffusion language model are positively adjusted based on the reward value to obtain the trained diffusion language model. If the reward value is less than the preset threshold, the parameters of the initial diffusion language model are adjusted in reverse based on the reward value to obtain the trained diffusion language model.

5. The method according to claim 4, characterized in that, After determining the reward value of any intermediate prediction result among the plurality of intermediate prediction results, the method further includes: Based on the reward values ​​of each of the intermediate prediction results, the preset threshold is determined, and the preset threshold includes at least the median or average value determined by each reward value.

6. The method according to claim 5, characterized in that, Determining the preset threshold based on the reward values ​​of each of the intermediate prediction results includes: Select the individual intermediate prediction results that need to be optimized from the plurality of intermediate prediction results; Based on the reward values ​​of each intermediate prediction result to be optimized, the preset threshold is determined.

7. The method according to claim 6, characterized in that, The final prediction result includes multiple final prediction results, and each final prediction result corresponds to an intermediate prediction result set. The intermediate prediction result set contains multiple intermediate prediction results generated by the initial diffusion language model in the process of generating the final prediction result for the sample prompt words. The step of selecting each intermediate prediction result to be optimized from the plurality of intermediate prediction results includes: The intermediate prediction results to be optimized are selected from each set of intermediate prediction results to obtain each set of intermediate prediction results to be optimized. Each set of intermediate prediction results to be optimized contains intermediate prediction results to be selected from the same set of intermediate prediction results. The step of determining the preset threshold based on each reward value of each intermediate prediction result to be optimized includes: The preset threshold is determined based on the reward value of the intermediate prediction results to be optimized in each set of intermediate prediction results to be optimized.

8. The method according to claim 7, characterized in that, The step of determining the preset threshold based on the reward value of the intermediate prediction results to be optimized in each set of intermediate prediction results to be optimized includes: For any intermediate prediction result to be optimized in any set of intermediate prediction results to be optimized, a target intermediate prediction result is determined from other intermediate prediction result sets; the other intermediate prediction result sets are the intermediate prediction result sets in each set of intermediate prediction results excluding the intermediate prediction result set containing any intermediate prediction result to be optimized; the target intermediate prediction result represents an intermediate prediction result that uses the same number of denoising steps as any intermediate prediction result to be optimized. The preset threshold is determined based on the reward value of each of the intermediate prediction results for each target and the reward value of any intermediate prediction result to be optimized.

9. The method according to claim 6, characterized in that, The method further includes: Obtain the final prediction result; Determine the reward value of the final prediction result; Determining the preset threshold based on the reward values ​​of each of the intermediate prediction results specifically includes: Select the prediction results to be optimized from the final prediction results and the multiple intermediate prediction results; Based on the reward values ​​of each prediction result to be optimized, the preset threshold is determined.

10. The method according to any one of claims 1-9, characterized in that, The multiple intermediate prediction results include a first intermediate prediction result and a second intermediate prediction result. The acquisition of multiple intermediate prediction results generated by the initial diffusion language model for the sample prompt words includes: The sample prompt words are input into the initial diffusion language model; wherein, the initial diffusion language model can determine, based on the sample prompt words, a first set of data to be processed containing a preset number of word positions to be filled, perform a denoising operation on the first set of data to be processed, generate a first intermediate prediction result, and convert the first intermediate prediction result into second data to be processed by fixing some word positions to be filled in the first intermediate prediction result and masking word positions to be filled in other word positions; and perform a denoising operation on the word positions to be masked in the second set of data to be processed to obtain the second intermediate prediction result. Obtain the first intermediate prediction result and the second intermediate prediction result.

11. The method according to claim 10, characterized in that, The step of fixing the lexical units at certain positions to be filled in the first intermediate prediction result includes: Obtain the generation probability value of each word element in the first intermediate prediction result; Sort each word in descending order of its generation probability value to obtain a first word sequence, and fix the first preset number of words in the first word sequence; or, sort each word in ascending order of its generation probability value to obtain a second word sequence, and fix the last preset number of words in the first word sequence.

12. A task processing method, characterized in that, include: Obtain tasks to be processed, including text understanding and generation tasks or multimodal semantic understanding tasks; The task to be processed is input into the diffusion language model to obtain the processing result for the task to be processed. The diffusion language model is a model trained by the method according to any one of claims 1 to 11.

13. A method for processing content generation tasks, characterized in that, include: Retrieve content to be processed and generate tasks; The content generation task to be processed includes at least one of the following: code completion task, text generation task, image generation task, audio generation task, and video generation task. The content to be processed is input into the diffusion language model to obtain the task generation result information; The diffusion language model is a model trained using the method described in any one of claims 1 to 11.

14. A model training platform, characterized in that, Includes model training units and response units; The model training unit is used to train the initial diffusion language model according to the training method of any one of claims 1 to 10 to obtain the diffusion language model. The response unit is used to output the diffusion language model.

15. The model training platform according to claim 14, characterized in that, It also includes a data receiving unit; The data receiving unit is used to receive sample prompt words input by the user and send the sample prompt words to the model training unit.

16. The model training platform according to claim 15, characterized in that, It also includes a model library, which stores multiple machine learning models; The model training unit is also used to determine the initial diffusion language model from the model library.

17. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

18. An electronic device, characterized in that, include: A memory and a processor, the memory and the processor being connected via a bus; The memory is used to store computer programs or instructions, and the processor is used to execute the computer programs or instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 11.

19. A computer-readable storage medium, characterized in that, It stores a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

20. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 13.

Images