Noise processing method and device in model training, equipment, medium and product
By introducing a noise addition module to process noise in parallel during model training, simulating errors in the inference stage, and using a dual loss mechanism to train the target model, the problem of poor model robustness caused by the Teacher Forcing training method is solved, and the model's predictive ability in noisy environments is improved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- MOORE THREADS TECH CO LTD
- Filing Date
- 2025-12-24
- Publication Date
- 2026-05-19
AI Technical Summary
During model training, the significant distribution difference between the inference and training phases caused by the Teacher Forcing training method results in poor model robustness. Once an erroneous token is generated, the subsequent generation process deviates further, leading to error accumulation.
By introducing a noise-adding module, noise that simulates errors in the inference phase is generated during the training phase, and noise is added to the data sequence. A dual loss mechanism is used to train the target model and the noise-adding module, achieving parallel processing of the entire sequence.
It reduces exposure bias during the training and inference phases, improves the training effectiveness and robustness of the model, and enhances the model's predictive ability in noisy environments.
Smart Images
Figure CN122065898A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of artificial intelligence technology, and in particular to methods, apparatus, equipment, media and products for noise processing in model training. Background Technology
[0002] Currently, in large-scale language models, the Teacher Forcing training method is commonly used. This method uses the previous real token (data unit) as the model input during the training phase to predict the current token. This method can accelerate model convergence and make the loss function calculation more stable.
[0003] However, in Teacher Forcing training, there is a significant difference in the distribution between the inference and training phases. During training, the model always uses real tokens as input, while during inference, the model can only use self-generated tokens as input. This difference is called exposure bias. Due to exposure bias, the model's training performance is affected, resulting in poor robustness during the inference phase. Once an incorrect token is generated, subsequent generation processes will deviate further from the incorrect foundation, leading to error accumulation. Summary of the Invention
[0004] In view of the above problems, methods, apparatus, devices, media, and products relating to noise handling in model training are proposed to overcome or at least partially solve the above problems, including: A method for noise handling during model training, the method comprising: Obtain the first data sequence; A noise addition module is used to generate noise to simulate the errors produced by the target model during the inference phase, and the noise is added to the first data sequence to obtain a second data sequence. The target model is trained based on the second data sequence.
[0005] Optionally, the first data sequence includes multiple first data units. A noise addition module is used to generate noise to simulate errors generated by the target model during the inference phase, and the noise is added to the first data sequence to obtain a second data sequence, including: A noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units. For a subset of the plurality of first data units, a target erroneous data unit is determined from the plurality of candidate erroneous data units according to the first probability distribution; The target erroneous data unit is used to replace the corresponding first data unit to obtain the second data sequence.
[0006] Optionally, a noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units, including: A noise addition module is used to process multiple first data units in the first data sequence in parallel to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units.
[0007] Optionally, before determining the target erroneous data unit from the plurality of candidate erroneous data units according to the first probability distribution for a subset of the plurality of first data units, the method further includes: Obtain the data unit replacement probability, and determine a portion of the first data units among the plurality of first data units based on the data unit replacement probability.
[0008] Optionally, training the target model based on the second data sequence includes: Using the target model, a predicted data sequence is generated for the second data sequence; Based on the predicted data sequence and the first data sequence, a first loss is determined, and the target model is adjusted based on the first loss.
[0009] Optionally, the second data sequence includes multiple second data units, and the target model is used to generate a predicted data sequence for the second data sequence, including: Using the target model, multiple predicted data units corresponding to each second data unit in the second data sequence and a second probability distribution of the multiple predicted data units are generated; Based on the second probability distribution, a target prediction data unit is determined from the plurality of prediction data units, and a prediction data sequence for the second data sequence is generated based on the target prediction data unit.
[0010] Optionally, it also includes: Based on the first probability distribution and the second probability distribution, a second loss is determined, and the noise addition module is adjusted according to the second loss.
[0011] Optionally, the target model is a language processing model, and the first data sequence is a text sequence; Alternatively, the target model may be a video processing model, and the first data sequence may be a video sequence.
[0012] An apparatus for noise processing during model training, the apparatus comprising: The first data sequence acquisition module is used to acquire the first data sequence; The second data sequence obtaining module is used to generate noise to simulate the errors generated by the target model during the inference phase using the noise adding module, and add the noise to the first data sequence to obtain the second data sequence. The model training module is used to train the target model based on the second data sequence.
[0013] An electronic device includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.
[0014] A computer-readable storage medium on which a computer program is stored, which, when executed by a processor, implements the method described above.
[0015] A computer program product includes a computer program that, when executed by a processor, implements the method described above. The embodiments of this application have the following advantages: In this embodiment, by acquiring a first data sequence, a noise addition module is used to generate noise to simulate the errors generated by the target model during the inference stage, and noise is added to the first data sequence to obtain a second data sequence. Based on the second data sequence, the target model is trained. This realizes the insertion of noise into the training data and the use of noisy data for model training, which reduces exposure bias, improves the training effect of the model, and thus improves the robustness of the model in the inference stage and enhances the model's predictive ability in noisy environments. Attached Figure Description
[0016] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0017] Figure 1 This is a flowchart of the steps of a method for noise processing in model training provided in some embodiments of this application; Figure 2 This is a schematic diagram of a system architecture provided by some embodiments of this application; Figure 3 This is a flowchart of the steps of a second method for noise processing in model training provided in some embodiments of this application; Figure 4 This is a structural block diagram of a noise processing device for model training provided in some embodiments of this application. Detailed Implementation
[0018] To make the above-mentioned objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.
[0019] In related technologies, such as Scheduled Sampling, during the training phase, tokens generated by the model itself (non-real tokens) are used as inputs for the next step with a certain probability. The model is gradually exposed to its own prediction errors and tries to learn how to recover from non-ideal inputs and adapt to its own predicted output environment, thereby narrowing the input gap between the training and inference phases.
[0020] However, while the related technologies have narrowed the gap between training and inference, they adopt a serial approach during the training phase. The model output must be generated token by token as the input for the next step (i.e., after waiting for the output of the previous token, the previous token is used as the input for the next step). This has a strong sequential dependency and cannot be trained in parallel, which greatly reduces training efficiency and hardware utilization, especially in large-scale training.
[0021] Based on this, this application proposes a noise addition module to simulate real noise. Noise is added during the training phase, allowing the model to learn how to cope with noise, reducing exposure bias during training and inference, improving training performance, and consequently enhancing robustness during inference and predictive ability in noisy environments. Furthermore, in this application, the simulated noise is generated in parallel step-by-step, avoiding the low training efficiency and low hardware utilization issues caused by a serial approach.
[0022] Specifically, the embodiments of this application include the following aspects: 1. Introduction of a noise addition module: Before the target model training process, a noise addition module is inserted to learn and simulate prediction errors, thereby generating a noisy input token sequence. Moreover, the noise addition process is decoupled from the target model, allowing for flexible replacement and expansion, and exhibiting good versatility and adaptability.
[0023] 2. Dual loss mechanism: Two independent loss functions are used to train the noise addition module and the target model separately to avoid loss confusion. The target model is only used to predict the correct tokens and always uses the input ground truth as the supervision signal, so the training objective is clear and stable. The noise module is only used to simulate the prediction errors, which can avoid gradient interference and ensure training stability and clear objectives.
[0024] 3. Supports parallel training of the entire sequence: The noise generation process does not depend on the inference results of the model itself, avoiding sequence dependence in related technologies. The noise sequence is generated and trained in full parallel, improving training efficiency.
[0025] 4. Enhance model error recovery capability: Systematically construct a training environment that exposes the target model to common erroneous token inputs during training, enabling it to learn how to handle erroneous token inputs, recover from errors, and significantly improve robustness during the inference stage.
[0026] Take a long text sequence of 50 tokens as an example: In related technologies, predictions must be generated token by token before deciding whether to replace them, resulting in low training efficiency.
[0027] In this embodiment of the application, the following solution is adopted: 1. The noise addition module calculates the replacement scenarios for 50 tokens in parallel at once, resulting in a noisy sequence. 2. The entire noisy sequence is simultaneously fed into the target model for parallel prediction. 3. Even when the 20th token is replaced with an incorrect input, the model can still correctly predict the target token at the 21st position based on contextual information, thus learning to recover from errors.
[0028] In this way, the target model not only obtains a training environment that is closer to the inference stage, but also maintains efficient parallel training under long text conditions, significantly improving training efficiency and robustness.
[0029] The following example illustrates this point: Suppose the input token sequence is: "The ship is sailing smoothly on the vast blue ocean". If the model makes a reasoning error during inference, mistakenly reasoning the token as "huge" instead of "vast", then the sentence will become: "The ship is floating in a huge space." The inference process of the specific model is as follows: Input the above text "This ship is in", and the model calculates the next token. There may be a 40% probability of "vast" and a 41% probability of "huge", so "huge" is selected as the output token. The error of one token will cause the subsequent predicted semantics to change completely.
[0030] To solve and alleviate this problem, so that even if the target model makes an error in inferring a token, the subsequent tokens can still be correctly predicted: "This ship is sailing smoothly on the vast blue ocean." In the related technology, noise is added in a serial manner for training, specifically as follows: Initialize an empty data sequence, and sequentially add training tokens to this empty data sequence. At each step, first decide whether to add a real token or a token inferred by the model, and decide according to the probability.
[0031] 1. Add "this" with a real token to get [this].
[0032] 2. Add "ship" with a real token to get [this ship].
[0033] 3. Add "boat" with a real token to get [this ship].
[0034] 4. Add "in" with a real token to get [this ship is in].
[0035] 5. Add "huge" with the inferred token (noisy) of the model to get [this ship is in huge].
[0036] And so on, the output data becomes "This ship is sailing smoothly on the vast blue ocean." It can be seen that this method requires adding data step by step in series, and at the same time the model also needs to infer step by step.
[0037] In the embodiment of the present application, noise is added in a parallel manner for training, specifically as follows: 1. Input "This ship is sailing smoothly on the vast blue ocean." to the noise addition module, and obtain the possible error probability of each token. This probability refers to the possible wrong tokens that the model may predict this token into, and at the same time gives the probability. For example, "vast" may be predicted as "huge".
[0038] 2. The noise addition module learns the probability of the model's prediction (40% probability of "vast", 41% probability of "huge", and 19% probability of other). Based on this probability, tokens are randomly sampled. At this point, it is possible to replace "vast" with "huge", and the entire training data becomes "This ship is sailing smoothly on a vast blue ocean."
[0039] 3. Feed the noisy data to the target model for training. This way, even if an error is made in the middle, the model can still make the subsequent predicted tokens correct, ensuring that the semantics of the whole sentence will not change significantly.
[0040] As can be seen, by inputting the entire sentence into the noise addition module, outputting the probability distribution of all token errors in parallel, and then replacing the tokens in parallel, the training efficiency is improved compared to the serial method.
[0041] Reference Figure 1 The diagram illustrates a flowchart of a method for noise processing in model training according to some embodiments of this application. This method can be used as an enhancement module in the training phase of a target model, such as for language model training optimization in generation tasks such as autoregressive video generation and language generation.
[0042] Specifically, it may include the following steps: Step 101: Obtain the first data sequence.
[0043] In some embodiments of this application, the target model to be trained can be a multimodal model, such as a language processing model, where the first data sequence can be a text sequence, or a video processing model, where the first data sequence can be a video sequence.
[0044] In practical applications, a first data sequence (i.e., a token sequence) can be obtained as training data. The first data sequence is used as the input of the ground truth. The first data sequence includes multiple first data units (tokens), that is, the first data sequence is a sequence of multiple first data units arranged in order.
[0045] In some examples, tokens in text sequences can be single Chinese characters, words, phrases, or sub-words, while tokens in video sequences can be video frames, video clips, or video features.
[0046] For example, the first data sequence X For inclusion N A long text sequence or video sequence for each token:
[0047] in, This represents the i-th token.
[0048] Step 102: Using a noise addition module, noise is generated to simulate the errors produced by the target model during the inference phase, and the noise is added to the first data sequence to obtain the second data sequence.
[0049] In practical applications, such as Figure 2 A noise injection layer can be pre-trained. This noise injection layer can simulate the prediction error of the target model by inserting noise into the first data sequence, generating a second data sequence containing the inserted noise. In other words, the second data sequence carries the token of the insertion error.
[0050] The noise-adding module is used to learn and simulate the prediction error distribution of the target model. In some examples, the noise-adding module is a model that gains the ability to simulate prediction errors through training with a loss. For example, the noise-adding module can be a lightweight neural network (such as a multilayer perceptron), whose parameters are optimized by a second loss. Of course, it can also be adjusted according to the actual model error distribution, making the structure flexible and scalable. In another example, the noise-adding module can also be implemented based on preset rules, for example, based on a word confusion matrix obtained from historical errors to determine other tokens that each token is most likely to be incorrectly predicted as, and generate a first probability distribution accordingly.
[0051] It should be noted that the "noise" mentioned in the embodiments of this application does not refer to a random signal, but rather a substitute data unit (token) generated to simulate the errors that the target model may produce during autoregressive inference. The distribution of this noise is designed to approximate the model's own prediction error distribution, thereby constructing input conditions that are closer to the inference environment during training.
[0052] In some embodiments of this application, a noise addition module is used to generate noise to simulate errors generated by the target model during the inference phase, and the noise is added to the first data sequence to obtain a second data sequence. This includes: using the noise addition module to generate multiple candidate error data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate error data units; for a portion of the multiple first data units, determining a target error data unit from the multiple candidate error data units according to the first probability distribution; and replacing the corresponding first data unit with the target error data unit to obtain the second data sequence.
[0053] In practical applications, the first data sequence can be input into the noise addition module. Since the first data sequence contains multiple first data units, the noise addition module can generate multiple possible candidate erroneous data units for each first data unit and determine the corresponding first probability distribution of these erroneous data units. This first probability distribution reflects which erroneous data units the model might incorrectly predict the first data unit as during the inference stage. For example, for the first data sequence being the text sequence "The ship sails smoothly on the vast blue ocean.", for the input first data unit "vast", there is a 40% probability of "vast", a 41% probability of "huge", and a 19% probability of "other". Similarly, if the first data sequence is a video sequence, and the input first data unit is a video sequence, such as a video frame, video segment, or video feature, the noise addition module can analyze other similar video frames, segments, or features that the unit might be incorrectly predicted as, and their corresponding first probability distributions.
[0054] Subsequently, for a subset of the first data units in the first data sequence, a target erroneous data unit can be selected from multiple candidate erroneous data units according to a first probability distribution. In some examples, the process of selecting the target erroneous data unit can be random, but it is constrained by the first probability distribution, making erroneous data units with higher probabilities more likely to be selected.
[0055] Once the target erroneous data unit is identified, it can be used to replace the original first data unit, thereby generating a new data sequence, namely the second data sequence. This achieves the insertion of noise into the first data sequence, simulating the non-ideal input situations that the model may encounter during the inference phase.
[0056] In this way, the model can be exposed to noisy data during the training phase, thereby learning how to maintain stable predictive performance even in the presence of noise or errors. This not only helps reduce the exposure bias of the model between the training and inference phases, but also improves the overall predictive ability and robustness of the model.
[0057] In some embodiments of this application, a noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units. This includes: using the noise addition module to process multiple first data units in the first data sequence in parallel to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units.
[0058] Scheduled Sampling employs a serial processing approach, meaning each token must be processed sequentially. The processing of the (t+1)th token depends on the output of the t-th token, making it impossible to process the entire sequence at once, resulting in low training efficiency. Specifically, the model first predicts the first token, then decides, based on a certain probability, whether to use the actual token or the model-predicted token as the next input, before predicting the second token, and so on, until the sequence ends.
[0059] In this embodiment, parallel processing is used, which allows the entire input sequence (such as all tokens in a sentence) to be processed at once without waiting for the output of the previous token before processing the next token. Specifically, before training begins, the noise addition module receives the entire sequence of real tokens at once, and then generates error tokens for each token in the sequence in parallel. The entire process is completed at once, without depending on or waiting for any intermediate prediction results from the target model.
[0060] In this embodiment, the noise is generated in parallel by an additional module. That is, the noise generation process of the token is completely independent. The noise addition module can process the entire sequence in parallel without generating a serial dependency per token. The target model can still be trained in parallel for the entire sequence and batch, which further improves the efficiency and effect of model training.
[0061] In some embodiments of this application, before determining the target erroneous data unit from the plurality of candidate erroneous data units according to the first probability distribution for a portion of the plurality of first data units, the method further includes: obtaining the data unit replacement probability, and determining a portion of the first data units from the plurality of first data units according to the data unit replacement probability.
[0062] The data unit replacement probability is a pre-defined probability value used to determine the proportion of first data units in the first data sequence that will be replaced with the target erroneous data unit. In some examples, the data unit replacement probability can be adjusted according to actual needs. For example, if it is desired that the model be exposed to noisy data more during the training phase to enhance its robustness to noise, a higher data unit replacement probability can be set; if it is desired that the model be trained more on real data to maintain its prediction accuracy on real data, a lower data unit replacement probability can be set.
[0063] In practical applications, it is not necessary to replace all the first data units in the first data sequence with the erroneous data units. A data unit replacement probability, such as 10%, can be preset. Then, based on the data unit replacement probability, it can be determined which first data unit in the first data sequence needs to be replaced with the erroneous data unit, that is, to determine a portion of the first data units in the first data sequence.
[0064] For the first data units that need to be replaced with erroneous data units, the target erroneous data units selected according to the first distribution probability can be used for replacement. For the other first data units, they can be retained in the second data sequence to obtain a data sequence carrying noise.
[0065] Step 103: Train the target model based on the second data sequence.
[0066] After obtaining the second data sequence, it can be input into the target model for training. The model's task remains predicting the next, noise-free token. When an incorrect token is input at a certain position, the model will attempt to recover and predict the correct target token based on the context. This training method enables the model to have the ability to "self-correct" and "recover from errors" when encountering incorrect input during the inference phase. This allows the target model to learn robust prediction capabilities under non-ideal conditions, specifically how to output the correct token even with incorrect input.
[0067] In some embodiments of this application, training the target model based on the second data sequence includes: using the target model to generate a predicted data sequence for the second data sequence; determining a first loss based on the predicted data sequence and the first data sequence; and adjusting the target model based on the first loss.
[0068] In practical applications, the target model can make predictions based on the second data sequence to obtain the predicted data sequence. Then, the output predicted data sequence is compared with the first data sequence, which is used as the input of the true value, to calculate the first loss. Based on the first loss value, the gradient and parameters of the target model are adjusted so that the target model can generate the true data sequence based on the input data sequence containing noise.
[0069] For example, the first loss can be obtained by calculating the cross-entropy between the predicted data sequence output by the target model and the first data sequence used as the true value input, using the following formula:
[0070] in, Let X be the first loss sequence, and X be the first data sequence. This is the second data sequence. To predict data sequences.
[0071] In this embodiment, by introducing a noise addition module, a second data sequence containing simulated error noise is provided to the target model during the training phase, and the prediction loss of the second data sequence is calculated. The target model is then adjusted based on the loss, which improves the training effect of the model and enhances its robustness during the inference phase. This improves the model's predictive ability in noisy environments, enabling the target model to more accurately predict the correct target token based on the experience accumulated during training when faced with real erroneous inputs during the inference phase. This effectively improves the prediction accuracy and stability of the model in complex environments.
[0072] In some embodiments of this application, the second data sequence includes a plurality of second data units. Using the target model, a predicted data sequence for the second data sequence is generated, including: Using the target model, multiple predicted data units corresponding to each second data unit in the second data sequence and a second probability distribution of the multiple predicted data units are generated; according to the second probability distribution, a target predicted data unit is determined from the multiple predicted data units, and a predicted data sequence for the second data sequence is generated according to the target predicted data unit.
[0073] In practical applications, after receiving the second data sequence, the target model can make predictions for each second data unit, generate multiple corresponding predicted data units, and determine the second probability distribution of these predicted data units. The second probability distribution reflects the model's assessment of the likelihood of each predicted data unit.
[0074] Subsequently, based on the second probability distribution, the model can select one target prediction data unit from multiple prediction data units. In some examples, the selection process for the target prediction data unit can be based on the maximum probability principle, i.e., selecting the prediction data unit with the highest probability, or other strategies can be adopted, such as random selection but constrained by the probability distribution.
[0075] After determining the target prediction data unit corresponding to each second data unit, the model combines these target prediction data units in sequence to generate a prediction data sequence for the entire second data sequence. The prediction data sequence is the result of the model's prediction of the second data sequence, which can be used for subsequent loss calculation and model adjustment.
[0076] In some embodiments of this application, it also includes: Based on the first probability distribution and the second probability distribution, a second loss is determined, and the noise addition module is adjusted according to the second loss.
[0077] In practical applications, the first probability distribution and the second probability distribution can be used to determine the second loss. The first probability distribution is the probability distribution of the erroneous data units generated by the noise addition module, while the second probability distribution is the probability distribution of the predicted data units generated by the target model.
[0078] After obtaining the second loss, the gradient and parameters of the noise addition module can be adjusted using the second loss, so that the noise addition module can learn the distribution of erroneous tokens that the target model may generate in noiseless training, and thus generate more reasonable noise in subsequent training.
[0079] For example, the cross-entropy of the first and second probability distributions is used to calculate the second loss, using the following formula:
[0080] in, This is the second loss. This is the first probability distribution. This represents the second probability distribution.
[0081] By simultaneously optimizing the first loss and the second loss, such as Figure 2 The model training method in this embodiment enables joint optimization of the target model and the noise addition module. This not only improves the target model's predictive ability in noisy environments but also allows the noise addition module to generate noise that better reflects the actual situation, thereby further improving the effectiveness and efficiency of model training. Furthermore, by decoupling the first and second losses, the gradient of the first loss applies only to the target model, and the gradient of the second loss applies only to the noise addition module, avoiding gradient interference and ensuring a stable and clear training process.
[0082] In this embodiment, by acquiring a first data sequence, a noise addition module is used to generate noise to simulate the errors generated by the target model during the inference stage, and noise is added to the first data sequence to obtain a second data sequence. Based on the second data sequence, the target model is trained. This realizes the insertion of noise into the training data and the use of noisy data for model training, which reduces exposure bias, improves the training effect of the model, and thus improves the robustness of the model in the inference stage and enhances the model's predictive ability in noisy environments.
[0083] Building upon the foregoing embodiments, when the target model is an autoregressive video generation model, the first data sequence can be a video frame sequence or a video feature sequence. The noise addition module is used to simulate errors that the video generation model may make when predicting the next frame or the next set of features. For example, for a data unit corresponding to a certain video frame, the noise module may learn that the model has a probability of incorrectly predicting it as a feature from another frame that is semantically similar but has different content. By training on such video sequences with simulated errors, the video generation model can acquire robustness to its own generation errors.
[0084] Reference Figure 3 The diagram illustrates a flowchart of another method for noise processing during model training provided in some embodiments of this application, which may specifically include the following steps: Step 301: Obtain a first data sequence; wherein the first data sequence includes a plurality of first data units.
[0085] In some embodiments of this application, the target model to be trained can be a multimodal model, such as a language processing model, where the first data sequence can be a text sequence, or a video processing model, where the first data sequence can be a video sequence.
[0086] In practical applications, a first data sequence (i.e., a token sequence) can be obtained as training data. The first data sequence is used as the input of the ground truth. The first data sequence includes multiple first data units (tokens), that is, the first data sequence is a sequence of multiple first data units arranged in order.
[0087] In some examples, tokens in text sequences can be single Chinese characters, words, phrases, or sub-words, while tokens in video sequences can be video frames, video clips, or video features.
[0088] Step 302: Using a noise addition module, generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units.
[0089] In practical applications, such as Figure 2 A noise injection layer can be pre-trained. This noise injection layer can simulate the prediction error of the target model by inserting noise into the first data sequence, generating a second data sequence containing the inserted noise. In other words, the second data sequence carries the token of the insertion error.
[0090] The noise-adding module is used to learn and simulate the prediction error distribution of the target model. In some examples, the noise-adding module is a model that gains the ability to simulate prediction errors through training with a loss. For example, the noise-adding module can be a lightweight neural network (such as a multilayer perceptron), whose parameters are optimized by a second loss. Of course, it can also be adjusted according to the actual model error distribution, making the structure flexible and scalable. In another example, the noise-adding module can also be implemented based on preset rules, for example, based on a word confusion matrix obtained from historical errors to determine other tokens that each token is most likely to be incorrectly predicted as, and generate a first probability distribution accordingly.
[0091] It should be noted that the "noise" mentioned in the embodiments of this application does not refer to a random signal, but rather a substitute data unit (token) generated to simulate the errors that the target model may produce during autoregressive inference. The distribution of this noise aims to approximate the model's own prediction error distribution, thereby constructing input conditions that are closer to the inference environment during training. In practical applications, a first data sequence can be input into the noise addition module. Since the first data sequence contains multiple first data units, the noise addition module can generate multiple possible candidate erroneous data units for each first data unit and determine the corresponding first probability distribution of these erroneous data units. This first probability distribution reflects which erroneous data units the model may incorrectly predict as the first data unit during the inference stage. For example, for the first data sequence being the text sequence "This ship sails smoothly on the vast blue ocean.", for the input first data unit "vast", there is a 40% probability of "vast", a 41% probability of "huge", and a 19% probability of other. For example, if the first data sequence is a video sequence, and the first input data unit is a video sequence, such as a certain video frame, video segment, or video feature, the noise addition module can analyze other similar video frames, segments, or features that may be incorrectly predicted as this unit and their corresponding first probability distributions.
[0092] Scheduled Sampling employs a serial processing approach, meaning each token must be processed sequentially. The processing of the (t+1)th token depends on the output of the t-th token, making it impossible to process the entire sequence at once, resulting in low training efficiency. Specifically, the model first predicts the first token, then decides, based on a certain probability, whether to use the actual token or the model-predicted token as the next input, before predicting the second token, and so on, until the sequence ends.
[0093] In this embodiment, parallel processing is used, which allows the entire input sequence (such as all tokens in a sentence) to be processed at once without waiting for the output of the previous token before processing the next token. Specifically, before training begins, the noise addition module receives the entire sequence of real tokens at once, and then generates error tokens for each token in the sequence in parallel. The entire process is completed at once, without depending on or waiting for any intermediate prediction results from the target model.
[0094] In this embodiment, the noise is generated in parallel by an additional module. That is, the noise generation process of the token is completely independent. The noise addition module can process the entire sequence in parallel without generating a serial dependency per token. The target model can still be trained in parallel for the entire sequence and batch, which further improves the efficiency and effect of model training.
[0095] Step 303: Obtain the data unit replacement probability, and determine a portion of the first data units among the plurality of first data units based on the data unit replacement probability.
[0096] The data unit replacement probability is a pre-defined probability value used to determine the proportion of first data units in the first data sequence that will be replaced with the target erroneous data unit. In some examples, the data unit replacement probability can be adjusted according to actual needs. For example, if it is desired that the model be exposed to noisy data more during the training phase to enhance its robustness to noise, a higher data unit replacement probability can be set; if it is desired that the model be trained more on real data to maintain its prediction accuracy on real data, a lower data unit replacement probability can be set.
[0097] In practical applications, it is not necessary to replace all the first data units in the first data sequence with the erroneous data units. A data unit replacement probability, such as 10%, can be preset. Then, based on the data unit replacement probability, it can be determined which first data unit in the first data sequence needs to be replaced with the erroneous data unit, that is, to determine a portion of the first data units in the first data sequence.
[0098] Step 304: For a portion of the plurality of first data units, determine the target erroneous data unit from the plurality of candidate erroneous data units according to the first probability distribution.
[0099] For a subset of first data units in the first data sequence, a target erroneous data unit can be selected from multiple candidate erroneous data units according to a first probability distribution. In some examples, the process of selecting the target erroneous data unit may be random, but it is constrained by the first probability distribution, making erroneous data units with higher probabilities more likely to be selected.
[0100] Step 305: Replace the corresponding first data unit with the target erroneous data unit to obtain the second data sequence.
[0101] Once the target erroneous data unit is identified, it can be used to replace the original first data unit, thereby generating a new data sequence, namely the second data sequence. This achieves the insertion of noise into the first data sequence, simulating the non-ideal input situations that the model may encounter during the inference phase.
[0102] In this way, the model can be exposed to noisy data during the training phase, thereby learning how to maintain stable predictive performance even in the presence of noise or errors. This not only helps reduce the exposure bias of the model between the training and inference phases, but also improves the overall predictive ability and robustness of the model.
[0103] Moreover, in the embodiments of this application, the noise is generated in parallel by an additional module, that is, the noise generation process of the token is completely independent, the noise addition module can process the entire sequence in parallel at the same time, without generating a serial dependency per token, and the target model can still be trained in parallel with the entire sequence and the entire batch, which further improves the efficiency and effect of model training.
[0104] Step 306: Train the target model according to the second data sequence.
[0105] After obtaining the second data sequence, it can be input into the target model for training. The model's task remains predicting the next, noise-free token. When an incorrect token is input at a certain position, the model will attempt to recover and predict the correct target token based on the context. This training method enables the model to have the ability to "self-correct" and "recover from errors" when encountering incorrect input during the inference phase. This allows the target model to learn robust prediction capabilities under non-ideal conditions, specifically how to output the correct token even with incorrect input.
[0106] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.
[0107] Reference Figure 4 The diagram shows a structural schematic of a noise processing apparatus for model training provided in some embodiments of this application, which may specifically include the following modules: First data sequence acquisition module 401 is used to acquire a first data sequence; The second data sequence obtaining module 402 is used to use a noise adding module to generate noise to simulate the errors generated by the target model during the inference stage, and to add the noise to the first data sequence to obtain the second data sequence. The model training module 403 is used to train the target model based on the second data sequence.
[0108] In some embodiments of this application, the first data sequence includes a plurality of first data units. A noise addition module is used to generate noise to simulate errors generated by the target model during the inference phase, and the noise is added to the first data sequence to obtain a second data sequence, including: A noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units. For a subset of the plurality of first data units, a target erroneous data unit is determined from the plurality of candidate erroneous data units according to the first probability distribution; The target erroneous data unit is used to replace the corresponding first data unit to obtain the second data sequence.
[0109] In some embodiments of this application, a noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units, including: A noise addition module is used to process multiple first data units in the first data sequence in parallel to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units.
[0110] In some embodiments of this application, it also includes: A partial first data unit determination module is used to obtain the data unit replacement probability and determine a portion of the first data units among the plurality of first data units based on the data unit replacement probability.
[0111] In some embodiments of this application, model training of the target model based on the second data sequence includes: Using the target model, a predicted data sequence is generated for the second data sequence; Based on the predicted data sequence and the first data sequence, a first loss is determined, and the target model is adjusted based on the first loss.
[0112] In some embodiments of this application, the second data sequence includes a plurality of second data units, and the target model is used to generate a predicted data sequence for the second data sequence, including: Using the target model, multiple predicted data units corresponding to each second data unit in the second data sequence and a second probability distribution of the multiple predicted data units are generated; Based on the second probability distribution, a target prediction data unit is determined from the plurality of prediction data units, and a prediction data sequence for the second data sequence is generated based on the target prediction data unit.
[0113] In some embodiments of this application, it also includes: The noise addition module adjustment module is used to determine a second loss based on the first probability distribution and the second probability distribution, and to adjust the noise addition module based on the second loss.
[0114] In some embodiments of this application, the target model is a language processing model and the first data sequence is a text sequence; or, the target model is a video processing model and the first data sequence is a video sequence.
[0115] In this embodiment, by acquiring a first data sequence, a noise addition module is used to generate noise to simulate the errors generated by the target model during the inference stage, and noise is added to the first data sequence to obtain a second data sequence. Based on the second data sequence, the target model is trained. This realizes the insertion of noise into the training data and the use of noisy data for model training, which reduces exposure bias, improves the training effect of the model, and thus improves the robustness of the model in the inference stage and enhances the model's predictive ability in noisy environments.
[0116] Some embodiments of this application also provide an electronic device, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method described above.
[0117] Some embodiments of this application also provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, it implements the method described above.
[0118] Some embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the method described above.
[0119] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.
[0120] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.
[0121] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.
[0122] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0123] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0124] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal equipment, causing a series of operational steps to be performed on the computer or other programmable terminal equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable terminal equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.
[0126] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.
[0127] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the aforementioned element.
[0128] The methods, apparatus, devices, media, and products for noise processing in model training have been described in detail above. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.
Claims
1. A method for noise processing during model training, characterized in that, The method includes: Obtain the first data sequence; A noise addition module is used to generate noise to simulate the errors produced by the target model during the inference phase, and the noise is added to the first data sequence to obtain a second data sequence. The target model is trained based on the second data sequence.
2. The method according to claim 1, characterized in that, The first data sequence includes multiple first data units. A noise addition module is used to generate noise to simulate errors produced by the target model during the inference phase. The noise is then added to the first data sequence to obtain a second data sequence, including: A noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units. For a subset of the plurality of first data units, a target erroneous data unit is determined from the plurality of candidate erroneous data units according to the first probability distribution; The target erroneous data unit is used to replace the corresponding first data unit to obtain the second data sequence.
3. The method according to claim 2, characterized in that, A noise addition module is used to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units, including: A noise addition module is used to process multiple first data units in the first data sequence in parallel to generate multiple candidate erroneous data units corresponding to each first data unit in the first data sequence and a first probability distribution of the multiple candidate erroneous data units.
4. The method according to claim 2, characterized in that, Before determining the target erroneous data unit from the plurality of candidate erroneous data units according to the first probability distribution for a subset of the plurality of first data units, the method further includes: Obtain the data unit replacement probability, and determine a portion of the first data units among the plurality of first data units based on the data unit replacement probability.
5. The method according to claim 2, 3, or 4, characterized in that, Based on the second data sequence, model training is performed on the target model, including: Using the target model, a predicted data sequence is generated for the second data sequence; Based on the predicted data sequence and the first data sequence, a first loss is determined, and the target model is adjusted based on the first loss.
6. The method according to claim 5, characterized in that, The second data sequence includes multiple second data units. Using the target model, a predicted data sequence is generated for the second data sequence, including: Using the target model, multiple predicted data units corresponding to each second data unit in the second data sequence and a second probability distribution of the multiple predicted data units are generated; Based on the second probability distribution, a target prediction data unit is determined from the plurality of prediction data units, and a prediction data sequence for the second data sequence is generated based on the target prediction data unit.
7. The method according to claim 6, characterized in that, Also includes: Based on the first probability distribution and the second probability distribution, a second loss is determined, and the noise addition module is adjusted according to the second loss.
8. The method according to claim 1, characterized in that, The target model is a language processing model, and the first data sequence is a text sequence. Alternatively, the target model may be a video processing model, and the first data sequence may be a video sequence.
9. A device for noise processing in model training, characterized in that, The device includes: The first data sequence acquisition module is used to acquire the first data sequence; The second data sequence obtaining module is used to generate noise to simulate the errors generated by the target model during the inference phase using the noise adding module, and add the noise to the first data sequence to obtain the second data sequence. The model training module is used to train the target model based on the second data sequence.
10. An electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the computer program, when executed by the processor, implements the method as described in any one of claims 1 to 8.
11. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the method as described in any one of claims 1 to 8.
12. A computer program product, characterized in that, It includes a computer program that, when executed by a processor, implements the method as described in any one of claims 1 to 8.