Model training method, data processing method, device, equipment, medium and product

Abstract

This disclosure relates to a model training method, data processing method, apparatus, device, medium, and product. The model training method includes: acquiring raw training data; constructing target training data based on the raw training data, the target training data including instruction data, input data, and output data, wherein the instruction data and the input data serve as input to a language model, and the output data serves as output to the language model; performing masking processing on the target training data according to a preset masking rule, the preset masking rule including multi-stage masking processing; and training the language model based on the target training data and the masked target training data to obtain a trained language model. Through this disclosure, training the model based on target training data with multi-stage masking processing improves the alignment effect between instruction data, input data, and output data, enhancing the generalization ability of the language model.

Description

Technical Field

[0001] This disclosure relates to the field of machine learning, and in particular to a model training method, data processing method, apparatus, device, medium, and product. Background Technology

[0002] With the development of artificial intelligence, language models such as Large Language Models (LLMs) have also developed and are increasingly being applied in various data processing applications. Before data processing, language models require training based on language model instruction alignment.

[0003] Language model instruction alignment is primarily achieved by collecting user instruction data, input data, and corresponding output data to train the language model for alignment. This alignment training enables the language model to understand and follow instructions, and to provide more reasonable responses.

[0004] In related technologies, alignment training of language models only uses the output data, enabling the language model to generate corresponding output data according to a unified format and content for both instruction and input data during the output process. However, the models in these technologies struggle to fully understand the instruction and input data during use, making it difficult to determine the output data to be generated. Consequently, the models fail to generate output data that matches the instruction and input data. Summary of the Invention

[0005] To overcome the problems existing in related technologies, this disclosure provides a model training method, data processing method, apparatus, equipment, medium and product.

[0006] According to a first aspect of the present disclosure, a model training method is provided, comprising: acquiring raw training data;

[0007] Target training data is constructed based on the original training data. The target training data includes instruction data, input data, and output data, wherein the instruction data and the input data serve as inputs to the language model, and the output data serves as the output of the language model. The target training data is then masked according to a preset masking rule, which includes multi-stage masking. Based on the target training data and the masked target training data, the language model is trained to obtain a trained language model.

[0008] In one embodiment, the multi-stage masking process includes a first-stage masking process and a second-stage masking process; the first-stage masking process represents masking the instruction data in the target training data, and the second-stage masking process represents masking the instruction data and input data in the target training data.

[0009] In one embodiment, the masked target training data includes target training data after a first stage of masking and target training data after a second stage of masking. The step of training the language model based on the target training data and the masked target training data to obtain a trained language model includes: performing initial training on the language model based on the target training data to obtain an initially trained language model; training the initially trained language model based on the first stage of masked target training data to obtain a second-trained language model; and training the second-trained language model based on the second stage of masked target training data to obtain a target language model, and using the target language model as the trained language model.

[0010] In one embodiment, the step of performing initial training on the language model based on the target training data to obtain the language model after initial training includes: obtaining first prediction data based on first target input data and the language model, wherein the first target input data includes instruction data, input data, and a portion of output data from the output data in the target training data, and the first prediction data includes first prediction instruction data, first prediction input data, and first prediction output data; determining a first loss value based on the portion of instruction data in the instruction data, the input data, the output data, and the first prediction data; and training and optimizing the language model based on the first loss value to obtain the language model after initial training.

[0011] In one embodiment, training the language model after the first training stage based on the target training data after the first stage masking to obtain the language model after the second training stage includes: obtaining second prediction data based on second target input data and the language model after the first training stage, wherein the second target input data includes instruction data, input data, and a portion of output data from the output data in the target training data, and the second prediction data includes second prediction instruction data, second prediction input data, and second prediction output data; masking the second prediction instruction data to obtain second prediction mask data, and masking the instruction data in the target training data to obtain first target mask training data; determining a second loss value based on the second prediction mask data and the first target mask training data; and adjusting the language model after the first training stage based on the second loss value to obtain the language model after the second training stage.

[0012] In one embodiment, training the language model after secondary training based on the target training data after the second-stage masking to obtain a target language model includes: obtaining third prediction data based on third target input data and the language model after secondary training, wherein the third target input data includes instruction data, input data, and a portion of output data from the output data in the target training data, and the third prediction data includes third prediction instruction data, third prediction input data, and third prediction output data; masking the third prediction instruction data and the third prediction input data to obtain third prediction mask data, and masking the instruction data and input data in the target training data to obtain second target mask training data; determining a third loss value based on the third prediction mask data and the second target mask training data; and adjusting the language model after secondary training based on the third loss value to obtain the target language model.

[0013] In one embodiment, obtaining the third prediction data based on the third target input data and the language model after secondary training includes: extracting features from the third target input data to obtain first feature data; adding noise data to the first feature data to obtain second feature data; and obtaining the third prediction data based on the second feature data and the language model after secondary training.

[0014] According to a second aspect of the present disclosure, a data processing method is provided, comprising: acquiring instruction data and input data; inputting the instruction data and the input data into a language model to obtain output data; wherein the language model is obtained based on the model training method according to any one of the first aspects of the claim.

[0015] According to a third aspect of the present disclosure, a model training apparatus is provided, comprising: a first acquisition unit for acquiring raw training data; and a first processing unit for constructing target training data based on the raw training data, performing masking processing on the target training data according to a preset masking rule, and training a language model based on the target training data and the masked target training data to obtain a trained language model, wherein the preset masking rule includes multi-stage masking processing, and the target training data includes instruction data, input data, and output data, wherein the instruction data and the input data serve as inputs to the language model, and the output data serves as outputs to the language model.

[0016] In one embodiment, the multi-stage masking process includes a first-stage masking process and a second-stage masking process; the first-stage masking process represents masking the instruction data in the target training data, and the second-stage masking process represents masking the instruction data and input data in the target training data.

[0017] In one embodiment, the masked target training data includes target training data after a first stage of masking and target training data after a second stage of masking. The first processing unit trains the language model based on the target training data and the masked target training data in the following manner to obtain a trained language model: the language model is trained for the first time based on the target training data to obtain a language model after the first training; the language model after the first training is trained based on the target training data after the first stage of masking to obtain a language model after a second training; the language model after the second training is trained based on the target training data after the second stage of masking to obtain a target language model, and the target language model is used as the trained language model.

[0018] In one embodiment, the first processing unit performs initial training on the language model based on the target training data in the following manner to obtain the language model after initial training: Based on the first target input data and the language model, first prediction data is obtained, wherein the first target input data includes instruction data, input data, and a portion of output data from the target training data, and the first prediction data includes first prediction instruction data, first prediction input data, and first prediction output data; based on the portion of instruction data from the instruction data, the input data, the output data, and the first prediction data, a first loss value is determined; based on the first loss value, the language model is trained and optimized to obtain the language model after initial training.

[0019] In one embodiment, the first processing unit trains the language model after the first training stage based on the target training data after the first stage masking process in the following manner to obtain a language model after secondary training: Based on the second target input data and the language model after the first training stage, second prediction data is obtained, wherein the second target input data includes instruction data, input data, and a portion of output data from the output data in the target training data, and the second prediction data includes second prediction instruction data, second prediction input data, and second prediction output data; the second prediction instruction data is masked to obtain second prediction mask data, and the instruction data in the target training data is masked to obtain first target mask training data; based on the second prediction mask data and the first target mask training data, a second loss value is determined; based on the second loss value, the language model after the first training stage is adjusted to obtain the language model after secondary training.

[0020] In one embodiment, the first processing unit trains the second-trained language model based on the target training data after the second-stage masking process to obtain a target language model in the following manner: Based on the third target input data and the second-trained language model, third prediction data is obtained, wherein the third target input data includes instruction data, input data, and a portion of output data from the target training data; the third prediction data includes third prediction instruction data, third prediction input data, and third prediction output data; the third prediction instruction data and the third prediction input data are masked to obtain third prediction mask data, and the instruction data and input data in the target training data are masked to obtain second target mask training data; based on the third prediction mask data and the second target mask training data, a third loss value is determined; based on the third loss value, the second-trained language model is adjusted to obtain the target language model.

[0021] In one embodiment, the processing unit obtains third prediction data based on the third target input data and the second-trained language model in the following manner: performing feature extraction on the third target input data to obtain first feature data; adding noise data to the first feature data to obtain second feature data; and obtaining the third prediction data based on the second feature data and the second-trained language model.

[0022] According to a fourth aspect of the present disclosure, a data processing apparatus is provided, comprising: a second acquisition unit for acquiring instruction data and input data; and a second processing unit for inputting the instruction data and the input data into a language model to obtain output data; wherein the language model is obtained based on the model training method according to any one of the first aspects of the claim.

[0023] According to a fifth aspect of the present disclosure, an electronic device is provided, comprising:

[0024] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the model training method described in the first aspect or any embodiment of the first aspect.

[0025] According to a sixth aspect of the present disclosure, an electronic device is provided, comprising:

[0026] A processor; a memory for storing processor-executable instructions; wherein the processor is configured to perform the data processing method described in the second aspect.

[0027] According to a seventh aspect of the present disclosure, a storage medium is provided, the storage medium storing instructions that, when executed by a processor of an electronic device, enable the electronic device to perform the model training method described in the first aspect or any embodiment of the first aspect.

[0028] According to an eighth aspect of the present disclosure, a storage medium is provided that stores instructions which, when executed by a processor of an electronic device, enable the electronic device to perform the data processing method described in the second aspect.

[0029] According to a ninth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the model training method described in the first aspect or any embodiment of the first aspect.

[0030] According to a tenth aspect of the present disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the data processing method described in the second aspect.

[0031] The technical solutions provided by the embodiments of this disclosure can include the following beneficial effects: Target training data is obtained by constructing data including instruction data, input data, and output data. Multi-stage masking processing is performed on the instruction data, input data, and output data in the target training data to obtain masked target training data. Using the target training data and the masked target training data, training is performed on the alignment relationships between the data, enabling the language model to learn and optimize the multi-stage masked data. This enhances the language model's understanding of instruction data and input data, and improves the alignment effect between masked and unmasked data in each stage, thereby improving the training effect of the language model. Furthermore, the trained language model can, based on the improved understanding and alignment effect of instruction data and input data during training, understand instruction data and input data that are not present in the target training data, or generate output data when instruction data is missing, improving the generalization ability and output stability of the language model, making the language model applicable to different scenarios.

[0032] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0033] The accompanying drawings, which are incorporated in and form a part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure.

[0034] Figure 1 This is a schematic diagram illustrating alignment training in a related art according to an exemplary embodiment.

[0035] Figure 2 This is a flowchart illustrating a model training method according to an exemplary embodiment.

[0036] Figure 3 This is a schematic diagram illustrating the process of aligning a language model based on target training data, according to an exemplary embodiment.

[0037] Figure 4 This is a schematic diagram illustrating target training data after the first stage of masking processing, according to an exemplary embodiment.

[0038] Figure 5 This is a schematic diagram illustrating target training data after undergoing a second-stage masking process, according to an exemplary embodiment.

[0039] Figure 6 This is a flowchart illustrating a model training method according to an exemplary embodiment.

[0040] Figure 7This is a flowchart illustrating a first-stage training according to an exemplary embodiment.

[0041] Figure 8 This is a flowchart illustrating a second-stage training according to an exemplary embodiment.

[0042] Figure 9 This is a schematic diagram illustrating the generation of a second prediction mask data according to an exemplary embodiment.

[0043] Figure 10 This is a flowchart illustrating a third-stage training according to an exemplary embodiment.

[0044] Figure 11 This is a schematic diagram illustrating the generation of third prediction mask data according to an exemplary embodiment.

[0045] Figure 12 This is a flowchart illustrating a third predictive data acquisition method according to an exemplary embodiment.

[0046] Figure 13 This is a schematic diagram illustrating a model training method according to an exemplary embodiment.

[0047] Figure 14 This is a flowchart illustrating a data processing method according to an exemplary embodiment.

[0048] Figure 15 This is a block diagram illustrating a model training apparatus according to an exemplary embodiment.

[0049] Figure 16 This is a block diagram illustrating a data processing apparatus according to an exemplary embodiment.

[0050] Figure 17 This is a block diagram illustrating an apparatus for model training or data processing according to an exemplary embodiment.

[0051] Figure 18 This is a block diagram illustrating an apparatus for model training or data processing according to an exemplary embodiment. Detailed Implementation

[0052] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure.

[0053] In the accompanying drawings, the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The described embodiments are some, but not all, of the embodiments of this disclosure. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain this disclosure, and should not be construed as limiting this disclosure. All other embodiments obtained by those skilled in the art based on the embodiments of this disclosure without inventive effort are within the scope of protection of this disclosure. The embodiments of this disclosure will be described in detail below with reference to the accompanying drawings.

[0054] The model training method provided in this disclosure is mainly applied to training language models and using language models for data processing. It should be understood that the model training method provided in this disclosure can be applied to various types of language models. For example, the language model can be a large language model (LLM) with architectures such as feedforward neural networks, convolutional neural networks (CNNs), recurrent neural networks (RNNs), long short-term memory networks (LSTMs), or Transformers, or it can be a statistical language model such as an n-gram model.

[0055] With the rapid development of artificial intelligence (AI) technology, alignment training has become an important technical means to improve the performance of language models in processing diverse data.

[0056] In related technologies, when aligning and training a language model, a portion of the output data from the prompt data, input data, and output data is input into the language model. Forward propagation is then performed to obtain the language model's output. The language model's output includes predicted prompt data, predicted input data, and predicted output data. Loss calculations are performed based on the output data and predicted output data, followed by backpropagation to adjust parameters or optimize the algorithm. The prompt data serves as the fixed initial input or context for the language model, providing a starting point or direction for text generation. It enables the language model to determine the type of task to be performed or the topic of the generated text; for example, it can be commands or guiding data. The input data consists of the data received by the language model during training that requires processing. The output data is the target result data generated by the language model based on the input data.

[0057] In a language model, instruction data and input data can be words, sentences, or paragraphs. The language model predicts words or sentences based on the instruction data and input data, obtaining the predicted words or sentences. The predicted words or sentences include content that matches the instruction data and input data input into the language model; that is, the predicted instruction data and predicted input data. Furthermore, the predicted words or sentences also include words or sentences related to the predicted instruction data and predicted input data, used to answer the questions raised by the predicted instruction data and predicted input data; this part of the words or sentences constitutes the predicted output data.

[0058] In an exemplary embodiment, the process of aligning and training a language model in the related art is as follows: Figure 1 As shown. Figure 1 This is a schematic diagram illustrating alignment training in a related art according to an exemplary embodiment. Figure 1In this model, training data includes instruction data, input data, and output data. Instruction data includes data 1 to data 4, input data includes data 5 to data 8, and output data includes data 8 to data 13. By inputting data 1 to data 12 into the language model, the language model makes predictions based on data 1 to data 12, generating prediction results including data 2 to data 13, thus completing the forward propagation of the language model. In related technologies, both the instruction data (corresponding to predicted instruction data) and the input data (corresponding to predicted input data) in the training data and prediction data are masked to obtain masked training data and masked prediction data. Specifically, the masked training data includes masked training data 1 to masked training data 8 and unmasked training data 9 to unmasked training data 13, and the masked prediction data includes masked prediction data 2 to masked prediction data 8 and unmasked prediction data 9 to unmasked prediction data 13. Based on the masked training data, the masked prediction data, and a preset loss function, the loss value of the language model is calculated. Based on the calculated loss value, the language model is backpropagated, including operations such as gradient calculation or parameter update.

[0059] Depend on Figure 1 It is known that in the related technologies, during the training of language models, the language model mainly calculates the loss based on the output data in the training data and the output data predicted by the language model based on the input data in the training data, and adjusts the language model accordingly. This results in the language model having poor understanding of the input instruction data and the input data, as well as poor alignment between the instruction data and the input data and the output data.

[0060] In this embodiment of the disclosure, masking processing, also known as masking, represents replacing the content in the language model input with mask markers.

[0061] In related technologies, instruction and input data in training and prediction data are masked. This causes the language model to primarily determine the loss value based on the output data from the training and prediction data during loss calculation. Consequently, backpropagation based on this loss value focuses on parameter adjustments to improve the accuracy of the output data. Therefore, language models trained using these technologies have limitations in their understanding and generalization abilities regarding instruction and input data. For example, when the input data to the language model includes the input data but lacks instruction data, the language model in these technologies cannot infer the corresponding instruction data based on the input data, nor can it generate output data that matches the input data.

[0062] In view of this, embodiments of this disclosure provide a model training method that constructs the data used for model training into a data format of instruction data, input data, and output data, and performs multi-stage masking processing on the instruction data and input data used for training. Based on the target training data and the target training data after multi-stage masking processing, a language model is trained, enabling the trained language model to enhance the alignment effect between instruction data, input data, and output data, improve the language model's understanding of different instruction data and input data, and enhance the training effect of the language model. By using the model training method provided in embodiments of this disclosure to train the language model, the trained language model can understand instruction data and input data that are not present in the training data, or generate output data that corresponds to the input data when instruction data is missing, improving the generalization ability and output stability of the language model, making the language model applicable to different scenarios.

[0063] Figure 2 This is a flowchart illustrating a model training method according to an exemplary embodiment, such as... Figure 2 As shown, it includes the following steps.

[0064] In step S11, the original training data is obtained.

[0065] In this embodiment of the disclosure, the language model can be trained using supervised learning. During supervised learning, the original training data of the language model includes instruction data, input data, and output data. By learning the mapping relationship from model inputs (including instruction data and input data) to model outputs (including predicted output data), the language model can accurately predict new model inputs and obtain model outputs.

[0066] It should be understood that the format of the original training data for a language model can vary. For example, for a large language model, the original training data can be multiple sets of text, including instruction text, input text, and output text. Alternatively, if a large language model can process multimodal data, the original training data can be multiple sets of images, including instruction text, input text, and output text.

[0067] In step S12, target training data is constructed based on the original training data.

[0068] In this embodiment of the disclosure, the target training data includes instruction data, input data, and output data. The instruction data and input data serve as input to the language model, and the output data serves as output to the language model. The instruction data represents one of multiple tasks that the language model can perform. For example, if the language model is a large language model, and the original training data of the large language model includes text comprising instruction text, input text, and output text, then the target training data may include instruction data, input data, and output data. The instruction data is the instruction text in the original training data, or the instruction data is encoded data obtained by encoding the instruction text in the original training data. The input data is the input text in the original training data, or the input data is encoded data obtained by encoding the input text in the original training data. The output data is the output text in the original training data, or the output data is encoded data obtained by encoding the output text in the original training data.

[0069] In this embodiment of the disclosure, there is a correspondence between the instruction data, input data, and output data in the target training data. The instruction data, input data, and output data included in the correspondence in the target training data are determined based on the original training data in the original training data.

[0070] In step S13, the target training data is masked according to a preset masking rule.

[0071] In this embodiment of the disclosure, the preset masking rule includes multi-stage masking processing, which performs multi-stage masking operations on the constructed target training data including instruction data, input data and output data. The masking processing at different stages can represent masking different data in the target training data.

[0072] In step S14, the language model is trained based on the target training data and the masked target training data to obtain the trained language model.

[0073] In this embodiment, a language model is trained using target training data and target training data processed through multi-stage masking. The language model learns the alignment relationships between the target training data based on the masked instruction data, input data, and output data, enabling it to learn from the target training data processed through multi-stage masking. Furthermore, by using the target training data processed through multi-stage masking, the language model is trained specifically, improving its ability to learn from the unmasked target training data in the multi-stage process. This enhances its ability to learn the semantic features of the instruction data, input data, and output data in the target training data, as well as the contextual relationships between the data, thereby improving the training effect of the language model.

[0074] In one example, multi-stage masking processing may include first-stage masking processing and second-stage masking processing. The first-stage masking processing represents masking the instruction data in the target training data, while the second-stage masking processing represents masking both the instruction data and the input data in the target training data.

[0075] It should be understood that, in the embodiments of this disclosure, the target training data can be treated as a stage mask even without masking, representing a stage mask where the data to be masked is 0.

[0076] In one exemplary embodiment, the process of aligning and training a language model using target training data is as follows: Figure 3 As shown, Figure 3 This is a schematic diagram illustrating a process for aligning a language model based on target training data, according to an exemplary embodiment. Figure 3 In this process, a predetermined amount of data is acquired as the original training data. This original training data is then constructed into a format of "instruction data + input data + output data" to obtain the target training data. Multi-stage masking processing includes first-stage masking and second-stage masking. First, the instruction data is masked, while the unmasked output and input data are retained, thus masking the instruction data in the target training data to obtain the target training data after the first stage of masking. Second, the instruction and input data are masked, while the unmasked output data is retained, thus masking the instruction and input data in the target training data to obtain the target training data after the second stage of masking. The language model is then trained using the target training data, the target training data after the first stage of masking, and the target training data after the second stage of masking, respectively, to achieve model alignment training. In the language model alignment training, the output loss value is calculated, and backpropagation is performed based on the loss value to obtain the trained language model, which serves as the language model after alignment training. The language model after alignment training can also be called the alignment model.

[0077] In this embodiment of the disclosure, the target training data is taken as text, and the instruction data, input data, and output data are taken as text segments, to illustrate the target training data after the first stage of masking processing and the target training data after the second stage of masking processing, respectively. Figure 4 and Figure 5 As shown. Figure 4 This is a schematic diagram illustrating target training data after the first stage of masking processing, according to an exemplary embodiment. Figure 4 In the code, the text corresponding to the instruction data is masked, while the text corresponding to the input data and the text corresponding to the output data are not masked. Figure 5This is a schematic diagram illustrating target training data after undergoing a second-stage masking process, according to an exemplary embodiment. Figure 5 In the target training data, the text corresponding to the instruction data and the text corresponding to the input data are masked, while the text corresponding to the output data is not masked.

[0078] In this embodiment, the target training data after multi-stage masking can be used to perform multiple iterative training operations on the language model. This allows the language model to output data that matches the output data in the target training data during use, improving the alignment effect of the language model obtained after iterative training and thus enhancing the accuracy of the output data generated by the trained language model. Furthermore, training the language model using the target training data enables the language model to learn from the instruction and input data in the target training data, enhancing its understanding of the features contained in the instruction and input data. It also allows the language model to understand instruction data not included in the target training data and unknown input data, improving the generalization ability of the language model.

[0079] In this embodiment of the disclosure, the target training data after masking processing may include target training data after a first-stage masking process and target training data after a second-stage masking process. The target training data after the first-stage masking process represents the data obtained by performing a first-stage masking process on the target training data, and the target training data after the second-stage masking process represents the data obtained by performing a second-stage masking process on the target training data.

[0080] For example, a language model can be trained in three stages using unmasked target training data, target training data after the first stage of masking, and target training data after the second stage of masking, to obtain a trained language model.

[0081] Figure 6 This is a flowchart illustrating a model training method according to an exemplary embodiment, such as... Figure 6 As shown, it includes the following steps.

[0082] In step S21, the language model is trained for the first time based on the target training data to obtain the language model after the first training.

[0083] In this embodiment of the disclosure, during the initial training of the language model, the language model is used to perform loss calculation and backpropagation on the predicted data generated by the language model and the instruction data, input data and output data in the target training data. This enables the language model to perform loss calculation and backpropagation based on the predicted data and the instruction data, input data and output data in the target training data, thereby enhancing the language model's ability to understand the semantic features in the instruction data, input data and output data.

[0084] In step S22, based on the target training data after the first stage masking process, the language model after the first training is trained to obtain the language model after the second training.

[0085] In this embodiment, the target training data after the first-stage masking process is used to perform a second training on the language model after its initial training. This training allows the language model to predict instruction data matching the input data, and generate output data matching both the input data and the predicted instruction data, even without instruction data. Furthermore, compared to the language model after the initial training, the language model after the second training can predict corresponding instruction data based on the input data, demonstrating an understanding of the instruction intent corresponding to the input data, and an improved alignment effect between the instruction data and the input / output data (including both input and output data). The alignment effect between the instruction data and the input / output data represents the ability to understand the corresponding instruction intent and generate output data based on the input data even when the instruction data is missing or inaccurate. Moreover, this training allows the language model to further learn from the knowledge in the input and output data, building upon its initial training.

[0086] In step S23, based on the target training data after the second-stage masking process, the language model after secondary training is trained to obtain the target language model, and the target language model is used as the trained language model.

[0087] In this embodiment, the target training data after the second-stage masking process is used to train the language model after secondary training, thereby improving the alignment effect between the language model and the instruction input data (including instruction data and input data) and the output data. The alignment effect between the instruction data and the input / output data characterizes the ability to generate output data that matches the features of the output data in the target training data, even when the instruction data and input data are missing or inaccurate.

[0088] In this embodiment of the disclosure, the language model is trained using target training data processed through multi-stage masking. This enables the language model to improve the alignment between instruction data and input / output data in stages, based on its semantic understanding capabilities. This alignment between the language model and the instruction input / output data enhances the language model's data understanding and data alignment, improves the accuracy of output data generation by the trained language model, and reduces overfitting of the language model to the target training data. The trained language model can then understand the information expressed by the positional instruction data based on the input data when receiving unknown instruction data, and generate output data corresponding to the instruction data and input data, thereby improving the generalization ability of the language model.

[0089] It should be understood that the initial training of the language model based on the target training data can also be called the first stage of training; the training of the language model after the initial training based on the target training data after the first stage of masking can also be called the second stage of training; and the training of the language model after the second training based on the target training data after the second stage of masking can also be called the third stage of training. Furthermore, this disclosure uses the above three-stage training as an example for illustration, but does not limit the number of training stages; other training stages besides three stages can also be performed on the language model.

[0090] In this embodiment of the disclosure, the first stage of training in the three-stage training of the language model is described below.

[0091] Figure 7 This is a flowchart illustrating a first-stage training according to an exemplary embodiment, such as... Figure 7 As shown, it includes the following steps.

[0092] In step S31, the first prediction data is obtained based on the first target input data and the language model.

[0093] In this embodiment of the disclosure, the target input data includes instruction data, input data, and a portion of output data from the target training data.

[0094] In this embodiment of the disclosure, the first prediction data includes first prediction instruction data, first prediction input data, and first prediction output data. The first target input data is input into a language model, and the language model performs predictions based on the target input data to obtain the first prediction data.

[0095] In step S32, a first loss value is determined based on a portion of the instruction data, input data, output data, and first prediction data in the instruction data.

[0096] In this embodiment of the disclosure, a portion of the instruction data in the instruction data corresponds to the first predicted instruction data, the input data corresponds to the first predicted input data, and the output data corresponds to the first predicted output data. By inputting the first predicted data, the portion of the instruction data corresponding to the first predicted data in the target training data, the input data, and the output data into a preset loss function, the loss value of the first training is obtained and used as the first loss value to determine the error of the instruction data, input data, and output data predicted by the language model.

[0097] In step S33, the language model is trained and optimized based on the first loss value to obtain the language model after the first training.

[0098] In this embodiment of the disclosure, a first loss value, which characterizes the error of the command data, input data and output data predicted by the language model, is used to optimize the command data prediction ability, input data prediction ability and output data prediction ability of the language model. This enables the language model to understand and learn the commands, needs or answers expressed by the command data, input data and output data, thereby improving the language model's ability to understand command data, input data and output data.

[0099] In this embodiment of the disclosure, after the language model is trained for the first time, the instruction data can be masked during the loss calculation process to optimize the alignment between the instruction data and the input / output data.

[0100] In this embodiment of the disclosure, the second stage of training in the three-stage training of the language model is described below.

[0101] Figure 8 This is a flowchart illustrating a second-stage training method according to an exemplary embodiment, such as... Figure 8 As shown, it includes the following steps.

[0102] In step S41, second prediction data is obtained based on the second target input data and the language model after the first training.

[0103] In this embodiment of the disclosure, the second target input data includes instruction data, input data, and a portion of output data from the target training data. The second prediction data includes second prediction instruction data, second prediction input data, and second prediction output data. The second target input data may be the same as or different from the first target input data.

[0104] It should be understood that the second target input data may also exclude some of the output data from the output data.

[0105] In this embodiment of the disclosure, a portion of the output data from the instruction data, input data, and output data is input into the language model after its initial training. The language model is then controlled to make predictions based on the target input data to obtain second prediction data. The second prediction data includes second prediction instruction data, second prediction input data, and second prediction output data.

[0106] In step S42, the second prediction instruction data is masked to obtain the second prediction mask data, and the instruction data in the target training data is masked to obtain the first target mask training data.

[0107] In this embodiment of the disclosure, second prediction mask data is obtained by masking the second prediction instruction data. The second prediction input data and the second prediction output data in the second prediction mask data can be used for loss calculation.

[0108] In step S43, a second loss value is determined based on the second predicted mask data and the first target mask training data.

[0109] In this embodiment of the disclosure, the second prediction mask data includes second prediction instruction data corresponding to a portion of the instruction data in the first target mask training data, second prediction input data corresponding to the input data in the first target mask training data, and second prediction output data corresponding to the output data in the first target mask training data. By inputting the second prediction mask data and the first target mask training data into a preset loss function, loss calculation is performed based on the input data, second prediction input data, output data, and second prediction output data to determine a second loss value. This second loss value is then used to determine the error between the input / output data predicted by the language model and the input / output data in the target training data.

[0110] In step S44, the language model after the first training is adjusted based on the second loss value to obtain the language model after the second training.

[0111] In one exemplary embodiment, the target training data includes instruction data 1 to instruction data 4, input data 5 to input data 8, and output data 9 to output data 13. For example, the language generates second prediction data based on the target training data and performs masking processing on the second prediction data to obtain second prediction mask data. Figure 9 As shown. Figure 9 This is a schematic diagram illustrating the generation of second prediction mask data according to an exemplary embodiment. Figure 9In this process, instruction data 1 to instruction data 4, input data 5 to input data 8, and output data 9 to output data 12 are used as target input data. The language model after the first training is controlled to make predictions based on the target input data, generating second prediction data, including second prediction instruction data (prediction instruction data 2 to prediction instruction data 4), second prediction input data (prediction input data 5 to prediction input data 8), and second prediction output data (output data 9 to output data 13). Prediction instruction data 2 to prediction instruction data 4 in the second prediction data are masked to obtain masked prediction instruction data 2 to masked prediction instruction data 4. The instruction data 2 to instruction data 4 in the second prediction data are replaced with the masked prediction instruction data 2 to masked prediction instruction data 4 to obtain the second prediction mask data.

[0112] In this embodiment of the disclosure, a second loss value, representing the error between the input and output data predicted by the language model and the input and output data in the target training data, is used to optimize the input data prediction ability and output data prediction ability of the language model after the first training. This enables the language model after the second training to ensure that the predicted input and output data are consistent for different instruction data, thereby learning the alignment relationship between instruction data and input and output data.

[0113] In this embodiment of the disclosure, after the language model is trained a second time, the instruction data and input data can be masked during the loss calculation process to optimize the alignment between the instruction input data and the output data.

[0114] In this embodiment of the disclosure, the third stage of training in the three-stage training of the language model is described below.

[0115] Figure 10 This is a flowchart illustrating a third-stage training method according to an exemplary embodiment, such as... Figure 10 As shown, it includes the following steps.

[0116] In step S51, the third prediction data is obtained based on the third target input data and the language model after secondary training.

[0117] In this embodiment of the disclosure, the third target input data includes instruction data, input data, and a portion of output data from the target training data; the third prediction data includes third prediction instruction data, third prediction input data, and third prediction output data. The third target input data may be the same as or different from the first target input data and / or the second target input data.

[0118] It should be understood that the third target input data may also exclude some of the output data from the output data.

[0119] In this embodiment of the disclosure, the third target input data is input into the language model after secondary training, and the language model is controlled to make predictions based on the third target input data to obtain third prediction data. The third prediction data includes third prediction instruction data, third prediction input data, and third prediction output data.

[0120] In step S52, the third prediction instruction data and the third prediction input data are masked to obtain the third prediction mask data, and the instruction data and input data in the target training data are masked to obtain the second target mask training data.

[0121] In this embodiment of the disclosure, the third prediction instruction data and the third prediction input data are masked, wherein the second prediction output data in the third prediction mask data can be used for loss calculation.

[0122] In step S53, a third loss value is determined based on the third predicted mask data and the second target mask training data.

[0123] In this embodiment of the disclosure, the third prediction mask data corresponds to a portion of the instruction data in the instruction data of the second target mask training data, the third prediction input data corresponds to the input data in the second target mask training data, and the third prediction output data corresponds to the output data in the second target mask training data. By inputting the third prediction mask data and the second target mask training data into a preset loss function, loss calculation is performed based on the second prediction output data and the output data in the target training data to determine a third loss value. This third loss value is then used to determine the error between the output data predicted by the language model and the output data in the target training data.

[0124] In step S54, the language model after secondary training is adjusted based on the third loss value to obtain the target language model.

[0125] In this embodiment of the disclosure, backpropagation using a third loss value is used to adjust the parameters of the language model, thereby enhancing the alignment between the language model and the instruction input data (including instruction data and input data) and the output data based on the initial and secondary training. This enables the language model to generate satisfactory output data based on the contextual relationships between instruction data and input data, even for instruction data and / or input data that have not been learned during use.

[0126] In an exemplary embodiment, taking the target training data including instruction data 1 to instruction data 4, input data 5 to input data 8, and output data 9 to output data 13 as an example, the language generates third prediction data based on the target training data, and performs masking processing on the third prediction data to obtain third prediction mask data. Figure 11 As shown. Figure 11 This is a schematic diagram illustrating the generation of third prediction mask data according to an exemplary embodiment. Figure 11 In this process, instruction data 1 to instruction data 4, input data 5 to input data 8, and output data 9 to output data 12 are used as target input data. The language model after the first training is controlled to make predictions based on the target input data, generating third prediction data, including third prediction instruction data (predicted instruction data 2 to predicted instruction data 4), third prediction input data (predicted input data 5 to predicted input data 8), and third prediction output data (output data 9 to output data 13). The predicted instruction data 2 to predicted instruction data 4 and the predicted input data 5 to predicted input data 8 in the third prediction data are masked to obtain masked predicted instruction data 2 to masked predicted instruction data 4 and masked predicted input data 2 to masked predicted input data 4. The instruction data 2 to instruction data 4 in the third prediction data are replaced with the masked instruction data 2 to masked predicted instruction data 4, and the input data 5 to input data 8 are replaced with the masked input data 5 to masked predicted input data 8, to obtain the third prediction mask data.

[0127] In this embodiment of the disclosure, a third loss value, which characterizes the error of the output data predicted by the language model, is used to optimize the output data prediction capability of the language model. This enables the target language model to ensure that the predicted output data is consistent for different instruction data and input data, thereby learning the alignment relationship between instruction input data and output data.

[0128] In this embodiment of the disclosure, during the training of the language model after secondary training, noise data can be added to the features corresponding to the third target input data to obtain noisy features. These noisy features are then used to train the language model after secondary training to obtain the target language model. This avoids overfitting of the language model to the output data and improves the generalization ability of the language model. The following describes the process of adding noise data to the features corresponding to the third target input data.

[0129] Figure 12 This is a flowchart illustrating a third predictive data acquisition method according to an exemplary embodiment, such as... Figure 12 As shown, it includes the following steps.

[0130] In step S61, feature extraction is performed on the third target input data to obtain the first feature data.

[0131] In this embodiment of the disclosure, the features (or representations) corresponding to the third target input data can be obtained during the feature extraction process of the language model on the target input data. For example, the feature vector obtained after performing a preset number of convolutional pooling operations on the third target input data can be used as the first feature data.

[0132] In step S62, noise data is added to the first feature data to obtain the second feature data.

[0133] In this embodiment of the disclosure, noise data can be added to the first feature data based on a preset noise addition rule. The noise data should be less than a preset noise threshold to avoid the feature information being completely covered due to excessive noise.

[0134] In step S63, the third prediction data is obtained based on the second feature data and the language model after secondary training.

[0135] In this embodiment of the disclosure, noise is added to the features corresponding to the target input data to obtain noisy second feature data. The language model learns based on the noisy second feature data, enabling the model to learn stable features in the data, avoiding the problem of the language model overfitting the target training data, and improving the generalization ability of the language model.

[0136] In this embodiment of the disclosure, the following exemplary description, in conjunction with exemplary embodiments, illustrates the process of performing multi-stage masking processing on the target training data and iteratively training the language model. The language model is a large language model based on the Transformer architecture. The instruction data is the prompt text, the input data is the question text, and the output data is the response text.

[0137] In one exemplary embodiment, a schematic diagram illustrating the training of a language model using target training data and target training data after multi-stage masking is shown below. Figure 13 As shown. Figure 13 This is a schematic diagram illustrating a model training method according to an exemplary embodiment. Figure 13In this process, data is collected for training a large language model. This collected data, also known as raw training data, can be text composed of prompt text, input text, and output text. Based on the collected data, the prompt text is used as instruction data, the input text as input data, and the output text as output data, constructing target training data in the format of "instruction data + input data + output data". A first-stage masking process is performed on the target training data, keeping the prompt text, input text, and output text unchanged, resulting in the masked data. The masked data is then used to perform alignment training on the large language model, completing the initial training of the large language model. After the initial training of the large model, a second-stage masking process is performed on the target training data. In this second-stage masking process, the prompt text in the target training data is masked, and the masked target training data is used as the masked data. The masked data is then used to perform alignment training on the large language model, completing the second training of the large language model. After secondary training of the large model, a two-stage masking process is performed on the target training data. During the third-stage masking process, the prompt text and question text in the target training data are masked, resulting in the masked data. Using this masked data, the large language model is trained for alignment, completing three training iterations. Furthermore, during this alignment training process, noise can be added to the learned feature representations (corresponding to the first feature data) to obtain second feature data. This second feature data is then used by the language model to generate the third prediction data, preventing overfitting, improving generalization ability, and outputting the final, three-stage trained language model as the target language model.

[0138] The alignment training process for the large language model at each stage is as follows: The large language model performs forward propagation based on the target training data to obtain prediction data including predicted prompt text, predicted question text, and predicted response text. A mask is constructed from the prediction data obtained at each stage. Using the masked data and the masked prediction data from each stage, loss is calculated to obtain the loss value. The large language model then performs backpropagation based on the loss value to adjust the parameters of the large language model.

[0139] In this embodiment, multi-stage masking processing is applied to the target training data, and the model is trained using the masked target training data from each stage. This enables the model to learn the knowledge included in the instruction data and input data, achieving accurate representation of instructions and inputs. Furthermore, by using masked target training data and prediction data for loss calculation and optimization, the alignment between instruction data and input / output data is effectively improved. Moreover, by using masked target training data and prediction data for loss calculation and optimization, the alignment between instruction input data and output data is further enhanced. Additionally, the trained language model can understand instruction data and input data not present in the training data and generate output data, improving the generalization ability and output stability of the language model, making it applicable to different scenarios.

[0140] Based on the same concept, this disclosure also provides a data processing method.

[0141] Figure 14 This is a flowchart illustrating a data processing method according to an exemplary embodiment, such as... Figure 14 As shown, it includes the following steps.

[0142] In step S71, instruction data and input data are acquired.

[0143] In this embodiment of the disclosure, the instruction data representation is used to indicate one of a variety of tasks, which are tasks that the language model can perform. The input data representation provides the language model with data to be processed or to be responded to.

[0144] In step S72, the instruction data and input data are input into the language model to obtain the output data.

[0145] In this embodiment of the disclosure, the language model is trained using target training data and target training data after multi-stage masking processing. The target training data includes instruction data, input data, and output data. Masking processing of the target training data may include masking the instruction data and / or input data within the target training data.

[0146] In this embodiment of the disclosure, a language model trained using target training data after multi-stage masking is used to enable the language model to understand instruction data and input data during the prediction of output data, and to maintain consistency of predicted output data for different instruction data and input data, thereby improving the accuracy and stability of predicted output data.

[0147] It is understood that the language model in the data processing method involved in the embodiments of this disclosure is obtained by the model training method provided in the above embodiments of this disclosure, and will not be described again here.

[0148] Based on the same concept, embodiments of this disclosure also provide a model training apparatus.

[0149] It is understood that the model training apparatus provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0150] Figure 15 This is a block diagram illustrating a model training apparatus 100 according to an exemplary embodiment. (Refer to...) Figure 15 The device includes a first acquisition unit 101 and a first processing unit 102.

[0151] The first acquisition unit 101 is used to acquire the original training data.

[0152] The first processing unit 102 is used to construct target training data based on the original training data, perform masking processing on the target training data according to a preset masking rule, and train the language model based on the target training data and the masked target training data to obtain the trained language model. The preset masking rule includes multi-stage masking processing. The target training data includes instruction data, input data and output data, wherein the instruction data and input data serve as the input of the language model, and the output data serves as the output of the language model.

[0153] In one embodiment, the multi-stage masking process includes a first-stage masking process and a second-stage masking process; the first-stage masking process represents masking the instruction data in the target training data, and the second-stage masking process represents masking the instruction data and input data in the target training data.

[0154] In one embodiment, the masked target training data includes target training data after a first stage of masking and target training data after a second stage of masking. The first processing unit 102 trains the language model based on the target training data and the masked target training data in the following manner to obtain a trained language model: the language model is trained for the first time based on the target training data to obtain a language model after the first training; the language model after the first training is trained based on the target training data after the first stage of masking to obtain a language model after the second training; the language model after the second training is trained based on the target training data after the second stage of masking to obtain a target language model, and the target language model is used as the trained language model.

[0155] In one embodiment, the first processing unit 102 performs initial training on the language model based on the target training data in the following manner to obtain the language model after initial training: Based on the first target input data and the language model, first prediction data is obtained, wherein the first target input data includes instruction data, input data, and a portion of output data from the target training data, and the first prediction data includes first prediction instruction data, first prediction input data, and first prediction output data; based on the portion of instruction data, input data, output data, and the first prediction data, a first loss value is determined; and based on the first loss value, the language model is trained and optimized to obtain the language model after initial training.

[0156] In one embodiment, the first processing unit 102 trains the language model after the first training based on the target training data after the first stage masking process in the following manner to obtain the language model after the second training: based on the second target input data and the language model after the first training, second prediction data is obtained, wherein the second target input data includes instruction data, input data and part of the output data in the target training data, and the second prediction data includes second prediction instruction data, second prediction input data and second prediction output data; the second prediction instruction data is masked to obtain second prediction mask data, and the instruction data in the target training data is masked to obtain first target mask training data; based on the second prediction mask data and the first target mask training data, a second loss value is determined; based on the second loss value, the language model after the first training is adjusted to obtain the language model after the second training.

[0157] In one embodiment, the first processing unit 102 trains the language model after secondary training based on the target training data after the second-stage masking process in the following manner to obtain a target language model: Based on the third target input data and the language model after secondary training, third prediction data is obtained. The third target input data includes instruction data, input data, and a portion of output data from the target training data. The third prediction data includes third prediction instruction data, third prediction input data, and third prediction output data. The third prediction instruction data and the third prediction input data are masked to obtain third prediction mask data. The instruction data and input data in the target training data are also masked to obtain second target mask training data. Based on the third prediction mask data and the second target mask training data, a third loss value is determined. Based on the third loss value, the language model after secondary training is adjusted to obtain the target language model.

[0158] In one embodiment, the processing unit obtains third prediction data based on the third target input data and the language model after secondary training in the following manner: extracting features from the third target input data to obtain first feature data; adding noise data to the first feature data to obtain second feature data; and obtaining third prediction data based on the second feature data and the language model after secondary training.

[0159] Based on the same concept, embodiments of this disclosure also provide a data processing apparatus.

[0160] It is understood that the model training apparatus provided in this disclosure includes hardware structures and / or software modules corresponding to each function in order to achieve the above-mentioned functions. In conjunction with the units and algorithm steps of the various examples disclosed in this disclosure, this disclosure can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the technical solutions of this disclosure.

[0161] Figure 16 This is a block diagram illustrating a data processing apparatus 200 according to an exemplary embodiment. (Refer to...) Figure 16 The device includes a second acquisition unit 201 and a second processing unit 202.

[0162] The second acquisition unit 201 is used to acquire instruction data and input data.

[0163] The second processing unit 202 is used to input instruction data and input data into the language model to obtain output data; wherein the language model is obtained based on the model training method provided in the above embodiments.

[0164] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0165] Figure 17 This is a block diagram illustrating an apparatus 300 for model training or data processing according to an exemplary embodiment. For example, apparatus 300 may be a mobile phone, computer, digital broadcasting terminal, messaging device, game console, tablet device, medical device, fitness equipment, personal digital assistant, etc.

[0166] Reference Figure 17 The device 300 may include one or more of the following components: processing component 302, memory 304, power component 306, multimedia component 308, audio component 310, input / output (I / O) interface 312, sensor component 314, and communication component 316.

[0167] Processing component 302 typically controls the overall operation of device 300, such as operations associated with display, telephone calls, data communication, camera operation, and recording. Processing component 302 may include one or more processors 320 to execute instructions to perform all or part of the steps of the methods described above. Furthermore, processing component 302 may include one or more modules to facilitate interaction between processing component 302 and other components. For example, processing component 302 may include a multimedia module to facilitate interaction between multimedia component 308 and processing component 302.

[0168] Memory 304 is configured to store various types of data to support the operation of device 300. Examples of such data include instructions for any application or method operating on device 300, contact data, phonebook data, messages, pictures, videos, etc. Memory 304 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk.

[0169] The power supply component 306 provides power to the various components of the device 300. The power supply component 306 may include a power management system, one or more power sources, and other components associated with generating, managing, and distributing power to the device 300.

[0170] Multimedia component 308 includes a screen that provides an output interface between device 300 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touchscreen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors may sense not only the boundaries of the touch or swipe action but also the duration and pressure associated with the touch or swipe operation. In some embodiments, multimedia component 308 includes a front-facing camera and / or a rear-facing camera. When device 300 is in an operating mode, such as a shooting mode or a video mode, the front-facing camera and / or the rear-facing camera may receive external multimedia data. Each front-facing camera and rear-facing camera may be a fixed optical lens system or have focal length and optical zoom capabilities.

[0171] Audio component 310 is configured to output and / or input audio signals. For example, audio component 310 includes a microphone (MIC) configured to receive external audio signals when device 300 is in an operating mode, such as call mode, recording mode, and voice recognition mode. The received audio signals may be further stored in memory 304 or transmitted via communication component 316. In some embodiments, audio component 310 also includes a speaker for outputting audio signals.

[0172] I / O interface 312 provides an interface between processing component 302 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include, but are not limited to, home buttons, volume buttons, power buttons, and lock buttons.

[0173] Sensor assembly 314 includes one or more sensors for providing status assessments of various aspects of device 300. For example, sensor assembly 314 may detect the on / off state of device 300, the relative positioning of components such as the display and keypad of device 300, changes in the position of device 300 or a component of device 300, the presence or absence of user contact with device 300, the orientation or acceleration / deceleration of device 300, and temperature changes of device 300. Sensor assembly 314 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. Sensor assembly 314 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, sensor assembly 314 may also include an accelerometer, a gyroscope, a magnetometer, a pressure sensor, or a temperature sensor.

[0174] Communication component 316 is configured to facilitate wired or wireless communication between device 300 and other devices. Device 300 can access wireless networks based on communication standards, such as WiFi, 3G, or a combination thereof. In one exemplary embodiment, communication component 316 receives broadcast signals or broadcast-related information from an external broadcast management system via a broadcast channel. In one exemplary embodiment, communication component 316 also includes a near-field communication (NFC) module to facilitate short-range communication. For example, the NFC module may be implemented based on radio frequency identification (RFID) technology, Infrared Data Association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.

[0175] In an exemplary embodiment, the apparatus 300 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the methods described above.

[0176] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 304 including instructions, which can be executed by a processor 320 of the device 300 to perform the above-described method. For example, the non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0177] Figure 18 This is a block diagram illustrating an apparatus 400 for model training or data processing according to an exemplary embodiment. For example, apparatus 400 may be provided as a server. (Refer to...) Figure 18 The device 400 includes a processing component 422, which further includes one or more processors, and memory resources represented by memory 432 for storing instructions, such as application programs, that can be executed by the processing component 422. The application programs stored in memory 432 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 422 is configured to execute instructions to perform the aforementioned model training method.

[0178] Device 400 may also include a power supply component 426 configured to perform power management of device 400, a wired or wireless network interface 450 configured to connect device 400 to a network, and an input / output (I / O) interface 458. Device 400 may operate on an operating system stored in memory 432, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or similar.

[0179] Furthermore, in conjunction with the model training and data processing methods described in the above embodiments, this disclosure can provide a storage medium for implementation. This storage medium stores computer program instructions; when these computer program instructions are executed by a processor, they implement any of the model training or data processing methods described in the above embodiments.

[0180] In an exemplary embodiment, the storage medium including the instructions may be a non-transitory computer-readable storage medium. For example, a non-transitory computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0181] This disclosure also provides a computer program product, including a computer program that, when executed by a processor, implements the model training method or data processing method involved in any of the above embodiments.

[0182] In one exemplary embodiment, the processor may be deployed, for example, in an electronic device. It is understood that in this disclosure, "multiple" refers to two or more, and other quantifiers are similarly used. "And / or" describes the relationship between related objects, indicating that three relationships can exist; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. The singular forms "a," "the," and "the" are also intended to include the plural forms unless the context clearly indicates otherwise.

[0183] It is further understood that the terms "first," "second," etc., are used to describe various types of information, but this information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another, and do not indicate a specific order or degree of importance. In fact, the expressions "first," "second," etc., are completely interchangeable. For example, without departing from the scope of this disclosure, first information can also be referred to as second information, and similarly, second information can also be referred to as first information.

[0184] It can be further understood that, unless otherwise specified, "connection" includes both direct connections where no other components exist between the two parties and indirect connections where other components exist between them.

[0185] It is further understood that although operations are described in a specific order in the accompanying drawings in the embodiments of this disclosure, this should not be construed as requiring these operations to be performed in the specific order or serial order shown, or requiring all of the shown operations to be performed to obtain the desired result. In certain environments, multitasking and parallel processing may be advantageous.

[0186] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein.

[0187] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A model training method, characterized in that, include: Obtain the raw training data; Target training data is constructed based on the original training data. The target training data includes instruction data, input data, and output data, wherein the instruction data and the input data serve as the input to the language model, and the output data serves as the output of the language model. The target training data is masked according to a preset masking rule, which includes multi-stage masking processing. Based on the target training data and the masked target training data, the language model is trained to obtain a trained language model.

2. The method according to claim 1, characterized in that, The multi-stage masking process includes a first-stage masking process and a second-stage masking process; The first stage of masking processing represents masking the instruction data in the target training data, and the second stage of masking processing represents masking the instruction data and input data in the target training data.

3. The method according to claim 2, characterized in that, The target training data after masking includes the target training data after the first stage of masking and the target training data after the second stage of masking. The step of training the language model based on the target training data and the masked target training data to obtain the trained language model includes: Based on the target training data, the language model is trained for the first time to obtain the language model after the first training. Based on the target training data after the first stage masking process, the language model after the first training is trained to obtain the language model after the second training. Based on the target training data after the second-stage masking process, the language model after secondary training is trained to obtain the target language model, and the target language model is used as the language model after training.

4. The method according to claim 3, characterized in that, The step of training the language model for the first time based on the target training data to obtain the language model after the first training includes: Based on the first target input data and the language model, first prediction data is obtained. The first target input data includes instruction data, input data and part of the output data in the target training data. The first prediction data includes first prediction instruction data, first prediction input data and first prediction output data. Based on a portion of the instruction data in the instruction data, the input data, the output data, and the first prediction data, a first loss value is determined; Based on the first loss value, the language model is trained and optimized to obtain the language model after the first training.

5. The method according to claim 3 or 4, characterized in that, The step of training the language model after the first stage of masking based on the target training data to obtain the language model after the second stage of training includes: Based on the second target input data and the language model after the first training, second prediction data is obtained. The second target input data includes the instruction data, input data and part of the output data in the target training data. The second prediction data includes the second prediction instruction data, the second prediction input data and the second prediction output data. The second prediction instruction data is masked to obtain the second prediction mask data, and the instruction data in the target training data is masked to obtain the first target mask training data. Based on the second predicted mask data and the first target mask training data, a second loss value is determined; Based on the second loss value, the language model after the first training is adjusted to obtain the language model after the second training.

6. The method according to claim 5, characterized in that, The step of training the language model after secondary training based on the target training data after the second stage masking to obtain the target language model includes: Based on the third target input data and the language model after secondary training, the third prediction data is obtained. The third target input data includes the instruction data, input data and part of the output data in the target training data. The third prediction data includes the third prediction instruction data, the third prediction input data and the third prediction output data. The third prediction instruction data and the third prediction input data are masked to obtain the third prediction mask data, and the instruction data and input data in the target training data are masked to obtain the second target mask training data. Based on the third predicted mask data and the second target mask training data, a third loss value is determined; Based on the third loss value, the language model after secondary training is adjusted to obtain the target language model.

7. The method according to claim 6, characterized in that, The third prediction data, obtained based on the third target input data and the language model after secondary training, includes: Feature extraction is performed on the third target input data to obtain the first feature data; Add noise data to the first feature data to obtain the second feature data; The third prediction data is obtained based on the second feature data and the language model after secondary training.

8. A data processing method, characterized in that, include: Acquire command data and input data; The instruction data and the input data are input into the language model to obtain the output data; The language model is obtained based on the model training method described in any one of claims 1 to 7.

9. A model training device, characterized in that, include: The first acquisition unit is used to acquire the raw training data; The first processing unit is configured to construct target training data based on the original training data, perform masking processing on the target training data according to a preset masking rule, and train the language model based on the target training data and the masked target training data to obtain a trained language model. The preset masking rule includes multi-stage masking processing. The target training data includes instruction data, input data, and output data, wherein the instruction data and the input data serve as the input to the language model, and the output data serves as the output of the language model.

10. A data processing apparatus, characterized in that, include: The second acquisition unit is used to acquire instruction data and input data; The second processing unit is used to input the instruction data and the input data into the language model to obtain output data; The language model is obtained based on the model training method described in any one of claims 1 to 7.

11. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to execute the model training method described in any one of claims 1 to 7 or the data processing method described in claim 8.

12. A storage medium, characterized in that, The storage medium stores instructions that, when executed by the terminal's processor, enable the electronic device to perform the model training method according to any one of claims 1 to 7 or the data processing method according to claim 8.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor on an electronic device, the electronic device performs the model training method as described in any one of claims 1 to 7 or the data processing method as described in claim 8.

Images