A method, device, storage medium and electronic device for generating a language model

Abstract

The present application discloses a language model generation method, device, storage medium and electronic device, wherein the method comprises: obtaining a fine-tuning language model trained based on a set of labeled samples corresponding to a target capability training task, obtaining a set of unlabeled samples corresponding to the target capability training task, inputting unlabeled samples in the unlabeled sample set into the fine-tuning language model, obtaining pseudo-label labeled samples, obtaining a credible score of the pseudo-label labeled samples, saving pseudo-label labeled samples with a credible score higher than a credible score threshold in a labeled sample set, and training the fine-tuning language model based on the labeled sample set until the number of model training times reaches a preset number of iterations to obtain a language model that meets the target capability training task. By adopting the present application, by using pseudo-label labeled samples with high credibility to train the model, samples with higher accuracy improve the model training effect and the generalization ability of the model, and reduce the demand for manually labeled samples.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a language model generation method, device, storage medium and electronic device. Background Art

[0002] Large-scale language model is an artificial intelligence technology that is now widely used in many different applications, such as speech recognition, text classification, and question-answering. The training of large-scale language models often requires a large amount of sample data to enable the model to learn grammatical semantics and contextual connections, thereby obtaining various model capabilities. In order to improve the model training effect, it is necessary to provide correctly labeled sample data. In the existing technology, manually labeled sample data is often used. If you want to obtain manually labeled sample data with a large enough scale for language model learning and training, it takes a lot of manpower and time, which increases the manpower cost of model training. It is necessary to provide a model training method that uses a small amount of manually labeled sample data for training while ensuring the model training effect. Summary of the invention

[0003] The embodiments of the present application provide a language model generation method, device, storage medium and electronic device, which can train the model by adding pseudo-labeled samples with high credibility to the labeled sample set, and the samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually labeled samples and reducing the model training cost. The technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a method for generating a language model, the method comprising:

[0005] Obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability;

[0006] Inputting the unlabeled samples in the unlabeled sample set into the fine-tuned language model to obtain pseudo-labeled samples;

[0007] Obtaining the credibility score of the pseudo-label annotated sample, and storing the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set;

[0008] The fine-tuning language model is trained based on the annotated sample set until the number of model training reaches a preset number of iterations, thereby obtaining a language model that meets the target capability training task.

[0009] In a second aspect, an embodiment of the present application provides a language model generation device, the device comprising:

[0010] A model fine-tuning module is used to obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability;

[0011] A pseudo-label acquisition module, used for inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples;

[0012] A credible score acquisition module, used to acquire the credible score of the pseudo-label annotated sample, and save the pseudo-label annotated sample whose credible score is higher than the credible score threshold in the annotated sample set;

[0013] The model training module is used to perform model training on the fine-tuning language model based on the labeled sample set until the number of model training reaches a preset number of iterations to obtain a language model that meets the target capability training task.

[0014] In a third aspect, an embodiment of the present application provides a computer storage medium, wherein the computer storage medium stores a plurality of instructions, wherein the instructions are suitable for being loaded by a processor and executing the above-mentioned method steps.

[0015] In a fourth aspect, an embodiment of the present application provides an electronic device, which may include: a processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the above-mentioned method steps.

[0016] In one or more embodiments of the present application, a fine-tuned language model is obtained by obtaining a set of labeled samples corresponding to a target capability training task, and an unlabeled sample set corresponding to the target capability training task is obtained, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are reply information for the question information under the target capability, and the unlabeled samples in the unlabeled sample set are input into the fine-tuned language model to obtain pseudo-label labeled samples, obtain the credibility score of the pseudo-label labeled samples, and save the pseudo-label labeled samples with credibility scores higher than the credibility score threshold in the labeled sample set, and perform model training on the fine-tuned language model based on the labeled sample set until the number of model training reaches a preset number of iterations, thereby obtaining a language model that meets the target capability training task. By adding pseudo-label labeled samples with high credibility to the labeled sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually labeled samples and the model training cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 is an example schematic diagram of a language model training provided in an embodiment of the present application;

[0019] Figure 2 It is a flow chart of a language model generation method provided in an embodiment of the present application;

[0020] Figure 3 It is a flow chart of a language model generation method provided in an embodiment of the present application;

[0021] Figure 4 This is an example schematic diagram of integrated label generation provided in an embodiment of the present application;

[0022] Figure 5 It is a structural diagram of a language model generation device provided in an embodiment of the present application;

[0023] Figure 6 It is a structural diagram of a language model generation device provided in an embodiment of the present application;

[0024] Figure 7 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0025] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0026] The language model generation method provided in the embodiment of the present application can be implemented by a computer program and can be run on a language model generation device based on the von Neumann system. The computer program can be integrated in an application or run as an independent tool application. The language model can be a large language model (LLM), which is a deep learning model trained using a large amount of text data, which can generate natural language text or understand the meaning of language text, and the large language model can handle a variety of natural language tasks. The language model generation device is used to perform model training on the fine-tuned language model and obtain a language model that meets the target ability training task after the training is completed. Among them, the target ability training task performs model training on the fine-tuned language model so that the language model obtained after the model training has a model training task with the target ability. The target ability can be the ability of the language model to process natural language tasks, for example, it can include classification, question and answer, dialogue, recommendation and other abilities, the classification ability can be to input multiple texts into the language model, and the language model can output the classification of these multiple texts, the question and answer ability can be to input questions into the language model, and the language model can output the answers to the questions, the dialogue ability can be to input dialogue sentences into the language model, and the language model can reply according to the dialogue sentences, etc. The fine-tuning language model is a language model that has initially achieved the target capability after fine-tuning. The language model generation device can use a large amount of sample data to perform model training on the fine-tuning language model to enhance the target capability of the fine-tuning language model, thereby obtaining a language model that meets the target capability training task.

[0027] Please also see Figure 1, an example schematic diagram of language model training is provided for an embodiment of the present application. The language model generation device can perform model training for a preset number of iterations on the fine-tuned language model in combination with the labeled sample set and the unlabeled sample set corresponding to the target capability training task. After the model training for the preset number of iterations, a language model can be obtained. The preset number of iterations represents the number of model trainings. It can be understood that the larger the preset number of iterations, the stronger the target capability of the language model, and the greater the computing power consumed by the language model generation device. The preset number of iterations can be the initial setting of the language model generation device, and can also be set by the user or relevant staff. Among them, the labeled sample set contains sample data that has been labeled, that is, labeled samples, and the unlabeled sample set contains sample data without labeled labels. The sample data in the labeled sample set and the unlabeled sample set are sample data used to train the target capability of the model. Since the language model is used to process natural language tasks, the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are the reply information for the question information under the target capability. The question information can be the content information that the user can input when using the language model for dialogue, which can be text, video, audio or picture. The question information is not limited to the text with the content of the question information. The reply information can be the reply content for the question information that the user expects the language model to output. The specific content of the reply information is related to the capabilities of the language model. For example, if the target capability is the dialogue capability of the model, the question information can be the dialogue sentence that the user can input when talking to the language model, and the reply information can be the reply sentence of the model language for the dialogue sentence. For example, the question information can be "Today is a good weather", and the language model can output the reply information "Today is indeed a good weather" based on the dialogue capability. If the target capability is the question-answering capability of the model, the question information may be the text of the question information, and the answer information may be the answer to the question information. For example, the question information may be "How is the weather today?", and the answer information may be "Today's weather is sunny, and the temperature is 28 degrees Celsius." It can be understood that the labeled samples in the labeled sample set are manually labeled samples.

[0028] In a model training of the fine-tuning language model, the language model generation device inputs the sample data without labeled labels in the unlabeled sample set, that is, the unlabeled samples, into the fine-tuning language model. The fine-tuning language model can output the pseudo-label of the unlabeled sample, that is, the unlabeled sample obtains the pseudo-label labeled sample through the fine-tuning language model, that is, the sample data labeled with the pseudo-label, and the pseudo-label is the reply information output by the fine-tuning language model for the question information that has not yet met the target ability training task. For example, the unlabeled sample data A is input into the fine-tuning language model to obtain the pseudo-label A corresponding to the sample data A, that is, the pseudo-label labeled sample data A-pseudo-label A is obtained. The language model generation device can obtain the pseudo-label labeled samples with a credible score higher than the credible score threshold in the pseudo-label labeled samples and save them in the labeled sample set, so as to achieve the purpose of gradually expanding the labeled sample set during the model training process. For example, if the credible score of the pseudo-label labeled sample data C-pseudo-label C is higher than the credible score threshold, it can be saved in the labeled sample set. At this time, the pseudo-label C can be confirmed as label C, and the sample data C-label C becomes a labeled sample.

[0029] Among them, the credible score is used to indicate the correctness of the pseudo-labeled sample. It can be understood that the larger the credible score, the more correct the pseudo-label annotated by the fine-tuned language model for the unlabeled sample. Conversely, the larger the credible score, the more incorrect the pseudo-label annotated by the fine-tuned language model for the unlabeled sample. The credible score threshold is used to determine whether the pseudo-labeled sample can be saved to the annotated sample set. It can be the initial setting of the language model generation device, or it can be set by the user or relevant staff. Then, the fine-tuned language model is trained using the expanded annotated sample set, and the parameters of the fine-tuned language model are adjusted. The purpose of gradually expanding the annotated sample set during the model training process is achieved, and the accuracy of the language model training is guaranteed while reducing the dependence on manually annotated samples.

[0030] The language model generation method provided in this application is described in detail below with reference to specific embodiments.

[0031] See also Figure 2 , which is a flow chart of a method for generating a language model according to an embodiment of the present application. Figure 2 As shown, the method of the embodiment of the present application may include the following steps S101-S104.

[0032] S101, obtaining a fine-tuned language model trained based on a set of labeled samples corresponding to a target capability training task, and obtaining a set of unlabeled samples corresponding to the target capability training task.

[0033] Specifically, the language model generation device can obtain a set of annotated samples corresponding to the target capability training task, and the annotated sample set includes sample data that has been manually annotated with labels, that is, annotated samples. The target capability training task performs model training on the fine-tuned language model so that the language model obtained after the model training has a model training task with the target capability. The target capability can be the ability of the language model to process natural language tasks, for example, it can include classification, question and answer, dialogue, recommendation and other capabilities. The annotated samples in the annotated sample set are question information, which can be content information that the user can input when using the language model dialogue, which can be text, video, audio or picture, and the question information is not limited to text with the content of question information. The label of the annotated sample is the reply information for the question information under the target capability, and the reply information can be the reply content for the question information output by the language model expected by the user, and the content of the reply information is related to the target capability.

[0034] The language model generation device can use the labeled sample set to perform fine-tuning processing to obtain a fine-tuned language model. Since the labeled sample set contains sample data corresponding to the target ability training task, the fine-tuned language model has preliminarily learned the target ability. The language model generation device can also obtain an unlabeled sample set corresponding to the target ability training task. The unlabeled sample set contains sample data without labeled labels, that is, unlabeled samples. The unlabeled samples can be question information without labeled answer information.

[0035] S102, inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples.

[0036] Specifically, the language model generating device can input the unlabeled samples in the unlabeled sample set into the fine-tuned language model, and the fine-tuned language model can output the pseudo-labels corresponding to each unlabeled sample, and label the unlabeled samples, thereby obtaining pseudo-labeled samples, that is, sample data annotated with pseudo-labels.

[0037] S103, obtaining the credibility scores of the pseudo-label annotated samples, and saving the pseudo-label annotated samples whose credibility scores are higher than the credibility score threshold in the annotated sample set.

[0038] Specifically, the language model generation device can obtain the credibility score of the pseudo-labeled sample, and the credibility score can represent the accuracy of the pseudo-labeled sample, that is, it can be used to represent the accuracy of the pseudo-label annotated by the fine-tuned language model for the unlabeled sample. The credibility score can be obtained based on the execution value of the fine-tuned language model when outputting the pseudo-label of the pseudo-labeled sample. The execution value can reflect the computing power, data size, etc. used by the fine-tuned language model when generating the pseudo-label, that is, the higher the execution value, the higher the credibility score of the pseudo-labeled sample.

[0039] The language model generation device can save the pseudo-label annotated samples with a credibility score higher than the credibility score threshold in the annotated sample set. The credibility score threshold is used to determine whether the pseudo-label annotated samples can be saved in the annotated sample set. It can be the initial setting of the language model generation device, or it can be set by the user or relevant staff. The language model generation device can confirm the pseudo-label annotated samples with a credibility score higher than the credibility score threshold as sample data annotated with the correct label. The pseudo-label annotated samples with a credibility score higher than the credibility score threshold can be saved in the annotated sample set for model training of the fine-tuned language model, thereby achieving the purpose of expanding the annotated sample set without manual annotation.

[0040] S104, performing model training on the fine-tuned language model based on the labeled sample set until the number of model training times reaches a preset number of iterations, thereby obtaining a language model that meets the target capability training task.

[0041] Specifically, the language model generation device can perform model training on the annotated language model based on the annotated sample set until the number of model training times reaches a preset number of iterations, and a language model that meets the target capability training task can be obtained. It is understandable that the greater the preset number of iterations, the stronger the target capability of the language model, and the greater the computing power consumed by the language model generation device. The preset number of iterations can be the initial setting of the language model generation device, and can also be set by the user or relevant staff. It is understandable that in each model training, the language model generation device can execute steps S102-S103 to expand the annotated sample set, and then use the expanded annotated sample set to train the fine-tuned language model, thereby gradually expanding the annotated sample set during the iterative training process and improving the model training effect.

[0042] In an embodiment of the present application, a fine-tuning language model trained by obtaining a set of labeled samples corresponding to a target capability training task is obtained, an unlabeled sample set corresponding to the target capability training task is obtained, the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are reply information to the question information under the target capability, the unlabeled samples in the unlabeled sample set are input into the fine-tuning language model, pseudo-label labeled samples are obtained, the credibility score of the pseudo-label labeled samples is obtained, the pseudo-label labeled samples with credibility scores higher than the credibility score threshold are saved in the labeled sample set, and the fine-tuning language model is trained based on the labeled sample set until the number of model training times reaches a preset number of iterations, and a language model that meets the target capability training task is obtained. By adding pseudo-label labeled samples with high credibility to the labeled sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually labeled samples and the cost of model training.

[0043] See also Figure 3 , which is a flow chart of a method for generating a language model according to an embodiment of the present application. Figure 3 As shown, the method of the embodiment of the present application may include the following steps S201-S211.

[0044] S201, creating an initial language model and obtaining a set of unlabeled unimodal samples.

[0045] Specifically, the language model generation device can create an initial language model and obtain an unlabeled unimodal sample set, wherein the unlabeled unimodal sample set includes unlabeled unimodal sample data, and the sample data in the unimodal sample set are all in the same modality, for example, they can all be sample data in the form of text, or they can all be sample data in the form of pictures or audio. It can be understood that the unlabeled unimodal sample set is all unlabeled sample data, so there are many unimodal sample data in the unlabeled unimodal sample set, which are used to help the initial model data learn common semantic feature expressions from a large number of unimodal sample data.

[0046] S202, using an unlabeled unimodal sample set to pre-train the initial language model to obtain a pre-trained model.

[0047] Specifically, the language model generation device can use an unlabeled unimodal sample set to pre-train the initial language model, thereby obtaining a pre-trained model. The pre-training process can be an unsupervised (Unsupervised Learning) training process, and the pre-training process can include unsupervised training methods such as masking mechanism, prediction order and pairing judgment to help the obtained pre-trained model learn general semantic feature expressions. For example, the masking mechanism can be applied when the unimodal sample data is text data. The language model generation device can randomly mask some words in the input text data, so that the initial language model predicts these masked words through the context, thereby learning a richer word representation. The prediction order can enable the initial language model to predict the content of the following content through the previous content, thereby learning the semantic connection between the contexts. The pairing judgment can enable the initial language model to find the corresponding following content for the previous content, or find the corresponding previous content for the following content, and further learn the semantic connection between the contexts.

[0048] S203, obtaining a set of labeled samples corresponding to the target capability training task, and using the set of labeled samples to fine-tune the pre-trained model to obtain a fine-tuned language model.

[0049] Specifically, the language model generation device can obtain a set of annotated samples corresponding to the target capability training task, and the annotated sample set includes sample data that has been manually annotated with labels, that is, annotated samples. The target capability training task performs model training on the fine-tuned language model so that the language model obtained after the model training has a model training task with the target capability. The target capability can be the ability of the language model to process natural language tasks, for example, it can include classification, question and answer, dialogue, recommendation and other capabilities. The annotated samples in the annotated sample set are question information, which can be content information that the user can input when using the language model dialogue, which can be text, video, audio or picture, and the question information is not limited to text with the content of question information. The label of the annotated sample is the reply information for the question information under the target capability, and the reply information can be the reply content for the question information output by the language model expected by the user, and the content of the reply information is related to the target capability. The language model generation device can use a labeled sample set to fine-tune the pre-trained model to obtain a fine-tuned language model. Since the labeled sample set contains sample data corresponding to the target capability training task, the fine-tuned language model has preliminarily learned the target capability, that is, the language model generation device can use a small amount of manually labeled samples to obtain a fine-tuned language model that has preliminarily learned the target capability.

[0050] S204, obtaining a set of unlabeled samples corresponding to the target capability training task.

[0051] Specifically, the language model generation device can also obtain an unlabeled sample set corresponding to the target ability training task, where the unlabeled sample set contains sample data without labeled labels, that is, unlabeled samples, which can be question information without labeled answer information.

[0052] S205 , inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples.

[0053] Specifically, the language model generating device can input the unlabeled samples in the unlabeled sample set into the fine-tuned language model, and the fine-tuned language model can output the pseudo-labels corresponding to each unlabeled sample, and label the unlabeled samples, thereby obtaining pseudo-labeled samples, that is, sample data annotated with pseudo-labels.

[0054] S206, performing credibility judgment processing on the pseudo-labeled sample to obtain a credibility score corresponding to the pseudo-labeled sample.

[0055] Specifically, the language model generating device can perform credibility judgment processing on the pseudo-label annotated samples to obtain the credibility score corresponding to the pseudo-label annotated samples. The credibility judgment processing is used to obtain the credibility score of the pseudo-label annotated samples. The credibility score can represent the accuracy of the pseudo-label annotated samples, that is, it can be used to represent the accuracy of the pseudo-label annotated by the fine-tuned language model for unlabeled samples. It can be understood that although the fine-tuned language model has initially learned the target ability, it still requires a lot of model training to strengthen the target ability. Therefore, the pseudo-labels annotated by the fine-tuned language model to the unlabeled samples are not necessarily accurate, that is, they are not necessarily the same as or similar to the manually annotated labels. The language model generating device can judge whether the pseudo-labels annotated by the fine-tuned language model are accurate based on the credibility score.

[0056] Optionally, the language model generation device can obtain the execution value when the initial fine-tuning language model outputs the pseudo-label annotated sample, wherein the execution value can reflect the computing power, data size, etc. used by the fine-tuning language model when generating the pseudo-label, that is, the higher the execution value, the higher the credibility score of the pseudo-label annotated sample. The language model generation device can also use a credibility decision tree to judge the pseudo-label annotated sample, so as to obtain the correctness value corresponding to the pseudo-label annotated sample. The credibility decision tree is a decision tree that can judge the correctness value of the pseudo-label annotated sample. It can be understood that the decision tree can be based on the known probability of occurrence of various situations. By constructing a decision tree to obtain the probability that the expected value of the net present value is greater than or equal to zero, the correctness value obtained by the credibility decision tree is the probability value of the pseudo-label annotated sample being correct. The larger the correctness value of the pseudo-label annotated sample, the larger the credibility score of the pseudo-label annotated sample. Conversely, the smaller the correctness value of the pseudo-label annotated sample, the smaller the credibility score of the pseudo-label annotated sample.

[0057] The language model generation device can directly use the execution degree value as the credibility score, or directly use the accuracy value as the credibility score. However, only using the model execution degree or decision degree to judge the accuracy of the pseudo-label annotated sample can easily lead to a single judgment angle and thus cause judgment errors. Therefore, the language model generation device can perform weighted sum processing on the execution degree value and the accuracy value to obtain the credibility score of the pseudo-label annotated sample.

[0058] Optionally, the language model generating device can obtain a first weighted parameter corresponding to the execution degree value and a second weighted parameter corresponding to the accuracy value. The first weighted parameter and the second weighted parameter can be the initial settings of the language model generating device, and can also be set by the user or relevant staff. The sum of the first weighted parameter and the second weighted parameter is 1. The execution degree value and the accuracy value are weighted and summed based on the first weighted parameter and the second weighted parameter, that is, the product of the first weighted parameter and the execution degree value is summed with the product of the second weighted parameter and the accuracy value to obtain the credible score of the pseudo-labeled sample.

[0059] S207, obtaining target pseudo-label annotated samples whose credibility scores are higher than a credibility score threshold from the pseudo-label annotated samples.

[0060] Specifically, the language model generation device can obtain a target pseudo-label annotated sample whose credibility score is higher than a credibility score threshold from the pseudo-label annotated sample. The credibility score threshold is used to determine whether the pseudo-label annotated sample is the target pseudo-label sample, that is, to determine whether the pseudo-label annotated sample can be saved to the annotated sample set. The language model generation device can determine that the pseudo-label annotated by the target pseudo-label annotated sample is correct and is the same or similar to the manually annotated label, which can help train the fine-tuned language model.

[0061] S208 , saving the target pseudo-label annotated samples in the annotated sample set, and deleting the target pseudo-label annotated samples from the unannotated sample set.

[0062] Specifically, the language model generation device can save the target pseudo-label annotated sample in the annotated sample set, that is, the target pseudo-label annotated sample is confirmed as an annotated sample in the annotated sample set, and can be used in the subsequent model training of the fine-tuning language model.

[0063] At the same time, the unlabeled samples corresponding to the target pseudo-labeled samples can be deleted from the unlabeled sample set, avoiding the need to annotate the pseudo-labels again when fine-tuning the language model during the next model training, thus saving the model training process.

[0064] S209: Perform model training on the fine-tuned language model based on the labeled sample set.

[0065] Specifically, the language model generation device can perform model training on the fine-tuning language model based on the labeled sample set, and the labeled sample set contains labeled samples, that is, sample data labeled with labels. The language model generation device can input the sample data in the labeled sample set into the fine-tuning language model, and the fine-tuning language model can output the predicted label corresponding to the sample data. The language model generation device can calculate the loss value of the fine-tuning language model based on the label and the predicted label, and adjust the parameters of the fine-tuning language model in the back-propagation training process based on the loss value, thereby realizing one-time model training. Among them, the loss value can be the difference between the label and the predicted label or the Euclidean distance, etc.

[0066] S210, determining whether the number of model training times reaches a preset number of iterations.

[0067] Specifically, the language model generation device can determine whether the model training times of the fine-tuning language model have reached the preset number of iterations. It can be understood that the greater the preset number of iterations, the stronger the target capability of the language model, and the greater the computing power consumed by the language model generation device. The preset number of iterations can be the initial setting of the language model generation device, and can also be set by the user or relevant staff. If the model training times have reached the preset number of iterations, step S211 is executed. If the model training times have not reached the preset number of iterations, step S205 is executed, that is, the fine-tuning language model continues to be trained. In each model training, the language model generation device can execute steps S205-S208 to expand the set of labeled samples, and then use the expanded set of labeled samples to train the fine-tuning language model, thereby gradually expanding the set of labeled samples during the iterative training process and improving the model training effect.

[0068] S211, obtaining a language model that meets the target ability training task.

[0069] Specifically, the language model generating device can perform model training on the annotated language model based on the annotated sample set until the number of model training times reaches a preset number of iterations, and a language model that meets the target capability training task can be obtained.

[0070] Optionally, the target capability of the fine-tuning language model has been strengthened by model training with a preset number of iterations, and a language model that meets the target capability training task has been obtained. On this basis, the target capability of the language model can be further strengthened. The language model generation device can obtain at least two language models that meet the target capability training task according to the steps of S201-S211. It can be understood that these language models are trained separately based on steps S201-S211. Although the model architectures are the same and both meet the target capability training task, there are still differences in the internal parameters of the models and certain differences in the target capability intensity. The language model generation device can construct a language model ensemble based on the at least two language models that meet the target capability training task. The language model ensemble can be a model ensemble (Model Ensemble), which is to fuse multiple trained models and realize multi-model fusion of test data based on a certain method, so that the final result can complement each other, fuse the learning ability of each model, and improve the generalization ability of the final model.

[0071] The language model generation device can also obtain an integrated model test sample set, at least one test sample in the integrated model test sample set, and the test samples are all labeled sample data, wherein the label labeled with the sample data can be a test label, and the test label can be manually labeled, and is used to test the target capability of each language model in the language model integration. The language model generation device can input each test sample in the integrated model test sample set into the language model integration to obtain an integrated test result, wherein the integrated test result is the integrated label labeled for the test sample output by the model integration, and the integrated label can be a label obtained by voting, averaging, etc. for all language models in the language model integration. The language model generation device can fuse at least two language models based on the integrated test result, so as to obtain a fused language model that meets the target capability training task. It can be understood that the fused language model can be a model after the language model is further strengthened in terms of the target capability, and the fusion process can be a fusion of the model parameters of each language model, so that the fused language model has a higher target capability and a wider applicability.

[0072] Optionally, the integrated label can also be the label of each language model in the language model integration for the test sample. Please refer to Figure 4, an integrated label generation method is provided for an embodiment of the present application, for example, a language model generation device can obtain three language models that meet the target ability training task, namely language model 1, language model 2 and language model 3, and the integrated model test sample set can include test samples such as test sample Q and test sample W, and the integrated label can be the label annotated by all language models for the test sample. For example, the integrated label of test sample Q may include label Q1, label Q1 and label Q2 output by language model 1, language model 2 and language model 3 to annotate test sample Q. Similarly, the integrated label of test sample W may include label W1, label W2 and label W3 output by language model 1, language model 2 and language model 3 to annotate test sample W. The labels annotated by different language models for the same test sample may be the same or different. For example, the integrated label of test sample Q has two identical labels Q1 and one different label Q2, and the labels in the integrated label of test sample W are all different.

[0073] The language model generation device can obtain test samples with high integration label identity rate in the integrated model test sample set based on the integrated test results. The integration label identity rate can be the probability of the same label appearing in the integrated label of the test sample. For example, the integration label identity rate of the test sample Q is 66.7%, while the integration label identity rate of the test sample W is 0%. It can be understood that the higher the integration label identity rate, the higher the correctness of the annotation of the language model integration for the test sample. The language model generation device can obtain target test samples with an integration label identity rate higher than the identity rate threshold in the integrated model test sample set. The identity rate threshold is used to determine whether the test sample is a target test sample. It can be the initial setting of the language model generation device, or it can be set by the user or relevant staff. For example, it can be 60%, and the test sample Q can be the target test sample. The fusion processing can be that the language model generation device uses the target test sample to continue model training on the language model, so as to obtain a fused language model.

[0074] It is understandable that the language model already has the target capability. If you want to obtain the update capability based on the language model that already has the target capability, the language model generation device can perform model training on the language model based on the update capability training task. The update capability training task is to perform model training on the language model so that the updated language model obtained after the model training has the update capability. Among them, the update capability can be the ability to process the results output by the model again on the basis of the target capability. For example, the target capability that the language model already has is the question-answering capability, that is, if the user inputs "how is the weather today" to the language model, the language model can output the reply information "today's weather is sunny, the temperature is 28 degrees Celsius", and the update capability can be the ability to score the reply information for the language model, for example, "80 points", and the score value indicates that today is a good weather, which helps users intuitively understand the weather conditions through the score value.

[0075] Optionally, the language model generation device can obtain an update annotation sample set corresponding to the update capability training task, wherein the update annotation sample set includes at least one update sample data that has been annotated with a label, wherein the update sample data can be a reply information, and the label of the update sample data can be a score value corresponding to the reply information under the update capability. The language model generation device can generate corresponding update layer data based on the update capability training task, and update the language model based on the update layer data, that is, add update layer data to the language model, and the update layer data can be a fully connected layer data that helps the language model have update capability, etc. The language model generation device can freeze the parameters of the updated language model except the update layer data after the update processing, and obtain the initial update language model, that is, in the subsequent fine-tuning process, the parameters related to the target capability that have been trained are not changed. The initial update language model is fine-tuned using the update annotation sample set to obtain an updated language model that meets the update capability training task, that is, the updated language model has the target capability, and the update capability is also learned on the basis of the target capability.

[0076] In an embodiment of the present application, an initial language model is created, an unlabeled unimodal sample set is obtained, and an unsupervised pre-training process is performed on the initial language model using the unlabeled unimodal sample set to obtain a pre-trained model, and a labeled sample set corresponding to the target capability training task is obtained, and the pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model. An unlabeled sample set corresponding to the target capability training task is obtained, and the unlabeled samples in the unlabeled sample set are input into the fine-tuned language model to obtain pseudo-label labeled samples, and the pseudo-label labeled samples are subjected to credibility judgment processing to obtain a credible score corresponding to the pseudo-label labeled samples, wherein the credible score can be obtained by combining the execution data and the accuracy value of the credibility decision tree to avoid the situation where a single judgment angle leads to a judgment error. A target pseudo-label labeled sample with a credible score higher than the credible score threshold is obtained from the pseudo-label labeled sample, and the target pseudo-label labeled sample is saved in the labeled sample set, and the target pseudo-label labeled sample is deleted from the unlabeled sample set, so as to avoid the need to label the pseudo-label again when fine-tuning the language model during the next model training, thereby saving the model training process. The fine-tuned language model is trained based on the annotated sample set, and the expanded annotated sample set is used to train the fine-tuned language model, so that the annotated sample set is gradually expanded during the iterative training process to improve the model training effect, until the model training times reach the preset number of iterations, and a language model that meets the target capability training task can be obtained. By adding pseudo-labeled annotated samples with high credibility to the annotated sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually annotated samples and reducing the model training cost. At the same time, model integration can be used to obtain a fused language model that further enhances the target capability, and the model can also learn to update capabilities without affecting the target capability of the language model by freezing parameters.

[0077] The following will be combined with the attached Figure 5 -Attached Figure 6 , the language model generation device provided in the embodiment of the present application is introduced in detail. It should be noted that the attached Figure 5 -Attached Figure 6 The language model generation device in the present application is used to execute Figure 2 and Figure 3 For the convenience of explanation, only the part related to the embodiment of the present application is shown. For the specific technical details not disclosed, please refer to the present application. Figure 2 and Figure 3 The embodiment shown.

[0078] See also Figure 5, which shows a schematic diagram of the structure of a language model generation device provided by an exemplary embodiment of the present application. The language model generation device can be implemented as all or part of the device through software, hardware or a combination of both. The device 1 includes a model fine-tuning module 11, a pseudo-label acquisition module 12, a credible score acquisition module 13 and a model training module 14.

[0079] The model fine-tuning module 11 is used to obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability;

[0080] A pseudo-label acquisition module 12 is used to input the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples;

[0081] A credibility score acquisition module 13 is used to acquire the credibility score of the pseudo-label annotated sample, and store the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set;

[0082] The model training module 14 is used to perform model training on the fine-tuning language model based on the annotated sample set until the number of model training reaches a preset number of iterations to obtain a language model that meets the target capability training task.

[0083] In this embodiment, a fine-tuning language model trained based on a labeled sample set corresponding to a target capability training task is obtained, an unlabeled sample set corresponding to the target capability training task is obtained, the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are reply information to the question information under the target capability, the unlabeled samples in the unlabeled sample set are input into the fine-tuning language model, pseudo-label labeled samples are obtained, the credibility score of the pseudo-label labeled samples is obtained, the pseudo-label labeled samples with credibility scores higher than the credibility score threshold are saved in the labeled sample set, and the fine-tuning language model is trained based on the labeled sample set until the number of model training times reaches a preset number of iterations, and a language model that meets the target capability training task is obtained. By adding pseudo-label labeled samples with high credibility to the labeled sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually labeled samples and the model training cost.

[0084] See also Figure 6, which shows a schematic diagram of the structure of a language model generation device provided by an exemplary embodiment of the present application. The language model generation device can be implemented as all or part of the device through software, hardware or a combination of both. The device 1 includes a model fine-tuning module 11, a pseudo-label acquisition module 12, a trust score acquisition module 13, a model training module 14, a fusion processing module 15 and an update capability training module 16.

[0085] The model fine-tuning module 11 is used to obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability;

[0086] Optionally, the model fine-tuning module 11 is specifically used to create an initial language model and obtain an unlabeled unimodal sample set, wherein the unlabeled unimodal sample set includes unlabeled unimodal sample data;

[0087] Using an unlabeled unimodal sample set to pre-train the initial language model to obtain a pre-trained model;

[0088] Obtain a set of labeled samples corresponding to the target ability training task;

[0089] The pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model.

[0090] A pseudo-label acquisition module 12 is used to input the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples;

[0091] A credibility score acquisition module 13 is used to acquire the credibility score of the pseudo-label annotated sample, and store the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set;

[0092] Optionally, the credibility score acquisition module 13 is specifically used to perform credibility judgment processing on the pseudo-labeled sample to obtain the credibility score corresponding to the pseudo-labeled sample;

[0093] Obtaining a target pseudo-label annotated sample whose credibility score is higher than a credibility score threshold from the pseudo-label annotated samples;

[0094] The target pseudo-label annotated sample is stored in the annotated sample set, and the target pseudo-label annotated sample is deleted from the unlabeled sample set.

[0095] Optionally, the credible score acquisition module 13 is specifically used to obtain the execution degree value when the initial fine-tuning language model outputs the pseudo-label annotated sample;

[0096] Using a credibility decision tree to judge the pseudo-labeled samples, and obtain the correctness value corresponding to the pseudo-labeled samples;

[0097] The execution degree value and the accuracy value are weighted and summed to obtain the credibility score of the pseudo-labeled sample.

[0098] The model training module 14 is used to perform model training on the fine-tuning language model based on the annotated sample set until the number of model training reaches a preset number of iterations to obtain a language model that meets the target capability training task.

[0099] Optionally, the model training module 14 is specifically used to perform model training on the fine-tuning language model based on the annotated sample set;

[0100] The step of inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples is performed until the model training times reaches a preset number of iterations to obtain a language model that meets the target capability training task.

[0101] A fusion processing module 15 is used to obtain at least two language models that meet the target capability training task and construct a language model integration;

[0102] Acquire an integrated model test sample set, wherein the integrated model test sample set includes at least one labeled sample data;

[0103] Inputting the integrated model test sample set into the language model integration to obtain an integrated test result;

[0104] Based on the integrated test result, at least two of the language models are fused to obtain a fused language model that meets the target capability training task.

[0105] An update capability training module 16 is used to obtain an update annotation sample set corresponding to the update capability training task;

[0106] Based on the update layer data corresponding to the update capability training task, updating the language model;

[0107] Freezing the parameters of the updated language model except the update layer data to obtain an initial updated language model;

[0108] The initial updated language model is fine-tuned using the updated annotated sample set to obtain an updated language model that meets the update capability training task.

[0109] In this embodiment, an initial language model is created, an unlabeled unimodal sample set is obtained, and an unsupervised pre-training process is performed on the initial language model using the unlabeled unimodal sample set to obtain a pre-trained model, and a labeled sample set corresponding to the target capability training task is obtained, and the pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model. An unlabeled sample set corresponding to the target capability training task is obtained, and the unlabeled samples in the unlabeled sample set are input into the fine-tuned language model to obtain pseudo-label labeled samples, and the pseudo-label labeled samples are subjected to credibility judgment processing to obtain a credibility score corresponding to the pseudo-label labeled samples, wherein the credibility score can be obtained by combining the execution data and the accuracy value of the credibility decision tree, so as to avoid the situation where a single judgment angle leads to a judgment error. A target pseudo-label labeled sample with a credibility score higher than the credibility score threshold is obtained from the pseudo-label labeled sample, and the target pseudo-label labeled sample is saved in the labeled sample set, and the target pseudo-label labeled sample is deleted from the unlabeled sample set, so as to avoid the need to label the pseudo-label again when fine-tuning the language model during the next model training, thereby saving the model training process. The fine-tuned language model is trained based on the annotated sample set, and the expanded annotated sample set is used to train the fine-tuned language model, so that the annotated sample set is gradually expanded during the iterative training process to improve the model training effect, until the model training times reach the preset number of iterations, and a language model that meets the target capability training task can be obtained. By adding pseudo-labeled annotated samples with high credibility to the annotated sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually annotated samples and reducing the model training cost. At the same time, model integration can be used to obtain a fused language model that further enhances the target capability, and the model can also learn to update capabilities without affecting the target capability of the language model by freezing parameters.

[0110] It should be noted that the language model generation device provided in the above embodiment only uses the division of the above functional modules as an example when executing the language model generation method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the language model generation device provided in the above embodiment and the language model generation method embodiment belong to the same concept, and the implementation process thereof is detailed in the method embodiment, which will not be repeated here.

[0111] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0112] The present application also provides a computer storage medium, which can store multiple instructions, and the instructions are suitable for being loaded and executed by a processor as described above. Figure 1-Figure 4 The language model generation method of the embodiment shown in the figure can be specifically executed by referring to Figure 1-Figure 4 The specific description of the illustrated embodiment will not be repeated here.

[0113] The present application also provides a computer program product, which stores at least one instruction, and the at least one instruction is loaded and executed by the processor as described above. Figure 1-Figure 4 The language model generation method of the embodiment shown in the figure can be specifically executed by referring to Figure 1-Figure 4 The specific description of the illustrated embodiment will not be repeated here.

[0114] Please refer to Figure 7 , which shows a block diagram of the structure of an electronic device provided by an exemplary embodiment of the present application. The electronic device in the present application may include one or more of the following components: a processor 110, a memory 120, an input device 130, an output device 140, and a bus 150. The processor 110, the memory 120, the input device 130, and the output device 140 may be connected via the bus 150.

[0115] The processor 110 may include one or more processing cores. The processor 110 uses various interfaces and lines to connect various parts of the entire electronic device, and executes various functions and processes data of the terminal 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 120, and calling data stored in the memory 120. Optionally, the processor 110 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 110 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), and a modem. Among them, the CPU mainly processes the operating system, user pages, and applications; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 110, but may be implemented separately through a communication chip.

[0116] The memory 120 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 120 includes a non-transitory computer-readable medium (Non-Transitory Computer-Readable Storage Medium). The memory 120 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 120 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the above-mentioned various method embodiments, etc. The operating system may be an Android system, including a system deeply developed based on the Android system, an IOS system developed by Apple, including a system deeply developed based on the IOS system or other systems.

[0117] The memory 120 can be divided into an operating system space and a user space. The operating system runs in the operating system space, and native and third-party applications run in the user space. In order to ensure that different third-party applications can achieve good operating results, the operating system allocates corresponding system resources to different third-party applications. However, the requirements for system resources in different application scenarios in the same third-party application are also different. For example, in the local resource loading scenario, the third-party application has higher requirements for disk reading speed; in the animation rendering scenario, the third-party application has higher requirements for GPU performance. The operating system and third-party applications are independent of each other, and the operating system often cannot perceive the current application scenario of the third-party application in a timely manner, resulting in the operating system being unable to perform targeted system resource adaptation according to the specific application scenario of the third-party application.

[0118] In order for the operating system to distinguish the specific application scenarios of third-party applications, it is necessary to open up data communication between third-party applications and the operating system so that the operating system can obtain the current scenario information of third-party applications at any time, and then perform targeted system resource adaptation based on the current scenario.

[0119] The input device 130 is used to receive input commands or data, and includes but is not limited to a keyboard, a mouse, a camera, a microphone, or a touch device. The output device 140 is used to output commands or data, and includes but is not limited to a display device and a speaker. In one example, the input device 130 and the output device 140 can be combined, and the input device 130 and the output device 140 are touch screen displays.

[0120] The touch display screen can be designed as a full screen, a curved screen or a special-shaped screen. The touch display screen can also be designed as a combination of a full screen and a curved screen, or a combination of a special-shaped screen and a curved screen, which is not limited in the embodiments of the present application.

[0121] In addition, those skilled in the art will appreciate that the structure of the electronic device shown in the above drawings does not constitute a limitation on the electronic device, and the electronic device may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the electronic device also includes a radio frequency circuit, an input unit, a sensor, an audio circuit, a wireless fidelity (WiFi) module, a power supply, a Bluetooth module and other components, which will not be described in detail here.

[0122] exist Figure 7 In the electronic device shown, the processor 110 may be used to call the language model generation application stored in the memory 120, and specifically perform the following operations:

[0123] Obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability;

[0124] Inputting the unlabeled samples in the unlabeled sample set into the fine-tuned language model to obtain pseudo-labeled samples;

[0125] Obtaining the credibility score of the pseudo-label annotated sample, and storing the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set;

[0126] The fine-tuning language model is trained based on the annotated sample set until the number of model training reaches a preset number of iterations, thereby obtaining a language model that meets the target capability training task.

[0127] In one embodiment, when the processor 110 executes the acquisition of the fine-tuned language model trained based on the annotated sample set corresponding to the target capability training task, the processor 110 specifically performs the following operations:

[0128] Creating an initial language model, obtaining an unlabeled unimodal sample set, wherein the unlabeled unimodal sample set includes unlabeled unimodal sample data;

[0129] Using an unlabeled unimodal sample set to pre-train the initial language model to obtain a pre-trained model;

[0130] Obtain a set of labeled samples corresponding to the target ability training task;

[0131] The pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model.

[0132] In one embodiment, the processor 110 performs the following operations when obtaining the credibility score of the pseudo-label annotated sample and storing the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set:

[0133] Performing credibility judgment processing on the pseudo-labeled sample to obtain a credibility score corresponding to the pseudo-labeled sample;

[0134] Obtaining a target pseudo-label annotated sample whose credibility score is higher than a credibility score threshold from the pseudo-label annotated samples;

[0135] The target pseudo-label annotated sample is stored in the annotated sample set, and the target pseudo-label annotated sample is deleted from the unlabeled sample set.

[0136] In one embodiment, the processor 110 performs the following operations when performing credibility judgment processing on the pseudo-labeled sample and obtaining the credibility score corresponding to the pseudo-labeled sample:

[0137] Obtaining an execution value when the initial fine-tuned language model outputs the pseudo-labeled sample;

[0138] Using a credibility decision tree to judge the pseudo-labeled samples, and obtain the correctness value corresponding to the pseudo-labeled samples;

[0139] The execution degree value and the accuracy value are weighted and summed to obtain the credibility score of the pseudo-labeled sample.

[0140] In one embodiment, when the processor 110 performs model training on the fine-tuning language model based on the annotated sample set until the number of model training times reaches a preset number of iterations and a language model that meets the target capability training task is obtained, the processor 110 specifically performs the following operations:

[0141] Performing model training on the fine-tuning language model based on the annotated sample set;

[0142] The step of inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples is performed until the model training times reaches a preset number of iterations to obtain a language model that meets the target capability training task.

[0143] In one embodiment, when executing the language model generation method, the processor 110 further performs the following operations:

[0144] Acquire at least two of the language models that meet the target capability training task, and construct a language model ensemble;

[0145] Acquire an integrated model test sample set, wherein the integrated model test sample set includes at least one labeled sample data;

[0146] Inputting the integrated model test sample set into the language model integration to obtain an integrated test result;

[0147] Based on the integrated test result, at least two of the language models are fused to obtain a fused language model that meets the target capability training task.

[0148] In one embodiment, when executing the language model generation method, the processor 110 further performs the following operations:

[0149] Obtain the updated annotation sample set corresponding to the update capability training task;

[0150] Based on the update layer data corresponding to the update capability training task, updating the language model;

[0151] Freezing the parameters of the updated language model except the update layer data to obtain an initial updated language model;

[0152] The initial updated language model is fine-tuned using the updated annotated sample set to obtain an updated language model that meets the update capability training task.

[0153] In this embodiment, an initial language model is created, an unlabeled unimodal sample set is obtained, and an unsupervised pre-training process is performed on the initial language model using the unlabeled unimodal sample set to obtain a pre-trained model, and a labeled sample set corresponding to the target capability training task is obtained, and the pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model. An unlabeled sample set corresponding to the target capability training task is obtained, and the unlabeled samples in the unlabeled sample set are input into the fine-tuned language model to obtain pseudo-label labeled samples, and the pseudo-label labeled samples are subjected to credibility judgment processing to obtain a credibility score corresponding to the pseudo-label labeled samples, wherein the credibility score can be obtained by combining the execution data and the accuracy value of the credibility decision tree, so as to avoid the situation where a single judgment angle leads to a judgment error. A target pseudo-label labeled sample with a credibility score higher than the credibility score threshold is obtained from the pseudo-label labeled sample, and the target pseudo-label labeled sample is saved in the labeled sample set, and the target pseudo-label labeled sample is deleted from the unlabeled sample set, so as to avoid the need to label the pseudo-label again when fine-tuning the language model during the next model training, thereby saving the model training process. The fine-tuned language model is trained based on the annotated sample set, and the expanded annotated sample set is used to train the fine-tuned language model, so that the annotated sample set is gradually expanded during the iterative training process to improve the model training effect, until the model training times reach the preset number of iterations, and a language model that meets the target capability training task can be obtained. By adding pseudo-labeled annotated samples with high credibility to the annotated sample set for model training, samples with higher accuracy improve the model training effect and the generalization ability of the model, while reducing the demand for manually annotated samples and reducing the model training cost. At the same time, model integration can be used to obtain a fused language model that further enhances the target capability, and the model can also learn to update capabilities without affecting the target capability of the language model by freezing parameters.

[0154] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in the embodiments of this specification are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions. For example, the initial language model, annotated samples, etc. involved in this specification are all obtained with full authorization.

[0155] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing related hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only storage memory, or a random access memory, etc.

[0156] The above disclosure is only the preferred embodiment of the present application, which certainly cannot be used to limit the scope of rights of the present application. Therefore, equivalent changes made according to the claims of the present application are still within the scope covered by the present application.

Claims

1. A language model generation method, characterized in that: The method comprises: Obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability; Inputting the unlabeled samples in the unlabeled sample set into the fine-tuned language model to obtain pseudo-labeled samples; Obtaining a credibility score of the pseudo-labeled sample, and storing the pseudo-labeled sample whose credibility score is higher than a credibility score threshold in the labeled sample set; Performing model training on the fine-tuning language model based on the annotated sample set until the number of model training times reaches a preset number of iterations, thereby obtaining a language model that meets the target capability training task; The obtaining of the credibility score of the pseudo-labeled sample includes: Obtaining an execution value when the initial fine-tuning language model outputs the pseudo-labeled sample; Using a credibility decision tree to judge the pseudo-labeled samples, and obtain the correctness value corresponding to the pseudo-labeled samples; The execution degree value and the accuracy value are weighted and summed to obtain the credibility score of the pseudo-labeled sample.

2. The method according to claim 1, characterized in that The step of obtaining a fine-tuned language model trained based on a set of labeled samples corresponding to the target capability training task includes: Creating an initial language model, obtaining an unlabeled unimodal sample set, wherein the unlabeled unimodal sample set includes unlabeled unimodal sample data; Using an unlabeled unimodal sample set to pre-train the initial language model to obtain a pre-trained model; Obtain a set of labeled samples corresponding to the target ability training task; The pre-trained model is fine-tuned using the labeled sample set to obtain a fine-tuned language model.

3. The method according to claim 1, characterized in that: The obtaining of the credibility score of the pseudo-label annotated sample and storing the pseudo-label annotated sample whose credibility score is higher than the credibility score threshold in the annotated sample set includes: Performing credibility judgment processing on the pseudo-labeled sample to obtain a credibility score corresponding to the pseudo-labeled sample; Obtaining a target pseudo-label annotated sample whose credibility score is higher than a credibility score threshold from the pseudo-label annotated samples; The target pseudo-label annotated sample is stored in the annotated sample set, and the target pseudo-label annotated sample is deleted from the unannotated sample set.

4. The method according to claim 1, characterized in that: The fine-tuning language model is trained based on the annotated sample set until the number of model training reaches a preset number of iterations to obtain a language model that meets the target capability training task, including: Performing model training on the fine-tuning language model based on the annotated sample set; The step of inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples is performed until the model training times reaches a preset number of iterations to obtain a language model that meets the target capability training task.

5. The method according to claim 1, characterized in that: The method further comprises: Acquire at least two of the language models that meet the target capability training task, and construct a language model ensemble; Acquire an integrated model test sample set, wherein the integrated model test sample set includes at least one labeled sample data; Inputting the integrated model test sample set into the language model integration to obtain an integrated test result; Based on the integrated test result, at least two of the language models are fused to obtain a fused language model that meets the target capability training task.

6. The method according to claim 1, characterized in that The method further comprises: Obtain the updated annotation sample set corresponding to the update capability training task; Based on the update layer data corresponding to the update capability training task, updating the language model; Freezing the parameters of the updated language model except the update layer data to obtain an initial updated language model; The initial updated language model is fine-tuned using the updated annotated sample set to obtain an updated language model that meets the update capability training task.

7. A language model generation device, characterized in that: The device comprises: A model fine-tuning module is used to obtain a fine-tuned language model trained based on a labeled sample set corresponding to a target capability training task, and obtain an unlabeled sample set corresponding to the target capability training task, wherein the labeled samples in the labeled sample set are question information, and the labels of the labeled samples are answer information for the question information under the target capability; A pseudo-label acquisition module, used for inputting the unlabeled samples in the unlabeled sample set into the fine-tuning language model to obtain pseudo-labeled samples; A credible score acquisition module, used to acquire the credible score of the pseudo-label annotated sample, and save the pseudo-label annotated sample whose credible score is higher than the credible score threshold in the annotated sample set; A model training module, used to perform model training on the fine-tuning language model based on the annotated sample set until the number of model training times reaches a preset number of iterations to obtain a language model that meets the target capability training task; The trust score acquisition module is specifically used to obtain the execution value when the initial fine-tuning language model outputs the pseudo-label annotated sample; Using a credibility decision tree to judge the pseudo-labeled samples, and obtain the correctness value corresponding to the pseudo-labeled samples; The execution degree value and the accuracy value are weighted and summed to obtain the credibility score of the pseudo-labeled sample.

8. A computer storage medium, characterized in that The computer storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor and executing the method steps as claimed in any one of claims 1 to 6.

9. An electronic device, characterized in that: include: A processor and a memory; wherein the memory stores a computer program, and the computer program is suitable for being loaded by the processor and executing the method steps as claimed in any one of claims 1 to 6.

Images