Language capability GPT model training method and device, computer device, and storage medium

Abstract

The application relates to the technical field of computers and discloses a language ability GPT model training method and device, computer equipment and a storage medium, the application determines the ability index of the pre-acquired sample data, improves the accuracy and generalization ability of the model; the ability index of the sample data is input into a preset model respectively, initial parameter weights of the sample data are obtained, the required language expression ability can be measured, and this is beneficial to the understanding of the GPT model; the initial parameter weights are input into the GPT model, first probability distribution and second probability distribution are obtained; the reward score of the GPT model is determined according to the first probability distribution and the second probability distribution, and the performance of the GPT model can be better evaluated; the reward score is input into a reinforcement learning model, adjustment parameters of the GPT model are obtained, the GPT model is adjusted according to the adjustment parameters, and until the loss function of the GPT model meets a preset condition, a high-quality GPT model can be trained more quickly, and the efficiency and precision of the model are improved.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and specifically to a method, apparatus, computer equipment, and storage medium for training a language ability GPT model. Background Technology

[0002] With the development of GPT technology, GPT models have achieved remarkable results in the field of natural language processing. However, current GPT models still lack accurate measurement and understanding of language expression ability, which leads to inaccurate output of GPT models. Therefore, it is urgent to propose a new training method for language ability GPT models to improve the ability of GPT models to understand the content of language expression. Summary of the Invention

[0003] In view of this, the present invention provides a method, apparatus, computer equipment and storage medium for training a language ability GPT model, so as to solve the problem of inaccurate measurement of language expression ability by the GPT model.

[0004] In a first aspect, the present invention provides a method for training a language ability GPT model, the method comprising:

[0005] Determine the capability indicators based on the sample data to be acquired;

[0006] The capability indicators of the sample data are input into the preset model to obtain the initial parameter weights of the sample data.

[0007] The initial parameter weights are input into the GPT model to obtain the first probability distribution and the second probability distribution corresponding to the initial parameter weights.

[0008] The reward score of the GPT model is determined based on the first probability distribution and the second probability distribution.

[0009] The reward score is input into the reinforcement learning model to obtain the adjustment parameters of the GPT model. The GPT model is adjusted according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and the trained GPT model is output.

[0010] Beneficial effects include: determining capability indicators through pre-acquired sample data, which better guides the training of the GPT model and improves its accuracy and generalization ability. Inputting the capability indicators of the sample data into the pre-defined model yields the initial parameter weights, allowing for the measurement of required language expression capabilities, thus benefiting the subsequent understanding of the GPT model. Inputting the initial parameter weights into the GPT model yields the corresponding first and second probability distributions, better simulating the distribution patterns of language data and improving the GPT model's language generation and comprehension abilities. Determining the GPT model's reward score based on the first and second probability distributions allows for better evaluation of the GPT model's performance, guiding the model towards optimal training. Inputting the reward score into the reinforcement learning model yields the GPT model's adjustment parameters. Adjusting the GPT model based on these parameters until the loss function meets preset conditions enables faster training of high-quality GPT models, improving both efficiency and accuracy.

[0011] In one optional implementation, the preset model includes at least one method for inputting capability indicators of the sample data into the preset model to obtain initial parameter weights for the sample data, specifically including:

[0012] The capability indicators of the sample data are input into the first preset model to obtain the first initial parameter weights corresponding to the first preset model. The first preset model is any one of at least one preset model, and the initial parameter weights corresponding to each preset model are the parameter weights for measuring the language attributes of the sample data in different dimensions.

[0013] In one alternative implementation, the reward score of the GPT model is calculated using the following formula:

[0014] R = aD KLn (P a ||Q a )+bD KL(n-1) (P n-1 ||Q n-1 )+…+·gD KL (P1||Q1)

[0015] Where n is the number of preset models, P a For the first probability distribution corresponding to the nth preset model, Q a This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively;

[0016]

[0017] Where P(x) is the first probability distribution of the corresponding preset model, and Q(x) is the second probability distribution of the corresponding preset model.

[0018] In one alternative implementation, the loss function of the GPT model is calculated using the following formula:

[0019] L=-∑(p(x)log(g(x))+aL n +bL n-1 +…+gL1)

[0020] Where p(x) is the labeled probability of the GPT model, g(x) is the predicted probability of the GPT model, L_n is the loss function corresponding to the nth preset model, n is the number of preset models, and n is a positive integer greater than or equal to 1.

[0021] In one alternative implementation, the preset models include a lexical richness model, a discourse coherence model, and a syntactic structure model.

[0022] Secondly, the present invention provides a language ability GPT model training device, characterized in that the device comprises:

[0023] The capability indicator determination module is used to determine the capability indicators for the pre-acquired sample data.

[0024] The prediction weight module is used to input the capability indicators of the sample data into the preset model to obtain the initial parameter weights of the sample data.

[0025] The prediction output probability module is used to input the initial parameter weights into the GPT model to obtain the first probability distribution and the second probability distribution corresponding to the initial parameter weights.

[0026] The reward determination module is used to determine the reward score of the GPT model based on the first probability distribution and the second probability distribution.

[0027] The reinforcement learning module is used to input the reward score into the reinforcement learning model to obtain the adjustment parameters of the GPT model. The GPT model is then adjusted according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and the trained GPT model is output.

[0028] In one optional implementation, the preset model includes at least one prediction weight module, specifically used for:

[0029] The capability indicators of the sample data are input into the first preset model to obtain the first initial parameter weights corresponding to the first preset model. The first preset model is any one of at least one preset model, and the initial parameter weights corresponding to each preset model are the parameter weights for measuring the language attributes of the sample data in different dimensions.

[0030] In one alternative implementation, the reward score of the GPT model is calculated using the following formula:

[0031] R = aD KLn (P a ||Q a )+bD KL(n-1) (P n-1 ||Q n-1 )+…+·gD KL (P1||Q1)

[0032] Where n is the number of preset models, P a For the first probability distribution corresponding to the nth preset model, Q a This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively;

[0033]

[0034] Where P(x) is the first probability distribution of the corresponding preset model, and Q(x) is the second probability distribution of the corresponding preset model.

[0035] Thirdly, the present invention provides a computer device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the language ability GPT model training method described in the first aspect or any corresponding embodiment thereof.

[0036] Fourthly, the present invention provides a computer-readable storage medium storing computer instructions for causing a computer to execute the language capability GPT model training method described in the first aspect or any corresponding embodiment thereof. Attached Figure Description

[0037] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0038] Figure 1 This is a flowchart of a language ability GPT model training method according to an embodiment of the present invention;

[0039] Figure 2 This is a schematic diagram of a language ability GPT model training device according to an embodiment of the present invention;

[0040] Figure 3 This is a schematic diagram of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] This embodiment provides a method for training a language ability GPT model. Figure 1 This is a flowchart of a language ability GPT model training method according to an embodiment of the present invention, such as... Figure 1 As shown, the method includes the following steps:

[0043] Step S101: Determine the sample data to be acquired for capability indicators.

[0044] For example, the sample data consists of a large amount of high-quality training data related to language expression ability, such as sample data related to students, which may include students' writings, oral expressions, papers, etc. After obtaining the sample data, it can be manually labeled to determine the corresponding language expression ability for measurement.

[0045] Specifically, when annotating sample data, lexical richness, discourse coherence, and syntactic complexity can be evaluated based on annotation criteria.

[0046] Develop a scoring guideline: Create a detailed scoring guideline that clarifies the scoring rules and examples for each annotation item to ensure consistency in annotation; Initial annotation: Annotators independently complete a round of initial annotation, marking relevant features and scores; Quality control and feedback: Review the annotation results, identify inconsistencies, and provide feedback to the annotators; Final annotation: Based on the feedback, revise the annotations until a certain consistency standard is met, at which point the annotation work is considered complete.

[0047] Lexical Richness: Vocabulary Variety: Indicates the diversity of words appearing in the text. It can be assessed by noting the use of synonyms, the appropriate use of technical terms, and avoiding excessive word repetition. Vocabulary Appropriateness: Indicates whether vocabulary is suitable for the context, topic, and target audience. It can be scored based on word accuracy and contextual relevance.

[0048] Discourse Coherence: Topic Consistency: Assess whether the text maintains its focus on the topic throughout. It can be marked based on whether the text deviates from the topic. Logical Flow: Demonstrates the smoothness of the connections and transitions between sentences and paragraphs. It is evaluated based on the use of transitional phrases, sentence connections, and paragraph links. Internal Logic: Indicates the internal logical structure and argumentative coherence of the text. It can be marked with the clarity of arguments and evidence, as well as the logical connections between them.

[0049] Grammatical Complexity: Sentence Variety: Reflects the diversity of sentence types (simple sentences, compound sentences, complex sentences, etc.) used in the text. Different levels of annotation can be applied to sentence types. Grammatical Correctness: Indicates whether a sentence follows grammatical rules. Grammatical errors can be marked, such as subject-verb disagreement, tense errors, spelling errors, etc. Sentence Structure Complexity: Assesses the complexity of a sentence's structure based on the number and type of clauses. The use of compound and complex sentences can be used as a reference.

[0050] The capability indicators of the sample data can be the text features in Table 1. Specifically, the methods for determining the capability indicators can adopt relevant technical means in the existing technology, which will not be elaborated here.

[0051]

[0052]

[0053]

[0054] Step S102: Input the capability indicators of the sample data into the preset model to obtain the initial parameter weights of the sample data.

[0055] For example, the preset model is a model related to the measurement and understanding of language ability. Specifically, it can be a model trained based on the parameters of language ability measurement from labeled sample data. It can be a preset model trained by Support Vector Machine (SVM), Random Forest, etc. The embodiments of the present invention do not limit the model type of the above preset model, and those skilled in the art can determine it according to the actual situation.

[0056] By inputting the ability indicators into a pre-trained model, the initial parameter weights corresponding to the language expression ability measure can be accurately predicted, such as information like the vocabulary richness of the sample data.

[0057] In a preferred embodiment, the preset model includes at least one method for inputting capability indicators of the sample data into the preset model to obtain initial parameter weights of the sample data, specifically including:

[0058] The capability indicators of the sample data are input into the first preset model to obtain the first initial parameter weights corresponding to the first preset model. The first preset model is any one of at least one preset model, and the initial parameter weights corresponding to each preset model are the parameter weights for measuring the language attributes of the sample data in different dimensions.

[0059] For example, the preset model is a prediction model for language expression ability during the overall training process. Therefore, the number of preset models can be set according to the actual situation. For example, it is necessary to determine the vocabulary richness model, discourse coherence model and syntactic structure model in the sample data. In this case, three models can be set, and the corresponding models can be trained according to these three indicators respectively.

[0060] Step S103: Input the initial parameter weights into the GPT model to obtain the first probability distribution and the second probability distribution corresponding to the initial parameter weights.

[0061] For example, the initial weight parameters are input into the GPT model, and the model is trained based on the initial weight parameters to obtain the first probability distribution and the second probability distribution corresponding to the initial language model and the current training model in the GPT model, respectively.

[0062] Step S104: Determine the reward score of the GPT model based on the first probability distribution and the second probability distribution.

[0063] For example, after obtaining the corresponding probability distribution, the reward score of the current GPT model is determined based on the corresponding probability distribution. Specifically, the reward score can be calculated using the following formula:

[0064]

[0065] Where P represents the first probability distribution and Q represents the second probability distribution.

[0066] In a preferred embodiment, when there are multiple preset models, the reward score of the GPT model is calculated using the following formula.

[0067] R = aD KLn (P a ||Q a )+bD KL(n-1) (P n-1 ||Q n-1 )+…+·gD KL (P1||Q1)

[0068] Where n is the number of preset models, P a For the first probability distribution corresponding to the nth preset model, Q a This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively;

[0069]

[0070] Where P(x) is the first probability distribution of the corresponding preset model, and Q(x) is the second probability distribution of the corresponding preset model.

[0071] Here, ag does not limit the number of corresponding preset models (the number of n); ag is only used to represent the initial weight parameters corresponding to the preset models.

[0072] Step S105: Input the reward score into the reinforcement learning model to obtain the adjustment parameters of the GPT model. Adjust the GPT model according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and output the trained GPT model.

[0073] For example, after obtaining the reward score corresponding to the GPT model, the reward score is input into the reinforcement learning model to obtain the adjustment parameters of the GPT model. The GPT model is optimized based on the adjustment parameters, specifically by adjusting parameters in the model, increasing or decreasing the number of network layers, or changing the network structure. The loss function value of the optimized GPT model is calculated, and it is checked whether it meets preset conditions. If the preset conditions are met, the trained GPT model is output; if the conditions are not met, the above steps need to be repeated until the model meets the preset conditions or reaches the preset number of training iterations. In this embodiment of the invention, the loss function corresponding to the GPT model needs to meet a preset standard.

[0074] In a preferred embodiment, the loss function of the GPT model is calculated using the following formula:

[0075] L=-∑(p(x)log(g(x))+aL n +bL n-1 +…+gL1)

[0076] Where p(x) is the labeled probability of the GPT model, g(x) is the predicted probability of the GPT model, L_n is the loss function corresponding to the nth preset model, n is the number of preset models, and n is a positive integer greater than or equal to 1.

[0077] This method, by determining capability indicators using pre-acquired sample data, can better guide the training of the GPT model, improving its accuracy and generalization ability. Inputting the capability indicators of the sample data into the pre-defined model yields the initial parameter weights, allowing for the measurement of required language expression capabilities, which is beneficial for the subsequent understanding of the GPT model. Inputting the initial parameter weights into the GPT model yields the first and second probability distributions corresponding to these weights, better simulating the distribution patterns of language data and improving the GPT model's language generation and comprehension capabilities. Based on the first and second probability distributions, the reward score of the GPT model is determined, enabling better evaluation of its performance and guiding the model towards optimal training. The reward score is then input into the reinforcement learning model to obtain the adjustment parameters for the GPT model. Adjusting the GPT model based on these parameters until its loss function meets preset conditions allows for faster training of a high-quality GPT model, improving both efficiency and accuracy.

[0078] This embodiment also provides a language ability GPT model training device, which is used to implement the above embodiments and preferred embodiments, and will not be repeated as already described. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0079] This embodiment provides a language ability GPT model training device, such as Figure 2 As shown, it includes:

[0080] The capability index determination module 201 is used to determine the capability index of the pre-acquired sample data;

[0081] The prediction weight module 202 is used to input the capability indicators of the sample data into the preset model to obtain the initial parameter weights of the sample data.

[0082] The prediction output probability module 203 is used to input the initial parameter weights into the GPT model to obtain the first probability distribution and the second probability distribution corresponding to the initial parameter weights.

[0083] The reward determination module 204 is used to determine the reward score of the GPT model based on the first probability distribution and the second probability distribution.

[0084] The reinforcement learning module 205 is used to input the reward score into the reinforcement learning model to obtain the adjustment parameters of the GPT model. The GPT model is adjusted according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and the trained GPT model is output.

[0085] In some optional implementations, the preset model includes at least one prediction weight module, specifically used for:

[0086] The capability indicators of the sample data are input into the first preset model to obtain the first initial parameter weights corresponding to the first preset model. The first preset model is any one of at least one preset model, and the initial parameter weights corresponding to each preset model are the parameter weights for measuring the language attributes of the sample data in different dimensions.

[0087] In some alternative implementations, the reward score of the GPT model is calculated using the following formula:

[0088] R = aD KLn (P a ||Q a )+bD KL(n-1) (P n-1 ||Q n-1 )+…+·gD KL (P1||Q1)

[0089] Where n is the number of preset models, P a For the first probability distribution corresponding to the nth preset model, Q a This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively;

[0090]

[0091] Where P(x) is the first probability distribution of the corresponding preset model, and Q(x) is the second probability distribution of the corresponding preset model.

[0092] In some alternative implementations, the loss function of the GPT model is calculated using the following formula:

[0093] L=-∑(p(x)log(g(x))+aL n +bL n-1 +…+gL_1)

[0094] Where p(x) is the labeled probability of the GPT model, g(x) is the predicted probability of the GPT model, L_n is the loss function corresponding to the nth preset model, n is the number of preset models, and n is a positive integer greater than or equal to 1.

[0095] In some alternative implementations, the preset models include a lexical richness model, a discourse coherence model, and a syntactic structure model.

[0096] Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0097] In this embodiment, the language ability GPT model training device is presented in the form of functional units. Here, a unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above functions.

[0098] This invention also provides a computer device having the above-described features. Figure 2 The language ability GPT model training device shown.

[0099] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a computer device provided in an optional embodiment of the present invention, such as... Figure 3 As shown, the computer device includes one or more processors 10, memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components communicate with each other via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on external input / output devices (such as display devices coupled to the interfaces). In some alternative implementations, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system). Figure 3 Take a processor 10 as an example.

[0100] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0101] The memory 20 stores instructions executable by at least one processor 10 to cause at least one processor 10 to perform the method shown in the above embodiments.

[0102] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0103] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0104] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0105] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0106] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method for training a language ability GPT model, characterized in that, The method includes: The pre-acquired sample data is used to determine the ability indicators, and the sample data includes written works, oral expression, and papers; The ability indicators of the sample data are input into a preset model to obtain the initial parameter weights of the sample data. The preset model is a prediction model for language expression ability during the overall training process. The initial parameter weights are input into the GPT model to obtain the first probability distribution and the second probability distribution corresponding to the initial parameter weights. The reward score of the GPT model is determined based on the first probability distribution and the second probability distribution. The reward score is input into the reinforcement learning model to obtain the adjustment parameters of the GPT model. The GPT model is adjusted according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and the trained GPT model is output. The language expression is processed based on the trained GPT model.

2. The method according to claim 1, characterized in that, The preset model includes at least one component, and the step of inputting the capability index of the sample data into the preset model to obtain the initial parameter weights of the sample data specifically includes: The capability index of the sample data is input into the first preset model to obtain the first initial parameter weight corresponding to the first preset model. The first preset model is any one of the at least one preset model. The initial parameter weight corresponding to each preset model is the parameter weight of measuring the language attribute of the sample data in different dimensions.

3. The method according to claim 2, characterized in that, The reward score of the GPT model is calculated using the following formula. Where n represents the number of preset models. This corresponds to the first probability distribution of the nth preset model. This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively; in, This corresponds to the first probability distribution of the preset model. This represents the second probability distribution corresponding to the preset model.

4. The method according to claim 3, characterized in that, The loss function of the GPT model is calculated using the following formula: in, The labeled probabilities of the GPT model are... The predicted probability of the GPT model. Let n be the loss function corresponding to the nth preset model, where n is the number of preset models and n is a positive integer greater than or equal to 1.

5. The method according to claim 1, characterized in that, The preset models include a lexical richness model, a discourse coherence model, and a syntactic structure model.

6. A language ability GPT model training device, characterized in that, The device includes: The ability indicator determination module is used to determine the ability indicators of the pre-acquired sample data, including written works, oral expression, and papers. The prediction weight module is used to input the ability indicators of the sample data into the preset model to obtain the initial parameter weights of the sample data. The preset model is a prediction model for language expression ability during the overall training process. The prediction output probability module is used to input the initial parameter weights into the GPT model to obtain a first probability distribution and a second probability distribution corresponding to the initial parameter weights. The reward determination module is used to determine the reward score of the GPT model based on the first probability distribution and the second probability distribution. The reinforcement learning module is used to input the reward score into the reinforcement learning model to obtain the adjustment parameters of the GPT model, adjust the GPT model according to the adjustment parameters until the loss function of the GPT model meets the preset conditions, and output the trained GPT model. The language expression is processed based on the trained GPT model.

7. The apparatus according to claim 6, characterized in that, The preset model includes at least one, and the prediction weight module is specifically used for: The capability index of the sample data is input into the first preset model to obtain the first initial parameter weight corresponding to the first preset model. The first preset model is any one of the at least one preset model. The initial parameter weight corresponding to each preset model is the parameter weight of measuring the language attribute of the sample data in different dimensions.

8. The apparatus according to claim 6, characterized in that, The reward score of the GPT model is calculated using the following formula. Where n represents the number of preset models. This corresponds to the first probability distribution of the nth preset model. This represents the second probability distribution corresponding to the nth preset model; n is a positive integer greater than or equal to 1, and ag represents the initial weight parameters corresponding to the n preset models respectively; in, This corresponds to the first probability distribution of the preset model. This represents the second probability distribution corresponding to the preset model.

9. A computer device, characterized in that, include: A memory and a processor are interconnected, the memory storing computer instructions, and the processor executing the computer instructions to perform the language competence GPT model training method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the language competence GPT model training method according to any one of claims 1 to 5.

Images