Language model training method and device, equipment and storage medium

Abstract

The invention belongs to the technical field of model training, and discloses a language model training method and device, equipment and a storage medium. According to the method, the strategy model is trained according to the pre-training model corresponding to the data field, and the field sampling ratio is determined; performing data sampling based on the domain sampling ratio to obtain a training data set; and training the language model according to the training data set. The strategy model is trained through the pre-training model corresponding to each data field, so that the field sampling ratio is determined, data sampling is performed from each data field according to the field sampling ratio, and the language model is trained based on the sampled data, so that the actually used language model can be ensured to better meet downstream task requirements; therefore, the model performance of the language model on the downstream task is improved.

Description

Technical Field

[0001] The present invention relates to the field of model training technology, and in particular to a language model training method, device, equipment and storage medium. Background Art

[0002] The training data set used for training language models (LM) is usually sampled from a mixed set of data from many different fields. The mixed ratio of domain data in each data field (such as encyclopedias, books, and online texts) will greatly affect the performance of the language model. When it is unclear how much domain data in each data field should be included to produce a model that can guarantee good performance for a variety of downstream tasks, the existing technology usually determines the domain sampling ratio (the domain sampling ratio includes the sampling weights of each data field) by using intuition or a small number of downstream tasks. However, the domain sampling ratio confirmed by this method is generally unreasonable, which seriously affects the performance of the model.

[0003] The above contents are only used to assist in understanding the technical solution of the present invention and do not constitute an admission that the above contents are prior art. Summary of the invention

[0004] The main purpose of the present invention is to provide a language model training method, device, equipment and storage medium, aiming to solve the technical problem that the prior art cannot reasonably determine the domain sampling ratio, resulting in poor language model performance.

[0005] To achieve the above object, the present invention provides a language model training method, which comprises the following steps:

[0006] Train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio;

[0007] Performing data sampling based on the field sampling ratio to obtain a training data set;

[0008] The language model is trained according to the training data set.

[0009] Optionally, the step of training the strategy model according to the pre-trained model corresponding to the data domain to determine the domain sampling ratio includes:

[0010] Aggregate the domain data corresponding to each data field to obtain the total data set;

[0011] The strategy model is trained according to the data set and the pre-trained models corresponding to each data field to determine the field sampling ratio.

[0012] Optionally, the step of training the strategy model according to the total data set and the pre-trained model corresponding to each data field to determine the field sampling ratio includes:

[0013] The strategy model is trained according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field;

[0014] Determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field;

[0015] If the strategy model converges, a domain sampling ratio is constructed according to the training sampling ratio.

[0016] Optionally, if the strategy model converges, the step of constructing a domain sampling ratio according to the training sampling ratio includes:

[0017] If the strategy model converges, reading the training sampling ratio used to train the strategy model to obtain a sampling ratio set;

[0018] The training sampling ratios in the sampling ratio set are averaged to obtain the domain sampling ratio.

[0019] Optionally, the step of determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field includes:

[0020] Get the model loss value in the pre-trained model corresponding to each data field;

[0021] Calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value;

[0022] Determine the maximum difference in difference based on the loss difference corresponding to each data field;

[0023] If the maximum difference of the difference values ​​is less than or equal to a preset threshold, it is determined that the strategy model has converged.

[0024] Optionally, the step of determining the maximum difference in difference according to the loss difference corresponding to each data field includes:

[0025] Obtain the maximum and minimum values ​​of the loss differences corresponding to each data field;

[0026] The difference between the maximum value and the minimum value is calculated to obtain the maximum difference of the difference.

[0027] Optionally, after the step of determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field, the step further includes:

[0028] If the strategy model has not converged, the sampling weights corresponding to each data field in the training sampling ratio are adjusted with the goal of reducing the loss difference to obtain an adjusted sampling ratio;

[0029] The adjusted sampling ratio is used as a new training sampling ratio, and the step of training the strategy model according to the total data set and the training sampling ratio is returned to obtain the strategy loss value corresponding to each data field.

[0030] Optionally, the step of aggregating the domain data corresponding to each data domain to obtain a total data set includes:

[0031] Obtain the field data corresponding to each data field;

[0032] Performing data format verification on the field data according to a preset data format template;

[0033] When the verification is passed, the field data is aggregated to obtain a total data set.

[0034] Optionally, the step of obtaining field data corresponding to each data field includes:

[0035] Obtain data extraction rules corresponding to each data field;

[0036] Constructing a data extraction request according to the data extraction rule;

[0037] Each data field is accessed through the data extraction request to obtain field data corresponding to each data field.

[0038] Optionally, the step of accessing each data field through the data extraction request to obtain field data corresponding to each data field includes:

[0039] Access each data field through the data extraction request to obtain the original data corresponding to each data field;

[0040] The original data is preprocessed to obtain domain data corresponding to each data field, and the preprocessing includes at least one of deduplication processing, error data processing, missing data processing, and abnormal data processing.

[0041] Optionally, the step of performing data sampling based on the field sampling ratio to obtain a training data set includes:

[0042] Determine the number of data samples corresponding to each data field according to the preset total number of data and the field sampling ratio;

[0043] Data sampling is performed from each data field according to the number of data sampling items to obtain a training data set.

[0044] Optionally, the step of determining the number of data samples corresponding to each data field according to the preset total number of data and the field sampling ratio includes:

[0045] Read the total weight value from the field sampling ratio and the sampling weight corresponding to each data field;

[0046] Determine the data sampling ratio corresponding to each data field according to the sampling weight corresponding to each data field and the total value of the weight;

[0047] The data sampling ratio is multiplied by the total number of preset data to obtain the number of data samples corresponding to each data field.

[0048] In addition, to achieve the above object, the present invention also proposes a language model training device, which includes the following modules:

[0049] A determination module is used to train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio;

[0050] A sampling module, used for performing data sampling based on the field sampling ratio to obtain a training data set;

[0051] The training module is used to train the language model according to the training data set.

[0052] Optionally, the determination module is further used to aggregate the domain data corresponding to each data domain to obtain a total data set; train the strategy model according to the total data set and the pre-trained model corresponding to each data domain to determine the domain sampling ratio.

[0053] Optionally, the determination module is also used to train the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field; determine whether the strategy model converges according to the strategy loss value and the pre-trained model corresponding to each data field; if the strategy model converges, construct the field sampling ratio according to the training sampling ratio.

[0054] Optionally, the determination module is also used to read the training sampling ratio used for training the strategy model to obtain a sampling ratio set if the strategy model converges; and to average the training sampling ratios in the sampling ratio set to obtain the domain sampling ratio.

[0055] Optionally, the determination module is also used to obtain the model loss value in the pre-trained model corresponding to each data field; calculate the loss difference corresponding to each data field based on the model loss value and the strategy loss value; determine the maximum difference based on the loss difference corresponding to each data field; if the maximum difference is less than or equal to a preset threshold, it is determined that the strategy model has converged.

[0056] Optionally, the determination module is further used to obtain the maximum value and the minimum value of the loss difference corresponding to each data field; calculate the difference between the maximum value and the minimum value to obtain the maximum difference.

[0057] Optionally, the determination module is also used to adjust the sampling weights corresponding to each data field in the training sampling ratio with the goal of reducing the loss difference if the strategy model has not converged, so as to obtain an adjusted sampling ratio; use the adjusted sampling ratio as a new training sampling ratio, and return to the step of training the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field.

[0058] Optionally, the sampling module is further used to determine the number of data sampling items corresponding to each data field according to a preset total number of data and the field sampling ratio; and to sample data from each data field according to the number of data sampling items to obtain a training data set.

[0059] In addition, to achieve the above-mentioned purpose, the present invention also proposes a language model training device, which includes: a processor, a memory, and a language model training program stored in the memory and executable on the processor, and the language model training program implements the steps of the language model training method described above when executed by the processor.

[0060] In addition, to achieve the above-mentioned purpose, the present invention also proposes a computer-readable storage medium, on which a language model training program is stored, and when the language model training program is executed, the steps of the language model training method as described above are implemented.

[0061] The present invention trains the strategy model according to the pre-trained model corresponding to the data domain to determine the domain sampling ratio; performs data sampling based on the domain sampling ratio to obtain a training data set; and trains the language model according to the training data set. Since the strategy model is trained by the pre-trained model corresponding to each data domain, the domain sampling ratio is determined, and data is sampled from each data domain according to the domain sampling ratio, and the language model is trained based on the sampled data, it can be ensured that the language model actually used is more in line with the downstream task requirements, thereby improving the model performance of the language model on downstream tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0062] Figure 1 It is a schematic diagram of the structure of an electronic device in a hardware operating environment involved in an embodiment of the present invention;

[0063] Figure 2 This is a flow chart of the first embodiment of the language model training method of the present invention;

[0064] Figure 3This is a flow chart of a second embodiment of the language model training method of the present invention;

[0065] Figure 4 This is a flow chart of a third embodiment of the language model training method of the present invention;

[0066] Figure 5 This is a flow chart of a fourth embodiment of the language model training method of the present invention;

[0067] Figure 6 This is a structural block diagram of the first embodiment of the language model training device of the present invention.

[0068] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0069] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0070] Reference Figure 1 , Figure 1 It is a schematic diagram of the structure of a language model training device in the hardware operating environment involved in the embodiment of the present invention.

[0071] like Figure 1 As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, WI-FI) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0072] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the electronic device, and may include more or less components than shown in the figure, or combine certain components, or arrange the components differently.

[0073] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a language model training program.

[0074] exist Figure 1 In the electronic device shown, the network interface 1004 is mainly used for data communication with a network server; the user interface 1003 is mainly used for data interaction with a user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in a language model training device, and the electronic device calls the language model training program stored in the memory 1005 through the processor 1001, and executes the language model training method provided in an embodiment of the present invention.

[0075] The embodiment of the present invention provides a language model training method, referring to Figure 2 , Figure 2 The figure is a flow chart of a first embodiment of a language model training method of the present invention.

[0076] In this embodiment, the language model training method includes the following steps:

[0077] Step S10: Train the strategy model according to the pre-trained model corresponding to the data domain to determine the domain sampling ratio.

[0078] It should be noted that the executor of this embodiment can be the language model training device, and the language model training device can be an electronic device such as a personal computer or a server, or other devices that can achieve the same or similar functions. This embodiment is not limited to this. In this embodiment and the following embodiments, the language model training method of the present invention is described by taking the language model training device as an example.

[0079] It should be noted that the pre-trained model corresponding to the data domain can be a domain model obtained by pre-training based on the domain data of the data domain. Among them, the model structure of the domain model and the strategy model can be consistent with the model structure of the language model. Of course, according to actual needs, they can also be inconsistent, and this embodiment does not limit this. The domain sampling ratio can include the sampling weights corresponding to each data domain.

[0080] In actual use, the policy model is trained according to the pre-trained model corresponding to the data domain, and the domain sampling ratio can be determined by collecting data from the data set according to the preset training sampling ratio to train the policy model, and comparing it with the pre-trained model corresponding to each data domain, until the performance gap between the policy model and the pre-trained model corresponding to each data domain is small, the training is determined to be completed, and the training sampling ratio currently used by the policy model is used as the domain sampling ratio. Among them, the data set can be constructed by aggregating the domain data of each data domain.

[0081] Of course, when the strategy model training is completed and the domain sampling ratio is determined, the training sampling ratio previously used by the strategy model can also be obtained, and a weighted calculation is performed based on the previously used training sampling ratio and the currently used training sampling ratio to obtain the domain sampling ratio.

[0082] Step S20: performing data sampling based on the field sampling ratio to obtain a training data set.

[0083] It should be noted that data sampling is performed based on the domain sampling ratio to obtain a training data set, which can be performed by extracting sampling weights corresponding to each data domain from the domain sampling ratio, sampling data from the domain data of each data domain according to the sampling weights, and constructing a set based on the sampled data to obtain the training data set.

[0084] Step S30: training the language model according to the training data set.

[0085] It should be noted that when training the language model based on the training data set, the training data set can also be split into a training set and a verification set according to a preset ratio. The language model is trained with the training set, and the trained language model is verified with the verification set. After the verification is passed, the language model is put into actual business use to ensure good performance of the model.

[0086] In this embodiment, the strategy model is trained according to the pre-trained model corresponding to the data domain to determine the domain sampling ratio; data sampling is performed based on the domain sampling ratio to obtain a training data set; and the language model is trained according to the training data set. Since the strategy model is trained by the pre-trained model corresponding to each data domain, the domain sampling ratio is determined, and data is sampled from each data domain according to the domain sampling ratio, and the language model is trained based on the sampled data, it can be ensured that the language model actually used is more in line with the downstream task requirements, thereby improving the model performance of the language model on downstream tasks.

[0087] refer to Figure 3 , Figure 3 The figure is a flow chart of a second embodiment of a language model training method according to the present invention.

[0088] Based on the above first embodiment, step S10 of the language model training method of this embodiment may include:

[0089] Step S101: Aggregate the domain data corresponding to each data domain to obtain a total data set.

[0090] Step S102: training the strategy model according to the data set and the pre-trained models corresponding to each data field, and determining the field sampling ratio.

[0091] It should be noted that there may be multiple data fields, which may be pre-set by the administrator of the language model training device, and the domain data corresponding to the data field may be data collected from the data field and may be used to train the language model.

[0092] In actual use, the domain data corresponding to each data field are aggregated to obtain the total data set, which can be adding the domain data corresponding to each data field to the same set for aggregation, and using the obtained set as the total data set.

[0093] In a specific implementation, step S102 of the language model training method of this embodiment may include:

[0094] Step S1021: Train the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field.

[0095] It should be noted that the strategy model is trained according to the total data set and the training sampling ratio, and the strategy loss value corresponding to each data field is obtained by sampling the field data of each data field contained in the total data set through the sampling weights of each data field in the training sampling ratio, and the strategy model is trained with the sampled data. After the training, the loss score of the strategy model in each data field is calculated, and the obtained score is used as the strategy loss value.

[0096] Step S1022: Determine whether the strategy model converges according to the strategy loss value and the pre-trained model corresponding to each data field.

[0097] It should be noted that determining whether the strategy model has converged based on the strategy loss value and the pre-trained models corresponding to each data field can be by comparing the strategy loss value corresponding to each data field with the model loss value of the pre-trained model corresponding to each data field to determine whether there is a pre-trained model that is significantly different from the strategy model, thereby determining whether the model has converged.

[0098] In a specific implementation, the difference between the models may be represented by the difference between the loss values. In this case, step S1022 in this embodiment may include:

[0099] Get the model loss value in the pre-trained model corresponding to each data field;

[0100] Calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value;

[0101] Determine the maximum difference in difference based on the loss difference corresponding to each data field;

[0102] If the maximum difference of the difference values ​​is less than or equal to a preset threshold, it is determined that the strategy model has converged.

[0103] It should be noted that the loss difference corresponding to the data field can be calculated by subtracting the strategy loss value corresponding to the data field from the model loss value corresponding to the data field, and the obtained difference is used as the loss difference corresponding to the data field.

[0104] In a specific implementation, determining the maximum difference in difference according to the loss difference corresponding to each data field can be performed by obtaining the maximum value and the minimum value of the loss difference corresponding to each data field, and then calculating the difference between the maximum value and the minimum value to obtain the maximum difference in difference.

[0105] It can be understood that if the maximum difference is less than or equal to the preset threshold, it means that the differences between the loss differences corresponding to each data field are small. At this time, there is no need to continue adjusting the training sampling ratio. Therefore, it can be determined that the strategy model converges.

[0106] Furthermore, if the policy model has not converged, it means that the policy model is still quite different from the pre-trained model in some data domains, and you can try to adjust the training sampling ratio and continue training. In this case, after step S1022 of this embodiment, the following may also be included:

[0107] If the strategy model has not converged, the sampling weights corresponding to each data field in the training sampling ratio are adjusted with the goal of reducing the loss difference to obtain an adjusted sampling ratio;

[0108] The adjusted sampling ratio is used as a new training sampling ratio, and the step of training the strategy model according to the total data set and the training sampling ratio is returned to obtain the strategy loss value corresponding to each data field.

[0109] It should be noted that if the loss difference corresponding to a certain data field is large, then when the training sampling ratio is adjusted, the sampling weight of the data field in the training sampling ratio can be increased; similarly, if the loss difference corresponding to a certain data field is small, then when the training sampling ratio is adjusted, the sampling weight of the data field in the training sampling ratio can be reduced.

[0110] In actual use, since the process of adjusting the training sampling ratio is a dynamic adjustment and pulling process, the data areas with larger loss differences may be different in each training cycle. At this time, the adjustment is aimed at reducing the loss difference. When adjusting the sampling weights corresponding to each data area in the training sampling ratio, the loss differences corresponding to each data area can be sorted from large to small first, and then the sampling weights of each data area in the training sampling ratio can be adjusted in turn according to the sorting results to ensure the stability of the sampling weights of each data area in the final training sampling ratio.

[0111] Step S1023: If the strategy model converges, construct a domain sampling ratio according to the training sampling ratio.

[0112] It should be noted that if the policy model converges, it means that the policy model has been trained. At this time, the domain sampling ratio can be constructed based on the training sampling ratio.

[0113] In actual use, constructing the domain sampling ratio according to the training sampling ratio may be to use the training sampling ratio currently used by the policy model as the domain sampling ratio.

[0114] Further, in order to reasonably construct the domain sampling ratio, step S1023 in this embodiment may include:

[0115] If the strategy model converges, reading the training sampling ratio used to train the strategy model to obtain a sampling ratio set;

[0116] The training sampling ratios in the sampling ratio set are averaged to obtain the domain sampling ratio.

[0117] It should be noted that reading the training sampling ratios used for training the policy model to obtain the sampling ratio set may be reading all the training sampling ratios used when training the policy model, and aggregating the obtained training sampling ratios into a set to obtain the sampling ratio set.

[0118] It is understandable that in the process of training the strategy model, the domain weights of each data domain in the training sampling ratio are constantly adjusted and pulled. If only the current training sampling ratio of the strategy model is used as the domain sampling ratio, its versatility will be poor. At this time, all the used training sampling ratios can be obtained, and then the average of each training sampling ratio can be calculated, and the average value is used as the domain sampling ratio to ensure the reasonable construction of the domain sampling ratio.

[0119] This embodiment obtains a total data set by aggregating the domain data corresponding to each data domain; trains the policy model according to the total data set and the training sampling ratio to obtain the policy loss value corresponding to each data domain; determines whether the policy model converges according to the policy loss value and the pre-trained model corresponding to each data domain; if the policy model converges, constructs the domain sampling ratio according to the training sampling ratio. Since the policy loss value of each data domain is compared with the model loss value of the pre-trained model of each data domain, it is determined whether there is a pre-trained model with a large performance difference with the policy model to determine whether the policy model converges, and it is ensured that when the policy model converges, the performance difference between the policy model and the pre-trained model corresponding to each data domain is small, so as to ensure that the domain sampling ratio constructed at this time is the current optimal sampling ratio, so as to ensure that the language model finally trained according to the domain sampling ratio has a high performance on downstream tasks.

[0120] refer to Figure 4 , Figure 4 It is a flow chart of the third embodiment of a language model training method of the present invention.

[0121] Based on the above first embodiment, step S20 of the language model training method of this embodiment includes:

[0122] Step S201: Determine the number of data samples corresponding to each data field according to the preset total number of data and the field sampling ratio.

[0123] It should be noted that the total number of preset data may be the total number of data items contained in the training data set used to train the language model, and the total number of preset data is pre-set by the administrator of the language model training device.

[0124] In actual use, the sampling weights corresponding to each data field stored in the field sampling ratio are generally normalized values, and their value range is generally (0,1). At this time, the number of data sampling items corresponding to each data field can be determined based on the preset total number of data and the field sampling ratio. The preset total number of data and the sampling weights corresponding to each data field in the field sampling ratio can be multiplied to obtain the number of data sampling items corresponding to each data field.

[0125] Furthermore, in some cases, the sampling weights in the field sampling ratio have not been normalized. In order to ensure that the number of data samples corresponding to each data field can still be obtained normally for such a scenario, step S201 in this embodiment may include:

[0126] Read the total weight value from the field sampling ratio and the sampling weight corresponding to each data field;

[0127] Determine the data sampling ratio corresponding to each data field according to the sampling weight corresponding to each data field and the total value of the weight;

[0128] The data sampling ratio is multiplied by the total number of preset data to obtain the number of data samples corresponding to each data field.

[0129] It should be noted that the total weight value may be the sum of the sampling weights corresponding to each data field in the field sampling ratio. Determining the data sampling ratio corresponding to each data field according to the sampling weights corresponding to each data field and the total weight value may be to divide the sampling weights corresponding to each data field by the total weight value, thereby obtaining the data sampling ratio corresponding to each data field. For example, assuming that the total weight value is 10, there are three data fields, A, B, and C, and the corresponding sampling weights are 3, 3, and 4, respectively. At this time, the corresponding data sampling ratios are 0.3, 0.3, and 0.4, respectively.

[0130] It can be understood that after obtaining the data sampling ratio corresponding to the data field, it is multiplied by the preset total number of data to obtain the number of data items sampled in the data field, that is, the number of data sampling items corresponding to the data field.

[0131] Step S202: sampling data from each data field according to the number of data sampling items to obtain a training data set.

[0132] It should be noted that data sampling is performed from each data field according to the number of data sampling items, and a training data set is obtained by collecting data from the field data corresponding to the data field, the number of which is consistent with the number of data sampling items corresponding to the data field, and obtaining the training data corresponding to the data field. The training data corresponding to each data field obtained are aggregated into a set to obtain the training data set.

[0133] In this embodiment, the number of data samplings corresponding to each data field is determined according to the preset total number of data and the field sampling ratio; data sampling is performed from each data field according to the number of data samplings to obtain a training data set. Since the number of data samplings corresponding to each data field is first determined according to the preset total number of data and the field sampling ratio, and data sampling is performed from each data field according to the number of data samplings, the butterfly training data set can ensure that the training data set finally obtained fits the field sampling ratio, so as to ensure that the performance of the trained language model on downstream tasks is high.

[0134] refer to Figure 5 , Figure 5 It is a flowchart of a fourth embodiment of a language model training method of the present invention.

[0135] Based on the above second embodiment, step S101 of the language model training method of this embodiment includes:

[0136] Step S1011: Acquire the field data corresponding to each data field.

[0137] It should be noted that obtaining the domain data corresponding to each data domain may be reading the domain data from a data source corresponding to each data domain, wherein the data source may be a data source such as a database, a server, or a forum.

[0138] Furthermore, in order to ensure that domain data can be correctly obtained for different data domains, step S1011 in this embodiment may include:

[0139] Obtain data extraction rules corresponding to each data field;

[0140] Constructing a data extraction request according to the data extraction rule;

[0141] Each data field is accessed through the data extraction request to obtain field data corresponding to each data field.

[0142] It should be noted that the data extraction rules may include information such as data request method, data extraction method, and data conversion method, wherein the data request method may be an access method for accessing a data source of a data domain, the data extraction method may be a method for extracting data from a data source of a data domain, and the data conversion method may be a conversion method for converting the extracted data into domain data.

[0143] It is understandable that by constructing a data extraction request according to the data extraction rules and then accessing the data field through the data extraction request, the data source of the data field can feedback the field data in a format that conforms to the data extraction rules included in the data extraction request.

[0144] In a specific implementation, in order to ensure the training effect of the model, the step of accessing each data field through the data extraction request and obtaining the field data corresponding to each data field in this embodiment may include:

[0145] Access each data field through the data extraction request to obtain the original data corresponding to each data field;

[0146] The original data is preprocessed to obtain field data corresponding to each data field.

[0147] It should be noted that, although accessing the data field according to the data extraction request containing the data extraction rules can make the data field feedback data that conforms to the format, the data feedback from the data field at this time may have some defects. If such defective data is used to train the model, it will affect the actual use effect of the model. Therefore, the data feedback from the data field can be used as the original data first, and the original data can be pre-processed to repair or exclude the defective data, so as to obtain the field data corresponding to each data field.

[0148] In actual use, preprocessing may include at least one of deduplication processing, erroneous data processing, missing data processing, and abnormal data processing.

[0149] Among them, deduplication processing can be to remove duplicate data in the original data. The basis for determining duplication can be data identifiers or other combined identifiers, which can be pre-set by the administrator of the language model training equipment; error data processing can include correction of error type data, truncation of over-long data, etc.; missing data processing can include deletion of missing values, replacement of missing values, or forward and backward deduction to complete missing values, etc.; abnormal data processing can include deletion of abnormal values, replacement of abnormal values, etc.

[0150] Step S1012: performing data format verification on the field data according to a preset data format template.

[0151] It should be noted that the preset data format template can be pre-set by the administrator of the language model training equipment according to the data input format of the language model. The preset data format template can include constraint information such as the number of data fields, the type of each data field, and the data length of each data field.

[0152] It can be understood that performing data format verification on domain data according to a preset data format template can be to detect whether there is data in the domain data that does not comply with the constraint information contained in the preset data format template. If not, the verification is determined to be passed; if included, the verification is determined to be failed.

[0153] Step S1013: When the verification is passed, the field data is aggregated to obtain a total data set.

[0154] It can be understood that if the verification is passed, it means that the domain data of each data field conforms to the data input format of the model. Therefore, the domain data can be aggregated into a data set and the subsequent steps can be continued.

[0155] This embodiment obtains the domain data corresponding to each data domain; performs data format verification on the domain data according to a preset data format template; when the verification passes, the domain data is aggregated to obtain a data set. Since the data format verification is first performed on the domain data according to the preset data format template, only when the verification passes, the data set is constructed according to the domain data corresponding to each data domain, ensuring that the data in the constructed data set is consistent with the data input format requirements of the model, avoiding model training anomalies caused by data format anomalies, and improving the reliability of the language model training method of the present invention.

[0156] In addition, an embodiment of the present invention further proposes a storage medium, on which a language model training program is stored. When the language model training program is executed by a processor, the steps of the language model training method described above are implemented.

[0157] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the language model training device of the present invention.

[0158] like Figure 6 As shown, the language model training device proposed in the embodiment of the present invention includes:

[0159] A determination module 10 is used to train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio;

[0160] A sampling module 20, configured to perform data sampling based on the field sampling ratio to obtain a training data set;

[0161] The training module 30 is used to train the language model according to the training data set.

[0162] In this embodiment, the strategy model is trained according to the pre-trained model corresponding to the data domain to determine the domain sampling ratio; data sampling is performed based on the domain sampling ratio to obtain a training data set; and the language model is trained according to the training data set. Since the strategy model is trained by the pre-trained model corresponding to each data domain, the domain sampling ratio is determined, and data is sampled from each data domain according to the domain sampling ratio, and the language model is trained based on the sampled data, it can be ensured that the language model actually used is more in line with the downstream task requirements, thereby improving the model performance of the language model on downstream tasks.

[0163] Furthermore, the determination module 10 is also used to aggregate the domain data corresponding to each data domain to obtain a total data set; train the strategy model according to the total data set and the pre-trained model corresponding to each data domain to determine the domain sampling ratio.

[0164] Furthermore, the determination module 10 is also used to train the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field;

[0165] Determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field;

[0166] If the strategy model converges, a domain sampling ratio is constructed according to the training sampling ratio.

[0167] Furthermore, the determination module 10 is also used to read the training sampling ratio used for training the strategy model to obtain a sampling ratio set if the strategy model converges; and to average the training sampling ratios in the sampling ratio set to obtain a domain sampling ratio.

[0168] Furthermore, the determination module 10 is also used to obtain the model loss value in the pre-trained model corresponding to each data field; calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value; determine the maximum difference according to the loss difference corresponding to each data field; if the maximum difference is less than or equal to a preset threshold, it is determined that the strategy model converges.

[0169] Furthermore, the determination module 10 is also used to obtain the maximum value and the minimum value of the loss difference corresponding to each data field; calculate the difference between the maximum value and the minimum value to obtain the maximum difference of the difference.

[0170] Furthermore, the determination module 10 is also used to adjust the sampling weights corresponding to each data field in the training sampling ratio with the goal of reducing the loss difference if the strategy model has not converged, so as to obtain an adjusted sampling ratio; use the adjusted sampling ratio as a new training sampling ratio, and return to the step of training the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field.

[0171] Furthermore, the determination module 10 is also used to obtain the field data corresponding to each data field; perform data format verification on the field data according to a preset data format template; and when the verification passes, aggregate the field data to obtain a total data set.

[0172] Furthermore, the determination module 10 is also used to obtain data extraction rules corresponding to each data field; construct a data extraction request according to the data extraction rules; access each data field through the data extraction request to obtain field data corresponding to each data field.

[0173] Furthermore, the determination module 10 is also used to access each data field through the data extraction request to obtain the original data corresponding to each data field; pre-process the original data to obtain the field data corresponding to each data field, and the pre-processing includes at least one of deduplication processing, erroneous data processing, missing data processing, and abnormal data processing.

[0174] Furthermore, the sampling module 20 is also used to determine the number of data sampling items corresponding to each data field according to the preset total number of data and the field sampling ratio; and to sample data from each data field according to the number of data sampling items to obtain a training data set.

[0175] Furthermore, the sampling module 20 is also used to read the total weight value and the sampling weight corresponding to each data field from the field sampling ratio; determine the data sampling ratio corresponding to each data field according to the sampling weight corresponding to each data field and the total weight value; multiply the data sampling ratio by the preset total number of data to obtain the number of data samples corresponding to each data field.

[0176] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present invention. In specific applications, technicians in this field can make settings as needed, and the present invention does not limit this.

[0177] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present invention. In practical applications, technicians in this field can select part or all of them according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0178] In addition, for technical details not fully described in this embodiment, reference can be made to the language model training method provided in any embodiment of the present invention, and will not be repeated here.

[0179] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0180] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0181] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0182] The above are only preferred embodiments of the present invention, and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

[0183] The present application also discloses A1, a language model training method, the language model training method comprising the following steps:

[0184] Train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio;

[0185] Performing data sampling based on the field sampling ratio to obtain a training data set;

[0186] The language model is trained according to the training data set.

[0187] A2. The language model training method as described in A1, wherein the step of training the strategy model according to the pre-trained model corresponding to the data domain and determining the domain sampling ratio comprises:

[0188] Aggregate the domain data corresponding to each data field to obtain the total data set;

[0189] The strategy model is trained according to the data set and the pre-trained models corresponding to each data field to determine the field sampling ratio.

[0190] A3. The language model training method as described in A2, wherein the step of training the strategy model according to the total data set and the pre-trained model corresponding to each data domain to determine the domain sampling ratio comprises:

[0191] The strategy model is trained according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field;

[0192] Determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field;

[0193] If the strategy model converges, a domain sampling ratio is constructed according to the training sampling ratio.

[0194] A4. The language model training method as described in A3, wherein if the strategy model converges, the step of constructing a domain sampling ratio according to the training sampling ratio comprises:

[0195] If the strategy model converges, reading the training sampling ratio used to train the strategy model to obtain a sampling ratio set;

[0196] The training sampling ratios in the sampling ratio set are averaged to obtain the domain sampling ratio.

[0197] A5. The language model training method as described in A3, wherein the step of determining whether the strategy model has converged based on the strategy loss value and the pre-trained model corresponding to each data field comprises:

[0198] Get the model loss value in the pre-trained model corresponding to each data field;

[0199] Calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value;

[0200] Determine the maximum difference in difference based on the loss difference corresponding to each data field;

[0201] If the maximum difference of the difference values ​​is less than or equal to a preset threshold, it is determined that the strategy model has converged.

[0202] A6. In the language model training method described in A5, the step of determining the maximum difference in difference according to the loss difference corresponding to each data field comprises:

[0203] Obtain the maximum and minimum values ​​of the loss differences corresponding to each data field;

[0204] The difference between the maximum value and the minimum value is calculated to obtain the maximum difference of the difference.

[0205] A7. The language model training method as described in A5, after the step of determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field, further includes:

[0206] If the strategy model has not converged, the sampling weights corresponding to each data field in the training sampling ratio are adjusted with the goal of reducing the loss difference to obtain an adjusted sampling ratio;

[0207] The adjusted sampling ratio is used as a new training sampling ratio, and the step of training the strategy model according to the total data set and the training sampling ratio is returned to obtain the strategy loss value corresponding to each data field.

[0208] A8. In the language model training method described in A2, the step of aggregating the domain data corresponding to each data domain to obtain a total data set comprises:

[0209] Obtain the field data corresponding to each data field;

[0210] Performing data format verification on the field data according to a preset data format template;

[0211] When the verification is passed, the field data is aggregated to obtain a total data set.

[0212] A9. In the language model training method described in A8, the step of obtaining domain data corresponding to each data domain comprises:

[0213] Obtain data extraction rules corresponding to each data field;

[0214] Constructing a data extraction request according to the data extraction rule;

[0215] Each data field is accessed through the data extraction request to obtain field data corresponding to each data field.

[0216] A10. The language model training method as described in A9, wherein the step of accessing each data field through the data extraction request to obtain field data corresponding to each data field comprises:

[0217] Access each data field through the data extraction request to obtain the original data corresponding to each data field;

[0218] The original data is preprocessed to obtain domain data corresponding to each data field, and the preprocessing includes at least one of deduplication processing, error data processing, missing data processing, and abnormal data processing.

[0219] A11. In the language model training method according to any one of A1 to A10, the step of performing data sampling based on the domain sampling ratio to obtain a training data set comprises:

[0220] Determine the number of data samples corresponding to each data field according to the preset total number of data and the field sampling ratio;

[0221] Data sampling is performed from each data field according to the number of data sampling items to obtain a training data set.

[0222] A12. In the language model training method described in A11, the step of determining the number of data samples corresponding to each data field according to the preset total number of data and the field sampling ratio comprises:

[0223] Read the total weight value from the field sampling ratio and the sampling weight corresponding to each data field;

[0224] Determine the data sampling ratio corresponding to each data field according to the sampling weight corresponding to each data field and the total value of the weight;

[0225] The data sampling ratio is multiplied by the total number of preset data to obtain the number of data samples corresponding to each data field.

[0226] The present application also discloses B13, a language model training device, the language model training device comprising the following modules:

[0227] A determination module is used to train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio;

[0228] A sampling module, used for performing data sampling based on the field sampling ratio to obtain a training data set;

[0229] The training module is used to train the language model according to the training data set.

[0230] B14. In the language model training device as described in B13, the determination module is also used to aggregate the domain data corresponding to each data domain to obtain a total data set; train the strategy model according to the total data set and the pre-trained model corresponding to each data domain to determine the domain sampling ratio.

[0231] B15. In the language model training device as described in B14, the determination module is also used to train the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field; determine whether the strategy model converges according to the strategy loss value and the pre-trained model corresponding to each data field; if the strategy model converges, construct the field sampling ratio according to the training sampling ratio.

[0232] B16. In the language model training device as described in B15, the determination module is also used to read the training sampling ratio used for training the strategy model to obtain a sampling ratio set if the strategy model converges; and to find the average of each training sampling ratio in the sampling ratio set to obtain the domain sampling ratio.

[0233] B17. In the language model training device as described in B15, the determination module is also used to obtain the model loss value in the pre-trained model corresponding to each data field; calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value; determine the maximum difference according to the loss difference corresponding to each data field; if the maximum difference is less than or equal to a preset threshold, it is determined that the strategy model has converged.

[0234] B18. In the language model training device as described in B17, the determination module is also used to obtain the maximum and minimum values ​​of the loss differences corresponding to each data field; calculate the difference between the maximum value and the minimum value to obtain the maximum difference.

[0235] B19. In the language model training device as described in B17, the determination module is also used for adjusting the sampling weights corresponding to each data field in the training sampling ratio with the goal of reducing the loss difference if the strategy model has not converged, so as to obtain an adjusted sampling ratio; using the adjusted sampling ratio as a new training sampling ratio, and returning to the step of training the strategy model according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field.

[0236] B20. In the language model training device as described in B13, the sampling module is further used to determine the number of data sampling items corresponding to each data field according to the preset total number of data and the field sampling ratio; and to sample data from each data field according to the number of data sampling items to obtain a training data set.

[0237] The present application also discloses C21, a language model training device, which includes: a processor, a memory, and a language model training program stored in the memory and executable on the processor, wherein the language model training program implements the steps of the language model training method described above when executed by the processor.

[0238] The present application also discloses D22, a computer-readable storage medium, on which a language model training program is stored, and when the language model training program is executed, the steps of the language model training method described above are implemented.

Claims

1. A language model training method, characterized in that: The language model training method comprises the following steps: Train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio; Performing data sampling based on the field sampling ratio to obtain a training data set; The language model is trained according to the training data set.

2. The language model training method according to claim 1, characterized in that: The step of training the strategy model according to the pre-trained model corresponding to the data domain and determining the domain sampling ratio includes: Aggregate the domain data corresponding to each data field to obtain the total data set; The strategy model is trained according to the data set and the pre-trained models corresponding to each data field to determine the field sampling ratio.

3. The language model training method according to claim 2, characterized in that: The step of training the strategy model according to the data set and the pre-trained model corresponding to each data field to determine the field sampling ratio includes: The strategy model is trained according to the total data set and the training sampling ratio to obtain the strategy loss value corresponding to each data field; Determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field; If the strategy model converges, a domain sampling ratio is constructed according to the training sampling ratio.

4. The language model training method according to claim 3, characterized in that: If the strategy model converges, the step of constructing a domain sampling ratio according to the training sampling ratio includes: If the strategy model converges, reading the training sampling ratio used to train the strategy model to obtain a sampling ratio set; The training sampling ratios in the sampling ratio set are averaged to obtain the domain sampling ratio.

5. The language model training method according to claim 3, characterized in that: The step of determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field includes: Get the model loss value in the pre-trained model corresponding to each data field; Calculate the loss difference corresponding to each data field according to the model loss value and the strategy loss value; Determine the maximum difference in difference based on the loss difference corresponding to each data field; If the maximum difference of the difference values ​​is less than or equal to a preset threshold, it is determined that the strategy model has converged.

6. The language model training method according to claim 5, characterized in that: The step of determining the maximum difference of the difference according to the loss difference corresponding to each data field includes: Obtain the maximum and minimum values ​​of the loss differences corresponding to each data field; The difference between the maximum value and the minimum value is calculated to obtain the maximum difference of the difference.

7. The language model training method according to claim 5, characterized in that: After the step of determining whether the strategy model has converged according to the strategy loss value and the pre-trained model corresponding to each data field, the method further includes: If the strategy model has not converged, the sampling weights corresponding to each data field in the training sampling ratio are adjusted with the goal of reducing the loss difference to obtain an adjusted sampling ratio; The adjusted sampling ratio is used as a new training sampling ratio, and the step of training the strategy model according to the total data set and the training sampling ratio is returned to obtain the strategy loss value corresponding to each data field.

8. A language model training device, characterized in that: The language model training device includes the following modules: A determination module is used to train the strategy model according to the pre-trained model corresponding to the data domain and determine the domain sampling ratio; A sampling module, used for performing data sampling based on the field sampling ratio to obtain a training data set; The training module is used to train the language model according to the training data set.

9. A language model training device, characterized in that: The language model training device includes: a processor, a memory, and a language model training program stored in the memory and executable on the processor. When the language model training program is executed by the processor, the steps of the language model training method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a language model training program, which, when executed, implements the steps of the language model training method according to any one of claims 1 to 7.

Images