Model training method and device

Abstract

The invention discloses a model training method and device. The method comprises the steps of obtaining a training data set; wherein each training sample data group in the training data set comprises fitting data and non-fitting data; inputting the input code of the non-fitting data and the first-layer target code of the fitting data into a large language model, and determining a first output code of the large language model; determining a second output code of the large language model based on all previously input codes and a second-layer target code of the input fitting data; determining an Nth output code of the large language model until all previously input codes of the large language model and an Nth layer of target codes of the input fitting data are based on the Nth layer of target codes; and based on the at least one N-layer target code of the fitting data and the corresponding number of output codes, performing iterative optimization on the large language model to generate a target model. According to the scheme, the model training difficulty can be reduced, and the model output integrity is improved.

Description

[0001] This application is a divisional application. The application number of the original application is 202411470413.6, the application date is October 21, 2024, and the name of the invention is “A large language model encoding training method and device”. Technical Field

[0002] The present invention relates to the field of machine learning technology, and in particular to a model training method and device. Background Art

[0003] Traditional model training generally encodes the training data in sequence and inputs it into the model one by one in sequence, and then the model outputs it one by one in sequence. However, this method of model training is difficult. The model needs to output all the information of one unit at a time, and the output information density is high; all output units are combined in sequence to generate the final output information, and the integrity of the output content is not high. Summary of the Invention

[0004] The present invention provides a model training method and device, which can provide richer data information. When performing model training, the training sample data can be output layer by layer, and fitting training can be performed layer by layer, starting from coarse to fine, and starting from the whole to the part, thereby reducing the difficulty of model training and improving the integrity of the model output.

[0005] According to one aspect of the present invention, a model training method is provided, comprising:

[0006] Acquire a training data set; wherein each training sample data group in the training data set includes input data and fitting data, or each training sample data group includes reference data, input data and fitting data;

[0007] Inputting the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determining a first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data;

[0008] Determining a second output code of the large language model based on all previously input codes and a second-level target code of the input fitting data; until determining an Nth output code of the large language model based on all previously input codes of the large language model and an Nth-level target code of the input fitting data;

[0009] Based on at least one N-layer target code of the fitting data and a corresponding number of output codes, the large language model is iteratively optimized to generate a target model.

[0010] According to another aspect of the present invention, there is provided a model training device, comprising:

[0011] A training data set acquisition module, configured to acquire a training data set; wherein each training sample data group in the training data set includes input data and fitting data, or each training sample data group includes reference data, input data, and fitting data;

[0012] a first output code determining module, configured to input the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determine a first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data;

[0013] a second output code determining module, configured to determine a second output code of the large language model based on all previously input codes and a second-level target code of the input fitting data; and finally, determining an Nth output code of the large language model based on all previously input codes of the large language model and an Nth-level target code of the input fitting data;

[0014] A target model generation module is used to iteratively optimize the large language model based on at least one N-layer target code and a corresponding number of output codes of the fitting data to generate a target model.

[0015] According to another aspect of the present invention, an electronic device is provided, comprising:

[0016] at least one processor; and

[0017] a memory communicatively connected to the at least one processor; wherein,

[0018] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the model training method described in any embodiment of the present invention.

[0019] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the model training method described in any embodiment of the present invention when executed.

[0020] The model training scheme of the embodiment of the present invention obtains a training data set; wherein each training sample data group in the training data set includes input data and fitting data, or each training sample data group includes reference data, input data and fitting data; inputs the input code of the non-fitting data and the first-layer target code of the fitting data into a large language model to determine the first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data; based on all previously input codes and the second-layer target code of the input fitting data, the second output code of the large language model is determined; until the Nth output code of the large language model is determined based on all previously input codes of the large language model and the Nth-layer target code of the input fitting data; based on at least one N-layer target code of the fitting data and the corresponding number of output codes, the large language model is iteratively optimized to generate a target model. The technical solution provided by the embodiment of the present invention can output the training sample data layer by layer during model training, and perform fitting training layer by layer, starting from coarse to fine, starting from the whole to the part, thereby reducing the difficulty of model training and improving the integrity of the model output.

[0021] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present invention, nor is it intended to limit the scope of the present invention. Other features of the present invention will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 A flowchart of a large language model encoding training method provided in Example 1 of the present invention;

[0024] Figure 2 A flowchart of a large language model encoding training method provided in Example 2 of the present invention;

[0025] Figure 3 A schematic diagram of the structure of a large language model encoding training device provided in Embodiment 3 of the present invention;

[0026] Figure 4 A schematic diagram of the structure of an electronic device for implementing the large language model encoding training method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] It should be noted that the terms "first", "second", etc. in the description and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Example 1

[0030] Figure 1 A flowchart of a large language model encoding training method is provided for embodiment 1 of the present invention. This embodiment is applicable to the case of encoding training of a large language model. The method can be performed by a large language model encoding training device. The large language model encoding training device can be implemented in the form of hardware and / or software. The large language model encoding training device can be configured in an electronic device.

[0031] like Figure 1 As shown, the method includes:

[0032] S110 , obtaining a training data set; wherein the training data set includes at least two training sample data groups, and each training sample data group consists of at least two corresponding training data.

[0033] In an embodiment of the present invention, a training data set is obtained, wherein the training data set includes multiple training sample data groups, and each training sample data group is composed of at least two corresponding data. For example, the training sample data group may include input data and fitting data; for another example, the training sample data group may include reference data, input data, and fitting data. The input data is data that needs to output corresponding generated content based on the input content when the large language model is subsequently trained, such as questions in question-and-answer data, and lyrics in music generation data; the reference data is data that needs to refer to part of the information in the subsequent large language model training to output corresponding generated content, such as reference pictures in literary image data, reference songs in music generation data, or reference accompaniment and reference singing style in the reference song, etc., which are used to enable the large language model to generate output data with the same style; the fitting data is standard answer data, which is used to compare one-to-one with the output results of the large language model in the subsequent large language model training, calculate the loss value, and thus correct the probability of the next output result of the large model. It is the ideal data that the large language model needs to fit. The training dataset is a dataset that matches the application scenario of the large language model used for subsequent training. For example, the large language model trained subsequently is a music generation model. Each training sample data group in the training dataset can include fitted music and target lyrics, or reference music, fitted music and target lyrics.

[0034] It should be noted that the embodiments of the present invention do not limit the amount of input data or reference data included in the training sample dataset. For example, when training a large language model for music generation, the input data in the training sample dataset can be a single lyric or multiple lyric segments, and the reference data in the training sample dataset can be a reference song or a reference accompaniment or singing style within the reference song.

[0035] S120. For at least one training data in each of the training sample data groups, determine a first-layer target encoding of the training data.

[0036] In an embodiment of the present invention, for each training sample data group in a training dataset, a first-level target encoding is determined for at least one type of training data in the training sample data group. It is understood that the first-level target encoding can be determined for any one or more or all of the training data in the training sample data group, wherein the types of training data for the first-level target encoding determined in each training sample data group in the training dataset can be the same or different. Exemplarily, an encoder can be used to encode at least one type of training data in the training sample data group to determine the first-level target encoding, wherein the encoder can include a MERT encoder and a Mel encoder.

[0037] Optionally, before determining the first-level target coding of at least one training data in each of the training sample data groups, the method further includes: discretizing the at least one training data to generate a plurality of minimum discrete units; determining the first-level target coding of the training data for at least one training data in each of the training sample data groups, including: encoding each minimum discrete unit of the at least one training data in each of the training sample data groups to determine the first-level target coding of the training data. Exemplarily, discretizing the at least one training data in the training sample data group to generate at least two minimum discrete units, and then encoding each of the at least two minimum discrete units by an encoder to determine the corresponding first target coding. It is understandable that the number of the first target coding is the same as the number of the minimum discrete units.

[0038] For example, a large language model to be subsequently trained is used as a music generation model, and the training sample data set consists of reference music, fitted music, and target lyrics. The fitted music is discretized to generate at least two frames of fitted audio data. For example, the fitted music can be discretized into 10 frames of fitted audio data per second. For example, if the fitted music is a 10-second segment, the fitted music can be discretized into 100 frames of fitted audio data. As another example, the fitted music as a whole can be discretized, for example, into 500 frames of fitted audio data. The number of discrete fitted audio data frames can be determined based on the length of the reference music; the longer the fitted music, the greater the number of discrete fitted audio data frames. Each frame of the fitted audio data is encoded using an encoder to generate a corresponding first-layer target encoding, where the number of first-layer target encodings is the same as the number of frames of the fitted audio data. Optionally, the same encoder can be used to encode each frame of the fitted audio data, or different encoders can be used to encode each frame of the fitted audio data. The encoder can be an open-source encoder, such as the MERT encoder or the Mel encoder. In an embodiment of the present invention, the discretization processing of the target lyrics may include: performing word segmentation processing on the target lyrics based on a preset word segmentation algorithm, dividing the target lyrics into multiple word segmentation units, wherein the word segmentation unit can be each character, each word, or each sentence. Encoding each word segmentation unit in the target lyrics based on the encoder generates a first-level target code corresponding to each word segmentation unit. The method for determining the first-level target code of the reference music can be similar to the method for determining the first-level target code of the fitted music, which will not be repeated here.

[0039] Optionally, determining the first-level target encoding of the training data includes: determining the first-level initial encoding of the training data; comparing the first-level initial encoding with each first code or a first standard code corresponding to each first code in a pre-set encoding codebook, determining a first target encoding corresponding to the first code in the encoding codebook that has the highest similarity to the first-level initial encoding; and determining the first-level target encoding based on the first target encoding. Exemplarily, an encoder is used to encode at least one training data item in the training sample data set, and the resulting encoding information is used as the first-level initial encoding of the training data. A pre-set encoding codebook is obtained, wherein the encoding codebook comprises multiple first codes. Optionally, the first code in the encoding codebook can be a standard code (also referred to as a special code), or a standard code can be replaced with a special character, such as using the first standard code as code 0, the second standard code as code 1, and so on, to generate an encoding codebook such as [0, 1, 2, ..., 1023]. This can greatly simplify the complexity of the codebook, and at the same time can isolate the codes (that is, special characters such as 0, 1, 2) and the standard codes, making it easier to adjust the format and / or content of the standard codes in the future. For example, the first standard code can be a multi-dimensional vector, or any one or more of an audio code, a graphic code, a character code, etc.; or the first standard code is originally a 256-dimensional code vector, and if it is subsequently adjusted to a 1024-dimensional code vector, the codebook is still [0, 1, 2, ..., 1023].

[0040] The first-layer initial code is sequentially compared with each first code in the encoding codebook or the first standard code corresponding to the first code, and the similarity between the two is calculated. The first code in the encoding codebook that has the highest similarity to the first-layer initial code is used as the first target code, and the first-layer target code of the training data is determined based on the first target code. The first target code in the encoding codebook can be directly used as the first-layer target code, or the first standard code corresponding to the first target code in the encoding codebook can be used as the first-layer target code. Alternatively, the first target code in the encoding codebook can be encoded based on a preset encoding strategy to generate the first-layer target code corresponding to the standard data.

[0041] S130: Use the first-layer target coding as the current-layer target coding, determine the current-layer coding deviation of the current-layer target coding, and determine the next-layer target coding based on the current-layer coding deviation.

[0042] In an embodiment of the present invention, the first-layer target coding of the training data is updated to the current-layer target coding, and the current-layer coding deviation of the current-layer target coding is determined, wherein the current-layer coding deviation can be the difference between the current-layer initial coding and the current-layer target coding. The next-layer target coding of the training data is determined based on the current-layer coding deviation. For example, the current-layer coding deviation can be encoded based on a preset coding algorithm to generate the next-layer target coding of the training data. It can be understood that the first-layer coding deviation of the first-layer target coding of the training data is determined, wherein the residual between the first-layer initial coding of the training data and the first-layer target coding can be used as the first-layer coding deviation. The second-layer target coding of the training data is determined based on the first-layer coding deviation.

[0043] Optionally, the current layer coding deviation of the current layer target coding is determined, and the next layer target coding is determined based on the current layer coding deviation, including: calculating the residual between the current layer initial coding and the current layer target coding, and using the residual as the current layer coding deviation of the current layer coding; comparing the current layer coding deviation with each second coding in a pre-set next layer deviation code book or the second standard coding corresponding to the second coding, determining the second target coding corresponding to the second coding in the next layer deviation code book with the highest similarity to the current layer coding deviation, and determining the next layer target coding based on the second target coding.

[0044] Exemplarily, the residual between the current layer initial code and the current layer target code is used as the current layer code deviation of the current layer code. A next layer deviation codebook is obtained, wherein the next layer deviation codebook is composed of multiple second codes, and the second codes can be understood as residual codes. Optionally, the second codes in the deviation codebook can be standard residual codes (also called special residual codes), or the standard residual codes can be replaced by special characters, such as using the first standard residual code as code 0 and the second standard residual code as code 1, and so on, to generate a deviation codebook such as [0, 1, 2, ..., 1023]. This can greatly simplify the complexity of the deviation codebook, and at the same time can isolate the codes (i.e., special characters such as 0, 1, 2) and the standard residual codes, making it easier to adjust the format and / or content of the standard residual codes in the future. For example, the first standard residual code can be a multi-dimensional vector, or any one or more of an audio code, a graphic code, a character code, etc.; or the first standard residual code is originally a 256-dimensional code vector, which is subsequently adjusted to be converted into a 1024-dimensional code vector. At this time, the deviation codebook is still [0, 1, 2, ..., 1023].

[0045] The current layer code deviation is sequentially compared with each second code in the next layer deviation codebook or the second standard code corresponding to the second code, and the similarity between the two is calculated. The second code in the next layer deviation codebook with the highest similarity to the current layer code deviation is used as the second target code, and the next layer target code of the training data is determined based on the second target code. The second target code in the next layer deviation codebook can be directly used as the next layer target code, or the second standard code corresponding to the second target code in the next layer deviation codebook can be used as the next layer target code. Alternatively, the second target code in the next layer deviation codebook can be encoded based on a preset encoding strategy to generate the next layer target code corresponding to the standard data.

[0046] S140. Update the next layer target code to the current layer target code, and return to execute the current layer code deviation for determining the current layer target code to determine the Nth layer target code; wherein N is an integer greater than or equal to 2.

[0047] In an embodiment of the present invention, the first-layer coding deviation of the first-layer target coding of the training data is compared with the second standard coding corresponding to each second coding in the second-layer deviation codebook, and the second target coding corresponding to the second coding with the highest similarity to the first-layer coding deviation in the second-layer deviation codebook is determined. The second-layer target coding is determined based on the second target coding. The first-layer coding deviation is used as the second-layer initial coding, and the residual between the second initial coding and the second-layer target coding is calculated, and the residual is used as the second-layer coding deviation. In other words, the residual between the first-layer coding deviation and the second-layer target coding is used as the second-layer coding deviation. For convenience of description, the second coding in the i-th layer deviation codebook may also be referred to as the i-th coding, and the standard coding corresponding to the i-th coding may be referred to as the i-th standard coding. Therefore, the second-layer coding deviation is compared with the third standard coding corresponding to each third coding in the third-layer deviation codebook, and the third target coding corresponding to the third coding with the highest similarity to the second-layer coding deviation in the third-layer deviation codebook is determined. The third-layer target coding is determined based on the third target coding. The second-layer encoding deviation is used as the third-layer initial encoding. The residual between the third initial encoding and the third-layer target encoding is calculated and used as the third-layer encoding deviation. Similarly, the residual between the second-layer encoding deviation and the third-layer target encoding is used as the third-layer encoding deviation. This method is repeated repeatedly until the Nth-layer target encoding of the training data is determined, where N is an integer greater than or equal to 2.

[0048] It is understandable that according to S120-S130, the N-layer target encoding corresponding to at least one training data in the training sample data group can be determined. It should be noted that the number of target encoding layers of at least one training data in each training sample data group in the training data set can be the same or different.

[0049] S150 , training the large language model based on the N layers of target encoding to generate a target model.

[0050] In an embodiment of the present invention, N-layer target codes corresponding to at least one training data in each training sample data group in the training data set are input into a large language model to train the large language model and generate a target model. The large language model can be any open source large model, such as GPT. It should be noted that the embodiment of the present invention does not limit the application scenario of the target model. For example, the target model can be a music generation model. In this case, each training sample data group in the training data set used to train the target model can include reference music, sample lyrics and fitted music. For another example, the target model can be a text-to-image model, that is, a model that generates pictures based on text. In this case, each training sample data group in the training data set used to train the target model can include sample text and fitted pictures.

[0051] The large language model encoding training method of the embodiment of the present invention obtains a training data set; wherein, the training data set contains at least two training sample data groups, and each of the training sample data groups is composed of at least two corresponding training data; for at least one training data in each of the training sample data groups, the first layer target encoding of the training data is determined; the first layer target encoding is used as the current layer target encoding, the current layer encoding deviation of the current layer target encoding is determined, and the next layer target encoding is determined based on the current layer encoding deviation; the next layer target encoding is updated to the current layer target encoding, and the current layer encoding deviation of the current layer target encoding is determined to determine the Nth layer target encoding; wherein N is an integer greater than or equal to 2; the large language model is trained based on the N layers of target encoding to generate a target model. The technical solution provided by the embodiment of the present invention can provide richer data information by encoding at least one training data in the training sample data group into multi-layer target encoding, thereby improving the degree of data discretization and fitting refinement, thereby improving the quality of the output data of the trained large language model.

[0052] In some embodiments, for at least one training data in each training sample data group, before determining the first layer of target coding for the training data, the method further includes: extracting partial training data of at least one dimension for the at least one training data; separately encoding the partial training data of at least one dimension to generate at least one layer of enhanced target coding; and training a large language model based on N layers of the target coding to generate a target model, including: superimposing at least one layer of the enhanced target coding with the N layers of the target coding to train the large language model to generate a target model. This arrangement has the advantage of enhancing the training effect of the large language model in certain dimensions, thereby achieving better training results for the large language model.

[0053] In an embodiment of the present invention, for at least one type of training data in each training sample data group, partial training data of at least one dimension is extracted from the training data, and the partial training data of each dimension is encoded separately to generate at least one layer of enhanced target code. The partial training data of each dimension can be encoded as a whole to generate at least one layer of enhanced target code, or the partial training data of each dimension can be discretized, and the generated minimum discrete units are encoded to generate at least one layer of enhanced target code. It should be noted that when the enhanced target code is multi-layered, the method for determining the multi-layer enhanced target code is the same as the method for determining the N-layer target code of the training data in the above embodiment, and will not be repeated here.

[0054] Exemplarily, if the training sample data set includes reference music, fitting music, and target lyrics, the fitting music can be split into a preset number of dimensions. For example, at least one dimension of fitting music parts (such as fitting music parts with two dimensions, namely, accompaniment and vocal parts) can be split from the fitting music, and each dimension of fitting music parts can be discretized to generate at least two frames of fitting audio data. Each frame of fitting audio data can be encoded by an encoder to generate at least one corresponding layer of enhanced target code. Similarly, the reference sample music can be split into a preset number of dimensions. For example, at least one dimension of reference music parts can be split from the reference music (such as reference music parts with two dimensions, namely, accompaniment and vocal parts), and the entire reference music parts of the split at least one dimension can be uniformly encoded to generate at least one corresponding layer of enhanced target code. Alternatively, the reference music parts of the split at least one dimension can be discretized to generate at least two frames of reference audio data. Each frame of reference audio data can be encoded by an encoder to generate at least one corresponding layer of enhanced target code. The target lyrics can be encoded to generate at least one corresponding layer of enhanced target code. As another example, the bass, mid-range, and treble in a frame of audio data are encoded at different layers to generate corresponding enhanced target codes, or different instruments and different voices in a frame of audio data are separated and encoded at different layers to generate corresponding enhanced target codes.

[0055] In an embodiment of the present invention, at least one layer of enhanced target code is superimposed with N layers of target code, and the superimposed code is input into a large language model to train the large language model and generate a target model. Optionally, superimposing at least one layer of the enhanced target code with N layers of the target code to train the large language model includes placing at least one layer of the enhanced target code before the N layers of the target code, and inputting at least a portion of the enhanced target code into the large language model prior to the N layers of the target code to train the large language model.

[0056] For example, taking the fitted music in the training sample data set as an example, the enhanced target code of the vocal part extracted from the fitted music is used as the first layer, and the four layers of target code corresponding to the fitted music are used as the second, third, fourth, and fifth layers respectively. The enhanced target code of the vocal part of the fitted music is preferentially input as the first layer code into the large language model for prediction output, and then the four layers of target code corresponding to the fitted music are sequentially input into the large language model to continue training the large language model based on the four layers of target code to generate a target model. The advantage of this setting is that the large language model can first be fitted and output by the vocal part corresponding to the simple fitted music, and then fit training for complex songs can be performed, obtaining a clearer fitting training of the vocal part than simply fitting the song, thereby achieving a better large language model training effect.

[0057] Example 2

[0058] Figure 2 This is a flowchart of a large language model encoding training method provided in Example 2 of the present invention, such as Figure 2 As shown, the method includes:

[0059] S210. Obtain a training data set; wherein the training data set includes at least two training sample data groups, each of the training sample data groups is composed of corresponding at least two types of training data, each of the training sample data includes input data and fitting data, or each of the training sample data groups includes reference data, input data and fitting data.

[0060] S220. For at least one training data in each of the training sample data groups, determine a first-layer target encoding of the training data.

[0061] S230: Use the first-layer target coding as the current-layer target coding, determine the current-layer coding deviation of the current-layer target coding, and determine the next-layer target coding based on the current-layer coding deviation.

[0062] S240. Update the next layer target code to the current layer target code, and return to execute the current layer code deviation for determining the current layer target code to determine the Nth layer target code; wherein N is an integer greater than or equal to 2.

[0063] S250: Inputting the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determining the first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data.

[0064] Each training sample data may include input data and fitting data, or may include reference data, input data, and fitting data. When the training sample data includes input data and fitting data, the input data is treated as non-fitting data; when the training sample data includes reference data, input data, and fitting data, the reference data and input data are treated as non-fitting data.

[0065] In this embodiment of the present invention, the N-layer target code of the unfitted data is used as the input code of the unfitted data. The input code of the unfitted data and the first-layer target code of the fitted data are input into the large language model to train the large language model and obtain the first output code of the large language model. It is understood that the first output code is the code output by the large language model after learning the input code of the unfitted data and the first-layer target code of the fitted data.

[0066] S260. Determine the second output code of the large language model based on all previously input codes and the second-layer target code of the input fitting data; until the Nth output code of the large language model is determined based on all previously input codes of the large language model and the Nth-layer target code of the input fitting data.

[0067] In this embodiment of the present invention, the second output code of the large language model is determined based on all previously input codes of the large language model and the second-level target code of the fitted data input to the large language model. Similarly, the third output code of the large language model is determined based on all previously input codes of the large language model and the third-level target code of the fitted data input to the large language model. The third output code of the large language model is determined based on all previously input codes of the large language model and the fourth-level target code of the fitted data input to the large language model. This process is deduced in this way until the Nth output code of the large language model is determined based on all previously input codes of the large language model and the Nth-level target code of the fitted data input to the large language model.

[0068] S270. Based on at least one N-layer target code of the fitting data and a corresponding number of output codes, iteratively optimize the large language model to generate a target model.

[0069] In this embodiment of the present invention, the N-layer target code for the fitted data may be one or more. When the fitted data corresponds to multiple N-layer target codes, output codes corresponding to each N-layer target code are obtained through S260-S270. Based on the at least one N-layer target code for the fitted data and the corresponding number of output codes, the large language model is continuously iteratively optimized to generate a target model.

[0070] Optionally, the fitting data corresponds to M N-layer target codes, where M is an integer greater than 1. The method further includes: inputting a large language model based on the input code of the non-fitting data and the first first-layer target code of the fitting data to determine a first output code of the large language model; determining a second output code of the large language model based on all previously input codes and the first second-layer target code and the second first-layer target code of the input fitting data, wherein the first second-layer target code and the second first-layer target code of the fitting data are input by superposition, summation or splicing; until the M+N-1th output code of the large language model is determined based on all previously input codes of the large language model and the Mth N-layer target code of the input fitting data; combining the M+N-1 output codes of the large language model into M N-layer final output codes; and iteratively optimizing the large language model based on at least M N-layer target codes and M N-layer final output codes of the fitting data to generate a target model.

[0071] In an embodiment of the present invention, since the output data of the large language model depends on the input data of the large language model corresponding to the previously fitted data, a delayed stacking method is used to input the M N-layer target codes corresponding to the fitted data into the large language model. Specifically, the fitted data corresponds to the M N-layer target codes, and all N-layer target codes of the non-fitted data are used as the input codes of the non-fitted data. The input codes of the non-fitted data and the first first-layer target code of the fitted data are input into the large language model, so that the large language model performs prediction and obtains the first output code of the large language model. It can be understood that the first output code is the code output by the large language model after learning the input code of the non-fitted data and the first first-layer target code of the fitted data. The second output code of the large language model is determined based on all previously input codes of the large language model, the first second-layer target code of the fitted data input into the large language model, and the second first-layer target code. Specifically, the first second-layer target code and the second first-layer target code of the fitted data are stacked, summed, or concatenated, and the second output code of the large language model is determined based on the stacked, summed, or concatenated code and all previously input codes. And so on, until the M+N-1th output code of the large language model is determined based on all the previously input codes of the large language model and the Mth-Nth layer target code of the input fitting data.

[0072] For example, M=4, N=4, and Table 1 is a table of delayed superposition inputs of four 4-layer target codes corresponding to fitting data provided by an embodiment of the present invention:

[0073] Table 1

[0074] t(14) t(24) t(34) t(44) t(13) t(23) t(33) t(43) t(12) t(22) t(32) t(42) t(11) t(21) t(31) t(41)

[0075] After the input code of the non-fitting data and the first first-layer target code t(11) of the fitting data are input into the large language model, the first output code of the large language model is determined. Then, the first second-layer target code t(12) and the second first-layer target code t(21) are superimposed and summed and input into the large language model. The large language model determines the second output code based on all the previously input codes and the codes after the superposition and summation of t(12) and t(21). The first third-layer target code t(13), the second second-layer target code t(22), and the third first-layer target code t(31) of the fitting data are superimposed and summed, and then input into the large language model. The large language model determines the third output code based on all the codes previously input and the codes after the superimposition and summation of t(13), t(22), and t(31); the first fourth-layer target code t(14), the second third-layer target code t(23), the third second-layer target code t(32), and the fourth first-layer target code t(41) of the fitting data are superimposed and summed, and then input into the large language model. The large language model determines the fourth output code based on all the codes previously input and the codes after the superimposition and summation of t(14), t(23), t(32), and t(41). And so on, according to the method described in Table 1, the 7th output code of the large language model is obtained. It can be understood that superimposing the M N-layer target codes of the fitting data into the large language model in a delayed manner can enable the large language model to first obtain the low-level target code of each unit and then obtain the high-level target code, which is beneficial to the training of the large language model.

[0076] Since the output data of the large language model depends on all the inputs of the non-fitting data (input data, or reference data and input data), the N-layer target codes corresponding to the non-fitting data can be input into the large language model in a conventional superposition or splicing manner, that is, all the N-layer target codes of the non-fitting data are input into the large language model at one time, so that the large model can obtain all the N-layer target codes of the non-fitting data at one time, thereby making the training of the large language model faster and more efficient. Optionally, the DELAY method can also be used to superimpose the N-layer target codes of the non-fitting data into the large language model to train the large language model. For example, Table 2 is a conventional superposition input table of 4 4-layer target codes corresponding to non-fitting data provided in an embodiment of the present invention:

[0077] Table 2

[0078] t(14) t(24) t(34) t(44) t(13) t(23) t(33) t(43) t(12) t(22) t(32) t(42) t(11) t(21) t(31) t(41)

[0079] Among them, when the N-layer target codes of the non-fitting data are superimposed and input into the large language model in a conventional manner, the multi-dimensional code vector obtained by adding or splicing the N-layer target codes of each unit can be obtained. For example, the 256-dimensional t(11), t(12), t(13), and t(14) are added to obtain a 256-dimensional vector, or spliced to obtain a 1024-dimensional vector. The 256-dimensional t(21), t(22), t(23), and t(24) are added to obtain a 256-dimensional vector, which is still a 256-dimensional vector, or spliced to obtain a 1024-dimensional vector. The multi-layer code vector is input into the large language model, wherein the output of the large language model is also a multi-layer code vector, thereby improving the accuracy of the output of the large language model.

[0080] The M+N-1 output codes of the large language model are combined into M N-layer final output codes. Then, based on at least M N-layer target codes and M N-layer final output codes of the fitted data, the large language model is iteratively optimized to generate a target model.

[0081] The technical solution provided by the embodiment of the present invention can provide richer data information by encoding at least one training data in the training sample data group into a multi-layer target code, thereby improving the degree of data discretization and fitting refinement, thereby improving the quality of the output data of the trained large language model.

[0082] Example 3

[0083] Figure 3 This is a schematic diagram of the structure of a large language model encoding training device provided by the third embodiment of the present invention. Figure 3 As shown, the device includes:

[0084] The training data set acquisition module 310 is used to acquire a training data set; wherein the training data set includes at least two training sample data groups, and each training sample data group is composed of corresponding at least two types of training data;

[0085] A first-layer target coding determination module 320 is configured to determine, for at least one training data in each of the training sample data groups, a first-layer target coding of the training data;

[0086] a next layer target coding determination module 330, configured to use the first layer target coding as the current layer target coding, determine a current layer coding deviation of the current layer target coding, and determine a next layer target coding based on the current layer coding deviation;

[0087] The coding deviation loop determination module 340 is configured to update the next layer target coding to the current layer target coding, and return to determine the current layer coding deviation of the current layer target coding to determine the Nth layer target coding, where N is an integer greater than or equal to 2;

[0088] The target model generation module 350 is used to train the large language model based on the N layers of target encoding to generate a target model.

[0089] Optionally, the device further includes:

[0090] a partial training data extraction module, configured to extract partial training data of at least one dimension for at least one training data in each of the training sample data groups before determining the first-layer target encoding of the training data;

[0091] An enhanced target code generation module is used to encode part of the training data of at least one dimension respectively to generate at least one layer of enhanced target code;

[0092] The target model generation module includes:

[0093] The target model generating unit is used to superimpose at least one layer of the enhanced target code with N layers of the target code, train the large language model, and generate a target model.

[0094] Optionally, the target model generating unit is configured to:

[0095] At least one layer of the enhanced target code is placed before the N layers of the target code, and at least part of the enhanced target code is input into the large language model before the N layers of the target code to perform the large language model training.

[0096] Optionally, the device further includes:

[0097] a minimum discrete unit generating module, configured to discretize the at least one training data in each of the training sample data groups before determining the first-layer target encoding of the training data to generate a plurality of minimum discrete units;

[0098] The first-layer target coding determination module is used to:

[0099] Each minimum discrete unit of at least one training data in each of the training sample data groups is encoded respectively to determine a first-layer target encoding of the training data.

[0100] Optionally, the first-layer target coding determination module is configured to:

[0101] Determining a first layer initial encoding of the training data;

[0102] Comparing the first layer initial code with each first code in a preset code book or a first standard code corresponding to the first code, and determining a first target code corresponding to the first code in the code book that has the highest similarity to the first layer initial code;

[0103] A first layer target coding is determined based on the first target coding.

[0104] Optionally, the next layer target coding determination module is configured to:

[0105] Calculating a residual between an initial code of a current layer and a target code of the current layer, and using the residual as a current layer code deviation of the current layer code;

[0106] Compare the current layer coding deviation with each second code or the second standard code corresponding to the second code in the preset next layer deviation code book, determine the second target code corresponding to the second code with the highest similarity to the current layer coding deviation in the next layer deviation code book, and determine the next layer target code based on the second target code.

[0107] Optionally, determining the corresponding target code based on the code with the highest similarity in the codebook includes:

[0108] directly determining the code with the highest similarity in the codebook as the target code; or,

[0109] The codes with the highest similarity in the codebook are encoded based on a preset encoding strategy to generate the target code.

[0110] Optionally, each of the training sample data includes input data and fitting data, or each of the training sample data groups includes reference data, input data and fitting data;

[0111] The target model generation module is used to:

[0112] Inputting the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determining a first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data;

[0113] Determining a second output code of the large language model based on all previously input codes and a second-level target code of the input fitting data; until determining an Nth output code of the large language model based on all previously input codes of the large language model and an Nth-level target code of the input fitting data;

[0114] Based on at least one N-layer target code of the fitting data and a corresponding number of output codes, the large language model is iteratively optimized to generate a target model.

[0115] Optionally, the fitting data corresponds to M N-layer target codes, where M is an integer greater than 1;

[0116] The target model generation module is used to:

[0117] Inputting the large language model based on the input code of the non-fitting data and the first first-layer target code of the fitting data, determining a first output code of the large language model;

[0118] Determining a second output code of the large language model based on all previously input codes and a first second-layer target code and a second first-layer target code of the input fitting data, wherein the first second-layer target code and the second first-layer target code of the fitting data are input by superposition, summation, or concatenation; until determining the M+N-1th output code of the large language model based on all previously input codes of the large language model and the Mth-Nth-layer target code of the input fitting data;

[0119] Combining the M+N-1 output codes of the large language model into M N-layer final output codes;

[0120] Based on at least M N-layer target codes and M N-layer final output codes of the fitting data, the large language model is iteratively optimized to generate a target model.

[0121] The large language model encoding training device provided in an embodiment of the present invention can execute the large language model encoding training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0122] Example 4

[0123] Figure 4 A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0124] like Figure 4 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., which is communicatively connected to the at least one processor 11. The memory stores a computer program that can be executed by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. Various programs and data required for the operation of the electronic device 10 can also be stored in the RAM 13. The processor 11, ROM 12, and RAM 13 are connected to each other via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] Multiple components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a magnetic disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0126] The processor 11 can be various general-purpose and / or specialized processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors that run machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the large language model encoding training method.

[0127] In some embodiments, the large language model encoding training method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model encoding training method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the large language model encoding training method in any other appropriate manner (e.g., by means of firmware).

[0128] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0129] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when the computer program is executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer program may be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0130] In the context of the present invention, computer-readable storage media can be tangible media that can contain or store a computer program for use with an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. Computer-readable storage media can include but are not limited to electronic, magnetic, optical, electromagnetic, infrared or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, computer-readable storage media can be machine-readable signal media. More specific examples of machine-readable storage media can include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0132] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] A computing system may include clients and servers. The clients and servers are typically remote from each other and typically interact via a communication network. This client-server relationship arises through computer programs running on the respective computers, creating a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host. This server is a hosting product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosting and VPS services.

[0134] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in the present invention can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solution of the present invention can be achieved. This is not limited herein.

[0135] The above specific embodiments do not limit the scope of protection of the present invention. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention are intended to be included within the scope of protection of the present invention.

Claims

1. A model training method, characterized in that: include: Acquire a training data set; wherein each training sample data group in the training data set includes input data and fitting data, or each training sample data group includes reference data, input data and fitting data; Inputting the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determining a first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data; Determining a second output code of the large language model based on all previously input codes and a second-level target code of the input fitting data; until determining an Nth output code of the large language model based on all previously input codes of the large language model and an Nth-level target code of the input fitting data; Based on at least one N-layer target code of the fitting data and a corresponding number of output codes, the large language model is iteratively optimized to generate a target model.

2. The method according to claim 1, characterized in that The training data set includes at least two training sample data groups, and each training sample data group consists of at least two corresponding training data; Before inputting the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model and determining the first output code of the large language model, the method further includes: For at least one training data in each of the training sample data groups, determining a first-layer target encoding of the training data; Taking the first layer target code as the current layer target code, determining a current layer code deviation of the current layer target code, and determining a next layer target code based on the current layer code deviation; wherein the current layer code deviation is a difference between the current layer initial code and the current layer target code; The next layer target code is updated to the current layer target code, and the current layer code deviation for determining the current layer target code is returned to determine the Nth layer target code; wherein N is an integer greater than or equal to 2.

3. The method according to claim 2, characterized in that For at least one training data in each of the training sample data groups, before determining the first-layer target coding of the training data, the method further includes: Extracting partial training data of at least one dimension for the at least one training data; Encode part of the training data of at least one dimension separately to generate at least one layer of enhanced target encoding; Iteratively optimizing the large language model based on at least one N-layer target code and a corresponding number of output codes of the fitting data to generate a target model includes: At least one layer of the enhanced target code is superimposed on N layers of the target code, and the large language model is trained to generate a target model.

4. The method according to claim 3, characterized in that The step of superimposing at least one layer of the enhanced target code and N layers of the target code to train the large language model includes: At least one layer of the enhanced target code is placed before the N layers of the target code, and at least part of the enhanced target code is input into the large language model before the N layers of the target code to perform the large language model training.

5. The method according to any one of claims 2 to 4, characterized in that: Before determining the first-layer target coding of at least one training data in each of the training sample data groups, the method further includes: performing discretization processing on the at least one training data to generate a plurality of minimum discrete units; For at least one training data in each of the training sample data groups, determining a first-layer target encoding of the training data comprises: Each minimum discrete unit of at least one training data in each of the training sample data groups is encoded respectively to determine a first-layer target encoding of the training data.

6. The method according to claim 2, characterized in that Determining a first-layer target encoding of the training data includes: Determining a first layer initial encoding of the training data; Comparing the first layer initial code with each first code in a preset code book or a first standard code corresponding to the first code, and determining a first target code corresponding to the first code in the code book that has the highest similarity to the first layer initial code; A first layer target coding is determined based on the first target coding.

7. The method according to claim 2, characterized in that Determining a current layer coding deviation of the current layer target coding, and determining a next layer target coding based on the current layer coding deviation, comprising: Calculating a residual between an initial code of a current layer and a target code of the current layer, and using the residual as a current layer code deviation of the current layer code; Compare the current layer coding deviation with each second code or the second standard code corresponding to the second code in the preset next layer deviation code book, determine the second target code corresponding to the second code with the highest similarity to the current layer coding deviation in the next layer deviation code book, and determine the next layer target code based on the second target code.

8. The method according to claim 6 or 7, characterized in that Determine the corresponding target code based on the code with the highest similarity in the codebook, including: directly determining the code with the highest similarity in the codebook as the target code; or, The codes with the highest similarity in the codebook are encoded based on a preset encoding strategy to generate the target code.

9. The method according to claim 1, characterized in that The fitting data corresponds to M N-layer target codes, where M is an integer greater than 1, and the method further includes: Inputting the large language model based on the input code of the non-fitting data and the first first-layer target code of the fitting data, determining a first output code of the large language model; Determining a second output code of the large language model based on all previously input codes and a first second-layer target code and a second first-layer target code of the input fitting data, wherein the first second-layer target code and the second first-layer target code of the fitting data are input by superposition, summation, or concatenation; until determining the M+N-1th output code of the large language model based on all previously input codes of the large language model and the Mth-Nth-layer target code of the input fitting data; Combining the M+N-1 output codes of the large language model into M N-layer final output codes; Based on at least M N-layer target codes and M N-layer final output codes of the fitting data, the large language model is iteratively optimized to generate a target model.

10. A model training device, characterized in that: include: A training data set acquisition module, configured to acquire a training data set; wherein each training sample data group in the training data set includes input data and fitting data, or each training sample data group includes reference data, input data, and fitting data; a first output code determining module, configured to input the input code of the non-fitting data and the first-layer target code of the fitting data into the large language model, and determine a first output code of the large language model; wherein the non-fitting data includes the input data, or the non-fitting data includes the reference data and the input data; a second output code determining module, configured to determine a second output code of the large language model based on all previously input codes and a second-level target code of the input fitting data; and finally, determining an Nth output code of the large language model based on all previously input codes of the large language model and an Nth-level target code of the input fitting data; A target model generation module is used to iteratively optimize the large language model based on at least one N-layer target code and a corresponding number of output codes of the fitting data to generate a target model.

Images