Large language model encoding training method and device

Abstract

The application discloses a large language model coding training method and device. The method comprises the following steps: extracting part training data of at least one dimension for at least one training data; coding the part training data of at least one dimension respectively to generate at least one layer of reinforced target coding; determining N layers of target coding of the training data for each training sample data group; superimposing at least one layer of the reinforced target coding and N layers of the target coding to train a large language model and generate a target model. The scheme can strengthen the training effect of the large language model in certain dimensions and obtain better large language model training effect.

Description

[0001] This application is a divisional application, the original application's application number is 202411470413.6, the application date is October 21, 2024, and the invention name is "a large language model coding training method and device". TECHNICAL FIELD

[0002] The application relates to the technical field of machine learning, in particular to a large language model coding training method and device. BACKGROUND

[0003] The traditional model training uniformly encodes the whole training data, so that the features of each dimension of the training data are compressed equally, the information density is high, the training difficulty is great, and the training efficiency is low; the trained model is balanced in all aspects, and cannot meet the higher performance requirements of the model in one or some aspects or dimensions. SUMMARY

[0004] The application provides a large language model coding training method and device, which can strengthen the training effect of the large language model in some dimensions and obtain better large language model training effect.

[0005] According to one aspect of the application, a large language model coding training method is provided, comprising:

[0006] extracting part training data of at least one dimension for at least one training data;

[0007] encoding the part training data of at least one dimension respectively to generate at least one layer of reinforced target coding;

[0008] determining the N-layer target coding of the training data for each training sample data group; wherein N is an integer greater than or equal to 2;

[0009] superimposing at least one layer of the reinforced target coding and N layers of the target coding to train the large language model and generate a target model.

[0010] According to another aspect of the application, a large language model coding training device is provided, comprising:

[0011] A part training data extraction module is configured to extract part training data of at least one dimension for at least one training data;

[0012] A reinforced target coding generation module is configured to encode the part training data of at least one dimension respectively to generate at least one layer of reinforced target coding;

[0013] The N-layer target encoding generation module is configured to determine N-layer target encoding of the training data for at least one kind of training data in each training sample data set, wherein N is an integer greater than or equal to 2.

[0014] The target model generation module is configured to superimpose at least one layer of the reinforced target encoding and N-layer target encoding, train the large language model, and generate a target model.

[0015] According to another aspect of the present application, an electronic device is provided, which comprises:

[0016] at least one processor; and

[0017] a memory in communication with the at least one processor; wherein

[0018] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to perform the large language model encoding training method according to any one of the embodiments of the present application.

[0019] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to perform the large language model encoding training method according to any one of the embodiments of the present application.

[0020] The large language model encoding training scheme according to the embodiments of the present application extracts at least one dimension of partial training data for at least one kind of training data, encodes the at least one dimension of partial training data to generate at least one layer of reinforced target encoding, determines N-layer target encoding of the training data for at least one kind of training data in each training sample data set, wherein N is an integer greater than or equal to 2, superimposes at least one layer of the reinforced target encoding and N-layer target encoding, trains the large language model, and generates a target model. The technical scheme provided by the embodiments of the present application can strengthen the training effect of the large language model in certain dimensions, and obtain better large language model training effect.

[0021] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments description. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained from these drawings without creative labor.

[0023] Figure 1 A flow chart of a large language model coding training method provided for the first embodiment of the present application;

[0024] Figure 2 A flow chart of a large language model coding training method provided for the second embodiment of the present application;

[0025] Figure 3 A structural schematic diagram of a large language model coding training device provided for the third embodiment of the present application;

[0026] Figure 4 A structural schematic diagram of an electronic device for implementing the large language model coding training method of the present application. DETAILED DESCRIPTION

[0027] In order to make the person in the art better understand the present application scheme, the following will combine the drawings in the embodiments of the present application, and clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments only constitute some embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should belong to the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and do not necessarily indicate a specific order or sequence. It should be understood that the data thus used can be interchanged under appropriate circumstances, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device including a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but can include other steps or units not clearly listed or inherent to these processes, methods, products or devices.

[0029] Embodiment one

[0030] Figure 1A flowchart of a large language model coding training method is provided for the first embodiment of the present application. The embodiment can be applied to the coding training of a large language model. The method can be executed by a large language model coding training device, which can be realized in the form of hardware and / or software and can be configured in an electronic device.

[0031] As shown in Figure 1 , the method comprises:

[0032] S110, obtaining a training data set; wherein the training data set contains at least two training sample data groups, and each training sample data group is composed of at least two corresponding training data.

[0033] In the embodiment of the present application, a training data set is obtained, wherein the training data set contains a plurality of training sample data groups, and each training sample data group is composed of at least two corresponding data. For example, the training sample data group can include input data and fitting data; for another example, the training sample data group can include reference data, input data and fitting data. The input data is data that needs to be output according to the input content to generate corresponding content during subsequent large language model training, such as the question in the question and answer data, the lyrics part in the music generation data; the reference data is data that needs to be referenced to output corresponding generated content during subsequent large language model training, such as the reference picture in the text-to-image data, the reference song in the music generation data, or the reference accompaniment, the reference singing style and other reference data in the reference song, which are used to make the large language model generate output data with the same style; the fitting data is the standard answer data, which is used to compare with the output result of the large language model one by one during subsequent large language model training to calculate the loss value, so as to correct the probability of the next output result of the large model, which is the ideal data that the large language model needs to fit. The training data set is a data set matched with the application scene of the large language model used for subsequent training, for example, the subsequent training large language model is a music generation model, and each training sample data group in the training data set can include fitting music and target lyrics, or reference music, fitting music and target lyrics.

[0034] It should be noted that the number of input data and the number of reference data contained in the training sample data group are not limited in the embodiment of the present application. For example, when training a large language model for music generation, the input data in the training sample data group can be a piece of lyrics, or multiple pieces of lyrics, and the reference data in the training sample data group can be a reference song, or reference accompaniment and reference singing style in the reference song.

[0035] S120, determining the first layer target encoding of the at least one training data in each of the training sample data sets.

[0036] In the embodiment of the present application, for each training sample data set in the training data set, the first layer target encoding of the at least one training data in the training sample data set is determined. It can be understood that the first layer target encoding of any one or more or all of the training data in the training sample data set can be determined, and the types of the training data whose first layer target encoding is determined in each training sample data set in the training data set can be the same or different. For example, the encoder can encode the at least one training data in the training sample data set to determine the first layer target encoding, wherein the encoder can include a MERT encoder and a Mel encoder.

[0037] Optionally, before determining the first layer target encoding of the at least one training data in each of the training sample data sets, the method further comprises: discretizing the at least one training data to generate a plurality of minimum discrete units; and determining the first layer target encoding of the at least one training data in each of the training sample data sets comprises: encoding each minimum discrete unit of the at least one training data in each of the training sample data sets to determine the first layer target encoding of the training data. For example, the at least one training data in the training sample data set is discretized to generate at least two minimum discrete units, and then each minimum discrete unit in the at least two minimum discrete units is encoded by the encoder to determine the corresponding first target encoding. It can be understood that the number of first target encodings is the same as the number of minimum discrete units.

[0038] With the subsequent large language model as the music generation model, the training sample data set is composed of reference music, fitting music and target lyrics, and is exemplarily described. The fitting music is discretized to generate at least two frames of fitting audio data. Exemplarily, the fitting music can be discretized into 10 frames of fitting audio data per second. For example, if the fitting music is a 10-second music, the fitting music can be discretized into 100 frames of fitting audio data. Exemplarily, the fitting music can also be discretized as a whole. For example, the fitting music can be discretized into 500 frames of fitting audio data. The number of frames of discretized fitting audio data can be determined according to the length of the reference music. The longer the fitting music, the more frames of discretized fitting audio data. Each frame of fitting audio data is encoded by an encoder to generate corresponding first layer target encoding. The number of first layer target encodings is the same as the number of frames of fitting audio data. Optionally, the same encoder can be used to encode each frame of fitting audio data, or different encoders can be used to encode each frame of fitting audio data. The encoder can be an open source encoder, such as MERT encoder or Mel encoder. In the embodiment of the present application, the discretization of the target lyrics can include: performing word segmentation on the target lyrics based on a preset word segmentation algorithm to divide the target lyrics into a plurality of word segmentation units. The word segmentation unit can be a word, a sentence or a word. Each word segmentation unit in the target lyrics is encoded by the encoder to generate a first layer target encoding corresponding to each word segmentation unit. The manner of determining the first layer target encoding of the reference music can be similar to that of the fitting music, which will not be described here.

[0039] Optionally, the first layer target encoding of the training data is determined by: determining a first layer initial encoding of the training data; comparing the first layer initial encoding with each first encoding in a preset encoding password book or a first standard encoding corresponding to the first encoding, to determine a first target encoding corresponding to a first encoding with the highest similarity to the first layer initial encoding in the encoding password book; and determining the first layer target encoding based on the first target encoding. For example, the training sample data set is encoded by an encoder, and the obtained encoding information is used as the first layer initial encoding of the training data. A preset encoding password book is obtained, wherein the encoding password book is composed of a plurality of first encodings. Optionally, the first encoding in the encoding password book can be a standard encoding (also referred to as a special encoding), or the standard encoding can be replaced by a special character, for example, the first standard encoding is used as encoding 0, the second standard encoding is used as encoding 1, and so on, to generate an encoding password book such as [0, 1, 2,..., 1023]. In this way, the complexity of the password book can be greatly simplified, and the encoding (i.e., special characters 0, 1, 2, etc.) and the standard encoding can be isolated, so that the format and / or content of the standard encoding can be adjusted in the future, for example, the first standard encoding can be a multi-dimensional vector, or an audio encoding, a graphic encoding, a character encoding, etc. Alternatively, the first standard encoding is originally a 256-dimensional encoding vector, and is subsequently converted into a 1024-dimensional encoding vector, while the password book is still [0, 1, 2,..., 1023].

[0040] The first layer initial encoding is compared with each first encoding in the encoding password book or a first standard encoding corresponding to the first encoding, and the similarity between the two is calculated. The first encoding with the highest similarity to the first layer initial encoding in the encoding password book is used as the first target encoding, and the first layer target encoding of the training data is determined based on the first target encoding. The first target encoding in the encoding password book can be directly used as the first layer target encoding, or the first standard encoding corresponding to the first target encoding in the encoding password book can be used as the first layer target encoding. Alternatively, the first target encoding in the encoding password book can be encoded based on a preset encoding strategy to generate the first layer target encoding corresponding to the standard data.

[0041] In S130, the first layer target encoding is used as a current layer target encoding, a current layer encoding deviation of the current layer target encoding is determined, and a next layer target encoding is determined based on the current layer encoding deviation.

[0042] In the embodiment of the present application, the first layer target code of the training data is updated to the current layer target code, and the current layer coding deviation of the current layer target code is determined, wherein the current layer coding deviation can be the difference between the current layer initial code and the current layer target code. The next layer target code of the training data is determined according to the current layer coding deviation, for example, the current layer coding deviation can be coded based on a preset coding algorithm to generate the next layer target code of the training data. It can be understood that the first layer coding deviation of the first layer target code of the training data is determined, wherein the residual error between the first layer initial code and the first layer target code of the training data can be taken as the first layer coding deviation. The second layer target code of the training data is determined according to the first layer coding deviation.

[0043] Optionally, the current layer coding deviation of the current layer target code is determined, and the next layer target code is determined based on the current layer coding deviation, including: calculating the residual error between the current layer initial code and the current layer target code, and taking the residual error as the current layer coding deviation of the current layer coding; comparing the current layer coding deviation with each second coding or second standard coding corresponding to the second coding in the preset next layer deviation password book, determining the second target coding corresponding to the second coding with the highest similarity to the current layer coding deviation in the next layer deviation password book, and determining the next layer target code based on the second target coding.

[0044] For example, the residual error between the current layer initial code and the current layer target code is taken as the current layer coding deviation of the current layer coding. The next layer deviation password book is obtained, wherein the next layer deviation password book is composed of a plurality of second codings, and the second coding can be understood as a residual error coding. Optionally, the second coding in the deviation password book can be a standard residual error coding (also referred to as a special residual error coding), and the standard residual error coding can be replaced by a special character, such as taking the first standard residual error coding as coding 0, and taking the second standard residual error coding as coding 1, and so on, to generate a deviation password book such as [0, 1, 2, …, 1023]. In this way, the complexity of the deviation password book can be greatly simplified, and the coding (i.e. 0, 1, 2, etc. special characters) and the standard residual error coding can be isolated, so as to facilitate the adjustment of the format and / or content of the standard residual error coding in the future, for example, the first standard residual error coding can be a multi-dimensional vector, or an audio coding, a graphic coding, a character coding, etc. one or more of them; or the first standard residual error coding is originally a 256-dimensional coding vector, and is subsequently adjusted and converted into a 1024-dimensional coding vector, and at this time, the deviation password book is still [0, 1, 2, …, 1023].

[0045] The current layer coding bias is sequentially compared with each second coding or second standard coding corresponding to the second coding in the next layer bias password book, the similarity between the two is calculated, the second coding with the highest similarity to the current layer coding bias in the next layer bias password book is taken as the second target coding, and the next layer target coding of the training data is determined based on the second target coding. Wherein, the second target coding in the next layer bias password book can be directly taken as the next layer target coding, or the second standard coding corresponding to the second target coding in the next layer bias password book can be taken as the next layer target coding, or the second target coding in the next layer bias password book can be encoded based on a preset coding strategy to generate the next layer target coding corresponding to the standard data.

[0046] S140, update the next layer target coding to the current layer target coding, and return to execute the current layer coding bias of the current layer target coding, and determine the Nth layer target coding; wherein N is an integer greater than or equal to 2.

[0047] In the embodiment of the application, the first layer coding bias of the first layer target coding of the training data is compared with the second standard coding corresponding to each second coding in the second layer bias password book, the second target coding corresponding to the second coding with the highest similarity to the first layer coding bias in the second layer bias password book is determined, and the second layer target coding is determined based on the second target coding. The first layer coding bias is taken as the second layer initial coding, the residual between the second initial coding and the second layer target coding is calculated, and the residual is taken as the second layer coding bias, that is, the residual between the first layer coding bias and the second layer target coding is taken as the second layer coding bias. In order to facilitate description, the second coding in the i-th layer bias password book can also be referred to as the i-th coding, and the standard coding corresponding to the i-th coding can also be referred to as the i-th standard coding. Therefore, the second layer coding bias is compared with the third standard coding corresponding to each third coding in the third layer bias password book, the third target coding corresponding to the third coding with the highest similarity to the second layer coding bias in the third layer bias password book is determined, and the third layer target coding is determined based on the third target coding. The second layer coding bias is taken as the third layer initial coding, the residual between the third initial coding and the third layer target coding is calculated, and the residual is taken as the third layer coding bias, that is, the residual between the second layer coding bias and the third layer target coding is taken as the third layer coding bias. According to the above method, the cycle is continuously repeated until the Nth layer target coding of the training data is determined, wherein N is an integer greater than or equal to 2.

[0048] It can be understood that the N layer target coding corresponding to at least one training data in the training sample data set can be determined according to S120-S130. It should be noted that the number of layers of the target coding of at least one training data in each training sample data set in the training data set can be the same or different.

[0049] S150, training the large language model based on the N-layer target encoding to generate a target model.

[0050] In the embodiment of the present application, at least one training data corresponding to the N-layer target encoding in each training sample data group in the training data set is input into the large language model to train the large language model and generate a target model. The large language model can be any open source large model, such as GPT, etc. It should be noted that the application scenario of the target model is not limited in the embodiment of the present application. For example, the target model can be a music generation model, in which case each training sample data group in the training data set used to train the target model can include reference music, sample lyrics and fitted music. For another example, the target model can be a text-to-image model, i.e. a model for generating images from text, in which case each training sample data group in the training data set used to train the target model can include sample text and fitted images.

[0051] The large language model encoding training method provided in the embodiment of the present application obtains a training data set, wherein the training data set contains at least two training sample data groups, each of which is composed of at least two kinds of training data. For at least one kind of training data in each training sample data group, the first-layer target encoding of the training data is determined. The first-layer target encoding is taken as the current-layer target encoding, the current-layer encoding bias of the current-layer target encoding is determined, and the next-layer target encoding is determined based on the current-layer encoding bias. The next-layer target encoding is updated as the current-layer target encoding, and the current-layer encoding bias of the current-layer target encoding is determined again. The N-layer target encoding is determined. N is an integer greater than or equal to 2. The large language model is trained based on the N-layer target encoding to generate a target model. The technical solution provided in the embodiment of the present application encodes at least one kind of training data in the training sample data group into multi-layer target encoding, which can provide more abundant data information, improve the fine degree of data discretization and fitting, and thus improve the quality of the output data of the large language model after training.

[0052] In some embodiments, before determining the first layer target encoding of the training data for each of the at least one training data in each of the training sample data sets, the method further comprises: extracting at least one dimension of partial training data from the at least one training data; encoding the at least one dimension of partial training data respectively to generate at least one layer of reinforced target encoding; and training the large language model based on the N layers of target encoding to generate a target model, including: superimposing the at least one layer of reinforced target encoding and the N layers of target encoding to train the large language model to generate the target model. The advantage of this setting is that the training effect of the large language model in certain dimensions can be strengthened, and better training effect of the large language model can be obtained.

[0053] In the embodiments of the present application, for at least one training data in each training sample data set, at least one dimension of partial training data is extracted from the training data, and each dimension of partial training data is encoded respectively to generate at least one layer of reinforced target encoding. Wherein, the at least one dimension of partial training data can be encoded as a whole to generate at least one layer of reinforced target encoding, or the at least one dimension of partial training data can be discretized to generate at least one layer of reinforced target encoding by encoding each minimum discrete unit. It should be noted that when the reinforced target encoding is multi-layer encoding, the determination method of the multi-layer reinforced target encoding is the same as the determination method of the N layers of target encoding of the training data in the above embodiments, which will not be repeated here.

[0054] Exemplarily, the training sample data set includes reference music, fitting music and target lyrics, and the fitting music can be subjected to preset number of dimension splitting operations, such as splitting at least one dimension of the fitting music part (such as two dimensions of the fitting music part of accompaniment part and vocal part) from the fitting music, and each dimension of the fitting music part is subjected to discretization processing to generate at least two frames of fitting audio data, each frame of fitting audio data is encoded by the encoder to generate at least one layer of reinforcement target code. Similarly, the reference sample music is subjected to preset number of dimension splitting operations, such as splitting at least one dimension of the reference music part (such as two dimensions of the reference music part of accompaniment part and vocal part) from the reference music, and the at least one dimension of the reference music part is subjected to unified coding to generate at least one layer of reinforcement target code. Or the at least one dimension of the reference music part is subjected to discretization processing to generate at least two frames of reference audio data, each frame of reference audio data is encoded by the encoder to generate at least one layer of reinforcement target code. The target lyrics are encoded to generate at least one layer of reinforcement target code. Exemplarily, the bass, the medium and the treble in one frame of audio data are respectively encoded in different layers to generate corresponding reinforcement target code, or different instruments and different voices in one frame of audio data are separated and individually encoded in different layers to generate corresponding reinforcement target code.

[0055] In the embodiment of the present application, at least one layer of reinforcement target code is superimposed on N layers of target code, and the superimposed code is input into the large language model to train the large language model to generate a target model. Optionally, the at least one layer of reinforcement target code is superimposed on the N layers of target code to train the large language model, which includes placing at least one layer of reinforcement target code before N layers of target code, and at least part of the reinforcement target code is input into the large language model in priority to the N layers of target code to train the large language model.

[0056] Exemplarily, taking the fitting music in the training sample data set as an example, the reinforcement target code of the vocal part extracted from the fitting music is taken as the first layer, and the corresponding 4 layers of target code of the fitting music are taken as the 2nd, 3rd, 4th and 5th layers, respectively. The reinforcement target code of the vocal part of the fitting music is input into the large language model as the first layer code in priority to predict the output, and then the 4 layers of target code of the fitting music are sequentially input into the large language model to continue training the large language model based on the 4 layers of target code to generate a target model. The advantage of such setting is that the large language model can first output the fitting of the simple vocal part corresponding to the fitting music, and then perform the fitting training of the complex song to obtain clearer fitting training of the vocal part than the pure fitting of the song, thereby achieving better training effect of the large language model.

[0057] Embodiment Two

[0058] Figure 2 A flowchart of a large language model encoding training method provided for Embodiment Two of the present application is shown in FIG. 2, which comprises the following steps: Figure 2

[0059] S210, obtaining a training data set; wherein the training data set comprises at least two training sample data groups, each training sample data group is composed of at least two corresponding training data, each training sample data comprises input data and fitting data, or each training sample data group comprises reference data, input data and fitting data.

[0060] S220, determining the first layer target encoding of at least one training data in each training sample data group.

[0061] S230, taking the first layer target encoding as the current layer target encoding, determining the current layer encoding deviation of the current layer target encoding, and determining the next layer target encoding based on the current layer encoding deviation.

[0062] S240, updating the next layer target encoding to the current layer target encoding, and returning to determine the current layer encoding deviation of the current layer target encoding to determine the Nth layer target encoding; wherein N is an integer greater than or equal to 2.

[0063] S250, inputting the input encoding of non-fitting data and the first layer target encoding of fitting data into a large language model to determine the first output encoding of the large language model; wherein the non-fitting data comprises the input data, or the non-fitting data comprises the reference data and the input data.

[0064] Each training sample data can comprise input data and fitting data, or reference data, input data and fitting data. When the training sample data comprises input data and fitting data, the input data is taken as non-fitting data; when the training sample data comprises reference data, input data and fitting data, the reference data and input data are taken as non-fitting data.

[0065] In the embodiment of the present application, the N layer target encoding of non-fitting data is taken as the input encoding of non-fitting data, and the input encoding of non-fitting data and the first layer target encoding of fitting data are input into a large language model to train the large language model and obtain the first output encoding of the large language model. It can be understood that the first output encoding is the encoding output by the large language model after learning the input encoding of non-fitting data and the first layer target encoding of fitting data.

[0066] ​S260, determining the second output encoding of the large language model based on all the previous input encodings and the second layer target encoding of the fitting data input into the large language model; until determining the Nth output encoding of the large language model based on all the previous input encodings and the Nth layer target encoding of the fitting data input into the large language model.

[0067] In the embodiment of the present application, the second output encoding of the large language model is determined based on all the previous input encodings and the second layer target encoding of the fitting data input into the large language model. Similarly, the third output encoding of the large language model is determined based on all the previous input encodings and the third layer target encoding of the fitting data input into the large language model; the fourth output encoding of the large language model is determined based on all the previous input encodings and the fourth layer target encoding of the fitting data input into the large language model. In this way, until the Nth output encoding of the large language model is determined based on all the previous input encodings and the Nth layer target encoding of the fitting data input into the large language model.

[0068] S270, iteratively optimizing the large language model based on at least one Nth layer target encoding of the fitting data and a corresponding number of output encodings, to generate a target model.

[0069] In the embodiment of the present application, the Nth layer target encoding of the fitting data can be one or multiple. When the fitting data corresponds to multiple Nth layer target encodings, the output encoding corresponding to each Nth layer target encoding is obtained through S260-S270. The large language model is iteratively optimized based on at least one Nth layer target encoding of the fitting data and a corresponding number of output encodings, to generate a target model.

[0070] Optionally, the fitting data corresponds to M Nth layer target encodings, M is an integer greater than 1, and the method further comprises: inputting the first input encoding of the non-fitting data and the first Nth layer target encoding of the fitting data into the large language model to determine the first output encoding of the large language model; determining the second output encoding of the large language model based on all the previous input encodings and the first second layer target encoding and the second first layer target encoding of the fitting data input into the large language model, wherein the first second layer target encoding and the second first layer target encoding of the fitting data are input in a superposition sum or splicing manner; until determining the M+N-1th output encoding of the large language model based on all the previous input encodings and the Mth Nth layer target encoding of the fitting data input into the large language model; combining the M+N-1th output encoding of the large language model into M Nth layer final output encodings; iteratively optimizing the large language model based on at least M Nth layer target encodings of the fitting data and M Nth layer final output encodings, to generate a target model.

[0071] In the embodiment of the present application, since the output data of the large language model depends on the input data of the large language model corresponding to the fitting data last time, the M N-layer target encodings corresponding to the fitting data are superimposed and input into the large language model in a delay manner. Specifically, the M N-layer target encodings corresponding to the fitting data are input into the large language model as input encodings of the non-fitting data, and the first first-layer target encoding of the non-fitting data and the first first-layer target encoding of the fitting data are input into the large language model to make the large language model predict and obtain the first output encoding of the large language model. It can be understood that the first output encoding is the encoding output by the large language model after learning the input encoding of the non-fitting data and the first first-layer target encoding of the fitting data. Based on all the encodings input into the large language model before, the first second-layer target encoding and the second first-layer target encoding of the fitting data input into the large language model, the second output encoding of the large language model is determined. Specifically, the first second-layer target encoding and the second first-layer target encoding of the fitting data are superimposed and summed or spliced, and based on the superimposed and summed or spliced encodings and all the encodings input before, the second output encoding of the large language model is determined. By analogy, until the (M+N-1)th output encoding of the large language model is determined based on all the encodings input into the large language model before and the Mth N-layer target encoding of the fitting data input.

[0072] For example, M=4, N=4, and Table 1 is a delay superimposition input table of 4 4-layer target encodings corresponding to the fitting data provided by the embodiment of the present application:

[0073] Table 1

[0074] t(14) t(24) t(34) t(44) t(13) t(23) t(33) t(43) t(12) t(22) t(32) t(42) t(11) t(21) t(31) t(41)

[0075] After inputting the input encoding of the non-fitting data and the target encoding t(11) of the first first layer of the fitting data into the large language model, the first output encoding of the large language model is determined, and then the first second layer target encoding t(12) and the second first layer target encoding t(21) are superimposed and input into the large language model. Based on all the encodings input before and the encoding after superimposing t(12) and t(21), the large language model determines the second output encoding. The first third layer target encoding t(13) of the fitting data, the second second layer target encoding t(22) and the third first layer target encoding t(31) are superimposed and input into the large language model. Based on all the encodings input before and the encoding after superimposing t(13), t(22) and t(31), the large language model determines the third output encoding. The first fourth layer target encoding t(14) of the fitting data, the second third layer target encoding t(23), the third second layer target encoding t(32) and the fourth first layer target encoding t(41) are superimposed and input into the large language model. Based on all the encodings input before and the encoding after superimposing t(14), t(23), t(32) and t(41), the large language model determines the fourth output encoding. In the same way, the seventh output encoding of the large language model is obtained according to Table 1. It can be understood that inputting the M N-layer target encodings of the fitting data into the large language model in a delayed manner can make the large language model first obtain the target encoding of each unit low level and then obtain the target encoding of the high level, which is beneficial to the training of the large language model.

[0076] Since the output data of the large language model depends on the input of all non-fitting data (input data or reference data and input data), the N-layer target encodings corresponding to the non-fitting data can be input into the large language model in a conventional superimposed or spliced manner, that is, all N-layer target encodings of the non-fitting data are input into the large language model at one time, so that the large model can obtain all N-layer target encodings of the non-fitting data at one time, thereby making the training of the large language model more efficient. Alternatively, the N-layer target encodings of the non-fitting data can be input into the large language model in a DELAY superimposed manner to train the large language model. For example, Table 2 is a table for inputting four 4-layer target encodings of non-fitting data in a conventional manner:

[0077] Table 2

[0078] t(14) t(24) t(34) t(44) t(13) t(23) t(33) t(43) t(12) t(22) t(32) t(42) t(11) t(21) t(31) t(41)

[0079] Wherein, when the N-layer target encoding of the non-fitting data is input into the large language model in a conventional manner, the multi-dimensional encoding vector obtained by adding or splicing the N-layer target encoding of each unit, such as adding 256-dimensional t(11), t(12), t(13), t(14) to obtain a 256-dimensional vector, or splicing to obtain a 1024-dimensional vector, adding 256-dimensional t(21), t(22), t(23), t(24) to obtain a 256-dimensional vector, still a 256-dimensional vector, or splicing to obtain a 1024-dimensional vector, the multi-layer encoding vector is input into the large language model, wherein the output of the large language model is also a multi-layer encoding vector, thereby improving the accuracy of the output of the large language model.

[0080] The M+N-1 output encoding of the large language model is combined into M N-layer final output encoding, and then the large language model is iteratively optimized based on at least M N-layer target encoding of the fitting data and M N-layer final output encoding, to generate a target model.

[0081] The technical scheme provided by the embodiment of the application can provide more abundant data information by encoding at least one training data in the training sample data set into multi-layer target encoding, improve the fine degree of data discretization and fitting, and thereby improve the quality of the output data of the large language model after training.

[0082] Embodiment three

[0083] Figure 3 A structural schematic diagram of a large language model encoding training device provided by the third embodiment of the application is shown in FIG. 3. Figure 3 As shown in the figure, the device comprises:

[0084] The training data set acquisition module 310 is configured to acquire a training data set, wherein the training data set comprises at least two training sample data groups, and each training sample data group comprises at least two corresponding training data;

[0085] The first layer target encoding determination module 320 is configured to determine the first layer target encoding of the training data in each training sample data group;

[0086] The next layer target encoding determination module 330 is configured to determine the first layer target encoding as the current layer target encoding, determine the current layer encoding deviation of the current layer target encoding, and determine the next layer target encoding based on the current layer encoding deviation;

[0087] The coding bias cycle determination module 340 is configured to update the next layer target coding to the current layer target coding, return to execute the current layer coding bias determination of the current layer target coding, and determine the Nth layer target coding; wherein N is an integer greater than or equal to 2.

[0088] The target model generation module 350 is configured to train the large language model based on the N-layer target coding, and generate a target model.

[0089] Optionally, the device further comprises:

[0090] The partial training data extraction module is configured to, before determining the first layer target coding of at least one type of training data in each of the training sample data groups, extract partial training data of at least one dimension for the at least one type of training data.

[0091] The reinforcement target coding generation module is configured to encode the partial training data of at least one dimension respectively, and generate at least one layer of reinforcement target coding.

[0092] The target model generation module comprises:

[0093] The target model generation unit is configured to superimpose the at least one layer of reinforcement target coding and the N-layer target coding, train the large language model, and generate a target model.

[0094] Optionally, the target model generation unit is configured to:

[0095] Place the at least one layer of reinforcement target coding before the N-layer target coding, and input at least part of the reinforcement target coding to the large language model in priority to the N-layer target coding, to train the large language model.

[0096] Optionally, the device further comprises:

[0097] The minimum discrete unit generation module is configured to, before determining the first layer target coding of at least one type of training data in each of the training sample data groups, perform discretization processing on the at least one type of training data, and generate a plurality of minimum discrete units.

[0098] The first layer target coding determination module is configured to:

[0099] Encode each minimum discrete unit of the at least one type of training data in each of the training sample data groups respectively, and determine the first layer target coding of the training data.

[0100] Optionally, the first layer target coding determination module is configured to:

[0101] determining a first layer initial encoding of the training data;

[0102] comparing the first layer initial encoding with each first encoding or a first standard encoding corresponding to a first encoding in a preset encoding password book, to determine a first target encoding corresponding to a first encoding with the highest similarity to the first layer initial encoding in the encoding password book;

[0103] determining a first layer target encoding based on the first target encoding.

[0104] Optionally, the next layer target encoding determination module is configured to:

[0105] calculating a residual between a current layer initial encoding and the current layer target encoding, and taking the residual as a current layer encoding deviation of the current layer encoding;

[0106] comparing the current layer encoding deviation with each second encoding or a second standard encoding corresponding to a second encoding in a preset next layer deviation password book, to determine a second target encoding corresponding to a second encoding with the highest similarity to the current layer encoding deviation in the next layer deviation password book, and determining a next layer target encoding based on the second target encoding.

[0107] Optionally, determining a target encoding corresponding to an encoding with the highest similarity in a password book comprises:

[0108] directly determining the encoding with the highest similarity in the password book as the target encoding; or,

[0109] encoding the encoding with the highest similarity in the password book based on a preset encoding strategy to generate the target encoding.

[0110] Optionally, each of the training sample data comprises input data and fitting data, or each of the training sample data groups comprises reference data, input data and fitting data.

[0111] The target model generation module is configured to:

[0112] inputting an input encoding of non-fitting data and a first layer target encoding of the fitting data into a large language model to determine a first output encoding of the large language model; wherein the non-fitting data comprises the input data, or the non-fitting data comprises the reference data and the input data.

[0113] determining a second output encoding of the large language model based on all previously input encodings and a second layer target encoding of the input fitting data; until determining an Nth output encoding of the large language model based on all previously input encodings of the large language model and an Nth layer target encoding of the input fitting data.

[0114] Based on at least one N-layer target code of the fitting data and a corresponding number of output codes, the large language model is iteratively optimized to generate a target model.

[0115] Optionally, the fitting data corresponds to M N-layer target codes, and M is an integer greater than 1.

[0116] The target model generation module is configured to:

[0117] Based on the input code of the non-fitting data and the first first-layer target code of the fitting data, the first output code of the large language model is determined by inputting the large language model.

[0118] Based on all previously input codes and the first second-layer target code and the second first-layer target code of the fitting data, the second output code of the large language model is determined, wherein the first second-layer target code and the second first-layer target code of the fitting data are input in a superposition summation or splicing manner; until the (M+N-1)th output code of the large language model is determined based on all previously input codes of the large language model and the Mth N-layer target code of the fitting data.

[0119] The M+N-1 output codes of the large language model are combined into M N-layer final output codes.

[0120] Based on at least M N-layer target codes of the fitting data and M N-layer final output codes, the large language model is iteratively optimized to generate a target model.

[0121] The large language model coding training device provided by the embodiments of the present application can perform the large language model coding training method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0122] Embodiment four

[0123] Figure 4 A structural schematic diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the applications described and / or claimed in this document.

[0124] AsFigure 4 As shown, the electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0125] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, a speaker, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0126] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as the large language model encoding training method.

[0127] In some embodiments, the large language model encoding training method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model encoding training method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the large language model encoding training method by any other appropriate means, such as by means of firmware.

[0128] The various embodiments of the systems and techniques described above can be implemented in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a load programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0129] Computer programs used to implement the processes of the application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus, such that the computer program

[0130] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store computer programs for use by or in connection with an instruction execution system, apparatus, or device. Computer-readable storage media can include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0131] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0132] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0133] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. A server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0134] It should be understood that the various forms of flow shown above can be re-ordered, added to, or deleted from without departing from the scope of the present disclosure. For example, the steps recited in the present disclosure can be executed in parallel, executed in sequence, or executed in a different order, as long as the desired results of the present disclosure are achieved, and the present disclosure is not limited herein.

[0135] The above detailed description does not limit the scope of the present disclosure. It is understood that various modifications, combinations, sub-combinations, and alternatives can be made to the detailed disclosure without departing from the spirit and principles of the present disclosure. Any modifications, equivalent substitutions, improvements, and the like that are made within the spirit and principles of the present disclosure are included in the scope of the present disclosure.

Claims

1. A large language model encoding training method, characterized in that, The method comprises the following steps: obtaining a training data set; wherein the training data set comprises at least two training sample data groups, each of which is composed of at least two corresponding training data; each of the training sample data groups comprises fitted music and target lyrics, or each of the training sample data groups comprises reference music, fitted music and target lyrics; extracting at least one dimension of partial training data for at least one training data; wherein if the training data is fitted music or reference music, the corresponding partial training data comprises two dimensions of accompaniment and vocals, or the corresponding partial training data comprises three dimensions of bass, middle and high, or the corresponding partial training data comprises multiple dimensions of different instruments and different vocals; encoding the partial training data of at least one dimension respectively to generate at least one layer of reinforced target encoding; determining N-layer target encoding of the training data for at least one training data in each training sample data group; wherein N is an integer greater than or equal to 2; superimposing at least one layer of the reinforced target encoding and N layers of the target encoding to train a large language model and generate a target model; wherein, for at least one training data in each training sample data group, determining N-layer target encoding of the training data comprises: determining the first layer target encoding of the training data for at least one training data in each training sample data group; determining the current layer encoding deviation of the current layer target encoding, and determining the next layer target encoding based on the current layer encoding deviation; wherein the current layer encoding deviation is the difference between the current layer initial encoding and the current layer target encoding; updating the next layer target encoding to the current layer target encoding, and returning to determine the current layer encoding deviation of the current layer target encoding to determine the N-layer target encoding; wherein, determining the first layer target encoding of the training data comprises: determining the first layer initial encoding of the training data; comparing the first layer initial encoding with each first encoding or the first standard encoding corresponding to the first encoding in a pre-set encoding password book to determine the first target encoding corresponding to the first encoding with the highest similarity in the encoding password book; determining the first layer target encoding based on the first target encoding; wherein, determining the current layer encoding deviation of the current layer target encoding, and determining the next layer target encoding based on the current layer encoding deviation comprises: calculating the residual between the current layer initial encoding and the current layer target encoding, and taking the residual as the current layer encoding deviation of the current layer encoding; comparing the current layer encoding deviation with each second encoding or the second standard encoding corresponding to the second encoding in a pre-set next layer deviation password book to determine the second target encoding corresponding to the second encoding with the highest similarity in the next layer deviation password book, and determining the next layer target encoding based on the second target encoding.

2. The method of claim 1, wherein, The at least one layer of the reinforced target encoding is superimposed on the N layers of the target encoding, and a large language model is trained, comprising: The at least one layer of the reinforced target encoding is placed before the N layers of the target encoding, and at least part of the reinforced target encoding is input into the large language model in priority to the N layers of the target encoding, so as to train the large language model.

3. The method of claim 1, wherein, Before determining the first layer target encoding of the training data for at least one kind of training data in each of the training sample data sets, further comprising: Discretization processing is performed on the at least one kind of training data to generate a plurality of minimum discrete units; For each of the at least one kind of training data in each of the training sample data sets, the first layer target encoding of the training data is determined, comprising: Each minimum discrete unit of the at least one kind of training data in each of the training sample data sets is encoded respectively to determine the first layer target encoding of the training data.

4. The method of claim 1, wherein, The corresponding target encoding is determined based on the highest similarity encoding in the codebook, comprising: The highest similarity encoding in the codebook is directly determined as the target encoding; or, The highest similarity encoding in the codebook is encoded based on a preset encoding strategy to generate the target encoding.

5. A large language model encoding training apparatus, characterized by comprising: Comprising: Obtaining a training data set; wherein the training data set contains at least two training sample data sets, each of the training sample data sets is composed of at least two corresponding training data; each of the training sample data sets comprises fitted music and target lyrics, or each of the training sample data sets comprises reference music, fitted music and target lyrics; A part of training data extraction module is used for extracting part of training data of at least one dimension for at least one kind of training data; A reinforced target encoding generation module is used for encoding part of training data of at least one dimension respectively to generate at least one layer of reinforced target encoding; wherein, if the training data is fitted music or reference music, the corresponding part of training data comprises two dimensions of accompaniment part and vocal part, or the corresponding part of training data comprises three dimensions of bass, medium and high, or the corresponding part of training data comprises multiple dimensions of different instruments and different voices; An N layer target encoding generation module is used for determining N layer target encoding of the training data for at least one kind of training data in each of the training sample data sets; wherein, N is an integer greater than or equal to 2; A target model generation module is used for superimposing at least one layer of the reinforced target encoding on N layers of the target encoding, training a large language model, and generating a target model; The N layer target encoding generation module is used for: Determining the first layer target encoding of the training data for at least one kind of training data in each of the training sample data sets; The first layer target encoding is taken as the current layer target encoding, the current layer encoding deviation of the current layer target encoding is determined, and the next layer target encoding is determined based on the current layer encoding deviation; wherein, the current layer encoding deviation is the difference between the current layer initial encoding and the current layer target encoding; updating the next layer target code as the current layer target code, and returning to execute determining a current layer coding bias of the current layer target code, and determining an Nth layer target code; wherein the determining the first layer target code of the training data comprises: determining a first layer initial code of the training data; comparing the first layer initial code with each first code or a first standard code corresponding to the first code in a preset coding password book, to determine a first target code corresponding to a first code with the highest similarity to the first layer initial code in the coding password book; determining a first layer target code based on the first target code; wherein the determining the current layer coding bias of the current layer target code and determining a next layer target code based on the current layer coding bias comprises: calculating a residual between a current layer initial code and the current layer target code, and taking the residual as the current layer coding bias of the current layer coding; comparing the current layer coding bias with each second code or a second standard code corresponding to the second code in a preset next layer bias password book, to determine a second target code corresponding to a second code with the highest similarity to the current layer coding bias in the next layer bias password book, and determining a next layer target code based on the second target code.

Images