Large language model training and reasoning method and device

Abstract

The invention discloses a big language model training and reasoning method and device. The large language model training method comprises the steps of obtaining a training data set; wherein the training data set comprises at least two training sample data sets, and each training sample data set is composed of at least two kinds of corresponding data; at least one first training sample data set is randomly extracted from the training data set according to a preset sequence, at least part of data is randomly removed from the first training sample data set, and target training data is generated; and performing model training on a large language model based on the training sample data set in the training data set and the target training data until a preset training condition is met, and generating a target model. According to the scheme, the large language model can still output a correct reasoning result when the input data are omitted or missing, the applicability of the large language model is improved, and the output result is more stable when the data are normally input.

Description

Technical Field

[0001] The present invention relates to the field of machine learning technology, and in particular to a large language model training and reasoning method and device. Background Art

[0002] The existing large model training generally achieves the model training effect by inputting training data into the large model so that the output of the large model is continuously fitted with the standard data in the training sample. In the model training process, the probability that the next output result is a specific value is calculated by the large model through all the previous input information (including the standard data input previously); in the model reasoning process, the probability that the next output result is a specific value is calculated by the large model through all the previous input information and the output token information. It can be seen that the output result of the large model is determined by probability. When the input information is omitted, the output result of the model may be completely different.

[0003] For example, when the model needs multiple input data to output the correct result, for example, the model needs to input reference accompaniment, reference singing style, and lyrics content to generate the corresponding music song, but in the actual reasoning process, the user only inputs the reference accompaniment and lyrics content, but does not input the reference singing style, or the user inputs the reference singing style but the input singing style is invalid or the extraction fails; or, part of the data in the user input data is missing, for example, part of the data in the reference accompaniment, reference singing style, and lyrics content input by the user is missing or lost. At this time, the model either cannot infer the output, or the output content deviates greatly from the user request. In addition, even if the input data of the model is exactly the same, the output result of the model each time may deviate greatly, making the model output unstable. Therefore, it is crucial to make the large model output correctly when there are omissions in the input data, and to make the output more stable when the input data is normal. Summary of the invention

[0004] The present invention provides a large language model training and reasoning method and device, which not only enables the large language model to output correct reasoning results when there are omissions or missing input data, thereby improving the applicability of the large language model, but also makes the output results more stable when normal input data is used.

[0005] According to one aspect of the present invention, a large language model training method is provided, comprising:

[0006] Acquire a training data set; wherein the training data set includes at least two training sample data groups, and each of the training sample data groups is composed of corresponding at least two types of data;

[0007] Selecting at least one first training sample data group from the training data set in a preset order or randomly, and randomly removing at least part of the data from the first training sample data group to generate target training data;

[0008] Model training is performed on the large language model based on the training sample data set and the target training data in the training data set until preset training conditions are met, thereby generating a target model.

[0009] According to another aspect of the present invention, a large language model reasoning method is provided, comprising:

[0010] In response to a model reasoning event being triggered, a first input data group is acquired; wherein the first input data group consists of at least one type of data;

[0011] Randomly remove at least part of the data from the first input data group to generate a second input data group;

[0012] Inputting the first input data group and the second input data group into a target model respectively, and obtaining a first probability distribution and a second probability distribution corresponding to the target model;

[0013] Based on the first probability distribution and the second probability distribution, a target probability distribution is obtained, and target output data is obtained based on the target probability distribution.

[0014] According to another aspect of the present invention, there is provided a large language model training device, comprising:

[0015] A training data set acquisition module is used to acquire a training data set; wherein the training data set includes at least two training sample data groups, and each training sample data group is composed of corresponding at least two types of data;

[0016] A target training data generating module is used to randomly extract a first training sample data group with a first preset ratio from the training data set, and randomly remove part of the data from the first training sample data group to generate target training data;

[0017] The target model generation module is used to train the large language model based on the target training data and determine whether the preset training conditions are met. If not, it returns to execute randomly extracting a first preset proportion of training sample data groups from the training data set until the preset training conditions are met to generate the target model.

[0018] According to another aspect of the present invention, there is provided a large language model reasoning device, comprising:

[0019] A first input data group acquisition module, configured to acquire a first input data group in response to a model reasoning event being triggered; wherein the first input data group is composed of at least one type of data;

[0020] A second input data group generating module, used for randomly removing at least part of the data from the first input data group to generate a second input data group;

[0021] A probability distribution acquisition module, used to input the first input data group and the second input data group into a target model respectively, and acquire a first probability distribution and a second probability distribution corresponding to the target model;

[0022] A target output data acquisition module is used to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and to acquire target output data based on the target probability distribution.

[0023] According to another aspect of the present invention, there is provided an electronic device, the electronic device comprising:

[0024] at least one processor; and

[0025] a memory communicatively connected to the at least one processor; wherein,

[0026] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the large language model training method or the large language model inference method described in any embodiment of the present invention.

[0027] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the large language model training method or the large language model inference method described in any embodiment of the present invention when executed.

[0028] The technical solution of the embodiment of the present invention includes: obtaining a training data set; wherein the training data set includes at least two training sample data groups, each of which is composed of at least two corresponding data; extracting at least one first training sample data group from the training data set in a preset order or randomly, and randomly removing at least part of the data from the first training sample data group to generate target training data; training a large language model based on the training sample data group and the target training data in the training data set until the preset training conditions are met to generate a target model. The technical solution provided by the embodiment of the present invention not only enables the large language model to output correct reasoning results when there are omissions or missing input data, thereby improving the applicability of the large language model, but also makes the output results more stable when normal input data is input.

[0029] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0031] Figure 1 A flowchart of a large language model training method provided in Embodiment 1 of the present invention;

[0032] Figure 2 A flowchart of a large language model inference method provided in Embodiment 2 of the present invention;

[0033] Figure 3 A schematic diagram of the structure of a large language model training device provided in Embodiment 3 of the present invention;

[0034] Figure 4 A schematic diagram of the structure of a large language model inference device provided in Embodiment 4 of the present invention;

[0035] Figure 5 A schematic diagram of the structure of an electronic device for implementing the large language model training method or large language model inference method of an embodiment of the present invention. DETAILED DESCRIPTION

[0036] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0037] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment 1

[0039] Figure 1 A flowchart of a large language model training method is provided for the first embodiment of the present invention. This embodiment is applicable to the case of training a large language model. The method can be performed by a large language model training device. The large language model training device can be implemented in the form of hardware and / or software. The large language model training device can be configured in an electronic device. Figure 1 As shown, the method includes:

[0040] S110, obtaining a training data set; wherein the training data set includes at least two training sample data groups, and each of the training sample data groups consists of at least two corresponding data.

[0041] In an embodiment of the present invention, a training data set is obtained, wherein the training data set includes a plurality of training sample data groups, and each training sample data group is composed of at least two corresponding data. For example, the training sample data group may include input data and fitting data; for another example, the training sample data group may include reference data, input data, and fitting data. Among them, the input data is data that needs to output corresponding generated content according to the input content when the large language model is subsequently trained, such as questions in question-and-answer data, and lyrics in music generation data; the reference data is data that needs to refer to part of the information in the subsequent large language model training to output corresponding generated content, such as reference pictures in literary image data, reference songs in music generation data, or reference accompaniment in reference songs, reference singing style and other reference data, which are used to enable the large language model to generate output data with the same style; the fitting data is standard answer data, which is used to compare with the output result of the large language model in the subsequent large language model training, calculate the loss value, and thus correct the probability of the next output result of the large model, which is the ideal data that the large language model needs to fit.

[0042] It should be noted that the embodiments of the present invention do not limit the number of input data and reference data contained in the training sample data set. For example, when training a large language model for music generation, the input data in the training sample data set can be a paragraph of lyrics or multiple paragraphs of lyrics, and the reference data in the training sample data set can be a reference song or a reference accompaniment and reference singing style in the reference song.

[0043] S120: Select at least one first training sample data group from the training data set in a preset order or randomly, and randomly remove at least part of the data from the first training sample data group to generate target training data.

[0044] In an embodiment of the present invention, at least one first training sample data group is extracted from at least two training sample data groups in the training data set in a preset order or randomly. Exemplarily, group numbers may be added to the respective training sample data groups in the training data set, and at least one first training sample data group is vertically extracted from the training data set in a sequence of group numbers from front to back or in a manner in which the group numbers are odd or even.

[0045] Optionally, extracting at least one first training sample data group from the training data set according to a preset order or randomly includes: extracting at least one training sample data group from the training data set according to a preset order or randomly as the first training sample data group; and / or, extracting at least one training sample data group of at least one type of data from the training data set according to a preset order or randomly as the first training sample data group; and / or, extracting at least one training sample data group of at least part of the data content of at least one type of data from the training data set according to a preset order or randomly as the first training sample data group.

[0046] Exemplarily, from the multiple training sample data groups included in the training data set, a first preset ratio of training sample data groups is extracted in a preset order or randomly, and the extracted first preset ratio of training sample data groups is used as the first training sample data group. For example, from the 1000 training sample data groups included in the training data set, 40% of the training sample data groups are extracted in a preset order or randomly, that is, 400 training sample data groups are extracted in a preset order or randomly, and the extracted 400 training sample data groups are used as the first training sample data group. It can be understood that each training sample data group in the training data set is extracted as a whole, and the data types contained in each extracted training sample data group can be the same or different.

[0047] In another exemplary embodiment, the training data set includes M training sample data groups, each training sample data group consists of at least two corresponding data, and at least one data of a second preset ratio is randomly extracted from the training data set in a preset order or randomly, and at least one training sample data group from which at least one data is extracted is used as the first training sample data group, that is, a second preset ratio is extracted for at least one data in the training data set, and at least one training sample data group from which data is extracted is used as the first training sample data group. For example, each training sample data group includes input data, reference data, and fitting data, and 20% of reference data, 20% of input data, and 20% of fitting data can be extracted from the training data set. If only one type of data is extracted from each training sample data group, one type of data will be extracted from 60% of the training sample data groups in the training data set; if two types of data are extracted from each training sample data group, the proportion of the training sample data groups extracted from the training sample data set will be reduced. It should be noted that at least one training sample data group from which at least one type of data is extracted is used as a first training sample group.

[0048] In another exemplary embodiment, a third preset proportion of data content in at least one data is extracted from the training data set in a preset order or randomly, and at least one training sample data group from which data content is extracted is used as the first training sample data group. For example, each training sample data group includes input data, reference data, and fitting data, then 30% of the input data content can be extracted from each input data in the training data set, 30% of the reference data content can be extracted from each reference data, and 30% of the fitting data content can be extracted from each fitting data, and then each training sample data group from which data content is extracted is used as a first training sample group. For another example, 30% of the data content can also be extracted from each of any two of the input data, reference data, and fitting data in the training data set. For another example, 30% of the data content can also be extracted from any one of the input data, reference data, and fitting data in the training data set. It should be noted that each training sample data group from which any data content is extracted is used as a first training sample group.

[0049] In an embodiment of the present invention, at least part of the data is randomly removed from the first training sample data group, and the first training sample data group after at least part of the data is removed is used as the target training data. Optionally, randomly removing at least part of the data from the first training sample data group includes: randomly removing at least one type of data from the first training sample data group; and / or randomly removing at least part of the data content of at least one type of data from the first training sample data group.

[0050] Exemplarily, if the first training sample data group contains input data and fitting data, the fourth preset proportion of input data and / or fitting data in the first training sample data group can be randomly eliminated, and the first training sample data group after elimination is used as the target training data. Optionally, the data content of the fifth preset proportion in at least part of the input data and / or fitting data in the first training sample data group can be randomly eliminated, and the first training sample data group after elimination is used as the target training data. Optionally, the fourth preset proportion of input data and / or fitting data in the first training sample data group is randomly eliminated, and the fifth preset proportion of data content in at least part of the input data and / or fitting data in the first training sample data group is randomly eliminated, and the first training sample data group after elimination is used as the target training data. Exemplarily, the fourth preset proportion of input data content in at least part of the input data, and / or the fifth preset proportion of fitting data in the fitting data can be randomly eliminated. For example, the input data includes "Who are you?", "Where are you from?", "Where are you going?", and the fitting data includes "I am XXX", "I come from XXXX", "I go to XXXXX", and the fitting data corresponds to the input data one by one. Randomly eliminating the fourth preset proportion of the input data content to obtain input content such as "Who are you?", "Where are you going?", and the fitting data includes "I am XXX", "I come from XXXX", "I go to XXXXX". For another example, randomly eliminating the fourth preset proportion of the input content in at least part of the input data to obtain input content such as "You are?", "Where are you from?", "Where are you going?", and the fitting data includes "I am XXX", "I come from XXXX", "I go to XXXXX".

[0051] Exemplarily, if the first training sample data group includes input data, reference data and fitting data, the fourth preset ratio of input data and / or reference data and / or fitting data in the first training sample data group can be randomly eliminated, that is, at least one of the fourth preset ratio of input data, reference data and fitting data in the first training sample data group is randomly eliminated, and the first training sample data group after elimination is used as the target training data. Optionally, the fifth preset ratio of data content in at least part of the input data and / or reference data and / or fitting data in the first training sample data group can also be randomly eliminated, that is, the fifth preset ratio of data content in at least part of the input data, reference data and fitting data in the first training sample data group is randomly eliminated, and the first training sample data group after elimination is used as the target training data. Optionally, a fourth preset proportion of input data and / or reference data and / or fitting data in the first training sample data group is randomly eliminated, and a fifth preset proportion of data content in at least part of the input data and / or reference data and / or fitting data in the first training sample data group is randomly eliminated, and the first training sample data group after elimination is used as target training data.

[0052] Exemplarily, a fourth preset proportion of reference data and / or input data in the reference data and input data in the first training sample data group can also be randomly eliminated. For example, the reference data is a reference song, or the reference data is the accompaniment data and singing style data in the reference song, the input data is the lyrics data of mature music, and the fitting data is a mature song. The fourth preset proportion of reference data and / or input data is randomly eliminated, and the results are as follows: for example, 10% or 20% of the reference songs are eliminated, so that only the lyrics data and fitting data are input in 10% or 20% of the first training sample data group for subsequent large language model training; 10% or 20% of the lyrics data are eliminated, so that only the reference songs and fitting data are input in 10% or 20% of the first training sample data group for subsequent large language model training. Training; and / or, randomly eliminating at least part of the reference data and / or a fifth preset proportion of data content in the input data, such as eliminating 10% or 20% of the reference song, so that only 9 seconds of the reference song are input out of 10 seconds of the reference song for subsequent large language model training; eliminating 10% or 20% of the input lyrics data, so that 80% or 90% of the lyrics data are input for subsequent large language model training; the input data can be lyrics data of mature music, and the fitting data can be mature songs, so as to achieve the output of the correct large language model generation result for subsequent training in the absence of data input.

[0053] S130, performing model training on a large language model based on the training sample data set in the training data set and the target training data until a preset training condition is met, thereby generating a target model.

[0054] In an embodiment of the present invention, model training is performed on a large language model based on a training sample data group and target training data in a training data set until a preset training condition is met, and the large language model corresponding to the preset training condition is used as a target model. Among them, the preset training condition may include that the loss function value of the large language model is less than a preset threshold, or that the number of model training times for the large language model reaches a preset number threshold, or that the model training duration for the large language model exceeds a preset duration threshold. It should be noted that the embodiment of the present invention does not limit the application scenario of the target model. Exemplarily, the target model may be a music generation model, in which case each training sample data group in the training data set used to train the target model may include reference music, sample lyrics, and fitting music; for another example, the target model may be a text-image model, that is, a model that generates pictures based on text, in which case each training sample data group in the training data set used to train the target model may include sample text and fitting pictures.

[0055] Optionally, model training is performed on a large language model based on the training sample data group and the target training data in the training data set until a preset training condition is met and a target model is generated, including: before model training is performed on the large language model, target training data is obtained N times based on the training sample data group in the training data set, where N is an integer greater than or equal to 1, and model training is performed on the large language model based on the target training data obtained each time until the preset training condition is met and a target model is generated; or, when model training is performed on the large language model, new target training data is obtained in real time based on the training sample data group in the training data set, and model training is performed on the large language model based on the new target training data obtained in real time and the training sample data group until the preset training condition is met and a target model is generated; or, the large language model is trained based on the target training data, and whether the preset training condition is met is determined during the model training of the large language model, and if not, the target training data for the next round is obtained based on the training sample data group in the training data set, and model training is performed on the large language model based on the target training data for the next round until the preset training condition is met and a target model is generated.

[0056] Exemplarily, since the target training data is generated by extracting at least one first training sample data group from the training data set in a preset order or randomly, and randomly removing at least part of the data from the first training sample data group, before the large language model is trained, S120 may be executed N times to obtain N target training data, where N is an integer greater than or equal to 1, and the target training data obtained each time is different. The large language model is iteratively trained based on the target training data obtained each time until the preset training conditions are met to generate the target model.

[0057] As another example, when the large language model is being trained, new target training data can be acquired in real time based on the training sample data set in the training data set. That is, when the large language model is being trained, S120 is executed in real time to acquire new target training data, and the large language model is trained based on the new target training data and the training sample data set acquired in real time until the preset training conditions are met and the target model is generated.

[0058] As another example, S120 is executed once to obtain the corresponding target training data, the large language model is trained based on the target training data, and it is determined whether the trained large language model meets the preset training conditions. If not, the next round of target training data is obtained based on the training sample data group in the training data set, that is, S120 is executed again to obtain the next round of target training data, and the large language model is trained based on the next round of target training data, and this cycle is iterated until the preset training conditions are met and the target model is generated.

[0059] It is understandable that when training the large language model, the 1 to 10 training data are not randomly removed from 1 or 2 pieces of data, and then input into the large language model in sequence for training. Instead, the training data are input into the large language model in sequence or randomly. 1 to 10 training data may be input into the large language model for training multiple times, and randomly removed before input, that is, sometimes removed, sometimes not removed, sometimes part of the input data is removed, and sometimes part of the fitting data is removed, so as to continuously adjust the network parameters of the large language model until the preset training conditions are met and the target model is generated.

[0060] The large language model training method of the embodiment of the present invention includes: obtaining a training data set; wherein the training data set contains at least two training sample data groups, each of which is composed of at least two corresponding data; extracting at least one first training sample data group from the training data set in a preset order or randomly, and randomly removing at least part of the data from the first training sample data group to generate target training data; performing model training on the large language model based on the training sample data group and the target training data in the training data set until the preset training conditions are met to generate a target model. The technical solution provided by the embodiment of the present invention can not only enable the large language model to output correct reasoning results when there are omissions or missing input data, thereby improving the applicability of the large language model, but also make the output results more stable when normal input data is input.

[0061] Embodiment 2

[0062] Figure 2 A flowchart of a large language model inference method is provided for the second embodiment of the present invention. This embodiment is applicable to the case of reasoning based on a large language model. The method can be executed by a large language model inference device. The large language model inference device can be implemented in the form of hardware and / or software. The large language model inference device can be configured in an electronic device. Figure 2 As shown, the method includes:

[0063] S210. In response to a model reasoning event being triggered, obtain a first input data group; wherein the first input data group consists of at least one type of data.

[0064] In an embodiment of the present invention, when a model inference request input by a user is received, it is determined that a model inference event is triggered. In response to the model inference event being triggered, a first input data group is obtained. Among them, the first input data group is composed of at least one kind of data, such as the first input data group may include input data when performing model inference, and may also include input data and reference data when performing model inference. It can be understood that the first input data group is composed of all data input by the user or specified input. Exemplarily, taking the large language model as a music generation model as an example, the first input data group may include only target lyrics, or may include reference music and target lyrics. For example, if the first input data group includes reference music and target lyrics, a certain music input by the user may be used as reference music, a certain music selected from a music database may be used as reference music, or a certain music selected by default may be used as reference music. Among them, the reference music may be a song containing vocals and accompaniment, or may be light music containing only accompaniment, or may be a cappella music containing only vocals. The length of the reference music can be a preset length, or it can be greater than a preset length (such as 5 seconds or 10 seconds). The reference music contains a certain amount of style information. For example, the reference music is a piece of music that can reflect the type of music or all the instruments played or all the singing styles. Optionally, the target lyrics can be a piece of lyrics input by the user in text, or a piece of lyrics input by the user in voice, or lyrics composed of subtitle information or voice information in the input video. The target lyrics can be any lyrics content, and the target lyrics can be content of any length. It should be noted that the embodiment of the present invention does not limit the content and length of the target lyrics.

[0065] S220: Randomly remove at least part of the data from the first input data group to generate a second input data group.

[0066] In an embodiment of the present invention, at least part of the data is randomly removed from the first input data group, and the first input data group after removing at least part of the data is used as the second input data group. It can be understood that the second input data group is incomplete data with some data missing relative to the first input data group. Optionally, randomly removing at least part of the data from the first input data group to generate the second input data group includes: randomly removing at least one kind of data from the first input data group; and / or randomly removing at least part of the data content of at least one kind of data from the first input data group to generate the second input data group. Taking the example that the first input data group includes input data and reference data when performing model inference, the reference data and / or input data in the first input data group can be randomly removed as a whole, or part of the data content in the reference data and / or input data can be randomly removed.

[0067] It should be noted that there is no limit on the number of times S220 is executed, and it can be executed once to obtain one second input data, or it can be executed multiple times to obtain multiple second input data. Since the second input data group is an input data group generated by randomly removing at least part of the data from the first input data group, when S220 is executed multiple times, the second input data group generated each time is different, so that multiple different second input data groups can be obtained.

[0068] S230: Input the first input data group and the second input data group into a target model respectively, and obtain a first probability distribution and a second probability distribution corresponding to the target model.

[0069] In an embodiment of the present invention, a first input data group is input into the target model provided in the first embodiment of the present invention to obtain a first probability distribution corresponding to the target model. A second input data group is input into the target model provided in the first embodiment of the present invention to obtain a second probability distribution corresponding to the target model. It is understandable that the first probability distribution is the probability distribution corresponding to the token output by the target model after all the data input by the user or the specified input is input into the target model; the second probability distribution is the probability distribution corresponding to the token output by the target model after incomplete data with some missing data is input into the target model. It should be noted that when there are multiple different second input data, there will be corresponding multiple second probability distributions.

[0070] Optionally, the first input data group and the second input data group are respectively input into the target model to obtain a first probability distribution and a second probability distribution corresponding to the target model, including: generating at least one corresponding first encoding vector based on each data in the first input data group, and generating at least one corresponding second encoding vector based on each data in the second input data group; wherein the first encoding vector and the second encoding vector are both encoding vectors of at least one layer; respectively input the first encoding vector and the second encoding vector into the target model to obtain a first probability distribution and a second probability distribution corresponding to the target model.

[0071] In an embodiment of the present invention, for each type of data in the first input data group, at least one corresponding encoding vector of at least one layer is generated based on each type of data. For the convenience of description, the encoding vector corresponding to each type of data in the first input data group is referred to as a first encoding vector. For each type of data in the second input data group, at least one corresponding encoding vector of at least one layer is generated based on each type of data. For the convenience of description, the encoding vector corresponding to each type of data in the second input data group is referred to as a second encoding vector. An exemplary description is given by taking the example of generating at least one corresponding first encoding vector of at least one layer based on each type of data in the first input data group. Exemplarily, the first input data group includes reference data and input data input or specified by the user during model inference, and the reference data can be encoded into at least one first reference encoding vector of at least one layer, and the input data can be encoded into at least one first input encoding vector of at least one layer through an encoder.

[0072] For example, when the target model is a music generation model, the first input data may include reference music and target lyrics. In this case, the reference music as a whole may be encoded into at least one global vector of at least one layer by an encoder, and the global vector may be used as a first reference encoding vector. It is also possible to intercept music of a preset length (such as 5 seconds or 10 seconds, etc.) from the reference music, and the intercepted music of the preset length may be encoded into at least one first reference encoding vector of at least one layer by an encoder. In another exemplary embodiment, the reference music is split into music parts of at least one dimension, and each music part of each dimension is encoded into at least one first reference encoding vector of at least one layer by an encoder. For example, the reference music is split into an accompaniment part and a vocal part, and the accompaniment part is encoded by a MERT encoder to generate a corresponding first reference encoding vector, and the vocal part is encoded by a Mel encoder to generate a corresponding first reference encoding vector. The target lyrics are segmented based on a preset segmentation algorithm, and the target lyrics are divided into a plurality of segmentation units, wherein the segmentation unit may be each character, each word, or each sentence. Based on the encoder, each word segmentation unit in the target lyrics is encoded to generate a corresponding first input encoding vector of at least one layer. The more layers of the first input encoding vector, the more it can represent the richer information in the target lyrics, such as emotional information.

[0073] It should be noted that the method of generating at least one corresponding layer of the second encoding vector based on each type of data in the second input data group is similar to the method of generating the first encoding vector, which will not be repeated here.

[0074] Optionally, before respectively inputting the first coding vector and the second coding vector into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model, it also includes: determining a preset coding vector corresponding to at least part of the data randomly removed from the first input data group; inputting the second coding vector into the target model to obtain the second probability distribution corresponding to the target model, including: inputting the second coding vector and the preset coding vector into the target model to obtain the second probability distribution corresponding to the target model.

[0075] In an embodiment of the present invention, after randomly removing at least part of the data from the first input data group, a preset coding vector corresponding to at least part of the data randomly removed from the first input data is determined. For example, at least part of the data randomly removed from the first input data can be directly set to a special coding vector (such as 0 or 1). When the second coding vector corresponding to the second input data group is input into the target model, the second coding vector and the preset coding vector corresponding to at least part of the data randomly removed from the first input data are simultaneously input into the target model to obtain a second probability distribution corresponding to the token output by the target model.

[0076] S240. Obtain a target probability distribution based on the first probability distribution and the second probability distribution, and obtain target output data based on the target probability distribution.

[0077] In an embodiment of the present invention, a target probability distribution is obtained according to a first probability distribution and a second probability distribution. For example, the mean of the second probability distribution and the second probability distribution can be used as the target probability distribution. Optionally, based on the first probability distribution and the second probability distribution, the target probability distribution is obtained, including: determining the target probability distribution according to the following formula: Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond))*C; wherein Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, C represents a preset coefficient, and C is not 0. It should be noted that when the number of the second probability distributions is n, the target probability distribution can be determined according to the following formula: Log(As)=Log(As cond)+(n*Log(As cond)-Log1(As uncond)-Log2(Asuncond)-…-Log n (As uncond))*C.

[0078] In the embodiment of the present invention, the target probability distribution is superior to the distribution probability of the token obtained by the traditional single reasoning method, thereby increasing the stability of the target model output. Moreover, since the target model is a model generated by training the large language model after randomly eliminating the data in the training sample set, even if some data is randomly eliminated during large language reasoning, the target model can still obtain a relatively correct token probability distribution. In the embodiment of the present invention, the target output data inferred by the target model is obtained according to the target probability distribution.

[0079] Optionally, when performing large language model inference output, the target model can sample and output the next token based on the optimized target probability distribution of each token. The probability distribution of each token is the probability of each token being sampled as an output token. Optimizing the probability distribution through two inferences can greatly increase the probability of the correct token being sampled and reduce the probability of the incorrect token being sampled, thereby achieving the purpose of improving the accuracy and stability of large language model inference.

[0080] The large language model inference method provided by the embodiment of the present invention includes: in response to a model inference event being triggered, obtaining a first input data group; wherein the first input data group is composed of at least one type of data; randomly eliminating at least part of the data from the first input data group to generate a second input data group; respectively inputting the first input data group and the second input data group into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model; based on the first probability distribution and the second probability distribution, obtaining the target probability distribution, and obtaining the target output data based on the target probability distribution. Through the technical solution provided by the embodiment of the present invention, by respectively inputting the complete data input or specified by the user during model inference and the incomplete data with part of the data missing into the target model, the probability distribution of at least 2 target model output tokens is obtained, and based on the probability distribution of at least 2 target model output tokens, the optimized target probability distribution is obtained, and then more accurate and stable target output data can be obtained according to the target probability distribution.

[0081] Embodiment 3

[0082] Figure 3 This is a schematic diagram of the structure of a large language model training device provided in Embodiment 3 of the present invention. Figure 3 As shown, the device comprises:

[0083] The training data set acquisition module 310 is used to acquire a training data set; wherein the training data set includes at least two training sample data groups, and each training sample data group is composed of at least two corresponding data;

[0084] A target training data generating module 320 is used to randomly extract a first preset ratio of a first training sample data group from the training data set, and randomly remove part of the data from the first training sample data group to generate target training data;

[0085] The target model generation module 330 is used to train the large language model based on the target training data and determine whether the preset training conditions are met. If not, it returns to execute randomly extracting a first preset proportion of training sample data groups from the training data set until the preset training conditions are met to generate the target model.

[0086] Optionally, the target training data generating module is used to:

[0087] Randomly remove at least one type of data from the first training sample data group; and / or,

[0088] At least part of the data content of at least one type of data is randomly eliminated from the first training sample data group.

[0089] Optionally, the target model generating module is used to:

[0090] Before performing model training on the large language model, target training data is obtained N times based on the training sample data group in the training data set, where N is an integer greater than or equal to 1, and model training is performed on the large language model based on the target training data obtained each time until a preset training condition is met to generate a target model; or,

[0091] When the large language model is trained, new target training data is acquired in real time based on the training sample data set in the training data set, and the large language model is trained based on the new target training data acquired in real time and the training sample data set until a preset training condition is met to generate a target model; or

[0092] The large language model is trained based on the target training data, and whether a preset training condition is met during the model training of the large language model. If not, the next round of target training data is obtained based on the training sample data group in the training data set, and the large language model is trained based on the next round of target training data until the preset training condition is met to generate a target model.

[0093] Optionally, the target training data generating module is used to:

[0094] At least one training sample data group is selected from the training data set in a preset order or randomly as a first training sample data group; and / or,

[0095] At least one training sample data set of at least one type of data is selected from the training data set in a preset order or randomly as a first training sample data set; and / or,

[0096] At least one training sample data group containing at least a portion of data content in at least one type of data is extracted from the training data set in a preset order or randomly as a first training sample data group.

[0097] The large language model training device provided in the embodiment of the present invention can execute the large language model training method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0098] Embodiment 4

[0099] Figure 4 This is a schematic diagram of the structure of a large language model inference device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device comprises:

[0100] A first input data group acquisition module 410 is used to acquire a first input data group in response to a model reasoning event being triggered; wherein the first input data group is composed of at least one type of data;

[0101] A second input data group generating module 420, configured to randomly remove at least part of the data from the first input data group to generate a second input data group;

[0102] A probability distribution acquisition module 430, used to input the first input data group and the second input data group into a target model respectively, and acquire a first probability distribution and a second probability distribution corresponding to the target model;

[0103] The target output data acquisition module 440 is used to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and acquire target output data based on the target probability distribution.

[0104] Optionally, the target output data acquisition module is used to:

[0105] The target probability distribution is determined according to the following formula:

[0106] Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond))*C;

[0107] Among them, Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, and C represents a preset coefficient, which is not 0.

[0108] Optionally, the probability distribution acquisition module includes:

[0109] A coding vector generating unit, configured to generate at least one first coding vector corresponding to each data in the first input data group, and to generate at least one second coding vector corresponding to each data in the second input data group; wherein the first coding vector and the second coding vector are both coding vectors of at least one layer;

[0110] The probability distribution acquisition unit is used to input the first encoding vector and the second encoding vector into the target model respectively to obtain the first probability distribution and the second probability distribution corresponding to the target model.

[0111] Optionally, the device further comprises:

[0112] a preset coding vector determining unit, configured to determine a preset coding vector corresponding to at least part of the data randomly removed from the first input data group before respectively inputting the first coding vector and the second coding vector into a target model to obtain a first probability distribution and a second probability distribution corresponding to the target model;

[0113] The probability distribution acquisition unit is used to:

[0114] The second encoding vector and the preset encoding vector are input into a target model to obtain a second probability distribution corresponding to the target model.

[0115] Optionally, the second input data group generating module is used to:

[0116] Randomly remove at least one type of data from the first input data group; and / or randomly remove at least part of the data content of at least one type of data from the first input data group to generate the second input data group.

[0117] The large language model inference device provided in the embodiment of the present invention can execute the large language model inference method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0118] Embodiment 5

[0119] Figure 5A schematic diagram of the structure of an electronic device 10 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0120] like Figure 5 As shown, the electronic device 10 includes at least one processor 11, and a memory connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 12 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] A number of components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0122] The processor 11 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The processor 11 executes the various methods and processes described above, such as a large language model training method or a large language model reasoning method.

[0123] In some embodiments, the large language model training method or the large language model reasoning method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the large language model training method or the large language model reasoning method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to execute the large language model training method or the large language model reasoning method in any other appropriate manner (e.g., by means of firmware).

[0124] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0125] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0126] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0128] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0130] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0131] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A large language model training method, characterized in that: include: Acquire a training data set; wherein the training data set includes at least two training sample data groups, and each of the training sample data groups is composed of corresponding at least two types of data; Selecting at least one first training sample data group from the training data set in a preset order or randomly, and randomly removing at least part of the data from the first training sample data group to generate target training data; Model training is performed on the large language model based on the training sample data group in the training data set and the target training data until preset training conditions are met, thereby generating a target model.

2. The method according to claim 1, characterized in that Randomly removing at least part of the data from the first training sample data group comprises: Randomly remove at least one type of data from the first training sample data group; and / or, At least part of the data content of at least one type of data is randomly eliminated from the first training sample data group.

3. The method according to claim 1, characterized in that The large language model is trained based on the training sample data set and the target training data in the training data set until a preset training condition is met to generate a target model, including: Before performing model training on the large language model, target training data is obtained N times based on the training sample data group in the training data set, where N is an integer greater than or equal to 1, and model training is performed on the large language model based on the target training data obtained each time until a preset training condition is met to generate a target model; or, When the large language model is trained, new target training data is acquired in real time based on the training sample data set in the training data set, and the large language model is trained based on the new target training data acquired in real time and the training sample data set until a preset training condition is met to generate a target model; or The large language model is trained based on the target training data, and whether a preset training condition is met during the model training of the large language model. If not, the next round of target training data is obtained based on the training sample data group in the training data set, and the large language model is trained based on the next round of target training data until the preset training condition is met to generate a target model.

4. The method according to any one of claims 1 to 3, characterized in that: At least one first training sample data group is extracted from the training data set in a preset order or randomly, comprising: At least one training sample data group is selected from the training data set in a preset order or randomly as a first training sample data group; and / or, At least one training sample data set of at least one type of data is selected from the training data set in a preset order or randomly as a first training sample data set; and / or, At least one training sample data group containing at least a portion of data content in at least one type of data is extracted from the training data set in a preset order or randomly as a first training sample data group.

5. A large language model reasoning method, characterized in that: include: In response to a model reasoning event being triggered, a first input data group is acquired; wherein the first input data group consists of at least one type of data; Randomly remove at least part of the data from the first input data group to generate a second input data group; Inputting the first input data group and the second input data group into a target model respectively, and obtaining a first probability distribution and a second probability distribution corresponding to the target model; Based on the first probability distribution and the second probability distribution, a target probability distribution is obtained, and target output data is obtained based on the target probability distribution.

6. The method according to claim 5, characterized in that Acquiring a target probability distribution based on the first probability distribution and the second probability distribution includes: The target probability distribution is determined according to the following formula: Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond))*C; Among them, Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(Asuncond) represents the second probability distribution, and C represents a preset coefficient, which is not 0.

7. The method according to claim 5, characterized in that Inputting the first input data group and the second input data group into a target model respectively, and obtaining a first probability distribution and a second probability distribution corresponding to the target model, comprises: Generate at least one first encoding vector corresponding to each type of data in the first input data group, and generate at least one second encoding vector corresponding to each type of data in the second input data group; wherein the first encoding vector and the second encoding vector are both encoding vectors of at least one layer; The first encoding vector and the second encoding vector are respectively input into a target model to obtain a first probability distribution and a second probability distribution corresponding to the target model.

8. The method according to claim 7, characterized in that Before respectively inputting the first encoding vector and the second encoding vector into a target model to obtain a first probability distribution and a second probability distribution corresponding to the target model, the method further includes: Determining a preset encoding vector corresponding to at least a portion of the data randomly removed from the first input data group; Inputting the second encoding vector into a target model to obtain a second probability distribution corresponding to the target model includes: The second encoding vector and the preset encoding vector are input into a target model to obtain a second probability distribution corresponding to the target model.

9. The method according to claim 5, characterized in that Randomly removing at least part of the data from the first input data group to generate a second input data group includes: Randomly remove at least one type of data from the first input data group; and / or randomly remove at least part of the data content of at least one type of data from the first input data group to generate the second input data group.

10. A large language model training device, characterized in that: include: A training data set acquisition module is used to acquire a training data set; wherein the training data set includes at least two training sample data groups, and each training sample data group is composed of corresponding at least two types of data; A target training data generating module is used to randomly extract a first training sample data group with a first preset ratio from the training data set, and randomly remove part of the data from the first training sample data group to generate target training data; The target model generation module is used to train the large language model based on the target training data and determine whether the preset training conditions are met. If not, it returns to execute randomly extracting a first preset proportion of training sample data groups from the training data set until the preset training conditions are met to generate the target model.

11. A large language model inference device, characterized in that: include: A first input data group acquisition module, configured to acquire a first input data group in response to a model reasoning event being triggered; wherein the first input data group is composed of at least one type of data; A second input data group generating module, used for randomly removing at least part of the data from the first input data group to generate a second input data group; A probability distribution acquisition module, used to input the first input data group and the second input data group into a target model respectively, and acquire a first probability distribution and a second probability distribution corresponding to the target model; A target output data acquisition module is used to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and to acquire target output data based on the target probability distribution.

Images