Large language model training and reasoning method and device

Abstract

The application discloses a large language model training and reasoning method and device. The large language model training method comprises the following steps: acquiring a training data set; wherein the training data set comprises at least two training sample data groups, and each training sample data group is composed of at least two corresponding data; at least one first training sample data group is extracted from the training data set according to a preset order or randomly, and at least part of data in the first training sample data group is randomly removed to generate target training data; a large language model is trained based on the training sample data groups in the training data set and the target training data until a preset training condition is met, and a target model is generated. According to the scheme, the large language model can still output correct reasoning results when facing input data with omissions or missing, the applicability of the large language model is improved, and the output result is more stable when normal input data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of machine learning, in particular to a large language model training and reasoning method and device. BACKGROUND

[0002] The existing large model training generally inputs training data into the large model, so that the output result of the large model is constantly fitted with the standard data in the training sample, thereby achieving the model training effect. In the model training process, the probability of the next output result being a specific value is calculated by the large model based on all the previous input information (including the previously input standard data); in the model reasoning process, the probability of the next output result being a specific value is calculated by the large model based on all the previous input information and the output token information. As can be seen, the output result of the large model is determined by the probability, and when the input information is missing, the output result of the model may be completely different.

[0003] For example, the model needs to input reference accompaniment, reference singing style, and lyrics content to generate a corresponding music song, but in the actual reasoning process, the user only inputs the reference accompaniment and lyrics content, and does not input the reference singing style or inputs the reference singing style but the input singing style is invalid or fails to be extracted; or, part of the data in the user input data is missing, for example, part of the data in the user input reference accompaniment, reference singing style, and lyrics content is missing or lost, at this time the model can not output or the output content deviates greatly from the user request. In addition, even if the input data of the model is completely the same, the output result of the model may also deviate greatly each time, so that the output of the model is not stable enough. Therefore, how to make the large model correctly output when facing missing input data and output more stably when the input data is normal becomes crucial. SUMMARY

[0004] The present application provides a large language model training and reasoning method and device, which can not only make the large language model output correct reasoning results when facing missing input data, thereby improving the applicability of the large language model, but also output more stable results when the input data is normal.

[0005] According to an aspect of the present application, a large language model training method is provided, comprising:

[0006] obtaining a training data set; wherein the training data set contains at least two training sample data groups, and each training sample data group is composed of at least two corresponding data;

[0007] extract at least one first training sample data set from the training data set according to a preset order or randomly, and randomly remove at least part of data from the first training sample data set to generate target training data;

[0008] perform model training on a large language model based on the training sample data set and the target training data in the training data set until a preset training condition is met to generate a target model.

[0009] According to another aspect of the present application, a large language model inference method is provided, comprising:

[0010] In response to a model inference event being triggered, a first input data set is obtained; wherein the first input data set is composed of at least one kind of data;

[0011] At least part of data is randomly removed from the first input data set to generate a second input data set;

[0012] The first input data set and the second input data set are respectively input into a target model to obtain a first probability distribution and a second probability distribution corresponding to the target model;

[0013] Based on the first probability distribution and the second probability distribution, a target probability distribution is obtained, and based on the target probability distribution, target output data is obtained.

[0014] According to another aspect of the present application, a large language model training device is provided, comprising:

[0015] A training data set acquisition module is configured to acquire a training data set; wherein the training data set contains at least two training sample data sets, and each training sample data set is composed of at least two corresponding kinds of data;

[0016] A target training data generation module is configured to extract a first training sample data set of a first preset proportion from the training data set randomly, and remove part of data from the first training sample data set to generate target training data;

[0017] A target model generation module is configured to train a large language model based on the target training data, and determine whether a preset training condition is met; if not, return to execute extraction of a training sample data set of a first preset proportion from the training data set until the preset training condition is met to generate a target model.

[0018] According to another aspect of the present application, a large language model inference device is provided, comprising:

[0019] The first input data set acquisition module is configured to acquire a first input data set in response to a model inference event being triggered, wherein the first input data set is composed of at least one data.

[0020] The second input data set generation module is configured to generate a second input data set by randomly removing at least part of data from the first input data set.

[0021] The probability distribution acquisition module is configured to input the first input data set and the second input data set into a target model respectively, and acquire a first probability distribution and a second probability distribution corresponding to the target model.

[0022] The target output data acquisition module is configured to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and acquire target output data based on the target probability distribution.

[0023] According to another aspect of the present application, an electronic device is provided, which comprises:

[0024] at least one processor; and

[0025] a memory connected to the at least one processor in communication; wherein

[0026] The memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the large language model training method or the large language model inference method according to any one of the embodiments of the present application.

[0027] According to another aspect of the present application, a computer readable storage medium is provided, which stores computer instructions for enabling a processor to implement the large language model training method or the large language model inference method according to any one of the embodiments of the present application when executed by the processor.

[0028] The technical scheme of the embodiment of the present application comprises: obtaining a training data set; wherein the training data set comprises at least two training sample data groups, each of the training sample data groups is composed of at least two corresponding data; at least one first training sample data group is extracted from the training data set according to a preset order or randomly, and at least part of data in the first training sample data group is randomly removed to generate target training data; a large language model is trained based on the training sample data groups in the training data set and the target training data until a preset training condition is met, and a target model is generated. Through the technical scheme provided by the embodiment of the present application, the large language model can still output correct reasoning results when facing input data with omissions or missing, and the applicability of the large language model is improved, and the output result is more stable when normal input data is input.

[0029] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present application, nor is it used to limit the scope of the present application. Other features of the present application will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0031] Figure 1 A flowchart of a large language model training method provided by the embodiment one of the present application;

[0032] Figure 2 A flowchart of a large language model reasoning method provided by the embodiment two of the present application;

[0033] Figure 3 A structural schematic diagram of a large language model training device provided by the embodiment three of the present application;

[0034] Figure 4 A structural schematic diagram of a large language model reasoning device provided by the embodiment four of the present application;

[0035] Figure 5 A structural schematic diagram of an electronic device for implementing the large language model training method or the large language model reasoning method of the present application. DETAILED DESCRIPTION

[0036] In the following, the technical solutions in the embodiments of the present application will be described clearly and completely with reference to the drawings in the embodiments of the present application in order to make the technical personnel in the technical field better understand the technical solutions. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by the ordinary skilled in the art without creative work should belong to the scope of protection of the present application.

[0037] It should be noted that the terms "first", "second" and the like in the specification and claims of the present application and the above-described drawings are used to distinguish similar objects, and do not necessarily have to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged under appropriate circumstances, so that the embodiments of the application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "include" and "have" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, system, product or device that includes a series of steps or units does not have to be limited to those steps or units clearly listed, but can include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0038] Embodiment one

[0039] Figure 1 A flowchart of a large language model training method is provided for the first embodiment of the present application. The present embodiment can be applicable to the training of a large language model. The method can be executed by a large language model training device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in the figure, the method comprises: Figure 1

[0040] S110, obtaining a training data set; wherein the training data set comprises at least two training sample data groups, and each training sample data group is composed of at least two corresponding data.

[0041] ​In this embodiment of the invention, a training dataset is obtained, which contains multiple training sample data groups, each consisting of at least two types of data. For example, a training sample data group may include input data and fitted data; or, alternatively, a training sample data group may include reference data, input data, and fitted data. Input data refers to the data that needs to be used to output corresponding generated content based on the input content during subsequent large language model training, such as questions in question-and-answer data or lyrics in music generation data. Reference data refers to the data that needs to be used to output corresponding generated content based on some information during subsequent large language model training, such as reference images in text-to-image data, reference songs in music generation data, or reference accompaniment, reference singing style, etc., in reference songs, used to ensure that the large language model generates output data with the same style. Fitted data is standard answer data, used to compare with the output results of the large language model during subsequent large language model training, calculate the loss value, and thus correct the probability of the next output result of the large model; it is the ideal data that the large language model needs to fit.

[0042] It should be noted that the embodiments of the present invention do not limit the number of input data and the number of reference data included in the training sample data set. For example, when training a large language model for music generation, the input data in the training sample data set can be a piece of lyrics or multiple pieces of lyrics, and the reference data in the training sample data set can be a reference song, or reference accompaniment and reference singing style in a reference song, etc.

[0043] S120. At least one first training sample data group is randomly selected from the training dataset according to a preset order, and at least some data is randomly removed from the first training sample data group to generate target training data.

[0044] In this embodiment of the invention, at least one first training sample data group is selected from at least two training sample data groups in the training dataset, either in a preset order or randomly. For example, group numbers can be added to each training sample data group in the training dataset, and at least one first training sample data group can be vertically selected from the training dataset according to the order of the group numbers from front to back or according to whether the group numbers are odd or even.

[0045] Optionally, the at least one first training sample data set is extracted from the training data set according to a preset order or randomly, including: extracting at least one training sample data set from the training data set as the first training sample data set according to a preset order or randomly; and / or extracting at least one training sample data set of at least one kind of data from the training data set as the first training sample data set according to a preset order or randomly; and / or extracting at least one training sample data set of at least part of data content of at least one kind of data from the training data set as the first training sample data set according to a preset order or randomly.

[0046] For example, 40% of the training sample data sets in the training data set are extracted according to a preset order or randomly, i.e., 400 training sample data sets are extracted according to a preset order or randomly, and the 400 extracted training sample data sets are taken as the first training sample data set. It can be understood that each training sample data set in the training data set is extracted as a whole, and the data types contained in each extracted training sample data set can be the same or different.

[0047] For example, each training sample data set contains input data, reference data and fitting data, 20% of the reference data, 20% of the input data and 20% of the fitting data can be extracted from the training data set, if only one kind of data is extracted from each training sample data set, then one kind of data is extracted from 60% of the training sample data sets in the training data set; if two kinds of data are extracted from each training sample data set, then the proportion of the extracted training sample data sets from the training sample data set will be reduced. It should be noted that at least one training sample data set of at least one kind of data is taken as a first training sample set.

[0048] For example, each training sample data set contains input data, reference data and fitting data, 30% of input data content can be extracted from each input data in the training data set, 30% of reference data content can be extracted from each reference data, and 30% of fitting data content can be extracted from each fitting data, and each training sample data set extracted from data content is taken as a first training sample set. For another example, 30% of data content can be extracted from each data in any two kinds of data of input data, reference data and fitting data in the training data set. For another example, 30% of data content can be extracted from each data in any one kind of data of input data, reference data and fitting data in the training data set. It should be noted that each training sample data set extracted from data content is taken as a first training sample set.

[0049] In the embodiment of the present application, at least part of data in the first training sample data set is randomly removed, and the first training sample data set after removing at least part of data is taken as target training data. Optionally, the removing at least part of data from the first training sample data set comprises: removing at least one kind of data from the first training sample data set; and / or removing at least part of data content in at least one kind of data from the first training sample data set.

[0050] For example, the first training sample data set includes input data and fitting data, and the fourth preset proportion of the input data and / or the fitting data in the first training sample data set can be randomly removed, and the first training sample data set after the removal can be used as the target training data. Alternatively, the fifth preset proportion of the data content in at least part of the input data and / or the fitting data in the first training sample data set can be randomly removed, and the first training sample data set after the removal can be used as the target training data. Alternatively, the fourth preset proportion of the input data and / or the fitting data in the first training sample data set can be randomly removed, and the fifth preset proportion of the data content in at least part of the input data and / or the fitting data in the first training sample data set can be randomly removed, and the first training sample data set after the removal can be used as the target training data. For example, the fourth preset proportion of the input data content in at least part of the input data and / or the fifth preset proportion of the fitting data can be randomly removed. For example, the input data includes "Who are you?", "Where do you come from?", and "Where do you go?", and the fitting data includes "I am XXX", "XXXX come", and "go to XXXX", and the fitting data corresponds to the input data one by one. The fourth preset proportion of the input data content is randomly removed to obtain input content such as "Who are you?", "Where do you go?", and the fitting data includes "I am XXX", "XXXX come", and "go to XXXX". For another example, the fourth preset proportion of the input content in at least part of the input data is randomly removed to obtain input content such as "Who are you?", "Where do you come from?", and "Where do you go?", and the fitting data includes "I am XXX", "XXXX come", and "go to XXXX".

[0051] For example, the first training sample data set includes input data, reference data and fitting data. The fourth preset proportion of the input data and / or the reference data and / or the fitting data in the first training sample data set can be randomly removed, that is, at least one of the fourth preset proportion of the input data, the reference data and the fitting data in the first training sample data set is randomly removed, and the first training sample data set after removal is used as target training data. Optionally, the fifth preset proportion of the data content in at least part of the input data and / or the reference data and / or the fitting data in the first training sample data set can be randomly removed, that is, the fifth preset proportion of the data content in at least one of the input data, the reference data and the fitting data in the first training sample data set is randomly removed, and the first training sample data set after removal is used as target training data. Optionally, the fourth preset proportion of the input data and / or the reference data and / or the fitting data in the first training sample data set is randomly removed, and the fifth preset proportion of the data content in at least part of the input data and / or the reference data and / or the fitting data in the first training sample data set is randomly removed, and the first training sample data set after removal is used as target training data.

[0052] For example, the fourth preset proportion of the reference data and / or the input data in the reference data and the input data in the first training sample data set can be randomly removed. For example, the reference data is a reference song, or the reference data is accompaniment data and singing style data in the reference song, the input data is lyrics data of mature music, and the fitting data is a mature song. The fourth preset proportion of the reference data and / or the input data is randomly removed to obtain, for example, 10% or 20% of the reference song, so that 10% or 20% of the first training sample data set only inputs lyrics data and fitting data for subsequent large language model training; 10% or 20% of the lyrics data is removed, so that 10% or 20% of the first training sample data set only inputs reference songs and fitting data for subsequent large language model training; and / or, the fifth preset proportion of the data content in at least part of the reference data and / or the input data is randomly removed, for example, 10% or 20% of the reference song is removed, so that 10 seconds of reference song only inputs 9 seconds of reference song for subsequent large language model training; 10% or 20% of the input lyrics data is removed, so that 80% or 90% of the lyrics data is input for subsequent large language model training; the input data can be lyrics data of mature music, and the fitting data is a mature song, so that the correct large language model generation result can be output in the case of missing data input.

[0053] In S130, a large language model is trained based on the training sample data set and the target training data in the training data set until a preset training condition is met, and a target model is generated.

[0054] In the embodiment of the present application, the large language model is trained based on the training sample data sets and the target training data in the training data set until the preset training condition is met, and the large language model corresponding to the time when the preset training condition is met is taken as the target model. The preset training condition can include that the loss function value of the large language model is less than a preset threshold, or the number of model training of the large language model reaches a preset number threshold, or the model training time of the large language model exceeds a preset time threshold. It should be noted that the application scenarios of the target model are not limited in the embodiment of the present application. For example, the target model can be a music generation model, at this time, each training sample data set in the training data set used to train the target model can include reference music, sample lyrics and fitted music; for another example, the target model can be a text-to-image model, that is, a model for generating pictures according to text, at this time, each training sample data set in the training data set used to train the target model can include sample text and fitted picture.

[0055] Optionally, the large language model is trained based on the training sample data sets and the target training data in the training data set until the preset training condition is met, and the target model is generated, including: before training the large language model, N times of target training data are obtained based on the training sample data sets in the training data set, N is an integer greater than or equal to 1, and the large language model is trained based on each obtained target training data until the preset training condition is met, and the target model is generated; or, when the large language model is trained, new target training data are obtained in real time based on the training sample data sets in the training data set, and the large language model is trained based on the real-time obtained new target training data and the training sample data set until the preset training condition is met, and the target model is generated; or, the large language model is trained based on the target training data, and whether the preset training condition is met is determined during the training of the large language model, if not, the next round of target training data is obtained based on the training sample data sets in the training data set, and the large language model is trained based on the next round of target training data until the preset training condition is met, and the target model is generated.

[0056] Exemplarily, since the target training data is generated by extracting at least one first training sample data set from the training data set according to a preset order or randomly, and randomly removing at least part of the data from the first training sample data set, before the model training of the large language model, S120 can be performed N times to obtain N times of target training data, N is an integer greater than or equal to 1, wherein the target training data obtained each time is different. The large language model is iteratively trained based on the target training data obtained each time until the preset training condition is met, and the target model is generated.

[0057] Exemplarily, the new target training data can also be obtained in real time based on the training sample data set in the training data set during the model training of the large language model, that is, S120 is executed in real time to obtain new target training data during the model training of the large language model, and the large language model is trained based on the real-time obtained new target training data and the training sample data set until the preset training condition is met, and the target model is generated.

[0058] Exemplarily, S120 is executed once to obtain corresponding target training data, the large language model is trained based on the target training data, and it is judged whether the trained large language model meets the preset training condition. If not, the next round of target training data is obtained based on the training sample data set in the training data set, that is, S120 is executed again to obtain the next round of target training data, and the large language model is trained based on the next round of target training data. This cycle is iterated until the preset training condition is met, and the target model is generated.

[0059] It can be understood that during the training of the large language model, not 1-10 training data are input into the large language model for training after 1 or 2 pieces of data are randomly removed, but the training data are input into the large language model in sequence or randomly, and 1-10 training data can be input into the large language model for training multiple times. Before input, randomly remove, that is, sometimes remove, sometimes do not remove, sometimes remove part of the input data, and sometimes remove part of the fitting data, so as to continuously adjust the network parameters of the large language model until the preset training condition is met, and the target model is generated.

[0060] The method for training a large language model according to the embodiment of the present application comprises: obtaining a training data set; wherein the training data set comprises at least two training sample data groups, and each training sample data group is composed of at least two corresponding data; at least one first training sample data group is extracted from the training data set according to a preset order or randomly, and at least part of the data in the first training sample data group is randomly removed to generate target training data; the large language model is trained based on the training sample data groups in the training data set and the target training data until a preset training condition is met, and a target model is generated. Through the technical scheme provided by the embodiment of the present application, the large language model can still output correct reasoning results when facing input data with omissions or missing, and the applicability of the large language model is improved, and the output results are more stable when normal input data is input.

[0061] Embodiment two

[0062] Figure 2 A flowchart of a large language model reasoning method is provided for the second embodiment of the present application. The present embodiment can be applied to the case of reasoning based on a large language model. The method can be executed by a large language model reasoning device, which can be realized in the form of hardware and / or software, and can be configured in an electronic device. As shown in Figure 2 The method comprises the following steps:

[0063] In S210, in response to a model reasoning event being triggered, a first input data group is obtained; wherein the first input data group is composed of at least one data.

[0064] In the embodiment of the present application, when a model inference request input by a user is received, it is determined that a model inference event is triggered. In response to the model inference event being triggered, a first input data set is obtained. The first input data set is composed of at least one data, for example, the first input data set can include input data for model inference, or can include input data and reference data for model inference. It can be understood that the first input data set is composed of all data input or specified by the user. For example, a large language model is a music generation model, the first input data set can only include target lyrics, or can include reference music and target lyrics. For example, the first input data set includes reference music and target lyrics, the user input music can be used as reference music, or a music selected from a music database can be used as reference music, or a default music can be used as reference music. The reference music can be a song containing vocals and accompaniment, or can be light music containing only accompaniment, or can be a cappella music containing only vocals. The length of the reference music can be a preset length, or can be greater than the preset length (such as 5 seconds or 10 seconds), and the reference music contains a certain amount of style information, for example, the reference music is a piece of music that can reflect the music type or all playing instruments or all singing styles. Optionally, the target lyrics can be a piece of lyrics input by the user in text, or can be a piece of lyrics input by the user in voice, or can be lyrics composed of subtitle information or voice information in the input video. The target lyrics can be any lyrics content, and the target lyrics can be any length of content. It should be noted that the embodiment of the present application does not limit the content and length of the target lyrics.

[0065] S220, randomly removing at least part of the data from the first input data set to generate a second input data set.

[0066] In the embodiment of the present application, at least part of the data in the first input data set is randomly removed, and the first input data set after removing at least part of the data is used as the second input data set. It can be understood that the second input data set is incomplete data that is missing part of the data relative to the first input data set. Optionally, the second input data set is generated by randomly removing at least part of the data from the first input data set, including: randomly removing at least one data from the first input data set; and / or, randomly removing at least part of the data content in the at least one data from the first input data set to generate the second input data set. For example, the first input data set includes input data and reference data for model inference, then the reference data and / or the input data in the first input data set can be randomly removed as a whole, or part of the data content in the reference data and / or the input data can be randomly removed.

[0067] It should be noted that the number of times of performing S220 is not limited, and S220 can be performed once to obtain one second input data, or S220 can be performed multiple times to obtain multiple second input data. Since the second input data set is generated by randomly removing at least part of the data from the first input data set, when S220 is performed multiple times, the second input data set generated each time is different, so that multiple different second input data sets can be obtained.

[0068] S230, respectively, input the first input data set and the second input data set into the target model to obtain a first probability distribution and a second probability distribution corresponding to the target model.

[0069] In the embodiment of the present application, the first input data set is input into the target model provided in the first embodiment of the present application to obtain a first probability distribution corresponding to the target model. The second input data set is input into the target model provided in the first embodiment of the present application to obtain a second probability distribution corresponding to the target model. It can be understood that the first probability distribution is a probability distribution corresponding to a token output by the target model after inputting all data input or specified by a user into the target model; and the second probability distribution is a probability distribution corresponding to a token output by the target model after inputting incomplete data with missing part into the target model. It should be noted that when there are multiple different second input data, there will be multiple corresponding second probability distributions.

[0070] Optionally, respectively inputting the first input data set and the second input data set into the target model to obtain a first probability distribution and a second probability distribution corresponding to the target model comprises: generating at least one first encoding vector corresponding to each data in the first input data set, and generating at least one second encoding vector corresponding to each data in the second input data set; wherein the first encoding vector and the second encoding vector are at least one layer of encoding vectors; respectively inputting the first encoding vector and the second encoding vector into the target model to obtain a first probability distribution and a second probability distribution corresponding to the target model.

[0071] In the embodiments of the present application, for each data in the first input data set, a corresponding at least one at least one layer encoding vector is generated based on each data. For the convenience of description, the encoding vector corresponding to each data in the first input data set is referred to as the first encoding vector. For each data in the second input data set, a corresponding at least one at least one layer encoding vector is generated based on each data. For the convenience of description, the encoding vector corresponding to each data in the second input data set is referred to as the second encoding vector. Taking the generation of the first encoding vector corresponding to each data in the first input data set as an example for illustrative description. For example, the first input data set includes reference data and input data input or specified by the user during model inference, and then the reference data can be encoded into a first reference encoding vector of at least one at least one layer by an encoder (Encoder), and the input data can be encoded into a first input encoding vector of at least one at least one layer by an encoder (Encoder).

[0072] For example, when the target model is a music generation model, the first input data can include reference music and target lyrics. At this time, the reference music can be encoded into a global vector of at least one at least one layer by an encoder (Encoder), and the global vector can be used as the first reference encoding vector. Alternatively, a preset length of music (such as 5 seconds or 10 seconds, etc.) can be extracted from the reference music, and the extracted preset length of music can be encoded into a first reference encoding vector of at least one at least one layer by an encoder. Alternatively, the reference music can be divided into at least one dimension of music part, and each dimension of music part can be encoded into a first reference encoding vector of at least one at least one layer by an encoder. For example, the reference music can be divided into an accompaniment part and a vocal part, the accompaniment part can be encoded into a corresponding first reference encoding vector by an MERT encoder (MERT Encoder), and the vocal part can be encoded into a corresponding first reference encoding vector by a Mel encoder (Mel Encoder). The target lyrics can be segmented into multiple segmentation units based on a preset segmentation algorithm, and the segmentation units can be each word, each character, or each sentence. Each segmentation unit in the target lyrics can be encoded into a first input encoding vector of at least one layer by an encoder. The more layers of the first input encoding vector, the more information, such as emotional information, can be represented in the target lyrics.

[0073] It should be noted that the manner of generating the second encoding vector corresponding to each data in the second input data set is similar to the manner of generating the first encoding vector, and will not be described here.

[0074] Optionally, before inputting the first encoding vector and the second encoding vector into the target model respectively to obtain the first probability distribution and the second probability distribution corresponding to the target model, the method further comprises: determining a preset encoding vector corresponding to the at least part of data randomly removed from the first input data set; and inputting the second encoding vector into the target model to obtain the second probability distribution corresponding to the target model, comprising: inputting the second encoding vector and the preset encoding vector into the target model to obtain the second probability distribution corresponding to the target model.

[0075] In the embodiment of the present application, after the at least part of data is randomly removed from the first input data set, the preset encoding vector corresponding to the at least part of data randomly removed from the first input data set is determined, for example, the at least part of data randomly removed from the first input data set can be directly set as a special encoding vector (such as 0 or 1). When the second encoding vector corresponding to the second input data set is input into the target model, the second encoding vector and the preset encoding vector corresponding to the at least part of data randomly removed from the first input data set are simultaneously input into the target model to obtain the second probability distribution corresponding to the token output by the target model.

[0076] S240, obtaining a target probability distribution based on the first probability distribution and the second probability distribution, and obtaining target output data based on the target probability distribution.

[0077] In the embodiment of the present application, the target probability distribution is obtained according to the first probability distribution and the second probability distribution. For example, the second probability distribution and the mean of the second probability distribution can be taken as the target probability distribution. Optionally, obtaining the target probability distribution based on the first probability distribution and the second probability distribution comprises: determining the target probability distribution according to the following formula: Log(As) = Log(As cond) + (Log(As cond) - Log(As uncond)) * C; wherein Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, and C represents a preset coefficient, and C is not 0. It should be noted that when the second probability distribution is n, the target probability distribution can be determined according to the following formula: Log(As) = Log(As cond) + (n * Log(As cond) - Log1(As uncond) - Log2(As uncond) - … - Logn(As uncond)) * C. n

[0078] ​In the embodiment of the present application, the target probability distribution is superior to the distribution probability of the token obtained by the traditional single inference mode, thereby increasing the stability of the target model output. Moreover, since the target model is a model generated by training the large language model after randomly removing data in the training sample set, even if part of the data is randomly removed during large language inference, the target model can still well obtain a relatively correct token probability distribution. In the embodiment of the present application, the target output data inferred by the target model is obtained according to the target probability distribution.

[0079] Optionally, when performing large language model inference output, the target model can sample and output the next token according to the target probability distribution of each token after optimization. The probability distribution of each token is the probability of each token being sampled as an output token. Optimizing the probability distribution twice can greatly increase the probability of correct tokens being sampled and reduce the probability of incorrect tokens being sampled, thereby achieving the purpose of improving the accuracy and stability of large language model inference.

[0080] The large language model inference method provided by the embodiment of the present application comprises: in response to a model inference event being triggered, obtaining a first input data set; wherein the first input data set is composed of at least one kind of data; removing at least part of the data from the first input data set to generate a second input data set; inputting the first input data set and the second input data set into a target model respectively to obtain a first probability distribution and a second probability distribution corresponding to the target model; obtaining a target probability distribution based on the first probability distribution and the second probability distribution, and obtaining target output data based on the target probability distribution. Through the technical scheme provided by the embodiment of the present application, by inputting the complete data input or specified by the user during model inference and the incomplete data missing part into the target model, the probability distribution of the token output by the target model at least twice is obtained, and the target probability distribution after optimization is obtained according to the probability distribution of the token output by the target model at least twice. Then, more accurate and stable target output data can be obtained according to the target probability distribution.

[0081] Embodiment three

[0082] Figure 3 A large language model training device provided for the third embodiment of the present application is shown in the structural schematic diagram. As shown in the figure, the device comprises: Figure 3

[0083] The training data set acquisition module 310 is configured to acquire a training data set; wherein the training data set comprises at least two training sample data sets, and each training sample data set is composed of at least two corresponding data sets;

[0084] ​The target training data generation module 320 is configured to randomly extract a first training sample data group in a first preset proportion from the training data set, and randomly remove part of data from the first training sample data group to generate target training data.

[0085] The target model generation module 330 is configured to train the large language model based on the target training data, and determine whether a preset training condition is met. If not, the method returns to the step of randomly extracting a training sample data group in a first preset proportion from the training data set until the preset training condition is met, and generates a target model.

[0086] Optionally, the target training data generation module is configured to:

[0087] randomly remove at least one type of data from the first training sample data group; and / or,

[0088] randomly remove at least part of data content in at least one type of data from the first training sample data group.

[0089] Optionally, the target model generation module is configured to:

[0090] Before model training of the large language model, N times of target training data are obtained based on the training sample data groups in the training data set, N is an integer greater than or equal to 1, and the large language model is trained based on each obtained target training data until a preset training condition is met, and a target model is generated; or,

[0091] During model training of the large language model, new target training data are obtained in real time based on the training sample data groups in the training data set, and the large language model is trained based on the real-time obtained new target training data and the training sample data group until a preset training condition is met, and a target model is generated; or,

[0092] The large language model is trained based on the target training data, and during the model training of the large language model, it is determined whether a preset training condition is met. If not, the next round of target training data is obtained based on the training sample data groups in the training data set, and the large language model is trained based on the next round of target training data until the preset training condition is met, and a target model is generated.

[0093] Optionally, the target training data generation module is configured to:

[0094] at least one training sample data group is extracted from the training data set in a preset order or randomly as a first training sample data group; and / or,

[0095] extract at least one training sample data set of at least one data from the training data set as the first training sample data set according to a preset order or randomly; and / or,

[0096] extract at least one training sample data set of at least part of data content in at least one data from the training data set as the first training sample data set according to a preset order or randomly.

[0097] The large language model training device provided by the embodiment of the present application can execute the large language model training method provided by any embodiment of the present application, and has the corresponding function modules and beneficial effects of the execution method.

[0098] Embodiment four

[0099] Figure 4 A structural schematic diagram of a large language model reasoning device provided by embodiment four of the present application is shown in FIG. 4. As shown in the figure, the device comprises: Figure 4

[0100] The first input data set acquisition module 410 is configured to acquire a first input data set in response to a model reasoning event being triggered, wherein the first input data set is composed of at least one data;

[0101] The second input data set generation module 420 is configured to randomly eliminate at least part of data from the first input data set to generate a second input data set;

[0102] The probability distribution acquisition module 430 is configured to input the first input data set and the second input data set into a target model respectively to acquire a first probability distribution and a second probability distribution corresponding to the target model;

[0103] The target output data acquisition module 440 is configured to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and acquire target output data based on the target probability distribution.

[0104] Optionally, the target output data acquisition module is configured to:

[0105] The target probability distribution is determined according to the following formula:

[0106] Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond))*C;

[0107] wherein Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, and C represents a preset coefficient, and C is not 0.

[0108] ​Optionally, the probability distribution obtaining module comprises:

[0109] The encoding vector generating unit is configured to generate at least one first encoding vector corresponding to each data in the first input data set based on the first input data set, and generate at least one second encoding vector corresponding to each data in the second input data set based on the second input data set; wherein the first encoding vector and the second encoding vector are both encoding vectors of at least one layer.

[0110] The probability distribution obtaining unit is configured to input the first encoding vector and the second encoding vector into a target model respectively, and obtain a first probability distribution and a second probability distribution corresponding to the target model.

[0111] Optionally, the device further comprises:

[0112] The preset encoding vector determining unit is configured to determine a preset encoding vector corresponding to at least part of the data randomly removed from the first input data set before the first encoding vector and the second encoding vector are input into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model.

[0113] The probability distribution obtaining unit is configured to:

[0114] input the second encoding vector and the preset encoding vector into the target model to obtain the second probability distribution corresponding to the target model.

[0115] Optionally, the second input data set generating module is configured to:

[0116] randomly remove at least one data from the first input data set; and / or, randomly remove at least part of the data content in at least one data from the first input data set to generate the second input data set.

[0117] The large language model reasoning device provided by the embodiment can execute the large language model reasoning method provided by any embodiment of the present application, and has the corresponding functional modules and beneficial effects of the execution method.

[0118] Embodiment five

[0119] Figure 5A structural diagram of an electronic device 10 that can be used to implement embodiments of the present application is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices (e.g., headsets, glasses, watches, etc.), and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not meant to limit implementations of the present application described and / or claimed in this document.

[0120] As shown in Figure 5 The electronic device 10 includes at least one processor 11, and a memory, such as a read-only memory (ROM) 12, a random access memory (RAM) 13, etc., communicatively connected to the at least one processor 11, where the memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer programs stored in the read-only memory (ROM) 12 or loaded into the random access memory (RAM) 13 from the storage unit 18. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0121] Various components in the electronic device 10 are connected to the I / O interface 15, including an input unit 16, such as a keyboard, a mouse, etc., an output unit 17, such as various types of displays, speakers, etc., a storage unit 18, such as a magnetic disk, an optical disk, etc., and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0122] The processor 11 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs various methods and processes described above, such as a large language model training method or a large language model inference method.

[0123] In some embodiments, the large language model training method or the large language model inference method can be implemented as a computer program tangibly embodied in a computer readable storage medium, such as the storage unit 18. In some embodiments, part or all of the computer program can be loaded onto and / or installed on the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded onto the RAM 13 and executed by the processor 11, one or more steps of the large language model training method or the large language model inference method described above can be performed. Alternatively, in other embodiments, the processor 11 can be configured to perform the large language model training method or the large language model inference method by any other appropriate means, for example, by means of firmware.

[0124] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0125] Computer programs used to implement the processes of the present application can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the computer program, when executed, implements the functions / acts specified in the flowcharts and / or block diagrams. The computer program can be executed entirely on a machine, partially on a machine, partially on a machine as part of a separate software package, or entirely on a remote machine or server.

[0126] In the context of the present application, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. A computer-readable storage medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of a machine-readable storage medium will include one or more lines of a program of instructions in a transitory signal, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0127] To provide for interaction with a user, the systems and techniques described here can be implemented on an electronic device having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0128] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0129] The computing system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system, to solve the defects of large management difficulty and weak business scalability in traditional physical host and VPS service.

[0130] It should be understood that the various forms of flow shown above can be reordered, added to, or have steps deleted. For example, the steps described in the present application can be performed in parallel, in series, or in a different order, as long as the desired results of the technical solutions of the present application can be achieved, which are not limited herein.

[0131] The above detailed description does not constitute a limitation on the protection scope of the present application. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a large language model, characterized in that, include: Obtain a training dataset; wherein the training dataset contains at least two training sample data groups, each training sample data group consisting of at least two corresponding data types; the training sample data group includes at least input data and fitted data; wherein the training sample data group includes at least sample lyrics, reference music, and fitted music; Multiple first training sample data groups are randomly selected from the training dataset in a preset order or randomly. For each first training sample data group, at least a portion of the data is randomly removed from the input data or the fitted data to generate target training data. The large language model is trained based on the training sample data group and the target training data in the training dataset until the preset training conditions are met, and the target model is generated; wherein, the target model is a music generation model. In response to a model inference event being triggered, multiple first input data sets are acquired; wherein each first input data set consists of at least one type of data; and the first input data set includes at least the target lyrics. For each first input data group, at least a portion of the data is randomly removed from the first input data group to generate a second input data group; The first input data set and the second input data set are respectively input into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model; Based on the first probability distribution and the second probability distribution, a target probability distribution is obtained, and target output data is obtained based on the target probability distribution; The step of obtaining the target probability distribution based on the first probability distribution and the second probability distribution includes: The target probability distribution is determined using the following formula: Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond)) C; Wherein, Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, and C represents a preset coefficient, which is not 0.

2. The method according to claim 1, characterized in that, At least a portion of the data is randomly removed from the first training sample data set, including: Randomly remove at least one type of data from the first training sample data set; and / or, At least a portion of the data content from at least one type of data is randomly removed from the first training sample data set.

3. The method according to claim 1, characterized in that, The large language model is trained based on the training sample data group and the target training data in the training dataset until the preset training conditions are met, generating the target model, including: Before training the large language model, N sets of target training data are obtained based on the training sample data groups in the training dataset, where N is an integer greater than or equal to 1. The large language model is then trained based on each set of target training data until preset training conditions are met, generating the target model; or, When training a large language model, new target training data is acquired in real time based on the training sample data set in the training dataset. The large language model is then trained based on the acquired new target training data and the training sample data set until preset training conditions are met, generating the target model; or... The large language model is trained based on the target training data. During the training process, it is determined whether the preset training conditions are met. If not, the target training data for the next round is obtained based on the training sample data group in the training dataset. The large language model is then trained based on the target training data for the next round until the preset training conditions are met, and the target model is generated.

4. The method according to any one of claims 1-3, characterized in that, At least one first training sample data group is selected from the training dataset in a preset order or randomly, including: At least one training sample data group is selected from the training dataset in a preset order or randomly as the first training sample data group; and / or, At least one training sample data group is selected from the training dataset according to a preset order or randomly as the first training sample data group; and / or, At least one training sample data group is selected from the training dataset in a preset order or randomly, and at least a portion of the data content is selected as the first training sample data group.

5. The method according to claim 1, characterized in that, The first input data set and the second input data set are respectively input into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model, including: At least one first encoding vector is generated based on each data in the first input data group, and at least one second encoding vector is generated based on each data in the second input data group; wherein, both the first encoding vector and the second encoding vector are encoding vectors with at least one layer. The first encoding vector and the second encoding vector are respectively input into the target model to obtain the first probability distribution and the second probability distribution corresponding to the target model.

6. The method according to claim 5, characterized in that, Before inputting the first encoding vector and the second encoding vector into the target model respectively to obtain the first probability distribution and the second probability distribution corresponding to the target model, the method further includes: Determine the preset encoding vector corresponding to at least a portion of the data randomly removed from the first input data group; The second encoded vector is input into the target model to obtain the second probability distribution corresponding to the target model, including: The second encoding vector and the preset encoding vector are input into the target model to obtain the second probability distribution corresponding to the target model.

7. The method according to claim 1, characterized in that, At least a portion of the data is randomly removed from the first input data set to generate a second input data set, including: Randomly remove at least one type of data from the first input data group; and / or randomly remove at least a portion of the data content from the first input data group to generate the second input data group.

8. A large language model training device, characterized in that, include: A training dataset acquisition module is used to acquire a training dataset; wherein the training dataset contains at least two training sample data groups, each training sample data group consists of at least two corresponding data types; the training sample data group includes at least input data and fitted data; wherein the training sample data group includes at least sample lyrics, reference music, and fitted music. The target training data generation module is used to randomly extract multiple first training sample data groups from the training dataset. For each first training sample data group, a portion of data is randomly removed from the input data or the fitted data to generate target training data. The target model generation module is used to train the large language model based on the target training data and determine whether the preset training conditions are met. If not, it returns to the process of randomly selecting multiple first training sample data groups from the training dataset until the preset training conditions are met, and then generating the target model; wherein, the target model is a music generation model. The first input data group acquisition module is used to acquire multiple first input data groups in response to a model inference event being triggered; wherein the first input data group consists of at least one type of data. The second input data group generation module is used to randomly remove at least a portion of the data from each first input data group to generate a second input data group. The first probability distribution and second probability distribution acquisition module is used to input the first input data group and the second input data group into the target model respectively, and acquire the first probability distribution and the second probability distribution corresponding to the target model; The target output data acquisition module is used to acquire a target probability distribution based on the first probability distribution and the second probability distribution, and to acquire target output data based on the target probability distribution; The target output data acquisition module is specifically used to determine the target probability distribution according to the following formula: Log(As)=Log(As cond)+(Log(As cond)-Log(As uncond)) C; Wherein, Log(As) represents the target probability distribution, Log(As cond) represents the first probability distribution, Log(As uncond) represents the second probability distribution, and C represents a preset coefficient, which is not 0.

Images