Training methods, inference methods, electronic devices and media for machine learning models

Abstract

The present disclosure relates to a training method, reasoning method, electronic device and medium of a machine learning model, and to the field of computer technology. The method comprises: obtaining an output vector generated by a fully connected layer before the last normalization layer of the machine learning model, wherein the output vector is used to characterize the original score of each node of the fully connected layer; calculating a corresponding first confidence parameter based on the original score of each node, and determining a second confidence parameter based at least on the maximum value of the first confidence parameter; using the ratio of the first confidence parameter to the second confidence parameter as the confidence coefficient of the node; and adjusting the loss of the machine learning model using the confidence coefficient, training the machine learning model based on the adjusted loss, and obtaining a trained machine learning model. The present disclosure can optimize the learning process of the machine learning model, improve the accuracy of model discrimination, and reduce the occurrence of out-of-set problems.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a training method, an inference method, an electronic device, and a medium for a machine learning model. Background Art

[0002] Machine learning models such as LLM (Large Language Model) or any multi-classification model can use methods such as Softmax function or its variants to achieve character prediction or category discrimination. The Softmax function can give the probability distribution of each input being divided into each predicted character / category, and will take out the index corresponding to the maximum score from many scores. The index corresponds to the predicted character or category as the output. Softmax is actually a relative discriminant function, and its score is a normalized value. That is, no matter what the circumstances, there will always be a maximum value among the scores of these categories when compared with each other, and this maximum value is used as the most likely output. At the same time, the scale of the output score obtained for each input is compressed to between 0 and 1.

[0003] However, using a normalization function such as Softmax will lose a lot of information about logits (the output of the final fully connected layer or linear layer of the machine learning model), making the model inaccurate, which can easily lead to out-of-set queries being mistakenly included in the set, resulting in prediction errors or classification errors. Summary of the invention

[0004] According to one aspect of an example of the present disclosure, a method for training a machine learning model is provided, the method comprising: obtaining an output vector generated by a fully connected layer before the last normalization layer of the machine learning model, wherein the output vector is used to represent the original score of each node of the fully connected layer; calculating a corresponding first confidence parameter based on the original score of each of the nodes, and determining a second confidence parameter based at least on a maximum value of the first confidence parameter; taking a ratio of the first confidence parameter to the second confidence parameter as a confidence coefficient of the node; and adjusting the loss of the machine learning model using the confidence coefficient, and training the machine learning model based on the adjusted loss to obtain a trained machine learning model.

[0005] According to another aspect of an example of the present disclosure, there is also provided an inference method for a machine learning model, including: obtaining data to be processed; inputting the data to be processed into a trained machine learning model to obtain an inference result; wherein the trained machine learning model is obtained after training using the above method.

[0006] According to another aspect of an example of the present disclosure, there is also provided an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements the above method when executing the computer program.

[0007] According to another aspect of an example of the present disclosure, a computer-readable storage medium is further provided, wherein a computer program is stored in the computer-readable storage medium, and the computer program implements the above method when executed by a processor.

[0008] According to another aspect of an example of the present disclosure, a computer program product is further provided, wherein the computer program product includes a computer program, and the computer program implements the above method when executed by a processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention, and for ordinary technicians in this field, other embodiments can be obtained based on these drawings without creative work.

[0010] Figure 1 is a block diagram of a computing system for performing reasoning according to at least one embodiment of the present disclosure;

[0011] Figure 2 is a flowchart of a method for training a machine learning model provided by at least one embodiment of the present disclosure;

[0012] Figure 3 is a schematic diagram of an LLM provided by at least one embodiment of the present disclosure;

[0013] Figure 4 is a flowchart of an inference method of a machine learning model provided by at least one embodiment of the present disclosure;

[0014] Figure 5 It is a schematic diagram of the structure of an electronic device provided by at least one embodiment of the present disclosure. DETAILED DESCRIPTION

[0015] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments described herein, which are instead provided for a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0016] Machine learning models are computer algorithms that use data to make estimates (educated guesses) or decisions. Machine learning models are designed differently from traditional algorithms. When traditional computer software needs improvement, people usually edit the software. In contrast, machine learning algorithms use data (such as training data) to get better at a specific task.

[0017] Figure 1 A block diagram of an example computing system 100 that performs inference according to an example embodiment of the present disclosure is depicted. System 100 includes a user computing device 110, a server computing device 130, and a training computing device 150 that are communicatively coupled via a network 170.

[0018] The user computing device 110 can be any type of computing device, such as a personal computing device (e.g., a laptop or desktop), a mobile computing device (e.g., a smartphone or tablet), a game console or controller, a wearable computing device (e.g., a smart watch or AR glasses or VR glasses or MR glasses), an embedded computing device, or any other type of computing device.

[0019] The user computing device 110 includes one or more processors 112 and memory 114. The one or more processors 112 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 114 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 114 may store data 116 and instructions 118 executed by the one or more processors 112 to cause the user computing device 110 to perform operations.

[0020] In some implementations, the user computing device 110 may store or include one or more machine learning models 120. For example, the machine learning model 120 may be or may otherwise include various machine learning models, such as a neural network (e.g., a deep neural network) or other types of machine learning models, including nonlinear models and / or linear models. Neural networks may include feedforward neural networks, recursive neural networks (e.g., long short-term memory recursive neural networks), convolutional neural networks, or other forms of neural networks. Some example machine learning models may utilize attention mechanisms such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model). Please refer to Figure 2 examples to train machine learning models.

[0021] In some implementations, one or more machine learning models 120 may be received from a server computing device 130 via a network 170 and stored in a user computing device memory 114 and then used or otherwise implemented by one or more processors 112. In some implementations, a user computing device 110 may implement multiple parallel instances of a single machine learning model 120 (e.g., multiple instances of a machine learning model 120 perform parallel inference).

[0022] More specifically, the machine learning model 120 may be used to generate predicted characters or classification results in response to user input. For example, the user may input content into an operating environment executed by the processor 112 of the user device 110 via a user input component 122 (e.g., keyboard / mic, etc.). It should be noted that the machine learning model 120 may generate predicted characters or classification results during a one-time reasoning time process, so that they may be provided to the user of the user computing device 110 in real time.

[0023] For example, the user input component 122 can be a touch-sensitive component (e.g., a touch-sensitive display screen or a touchpad) that is sensitive to the touch of a user input object (e.g., a finger or a stylus). The touch-sensitive component can be used to implement a virtual keyboard. Other example user input components include a microphone, a traditional keyboard, or other devices that a user can use to provide user input.

[0024] In addition, one or more machine learning models 140 may be included in a server computing device 130 that communicates with the user computing device 110 according to a client-server relationship, and the capabilities provided by the machine learning models 140 are stored and implemented by the server computing device 130. For example, the machine learning model 140 may be implemented by the server computing device 130 as part of a network service (e.g., an inference service or a conversational robot service). Thus, one or more machine learning models 120 may be stored and implemented at the user computing device 110, and / or one or more models 140 may be stored and implemented at the server computing device 130.

[0025] The server computing device 130 includes one or more processors 132 and memory 134. The one or more processors 132 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operatively connected. The memory 134 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, magnetic disks, etc., and combinations thereof. The memory 134 may store data 136 and instructions 138 executed by the processor 132 to cause the server computing system 130 to perform operations.

[0026] In some implementations, server computing device 130 includes or is implemented by one or more server computing devices. Where server computing device 130 includes multiple server computing devices, such server computing devices may operate according to a sequential computing architecture, a parallel computing architecture, or some combination thereof.

[0027] As described above, the server computing device 130 may store or otherwise include one or more machine learning models 140. For example, the machine learning model 140 may include various machine learning models. Example machine learning models include neural networks or other multi-layer nonlinear models. Example neural networks include feedforward neural networks, deep neural networks, recursive neural networks, and convolutional neural networks. Some example machine learning models may be aided by attention mechanisms such as self-attention. For example, some example machine learning models may include a multi-head self-attention model (e.g., a transformer model).

[0028] The user computing device 110 and / or the server computing device 130 may train the machine learning model 120 and / or the machine learning model 140 by interacting with the training computing device 150 that is communicatively coupled via the network 170. The training computing device 150 may be separate from the server computing device 130, or may be part of the server computing device 130. The specific training process may be referred to Figure 2 An exemplary description of .

[0029] The training computing device 150 includes one or more processors 152 and a memory 154. The one or more processors 152 may be any suitable processing device (e.g., a processor core, a microprocessor, an ASIC, an FPGA, a controller, a microcontroller, etc.) and may be one processor or a plurality of processors operably connected. The memory 154 may include one or more non-transitory computer-readable storage media, such as RAM, ROM, EEPROM, EPROM, flash memory devices, disks, etc., and combinations thereof. The memory 154 may store data 156 and instructions 158 executed by the processor 152 to cause the training computing device 150 to perform operations. In some implementations, the training computing device 150 includes or is otherwise implemented by one or more server computing devices.

[0030] The training computing system device may include a model trainer 160 that uses various training or learning techniques, such as error back propagation, to train the machine learning model 120 and / or the machine learning model 140 stored at the user computing device 110 and / or the server computing device 130. For example, a loss function may be back propagated through the (multiple) machine learning models to update one or more parameters of the (multiple) machine learning models (e.g., based on the gradient of the loss function). Various loss functions may be used, such as mean squared error, likelihood loss, cross entropy loss, hinge loss, and / or various other loss functions. Gradient descent techniques may be used to iteratively update parameters over multiple training iterations.

[0031] In some implementations, performing back propagation of errors may include performing truncated back propagation over time. The model trainer 160 may perform a variety of generalization techniques (eg, weight decay, dropout, etc.) to improve the generalization ability of the trained model.

[0032] Specifically, the model trainer 160 can train the machine learning model 120 and / or the machine learning model 140 based on a set of training data 162 to obtain a trained machine learning model. The training data 162 may include, for example, label data indicating input data that needs to be classified and indicating the classification result of the input data. In other examples, the training data 162 may be input data that does not include labels, and the machine learning model 120 and / or the machine learning model 140 may use these input data that do not include labels to learn, for example, the structure, pattern, and usage habits of the input data. For example, the input data is input into the machine learning model 120 and / or the machine learning model 140 after mapping, and the output vector generated by the last fully connected layer before the normalization layer of the machine learning model 120 and / or the machine learning model 140 is obtained. The above output vector is, for example, a single-dimensional vector, and the value on each node (position) represents the score of the character / classification corresponding to the position output by the machine learning model 120 and / or the machine learning model 140. The above score is then used to calculate the first confidence parameter corresponding to the character / category score of the node (position), and the second confidence parameter is determined by the maximum value of all the first confidence parameters. The ratio of the two is used as the confidence coefficient of the node (position), and this confidence coefficient is used to adjust the loss of this batch of training data.

[0033] In some implementations, if the user has provided consent to use private data for training, the training examples may be provided by the user computing device 110. Thus, in such implementations, the machine learning model 120 provided to the user computing device 110 may be trained by the training computing system 150 based on user-specific data received from the user computing device 110. In some cases, this process may be referred to as personalized training of the model.

[0034] The model trainer 160 includes computer logic for providing the desired functionality. The model trainer 160 can be implemented in hardware, firmware, and / or software that controls a general purpose processor. For example, in some implementations, the model trainer 160 includes a program file stored on a storage device, loaded into a memory, and executed by one or more processors. In other implementations, the model trainer 160 includes one or more sets of computer executable instructions stored in a tangible computer readable storage medium, such as RAM, a hard disk, or an optical or magnetic medium.

[0035] Network 170 may be any type of communication network, such as a local area network (e.g., an intranet), a wide area network (e.g., the Internet), or some combination thereof, and may include any number of wired or wireless links. In general, communications on network 180 may be carried via any type of wired and / or wireless connection using a variety of communication protocols (e.g., TCP / IP, HTTP, SMTP, FTP), encodings or formats (e.g., HTML, XML), and / or protection schemes (e.g., VPN, secure HTTP, SSL).

[0036] As described above, the machine learning model may be a neural network model or a converter model, etc. In some examples, the neural network model or the converter model includes an input layer (e.g., a mapping layer for performing mapping), a convolutional layer (or an encoder structure, a decoder structure, or an encoder-decoder structure including an encoder structure and a decoder structure in the converter), and an output layer. For example, the output layer may include at least one fully connected layer (e.g., a feedforward layer) and a final normalization layer (e.g., a classification layer) connected after the fully connected layer. The output of each node (position) of the fully connected layer is referred to as logits in this article. The normalization layer may use, for example, the Softmax function to give the probability distribution of the predicted character / category corresponding to each node (position) of the final normalization layer to which each input is divided, and the function argmax() is used to output the index with the maximum value of the probability distribution. The index may be understood as indicating the predicted character / category corresponding to the node (position) of the maximum value. In some examples, the formula of the Softmax function may refer to the formula: , Softmax will calculate the score for i representing each node (position). In this formula, the flatness of the distribution can be controlled by the temperature coefficient T. The temperature coefficient T can affect the output of the machine learning model to be more inclined to select outputs with high probability or to select diverse outputs. For example, when the temperature coefficient T is high (for example, greater than 1), the machine learning model will select diverse outputs; for example, when the temperature coefficient T is low (for example, less than or equal to 1), the machine learning model will select outputs from words with higher probabilities.

[0037] In some examples, when using, for example, the Softmax function as the output function, the logits of each node (position) output by the fully connected layer of the machine learning model output layer are not high, but in contrast, a maximum value can be selected, and the value is not low after normalization. For example, when using a machine learning model for classification, each node (position) output by the fully connected layer corresponds to a classification category. For example, the input "open the car window" is labeled as the "control" class, while the input "ahaha window" has not been trained and has no corresponding label. Since there is no corresponding classification label set and it has not been trained, the input "ahaha window" may not have high logits in each category, but because the final normalization layer considers relative scores, the score of the sentence "ahaha window" in a certain category may not be low, resulting in the OOD (Out-Of-Distribution) problem (in the field of machine learning, it refers to the situation where the data encountered by the model is significantly different from the distribution of the training data, resulting in a decrease in model performance (for example, the classification results of the machine learning model are significantly inaccurate, unreliable or unstable)). Similarly, using machine learning models for character prediction, such as the LLM-based reasoning process, will also have such a problem. When predicting the next token (the smallest meaningful unit in the text, such as a word, number, or punctuation mark), the absolute score of each word is not high, but the relative score is not low. LLM continues to reason and eventually produces hallucinations (such as wrong output).

[0038] Based on this, the present disclosure provides a training method, an inference method, an electronic device and a storage medium for a machine learning model, aiming to alleviate the OOD problem that occurs in the prediction, classification or other inference processes of the machine learning model.

[0039] A method for training a machine learning model is provided in an embodiment of the present disclosure. Figure 2 is a flowchart of a method for training a machine learning model according to an embodiment of the present disclosure. Figure 2 The method steps involved are described.

[0040] Step S210, obtaining an output vector generated by a fully connected layer before the last normalization layer of the machine learning model, wherein the output vector is used to represent the original score of each node in the fully connected layer.

[0041] For example, when a trained machine learning model performs a classification task, the machine learning model can infer the probability that the input data falls into each classification category, and the model can select the classification category with the maximum probability as the output category. The goal of the classification task is to predict the category to which the given input data belongs. For example, when a trained machine learning model performs a prediction task, such as performing a conversational robot task to output content (such as text) to the user, the machine learning model can infer multiple inference results (such as corresponding to multiple tokens) for the next output order based on the user input and the inferred output, and can select the inference result with the maximum probability as the output content among the multiple inference results.

[0042] The machine learning model can be, for example, a natural language processing model, such as a large language model (LLM), which can be used to understand and generate natural language text. It should be noted that LLM or any machine learning model can generally use softmax or its variants to achieve text prediction or category discrimination. Softmax will take the index corresponding to the maximum score from many scores, and the index corresponds to the inference result or category, which is the output.

[0043] See attached Figure 3 As shown, LLM can be, for example, a model that only includes a Transformer decoder structure, which includes an input, a mapping layer, multiple decoders, a linear layer (such as a fully connected layer or a feedforward layer), a normalization layer (such as a Softmax layer), and an output. In some examples, the output of the mapping layer and the positional encoding of the input together constitute the input of the decoder. In other examples, the decoder structure also includes a residual connection, for example, the output of the mapping layer is also residually connected to the first layer normalization, and the output of the first layer normalization is also residually connected to the second layer normalization. The linear layer or fully connected layer can be used to generate a multidimensional vector, and each node (position) of the linear layer or the fully connected layer generates an element of the multidimensional vector. Each element can be, for example, a specific value (such as a raw score). The values ​​generated by these nodes (positions) are normalized in the normalization layer so that the sum of all normalized elements is equal to 1. It can be understood that in the case of Figure 3When the exemplary LLM shown is trained, the output vector generated by the fully connected layer (also called the linear layer) before the last normalization layer (such as the Softmax layer) of the LLM can be obtained, and the output vector is used to represent the original score of each node in the fully connected layer.

[0044] The machine learning model may be, for example, a neural network, which may include, for example, an input layer, a hidden layer, and an output layer. The input layer may be used, for example, to receive image input, text input, or any other suitable input. The hidden layer may include one or more fully connected layers. When the output layer of the neural network is used for classification, it is called a classification layer. The output layer is located at the end of the neural network and is responsible for converting the output of the previous layer into a node probability distribution. The fully connected layer closest to the classification layer is used as the fully connected layer before the classification layer. In some examples, after input data, such as training data, is input into the neural network, the output vector generated by the fully connected layer before the classification layer of the neural network is obtained, wherein the output vector is used to characterize the original score of each node.

[0045] In some embodiments, the output vector, i.e., the logits of the output layer, refers to the original values ​​that have been transformed by the last layer of the machine learning model but have not been normalized. These values ​​can be regarded as the unnormalized scores or logarithmic probabilities of the model for each node (position).

[0046] In this step, by retaining the original score, more information of the output vector itself is retained.

[0047] Step S220: Calculate a corresponding first confidence parameter based on the original score of each node, and determine a second confidence parameter based at least on a maximum value of the first confidence parameter.

[0048] In some embodiments, a first confidence parameter may be calculated for the raw score output by each node (position). When calculating the first confidence parameter corresponding to a node (position), the calculation is performed based on the raw score of the node (position). After obtaining the first confidence parameter, a second confidence parameter corresponding to the current reasoning step of the machine learning model is determined based at least on the maximum value of the first confidence parameters of each node (position).

[0049] It should be noted that the second confidence parameter is a parameter that is continuously updated with the training batch (or training process), which represents the maximum value of all first confidence parameters generated in the training batch that has been carried out. For example, if the current training batch is the 8th batch, the second confidence parameter is the maximum value of all first confidence parameters generated in the previous 8 training batches that have been carried out.

[0050] Step S230, taking the ratio of the first confidence parameter to the second confidence parameter as the confidence coefficient of the node, and using the confidence coefficient to adjust the loss of the machine learning model, training the machine learning model based on the adjusted loss, and obtaining a trained machine learning model.

[0051] In some embodiments, the ratio of the first confidence parameter to the second confidence parameter is used as the confidence coefficient of the node (position), so as to obtain the confidence coefficient corresponding to each node (position). The confidence coefficient can be added to the loss calculation process of the machine learning model, that is, when calculating the loss of the reasoning process, the confidence coefficient is used to adjust the loss of the machine learning model, wherein the loss can be described by a loss function. The loss function is a key indicator for evaluating the difference between the model prediction result and the actual label. The specific loss functions that can be used include but are not limited to: cross entropy loss function, Hinge loss (Hinge Loss) function, multi-class Hinge loss (Multi-Class Hinge Loss) function, log-likelihood loss function, etc. Afterwards, for example, back propagation can be performed based on the adjusted loss to train the machine learning model to change the weight parameters of the machine learning model, and the training process is iterated until the trained machine learning model is obtained. For example, the stopping condition of the iteration may be that the number of training times is reached, the loss function no longer decreases, etc.

[0052] It can be understood that for the normalization layer, for example, using softmax for normalization to reduce the score difference causes the probability of the node (position) being selected to change, thereby causing the low-confidence node (position) to generate a relatively high-probability output after normalization, resulting in the phenomenon that the output with low confidence but high probability is selected. The exemplary embodiment of the present disclosure introduces the original confidence of the output of the last linear layer (fully connected layer) when calculating the loss to alleviate this problem. The logits output by the last linear layer (fully connected layer) of the model is used as a confidence reference parameter, a first confidence parameter is calculated based on the original score, and a confidence coefficient is obtained based on the first confidence parameter and a second confidence parameter associated with the first confidence parameter, and the confidence coefficient is introduced into the calculation of the loss, which can effectively utilize the original confidence and reduce the possibility of hallucinations in the model.

[0053] The training method of the machine learning model provided in the embodiment of the present disclosure includes:

[0054] Obtain the output vector generated by the fully connected layer before the last normalization layer of the machine learning model, wherein the output vector is used to characterize the original score of each node of the fully connected layer, calculate the corresponding first confidence parameter based on the original score of each node, and determine the second confidence parameter based on at least the first confidence parameter; use the ratio of the first confidence parameter to the second confidence parameter as the confidence coefficient of the node, use the confidence coefficient to adjust the loss of the machine learning model, and train the machine learning model based on the adjusted loss to obtain a trained machine learning model. The disclosed embodiment calculates the confidence coefficient based on the output vector, thereby retaining more information of the fully connected layer output vector itself (such as the confidence information of each node), and adjusts the loss of the machine learning model based on the confidence coefficient, thereby optimizing the learning process of the machine learning model, improving the accuracy of model discrimination, and reducing the occurrence of out-of-set problems.

[0055] In a possible implementation, the corresponding first confidence parameter is calculated based on the original score of each node, which can be performed according to the following steps: using N as the base and the original score of each node as the exponent to calculate the first confidence parameter, where N is greater than 1.

[0056] In some embodiments, N is set to a natural logarithm.

[0057] In a possible implementation, the first confidence parameter may be calculated according to the following formula: xi , where e represents the natural logarithm, x i represents the raw score of the node (position), e xi Indicates the calculated first confidence parameter of the node (position).

[0058] In a possible implementation, determining the second confidence parameter based at least on the maximum value of the first confidence parameter can be performed according to the following steps: in response to the current training batch being the first training batch, selecting the maximum value from the first confidence parameters as the second confidence parameter; or in response to the current training batch being not the first training batch, selecting the maximum value from the first confidence parameters and comparing it with a past second confidence parameter determined in the previous training batch; and selecting the larger value from the maximum value and the past second confidence parameter as the new second confidence parameter.

[0059] During the training process of a machine learning model, data is usually divided into multiple batches, each of which contains a portion of training samples. In this possible implementation, the machine learning model is trained using each batch in turn, and the first batch of samples used for training is referred to as the first training batch, and the second and subsequent batches of samples used for training are referred to as non-first training batches.

[0060] Each group of training data in a training batch is sequentially input into the machine learning model to train the machine learning model. Each time a group of training data is input, an output vector of the machine learning model and the probability of each node (position) obtained after normalizing the output vector and the output (such as a classification result, or, for example, a character) selected based on the probability are generated based on the training data. The output vector includes the original score of each node. Based on the output vector, the first confidence parameter of each node can be obtained. In the case where the batch is the first training batch, the largest first confidence parameter in the batch, that is, the maximum value, is used as the second confidence parameter of the current training batch. That is, in response to the current training batch being the first training batch, the maximum value is selected from the first confidence parameter as the second confidence parameter. If the batch is not the first training batch, the largest first confidence parameter in the batch, that is, the larger value of the maximum value and the second confidence parameter determined in the previous batch, is used as the second confidence parameter of the current batch, that is, the updated second confidence parameter, that is, in response to the current training batch being not the first training batch, the maximum value is selected from the first confidence parameters and compared with the past second confidence parameter determined in the previous training batch; the larger value is selected from the maximum value and the past second confidence parameter as the new second confidence parameter.

[0061] In a possible implementation, the loss is a cross entropy loss function, and the confidence coefficient is used to adjust the loss of the machine learning model, which can be performed according to the following steps:

[0062] Calculate the loss of the machine learning model; obtain the maximum confidence coefficient; adjust the loss using the maximum confidence coefficient to obtain the adjusted loss.

[0063] In this possible implementation, the cross entropy loss function is used to measure the gap between the model prediction results and the actual labels in the machine learning model training.

[0064] As mentioned earlier in this article, during the training process of a machine learning model, the model can calculate a loss value between the normalized probability of each node (position) and the sample label, for example, using represents the logarithmic probability corresponding to the node (position), Indicates the probability of the model selecting this node (position) output, using The vector representing the encoded and / or mapped labels of the training samples is expressed using the formula To calculate the loss of the model, 1-n represents the number of output nodes of the model.

[0065] Among them, the probability of the output corresponding to the node (position) , which can be calculated by the softmax function. It should be noted that, assuming that the probability distribution of a sample predicted by the model is , where n represents the number of output nodes of the model, It indicates the probability that the model predicts that the sample belongs to the i-th category. It should be noted that in the calculation of the cross entropy loss function, the true label can be represented by one-hot encoding. For the i-th sample, its true label vector is.

[0066] For example, the output nodes of the last normalized layer of the model include three, and it can be understood that the corresponding output nodes of the previous fully connected layer also include three. The three-dimensional vector output by the training sample input machine learning model is (0.8, 0.1, 0.1), and the label of the training sample is (1, 0, 0). In other examples, the label of the training sample can also be (0.95, 0.03, 0.02) or other suitable formats to introduce other suitable features of the adjustment model. Then the loss calculated by the model under this training sample is approximately equal to 0.0969. However, there may be multiple logits output by the fully connected layer corresponding to the three-dimensional vector. In one case, the values ​​of logits are relatively small (for example, all less than 10) or even have negative values. In another case, logits are relatively large (for example, all greater than 100). In the second case, the output probability reliability is relatively high. Therefore, the present disclosure introduces the confidence coefficient associated with logits to adjust the calculated loss.

[0067] See also Figure 2 The schematic diagram of the implementation steps of the training method of the machine learning model shown is a schematic diagram of the implementation steps of the method. The implementation of the method is described below with a specific embodiment.

[0068] Step S310: Dynamic statistics max ;

[0069] On the training set, during the training process, the output logits of the last fully connected layer of the dynamic statistical model are based on the raw scores of each node (position) in the fully connected layer. Calculate the value of each node (position) , get all e logits The maximum value of e max , and record the final stage of training e maxIt should be noted that this step is a dynamic process, and for each batch of training samples, a maximum value e will be generated. max , the training set is continuously input into the model and the maximum value will be continuously updated.

[0070] Step S320, calculating the confidence coefficient;

[0071] The aforementioned step S310 has calculated the first confidence parameter of each node (position) , then the confidence coefficient of the node (position) is:

[0072] Step S330, calculating and adjusting loss;

[0073] Add confidence terms to the loss function and train the machine learning model: The calculation of loss can be illustrated by the formula Then select the largest confidence coefficient from the confidence coefficients calculated from all nodes (positions) The index of the category or character corresponding to the selected node represents the most likely output selected by the model in this training sample, using the maximum confidence coefficient Get new loss , new A confidence coefficient is added to the original loss calculation, thereby introducing the characteristics of the fully connected layer in the process of updating model parameters by backpropagation.

[0074] When the machine learning model is trained, it may happen that each value in the output vector of the fully connected layer is a small value or a negative value. After the model normalizes the output vector, it will still select an index corresponding to a node with a relatively high probability to output, resulting in the OOD problem. In one example, the confidence coefficient of the node (position) and the probability of the softmax output can be used to evaluate the score of the current node (position), that is: p i_new =C i *p i ; Set the confidence threshold T and the maximum confidence coefficient For comparison, , the current score is not adopted, or the input is considered to be out-of-set OOD.

[0075] The present disclosure also provides a method for reasoning about a machine learning model. Figure 4 is a flowchart of the inference method of the machine learning model according to the embodiment of the present disclosure. Figure 4 The method steps involved are described.

[0076] Step S410, obtaining data to be processed.

[0077] In the embodiment of the present disclosure, the data to be processed may include but is not limited to the following types of data: image data, text data, audio data, video data and code. The data type used by the data to be processed can be selected according to actual needs, and the embodiment of the present disclosure does not specifically limit this.

[0078] Step S420, inputting the data to be processed into a trained machine learning model to obtain an inference result; wherein the trained machine learning model is obtained by training through any exemplary method of the present disclosure.

[0079] In the embodiments of the present disclosure, the inference result is used to describe the machine learning model's prediction of the data to be processed to obtain the next character or the category to which the data to be processed belongs or other suitable results. The data to be processed is input into the trained machine learning model exemplarily described in the present disclosure, and the inference result obtained has a higher accuracy, and the possibility of OOD in the inference result is also reduced.

[0080] In some embodiments, the input to the machine learning model of the present disclosure may be image data. The machine learning model may process the image data to generate an output. As an example, the machine learning model may process the image data to generate an image recognition output (e.g., recognition of image data, potential embedding of image data, encoded representation of image data, hashing of image data, etc.). As another example, the machine learning model may process the image data to generate an image segmentation output. As another example, the machine learning model may process the image data to generate an image classification output. As another example, the machine learning model may process the image data to generate an image data modification output (e.g., a change in image data, etc.). As another example, the machine learning model may process the image data to generate an encoded image data output (e.g., an encoded and / or compressed representation of image data, etc.). As another example, the machine learning model may process the image data to generate a prediction output.

[0081] In some embodiments, the input to the machine learning model of the present disclosure may be text or natural language data. The machine learning model may process text or natural language data to generate an output. As an example, the machine learning model may process natural language data to generate a language encoding output. As another example, the machine learning model may process text or natural language data to generate a potential text embedding output. As another example, the machine learning model may process text or natural language data to generate a classification output. As another example, the machine learning model may process text or natural language data to generate a text segmentation output. As another example, the machine learning model may process text or natural language data to generate a semantic intent output. As another example, the machine learning model may process text or natural language data to generate an upgraded text or natural language output (e.g., text or natural language data of higher quality than the input text or natural language, etc.). As another example, the machine learning model may process text or natural language data to generate a prediction output.

[0082] In some embodiments, the input to the machine learning model of the present disclosure may be speech data. The machine learning model may process speech data to generate an output. As an example, the machine learning model may process speech data to generate a speech recognition output. As another example, the machine learning model may process speech data to generate a speech translation output. As another example, the machine learning model may process speech data to generate a potential embedding output. As another example, the machine learning model may process speech data to generate an encoded speech output (e.g., an encoded and / or compressed representation of speech data, etc.). As another example, the machine learning model may process speech data to generate a text representation output (e.g., a text representation of input speech data, etc.). As another example, the machine learning model may process speech data to generate a prediction output.

[0083] In some embodiments, the input to the machine learning model of the present disclosure may be potentially encoded data (e.g., a latent space representation of an input, etc.). The machine learning model may process the potentially encoded data to generate an output. As an example, the machine learning model may process the potentially encoded data to generate a recognition output. As another example, the machine learning model may process the potentially encoded data to generate a reconstruction output. As another example, the machine learning model may process the potentially encoded data to generate a search output. As another example, the machine learning model may process the potentially encoded data to generate a re-clustering output. As another example, the machine learning model may process the potentially encoded data to generate a prediction output.

[0084] In some embodiments, the input to the machine learning model of the present disclosure may be statistical data. The machine learning model may process the statistical data to generate an output. As an example, the machine learning model may process the statistical data to generate a recognition output. As another example, the machine learning model may process the statistical data to generate a prediction output. As another example, the machine learning model may process the statistical data to generate a classification output. As another example, the machine learning model may process the statistical data to generate a segmentation output. As another example, the machine learning model may process the statistical data to generate a segmentation output. As another example, the machine learning model may process the statistical data to generate a visualization output. As another example, the machine learning model may process the statistical data to generate a diagnostic output.

[0085] In some cases, the machine learning model can be configured to perform a task including encoding input data for reliable and / or efficient transmission or storage (and / or corresponding decoding). For example, the task can be an audio compression task. The input can include audio data, and the output can include compressed audio data. In another example, the input includes visual data (e.g., one or more images or videos), the output includes compressed visual data, and the task is a visual data compression task. In another example, the task can include generating an embedding for input data (e.g., input audio or visual data).

[0086] In one possible implementation, the data to be processed is input into a trained machine learning model to obtain an inference result, which can be performed according to the following steps: input the data to be processed into a trained machine learning model to obtain a confidence coefficient and a probability distribution result of each node; determine the score of the node based on the confidence coefficient and the probability distribution result; in response to the confidence coefficient of the highest score being greater than a confidence threshold, take the content represented by the node corresponding to the highest score as the inference result; or in response to the confidence coefficient of the highest score being not greater than a confidence threshold, determine that the data to be processed is invalid data.

[0087] In this possible implementation, the data to be processed is input into a trained machine learning model, and the confidence coefficient of each node (position) is obtained based on the output vector generated by the fully connected layer before the last normalization layer of the machine learning model, and the probability corresponding to each node (position) is generated after the output vector passes through the last normalization layer.

[0088] It should be noted that the content of the node representation can be, for example, a predicted character, a classified category, an embedded representation, or other suitable content, and the last normalization layer can output, for example, an index to indicate the model's inference result for the data to be processed.

[0089] The product of the confidence coefficient and the probability distribution value corresponding to the node (position) is used as the reasoning score of the node (position), that is, the score of the node is determined based on the confidence coefficient and the probability distribution result.

[0090] In response to the confidence coefficient of the highest score being greater than a confidence threshold, the node corresponding to the highest score is taken as the inference result, or in response to the confidence coefficient of the highest score being not greater than a confidence threshold, the data to be processed is determined to be invalid data, and the invalid data can be used to characterize that the data to be processed is out-of-set data.

[0091] It should be noted that in this step, the confidence coefficient corresponding to the node (position) may be compared with the confidence threshold. When the confidence coefficient corresponding to the node (position) is not greater than the confidence threshold, the data to be processed may be determined to be invalid data.

[0092] In this possible implementation, based on the confidence threshold, the data to be processed can be quickly screened out as invalid data, thereby reducing the occurrence of out-of-set problems.

[0093] In one possible implementation, the data to be processed is input into a trained machine learning model to obtain an inference result, which can be performed according to the following steps: using the trained machine learning model to perform multiple inferences on the data to be processed to obtain multiple first inference results on each output order in at least part of the output order; using the trained machine learning model to perform one inference on another part of the output order to obtain a second inference result on each output order in the other part of the output order; using the first inference result with the largest number among the multiple first inference results as the third inference result on the output order; and combining the third inference result and the second inference result based on the output order to obtain the inference result.

[0094] For example, the data to be processed is a natural language text, and a machine learning model is used to reply to the natural language text. For example, the machine learning model outputs a total of 100 characters as a reply to the natural language text. For example, each output order of the 1st to 10th output order is selected for multiple reasoning, that is, the machine learning model performs multiple reasoning on the first output order. For example, 5 reasonings obtain 5 candidate characters (a, b, a, c, a), and then selects the character with the largest number (a) from the 5 candidate characters as the reasoning result of the first output order. Then the machine learning model continues to perform multiple reasoning for the second output order until the model completes multiple reasoning for the tenth output order. Multiple reasoning for an output order can alleviate the uncertainty caused by random hallucinations and improve the reliability of the model output.

[0095] In some embodiments, the output order of multiple inferences can be freely selected, for example, fewer (at least one) or more output orders are selected for multiple inferences. In some embodiments, in order to avoid the computing resource overhead caused by more multiple inferences, the output order of multiple inferences can be an appropriate value, and the remaining output orders are only performed once.

[0096] In other embodiments, as the output order moves backward, the number of inferences of multiple inferences can be reduced to a level not less than 1. For example, the first output order is inferred 5 times, the second output order is inferred 4 times, the third output order is inferred 3 times, until all output orders are inferred only once.

[0097] In some embodiments, the machine learning model can combine the final inference results at each output order to output a complete response to the user.

[0098] In some embodiments, the data to be processed is data to be classified, the inference result is a classification result, and the data to be processed is input into a trained machine learning model to obtain an inference result, including:

[0099] Using the trained machine learning model to perform multiple inferences on the data to be classified to obtain multiple classification results to be processed;

[0100] The pending classification results with the most identical classification results are taken as the classification results.

[0101] For example, the data to be processed is image data / natural language text to be classified, etc. The machine learning model can perform multiple inferences on each data to be classified. For example, five inferences are performed on the image data to be classified to obtain five inference results (cat, cat, dog, cat, car). The model outputs the "cat" class with the most identical classification among the five inference results as the classification result of the image data to be classified.

[0102] An embodiment of the present disclosure further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any example of the present disclosure is implemented.

[0103] An embodiment of the present disclosure further provides a computer program product, including a computer program, wherein the computer program, when executed by a processor of a computer, is used to cause the computer to execute a method described in any example of the present disclosure.

[0104] The present disclosure also provides an electronic device, including: at least one processor; and a memory connected to the at least one processor. The memory stores a computer program executable by the at least one processor, and the computer program is used to enable the electronic device to perform the method described in any example of the present disclosure when executed by the at least one processor.

[0105] refer to Figure 5 , a block diagram of an electronic device that can be used as a server or client of an embodiment of the present disclosure will now be described, which is an example of a hardware device that can be applied to various aspects of the present disclosure. The electronic device is intended to represent various forms of digital electronic computer devices, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention created by and / or claimed herein.

[0106] like Figure 5 As shown, the electronic device includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded from a storage unit 508 to a random access memory (RAM) 503. In the RAM 503, various programs and data required for the operation of the electronic device can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0107] Multiple components in the electronic device are connected to the I / O interface 505, including: an input unit 506, an output unit 507, a storage unit 508, and a communication unit 509. The input unit 506 can be any type of device that can input information to the electronic device, and the input unit 506 can receive input digital or character information, and generate key signal input related to user settings and / or function control of the electronic device. The output unit 507 can be any type of device that can present information, and can include but is not limited to a display, a speaker, a video / audio output terminal, a vibrator, and / or a printer. The storage unit 508 can include but is not limited to a disk, an optical disk. The communication unit 509 allows the electronic device to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks, and can include but is not limited to a modem, a network card, an infrared communication device, and / or a wireless communication transceiver, such as a Bluetooth device, a WiFi device, a WiMax device, a cellular communication device, and / or the like.

[0108] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a CPU, a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing units, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 501 performs the various methods and processes described above. For example, in some embodiments, the method embodiments of the present disclosure may be implemented as a computer program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on an electronic device via ROM 502 and / or a communication unit 509. In some embodiments, the computing unit 501 may be configured to perform the above-described method in any other appropriate manner (e.g., by means of firmware).

[0109] The computer programs for implementing the methods of the embodiments of the present disclosure may be written in any combination of one or more programming languages. These computer programs may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine as a stand-alone software package and partially on a remote machine, or entirely on a remote machine or server.

[0110] In the context of the embodiments of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or apparatus. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable signal medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, or infrared systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, random access memories (RAM), read-only memories (ROM), erasable programmable read-only memories (EPROM or flash memory), optical fibers, portable compact disk read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0111] It should be noted that the term "including" and its variations used in the embodiments created by the present disclosure are open inclusions, that is, "including but not limited to". The term "based on" means "based at least in part on". The term "one embodiment" means "at least one embodiment"; the term "another embodiment" means "at least one other embodiment"; the term "some embodiments" means "at least some embodiments". The modifications of "one" and "multiple" mentioned in the embodiments of the present disclosure are illustrative and not restrictive. Those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0112] The user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in the embodiments created by the present disclosure are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0113] The various steps described in the method implementation methods provided by the embodiments of the present disclosure may be performed in different orders and / or in parallel. In addition, the method implementation methods may include additional steps and / or omit the steps shown. The scope of protection of the present disclosure is not limited in this respect.

[0114] The term "embodiment" in this specification refers to specific features, structures or characteristics described in conjunction with the embodiment that can be included in at least one embodiment of the present disclosure. The appearance of this phrase in various places in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. The various embodiments in this specification are described in a related manner, and the same and similar parts between the various embodiments refer to each other. In particular, for the device, equipment, and system embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts refer to the partial description of the method embodiment.

[0115] The above-described embodiments only express several implementation methods of the present disclosure, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of protection. It should be pointed out that, for a person of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present disclosure, and these all belong to the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure shall be subject to the attached claims.

Claims

1. A method for training a machine learning model, wherein the machine learning model is a natural language processing model, comprising: Obtain an output vector generated by a fully connected layer before the last normalization layer of the machine learning model, wherein the output vector is used to represent the original score of each node of the fully connected layer; Calculate a corresponding first confidence parameter based on the original score of each of the nodes, and determine a second confidence parameter based at least on a maximum value of the first confidence parameters, wherein the second confidence parameter is a maximum value among all first confidence parameters generated in a currently performed training batch; using the ratio of the first confidence parameter to the second confidence parameter as the confidence coefficient of the node; and The loss of the machine learning model is adjusted using the maximum confidence coefficient in a training batch, and the machine learning model is trained based on the adjusted loss to obtain a trained machine learning model.

2. The method according to claim 1, wherein: Calculating the corresponding first confidence parameter based on the original score of each node includes: A first confidence parameter is calculated using N as a base and an original score of each node as an exponent, wherein N is greater than 1.

3. The method according to claim 1, wherein: Determining a second confidence parameter based at least on a maximum value of the first confidence parameter comprises: In response to the current training batch being the first training batch, selecting the maximum value from the first confidence parameters as the second confidence parameter; or In response to the current training batch being not the first training batch, selecting the maximum value from the first confidence parameters and comparing it with a second confidence parameter determined in the past in a previous training batch; A larger value is selected from the maximum value and the past second confidence parameter as a new second confidence parameter.

4. The method according to claim 1, wherein: The loss is a cross entropy loss function, and adjusting the loss of the machine learning model using the maximum confidence coefficient in a training batch includes: Calculating the loss of the machine learning model; Obtaining the maximum confidence coefficient; The loss is adjusted using the maximum confidence coefficient to obtain an adjusted loss.

5. A method for reasoning about a machine learning model, comprising: Acquire data to be processed, wherein the data to be processed is natural language text; The data to be processed is input into a trained machine learning model to obtain an inference result; wherein the trained machine learning model is obtained by training using the method described in any one of claims 1 to 4.

6. The method according to claim 5, wherein: The data to be processed is input into the trained machine learning model to obtain the inference results including: Inputting the data to be processed into a trained machine learning model to obtain a confidence coefficient and a probability distribution result of each node; determining a score of the node based on the confidence coefficient and the probability distribution result; In response to the confidence coefficient of the highest score being greater than a confidence threshold, taking the content represented by the node corresponding to the highest score as the inference result; or In response to the confidence coefficient of the highest score being not greater than a confidence threshold, the data to be processed is determined to be invalid data.

7. The method according to claim 5, wherein: Inputting the data to be processed into a trained machine learning model to obtain an inference result includes: performing multiple inferences on the partial output order using the trained machine learning model to obtain multiple first inference results on each output order in at least the partial output order; Performing an inference on another partial output sequence using the trained machine learning model to obtain a second inference result on each output sequence in the other partial output sequence; taking the first inference result with the largest number among the plurality of first inference results as the third inference result in the output order; The inference result is obtained by combining the third inference result and the second inference result based on an output order.

8. The method according to claim 5, wherein: The data to be processed is data to be classified, and the inference result is a classification result. The data to be processed is input into a trained machine learning model, and the inference result obtained includes: Using the trained machine learning model to perform multiple inferences on the data to be classified to obtain multiple classification results to be processed; The pending classification results with the most identical classification results are taken as the classification results.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 8 when executing the computer program.

10. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

11. A computer program product, wherein: The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images