Machine translation model training method and apparatus, electronic device, and computer-readable storage medium

Abstract

The disclosure discloses a machine translation model training method and device, electronic equipment and a computer readable storage medium, relating to the fields of deep learning, natural language processing and artificial intelligence. The method can include: in the model training process using a label smoothing method, dynamically adjusting the generated smoothing label according to the model output; determining the loss according to the smoothing label, and updating the model parameters according to the loss. The application of the scheme can better alleviate the overfitting problem in the model training process and improve the generalization ability of the model.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of artificial intelligence, in particular to a machine translation model training method and device, an electronic device and a computer readable storage medium in the fields of deep learning and natural language processing. BACKGROUND

[0002] Machine translation refers to a process of translating a natural language text (i.e., a source language) into another natural language text (i.e., a target language) by means of a machine, and is an important research field of natural language processing.

[0003] In recent years, neural network models have made great progress in machine translation tasks. However, as the models become more and more complex, the problem of overfitting in model training is increasingly prominent.

[0004] In order to alleviate this problem, a label smoothing method can be introduced in the model training process. However, the current label smoothing method usually adopts a fixed smoothing method, and the actual effect is often not ideal. SUMMARY

[0005] The present disclosure provides a machine translation model training method, device, electronic device and computer readable storage medium.

[0006] A machine translation model training method comprises:

[0007] In the model training process using the label smoothing method, the generated smoothing label is dynamically adjusted according to the model output.

[0008] The loss is determined according to the smoothing label, and the model parameters are updated according to the loss.

[0009] A machine translation model training device comprises a label determination module and a model updating module.

[0010] The label determination module is configured to dynamically adjust the generated smoothing label according to the model output in the model training process using the label smoothing method.

[0011] The model updating module is configured to determine the loss according to the smoothing label, and update the model parameters according to the loss.

[0012] An electronic device comprises:

[0013] At least one processor; and

[0014] A memory connected in communication with the at least one processor; wherein

[0015] The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method as described above.

[0016] A non-transitory computer-readable storage medium storing computer instructions for causing a computer to perform the method as described above.

[0017] A computer program product comprising a computer program which, when executed by a processor, implements the method as described above.

[0018] One embodiment in the above disclosure has the following advantages or beneficial effects: in the model training process using the label smoothing method, the generated smoothing label can be dynamically adjusted according to the model output, that is, an adaptive label smoothing method is used, so as to better alleviate the overfitting problem in the model training process and improve the generalization ability of the model, etc.

[0019] It should be understood that the content described in this part is not intended to identify key or important features of the embodiments of the present disclosure, nor to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0020] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0021] Figure 1 Flowchart of the first embodiment of the machine translation model training method according to the present disclosure;

[0022] Figure 2 Schematic diagram of the Gaussian attenuation function with different attenuation rates according to the present disclosure;

[0023] Figure 3 Flowchart of the second embodiment of the machine translation model training method according to the present disclosure;

[0024] Figure 4 Schematic diagram of the component structure of the machine translation model training device embodiment 400 according to the present disclosure;

[0025] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. DETAILED DESCRIPTION

[0026] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are cited as illustrative examples. Various details of the embodiments of the present disclosure are described herein in order to provide a thorough understanding of the present disclosure. It will be understood by those of ordinary skill in the art that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Also, in the following description, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0027] In addition, it should be understood that the term "and / or" herein merely describes an association relationship of associated objects, which means that there can be three relationships, for example, A and / or B can represent the three cases of A alone, A and B together, and B alone. In addition, the character " / " herein generally represents an "or" relationship between the front and rear associated objects.

[0028] Figure 1 A flowchart of the machine translation model training method according to the first embodiment of the present disclosure is shown in FIG. 1. As shown in FIG. 1, the following specific implementation is included. Figure 1

[0029] In step 101, in the process of model training by using the label smoothing method, the generated smoothing label is dynamically adjusted according to the model output.

[0030] In step 102, the loss is determined according to the smoothing label, and the model parameters are updated according to the loss.

[0031] As can be seen, in the scheme of the above method embodiment, in the process of model training by using the label smoothing method, the generated smoothing label can be dynamically adjusted according to the model output, that is, an adaptive label smoothing method is used, so as to better alleviate the overfitting problem in the process of model training and improve the generalization ability of the model, etc.

[0032] In actual application, in the training process of each round (epoch), for each batch, the following processing can be performed respectively: for each training sample (or training data, etc.) in the batch, the model output corresponding to the training sample is obtained respectively, and the smoothing label corresponding to the training sample can be determined according to the model output corresponding to the training sample.

[0033] Suppose that each batch respectively includes 100 training samples, then for each training sample, its corresponding smoothing label can be obtained according to the above method.

[0034] ​Specifically, for each training sample, the one-hot label corresponding to the training sample can be uniformly label smoothed to obtain a first smoothing result, and the model output corresponding to the training sample can be smoothed to obtain a second smoothing result, and then the smoothing label corresponding to the training sample can be determined according to the first smoothing result and the second smoothing result. How to obtain the one-hot label is prior art.

[0035] That is, to generate a smoothing label, two smoothing processes can be performed, one is uniform label smoothing of the one-hot label corresponding to the training sample, and the other is smoothing of the model output. Among them, the uniform label smoothing belongs to the existing fixed smoothing method, and the specific implementation is prior art. The specific implementation of the smoothing of the model output is not limited, which can be determined according to actual needs.

[0036] Through the above processing, the generated smoothing label can change dynamically according to the model output, so as to better alleviate the overfitting problem in the model training process and improve the generalization ability of the model.

[0037] As described above, for any training sample, after obtaining the first smoothing result and the second smoothing result, the smoothing label corresponding to the training sample can be further determined in combination with the two processing results. Specifically, the first smoothing result and the second smoothing result can be multiplied by the corresponding weight respectively, and the two products obtained can be added, and the sum is taken as the smoothing label corresponding to the training sample, wherein the weight corresponding to the first smoothing result can be a hyperparameter λ, and the weight corresponding to the second smoothing result can be 1-λ.

[0038] That is, the final smoothing label can be a convex combination of the first smoothing result and the second smoothing result, and the change of the convex combination is controlled by the hyperparameter λ.

[0039] That is,

[0040] Among them, denotes the one-hot label, β denotes the label smoothing rate, which can be pre-set, Φ1 is a uniform label smoothing function, which means uniform label smoothing, p denotes the model output, Φ2 is a model output smoothing function, which means smoothing of the model output, denotes the smoothing label.

[0041] The hyperparameter λ can be defined as a Gaussian decay function of the number of rounds in the training process. Figure 2This is a schematic diagram of the Gaussian decay functions for different decay rates (σ shown in the figure) described in this disclosure, as follows: Figure 2 As shown, the horizontal axis represents the number of rounds, i.e., the number of epochs, and the vertical axis represents λ.

[0042] like Figure 2 As shown, in the early stages of model training, λ is close to 1, and the model will focus more on the real training sample labels. As the model training process progresses, the quality of the model's output becomes higher and higher. Due to the decay of λ, the model will pay more and more attention to its own output, thereby dynamically adjusting the smooth labels used for training.

[0043] It can be seen that if λ is always 1 during training, the adaptive label smoothing method described in this disclosure will be simplified to the existing uniform label smoothing method.

[0044] Accordingly, λ can be determined based on the current round number and the preset decay rate.

[0045] Specifically, it may include:

[0046] Where t represents the current round number and σ represents the decay rate.

[0047] As can be seen, based on the above λ, the focus of the model can be dynamically adjusted, thereby improving the model training effect and model performance.

[0048] Furthermore, in each round of training, the following processing can be performed on each batch: for each training sample in the batch, the loss corresponding to the training sample is determined according to the model output and smoothing label corresponding to the training sample; the mean of the loss corresponding to each training sample in the batch is obtained, and the model parameters are updated according to the mean.

[0049] For any training sample, the loss corresponding to the training sample can be determined based on the model output, smoothed label, and loss function. The specific form of the loss function is not limited and can be determined according to actual needs.

[0050] Suppose a batch contains 100 training samples. The mean of the loss corresponding to these 100 training samples can be calculated. Based on the obtained mean, the model parameters can be updated using stochastic gradient descent. How to update the model parameters is a current technique.

[0051] Each batch in each training round can be processed in the manner described above until the model training is completed.

[0052] The following further describes the scheme of the present disclosure by taking the model as a machine translation model as an example.

[0053] Figure 3 The flowchart of the second embodiment of the machine translation model training method of the present disclosure is shown in FIG. 3. Figure 3 The following describes the specific implementation.

[0054] In step 301, in each round of training of the machine translation model, for each batch, the processing shown in steps 302-308 is performed respectively.

[0055] In step 302, for each training sample in the batch, the processing shown in steps 303-307 is performed respectively.

[0056] Each training sample can be composed of a source language and a target language.

[0057] In step 303, the model output corresponding to the training sample is obtained.

[0058] If p = f(x, y, θ); (3)

[0059] Where x represents the source language, y represents the target language, θ represents the model parameters, and f represents the model itself.

[0060] In step 304, uniform label smoothing processing is performed on the one-hot label corresponding to the training sample to obtain a first smoothing result, and smoothing processing is performed on the model output corresponding to the training sample to obtain a second smoothing result.

[0061] That is,

[0062] ω2 = Φ2(p); (5)

[0063] Where ω1 represents the first smoothing result, and ω2 represents the second smoothing result.

[0064] In step 305, the hyperparameter λ is determined.

[0065] For example, λ can be calculated according to formula (2).

[0066] In step 306, the first smoothing result and the second smoothing result are multiplied by the corresponding weights respectively, the two products obtained are added, and the sum is taken as the smoothing label corresponding to the training sample; where the weight corresponding to the first smoothing result is λ, and the weight corresponding to the second smoothing result is 1-λ.

[0067] The smoothing label can be calculated according to formula (1).

[0068] In step 307, the loss corresponding to the training sample is determined according to the model output corresponding to the training sample and the smoothed label.

[0069] For example, the loss corresponding to the training sample can be determined according to the model output corresponding to the training sample, the smoothed label, and a loss function, and the specific form of the loss function is not limited and can be determined according to actual needs.

[0070] In step 308, the mean of the loss corresponding to each training sample in the batch is obtained, and the model parameters are updated according to the obtained mean.

[0071] For example, the model parameters can be updated by using the stochastic gradient descent method according to the obtained mean.

[0072] Through the above processing, the generalization ability and translation effect (translation quality) of the machine translation model can be better improved. Experiments show that the machine translation model trained by the method of the present disclosure has a greater improvement in the Bilingual Evaluation Understudy (BLEU) score compared to the machine translation model trained by the existing label smoothing method, and the translation effect on various standard data sets is also significantly improved.

[0073] It should be noted that, for the foregoing method embodiments, in order to simply describe, they are expressed as a series of action combinations, but those skilled in the art should know that the present disclosure is not limited by the order of the described actions, because according to the present disclosure, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should know that the embodiments described in the specification all belong to preferred embodiments, and the actions and modules involved are not necessarily necessary for the present disclosure. In addition, the parts not described in detail in a certain embodiment can refer to the related description in other embodiments.

[0074] The above is the introduction of the method embodiment, and the scheme of the present disclosure will be further described through the device embodiment.

[0075] Figure 4 The constituent structure of the machine translation model training device embodiment 400 of the present disclosure is shown in FIG. 4. As shown in FIG. 4, it includes a label determination module 401 and a model updating module 402. Figure 4

[0076] The label determination module 401 is configured to dynamically adjust the generated smoothed label according to the model output during the model training process by using the label smoothing method.

[0077] ​The model updating module 402 is configured to determine a loss according to the generated smooth label, and update the model parameter according to the determined loss.

[0078] The label determining module 401 can perform the following processing for each batch in each round of training: for each training sample in the batch, obtain the model output corresponding to the training sample, and determine the smooth label corresponding to the training sample according to the model output.

[0079] Specifically, the label determining module 401 can perform uniform label smoothing processing on the one-hot label corresponding to any training sample to obtain a first smoothing processing result, and perform smoothing processing on the model output corresponding to the training sample to obtain a second smoothing processing result, and determine the smooth label corresponding to the training sample according to the first smoothing processing result and the second smoothing processing result.

[0080] That is, to generate a smooth label, two kinds of smoothing processing can be performed, one of which is uniform label smoothing processing on the one-hot label corresponding to the training sample, and the other of which is smoothing processing on the model output.

[0081] The label determining module 401 can determine the smooth label corresponding to any training sample in combination with the two processing results. Specifically, the first smoothing processing result and the second smoothing processing result can be multiplied by corresponding weights, and the two products obtained can be added, and the sum is taken as the smooth label corresponding to the training sample, wherein the weight corresponding to the first smoothing processing result can be a hyperparameter λ, and the weight corresponding to the second smoothing processing result can be 1-λ.

[0082] In addition, the label determining module 401 can determine λ according to the current round number and a pre-set decay rate.

[0083] Further, the model updating module 402 can perform the following processing for each batch in each round of training: for each training sample in the batch, determine the loss corresponding to the training sample according to the model output and the smooth label corresponding to the training sample; obtain the mean of the losses corresponding to the training samples in the batch, and update the model parameter according to the mean.

[0084] In actual application, for any training sample, the loss corresponding to the training sample can be determined according to the model output and the smooth label corresponding to the training sample and a loss function, etc. In addition, the model parameter can be updated by using a stochastic gradient descent method according to the obtained mean.

[0085] Figure 4The specific working procedure of the device embodiment shown is described in the foregoing method embodiment, and will not be described again.

[0086] In summary, the scheme described in the device embodiment of the present disclosure adopts an adaptive label smoothing manner, thereby better alleviating the overfitting problem in the model training process and improving the generalization ability of the model.

[0087] The scheme described in the present disclosure can be applied to the field of artificial intelligence, and particularly relates to the fields of deep learning and natural language processing.

[0088] Artificial intelligence is a discipline that studies enabling computers to simulate some thinking processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.) of people, and includes both hardware-level technologies and software-level technologies. The artificial intelligence hardware technologies generally include technologies such as sensors, special artificial intelligence chips, cloud computing, distributed storage, and big data processing, and the artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, and machine learning / deep learning, big data processing technology, knowledge graph technology, and several other directions.

[0089] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium, and a computer program product.

[0090] Figure 5 A schematic block diagram of an example electronic device 500 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, servers, servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are merely examples and are not intended to limit the implementations of the present disclosure described and / or claimed in this document.

[0091] As Figure 5 shown, the device 500 includes a computing unit 501 that can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 502 or a computer program loaded into a random access memory (RAM) 503 from a storage unit 508. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other through a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0092] A plurality of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a magnetic disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0093] The computing unit 501 can be various general and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs various methods and processes described above, such as the methods described in the present disclosure. For example, in some embodiments, the methods described in the present disclosure can be implemented as a computer software program, which is tangibly embodied in a machine-readable medium, such as the storage unit 508. In some embodiments, part or all of the computer program can be loaded and / or installed onto the device 500 via the ROM 502 and / or the communication unit 509. When the computer program is loaded onto the RAM 503 and executed by the computing unit 501, one or more steps of the methods described in the present disclosure can be performed. Alternatively, in other embodiments, the computing unit 501 can be configured to perform the methods described in the present disclosure by any other appropriate means, such as by means of firmware.

[0094] The various implementations of the systems and techniques described above can be realized in a digital electronic circuit system, an integrated circuit system, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), an application specific standard product (ASSP), a system on a chip system (SOC), a complex programmable logic device (CPLD), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0095] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a standalone software package, or entirely on a remote machine or server.

[0096] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0097] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0098] The systems and techniques described here can be implemented in a computing system that includes a back end component, e.g., as a data server, or that includes a middleware component, e.g., an application server, or that includes a front end component, e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here, or any combination of these, e.g., a client server. The components of the system can be interconnected by any form or medium of digital data communication, e.g., a communication network. Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0099] The computer system can include clients and servers. The clients and servers are generally remote from each other and typically interact through a communication network. The relationship of client and server can arise by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are a host product in the cloud computing service system to solve the defects of large management difficulty and weak business scalability in traditional physical hosts and virtual private servers (VPS). The servers can also be servers of a distributed system, or servers combined with a blockchain. Cloud computing refers to accessing elastic scalable shared physical or virtual resource pools through a network, resources can include servers, operating systems, networks, software, applications and storage devices, etc., and resources can be deployed and managed in a demand-driven and self-service manner. Through cloud computing technology, powerful data processing capabilities can be provided for artificial intelligence, blockchain and other technology applications and model training.

[0100] It should be understood that various forms of flow shown above can be used to reorder, add or delete steps. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions of the present disclosure can be achieved, which are not limited herein.

[0101] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A method for training a machine translation model, comprising: in a model training process using a label smoothing manner, dynamically adjusting a generated smoothed label according to a first smoothing result obtained based on a training sample and a second smoothing result obtained based on a model output corresponding to the training sample; wherein a final smoothed label is a convex combination of the first smoothing result and the second smoothing result, and a change of the convex combination is controlled by a hyperparameter λ; a weight corresponding to the first smoothing result is the hyperparameter λ, and a weight corresponding to the second smoothing result is 1-λ; the first smoothing result is a result of uniformly smoothing a one-hot label corresponding to the training sample; the second smoothing result is a result of smoothing the model output corresponding to the training sample; the λ is a Gaussian decay function of a round number in the training process, and a value of the λ will decay as the model training process proceeds; wherein the model output corresponding to the training sample is a language translation result; determining a loss according to the model output corresponding to the training sample, a loss function, and the smoothed label, and updating a model parameter according to the loss; wherein the training sample includes a source language and a target language, and the model corresponding to the training sample is a machine translation model.

2. The method of claim 1, wherein, The dynamically adjusting the generated smoothed label according to the first smoothing result obtained based on a training sample and the second smoothing result obtained based on a model output corresponding to the training sample comprises: in each round of the training process, for each batch, the following processing is performed respectively: for each training sample in the batch, the model output corresponding to the training sample is obtained, and the smoothed label corresponding to the training sample is determined according to the model output corresponding to the training sample. 3.The method of claim 1, further comprising: determining the λ according to a current round number and a pre-set decay rate.

4. The method of claim 2, wherein, The updating the model parameter according to the loss comprises: in each round of the training process, for each batch, the following processing is performed respectively: for each training sample in the batch, the loss corresponding to the training sample is determined according to the model output corresponding to the training sample and the smoothed label corresponding to the training sample; obtaining a mean value of the loss corresponding to each training sample in the batch, and updating the model parameter according to the mean value.

5. A machine translation model training apparatus comprising: a label determining module and a model updating module. The label determination module is configured to dynamically adjust the generated smoothing label according to a first smoothing processing result obtained based on a training sample and a second smoothing processing result obtained based on model output corresponding to the training sample in a model training process using a label smoothing manner; wherein a final smoothing label is a convex combination of the first smoothing processing result and the second smoothing processing result, and a change of the convex combination is controlled by a hyperparameter λ; a weight corresponding to the first smoothing processing result is the hyperparameter λ, and a weight corresponding to the second smoothing processing result is 1-λ; the first smoothing processing result is a result of uniformly smoothing a one-hot label corresponding to the training sample; the second smoothing processing result is a result of smoothing the model output corresponding to the training sample; the λ is a Gaussian decay function of a round number in the training process, and the value of λ will decay as the model training process proceeds; wherein the model output corresponding to the training sample is a language translation result. The model updating module is configured to determine a loss according to the model output corresponding to the training sample, a loss function and the smoothing label, and update a model parameter according to the loss. The training sample includes a source language and a target language, and the model corresponding to the training sample is a machine translation model.

6. The apparatus of claim 5, wherein, The label determination module is configured to, in each round of the training process, for each batch, respectively perform the following processing: for each training sample in the batch, respectively obtain model output corresponding to the training sample, and determine a smoothing label corresponding to the training sample according to the model output corresponding to the training sample.

7. The apparatus of claim 5, wherein, The label determination module is further configured to determine the λ according to a current round number and a pre-set decay rate.

8. The apparatus of claim 6, wherein, The model updating module is configured to, in each round of the training process, for each batch, respectively perform the following processing: for each training sample in the batch, respectively determine a loss corresponding to the training sample according to the model output corresponding to the training sample and the smoothing label; obtain a mean value of the losses corresponding to the training samples in the batch, and update the model parameter according to the mean value.

9. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-4.

10. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-4.

Images