A comprehensive method for extracting text from teaching video speech

Abstract

The present invention discloses a comprehensive teaching video speech text extraction method. First, the speech in the video is recognized based on the trained CLDNN model; then, the speech extracted in step 1 is de-noised based on the trained dual adversarial network to obtain the denoised speech. Finally, the trained end-to-end speech recognition model is used for back-end processing, wherein the end-to-end speech recognition model is established based on Conformer, the denoised speech is input into the speech recognition model, and then the output result is subjected to natural language processing to obtain the text content. Aiming at the characteristics of teaching videos, the present invention realizes text extraction through a comprehensive solution including three parts: speech activity detection, speech enhancement and back-end processing, and introduces the Noisy Student Training semi-supervised learning method so that the model can learn under large-scale unlabeled data. Experiments have proved that the scheme proposed in this article can effectively improve the efficiency and accuracy of text extraction.

Description

Technical Field

[0001] The invention relates to a comprehensive teaching video speech text extraction method. Background Art

[0002] When the Internet first emerged in the 1990s, the online education industry began to develop. Its biggest feature was the creation of a cross-time and space learning method. This type of education initially relied on recorded teaching content, with teachers recording videos and uploading them to the Internet for students to watch online or download. To this day, teaching videos are still an important carrier of online education.

[0003] However, video is a kind of unstructured data. Traditional information retrieval mainly uses text as the object, which makes it very difficult to search and browse teaching video content. How to obtain and manage video data to meet the search needs of users has become a hot research area. Content-Based Video Analysis (CBVA), proposed in the 1990s, extracts semantic content units from unstructured video data as retrieval objects by analyzing the video structure and semantic content.

[0004] Speech recognition is the process of converting speech signals into text. In 1952, KH Davis and others from Bell Labs developed the world's first Audry speech recognition system that could recognize ten English digits based on their research on the resonance spectrum of the digital vowel area, marking the beginning of modern speech recognition technology. [2] In the 1970s, speech recognition made substantial progress, and the Hidden Markov Model (HMM) technology gradually became the mainstream. In the late 1980s, speech recognition research solved the three major obstacles of large vocabulary, continuous speech and non-specific speakers. The representative one was the Sphinx system developed by Carnegie Mellon University.

[0005] However, teaching videos have the following characteristics: (1) Separation of audio from video stream: Voice signals contain many complex factors such as background noise, microphone quality, environmental factors, etc. Therefore, it is necessary to perform good noise reduction on the audio in voice preprocessing to enhance the voice signal. (2) Long and continuous audio: Teaching videos often contain long continuous audio streams, which requires that such audio files be "sentenced" smoothly before calling the recognition kernel, so more optimization is needed in endpoint detection.

[0006] Currently, many teaching videos are equipped with subtitles for users to watch. There are also automatic subtitle generation systems based on speech recognition to automatically add subtitles to videos and synchronize positioning. However, the separation of text content and video streams has not yet been achieved, and there are still deficiencies in the application of online education, lacking an automated method to obtain teaching video content text. If teaching video data is extracted through speech recognition, the scope of information retrieval can be effectively expanded, resource sharing can be achieved, and users can locate video clips based on text clips, which to a certain extent avoids loading videos to find teaching content. Summary of the invention

[0007] In order to solve the current technical problem that there is no way to automatically extract the text content of teaching video data, the present invention provides a comprehensive solution that can extract text from the audio stream of the teaching video.

[0008] In order to achieve the above technical purpose, the technical solution of the present invention is:

[0009] A comprehensive teaching video speech extraction method comprises the following steps:

[0010] Step 1: Recognize the speech in the video based on the trained CLDNN, i.e., convolutional-LSTM-fully connected neural network model. The CLDNN model is based on the CLDNN_64 model, and the original LSTM in CLDNN_64 is replaced by an attention module that can obtain feature information in both time and frequency.

[0011] Step 2: De-noise the speech extracted in step 1 based on the trained dual adversarial network to obtain denoised speech.

[0012] In the method, in step 1, the attention module is used to perform pooling operations on the input acoustic features in the hidden feature layer in three ways: maximum, average and standard deviation, to obtain three feature vectors, and then connect the three feature vectors and convolve them by four layers of one-dimensional convolutional layers, and activate the output of the last convolutional layer by a sigmoid function to obtain time-related features and frequency-related features, and finally sum and merge them with the hidden feature map H to obtain an improved hidden feature map H′. The expression of the attention module is:

[0013]

[0014]

[0015]

[0016] in, and ∈R T×1, and represent the maximum, mean, and standard deviation pooling results of the hidden feature map H along the frequency dimension, with dimensions of T×1, H temp f temp Output: and ∈R 1×D , and represent the maximum, mean, and standard deviation pooling results of the feature map H along the time dimension, with a dimension of 1×D, H freq represents f freq Output of f temp represents 4 layers of one-dimensional convolution operations. The number of convolution kernels in each layer is 3, 5, 5, and 1 respectively, and the convolution kernel size is 11×11; σ represents the sigmoid function. represents the sum of elements; f freq Represents and f temp Convolution operations with the same number of layers and kernels, but f freq The convolution kernel size is 21×21.

[0017] In the method, in step 2, the dual adversarial network includes an enhanced model E, a generator G and a discriminator D; wherein the enhanced model E is a convolutional recurrent network, the generator and the discriminator are both established based on the DC-GAN architecture, and the loss function L is defined for the enhanced model E E , define the loss function L for the discriminator D D(E) , that is, L D(E) Represents the method used to measure the difference between E(x) and s

[0018]

[0019]

[0020] s represents the noise-free speech sequence, i.e., clean speech, and p clean Corresponding to the speech sequence generated by the enhanced model, s~p clean represents s obeys p clean The sequence distribution of x represents the noisy speech sequence, E(x) represents the speech sequence generated by the enhanced model, and p noisy Represents a noisy speech sequence, x~p noisy represents x obeys p noisy The sequence distribution of is the mean operation; (s, x) represents the joint distribution of s and x, and λ is a hyperparameter used to balance reconstruction and adversarial losses;

[0021] Loss function L D(G) and the loss function L of G G They are:

[0022]

[0023]

[0024] z is a d-dimensional random vector following a standard Gaussian distribution N(0, 1), p z represents the probability distribution of z, G(z) is the generated speech sequence, so L D(G) represents a measure of the difference between G(z) and s.

[0025] In the method, the CLDNN model training process in step 1 includes:

[0026] The acoustic feature sequence with a frame length of 25ms and a shift length of 10ms output by a 40-dimensional logarithmic Mel filter bank is input into the CLDNN model, and the length T of the acoustic feature sequence is a fixed value to construct a data set for training; at the same time, the focal loss function is used to adjust the balance of speech and non-speech data in the training set, where the focal loss function l FL (y t )for:

[0027] l FL (y t )=-(1-y t ) γ log(y t )

[0028] Where γ is an adjustable parameter, and y represents the sequence label, y t is the label corresponding to the sequence at the current moment, and is the same as the cross entropy loss; during the execution, the modulation factor (1-y t ) γ The modulation factor is multiplied by the cross entropy; when the difference between the target value and the predicted value increases, the modulation factor increases; when the difference decreases, the modulation factor decreases, thereby strengthening the correction of misclassified instances and reducing the bias of the dominant class.

[0029] The method described above, the dual adversarial network training process in step 2 includes:

[0030] Adversarial enhancement: The enhanced model generates enhanced speech similar to the actual clean speech to deceive the discriminator, and allows the discriminator to learn the maximum distance between the enhanced speech and the clean speech; at the same time, the functional mean square error f-MSE is used to replace the adversarial loss term to correct the loss function of the enhanced model;

[0031] Adversarial generation: The discriminator learns the difference between clean and enhanced speech, as well as the distribution of clean speech, and uses a positive loss function;

[0032] At the same time, during the training process, gradient penalty regularization and adjustment of the update strategy are used to stabilize the training process: if the gradient of the output relative to the input is too large, the regularization will penalize the discriminator

[0033]

[0034] where y is sampled from a convex combination of real clean speech and fake speech: is the gradient of y; ε is a scalar sampled from a uniform distribution; in the dual adversarial network, the enhanced speech and the generated speech are considered as pseudo samples

[0035] During the training process, the model is updated using an adjusted update strategy: the discriminator is updated five times, and then the enhanced model and generator are updated once.

[0036] A comprehensive teaching video speech extraction text method, based on the denoised speech obtained by the above method, includes the following steps:

[0037] The trained end-to-end speech recognition model is used for back-end processing, where the end-to-end speech recognition model is built based on Conformer. The denoised speech is input into the speech recognition model, and then the output result is subjected to natural language processing to obtain text content.

[0038] In the method, the training process of the end-to-end speech recognition model includes:

[0039] Step 1: Use the SpecAugment method to train the Conformer network structure on the labeled dataset S to obtain the initial model M0, denoted as M;

[0040] Step 2: shallowly fuse M and the language model LSTM to integrate and evaluate the performance;

[0041] Step 3: Use the fused model to label the unlabeled dataset U;

[0042] Step 4: Set a confidence threshold, and then remove data below the confidence threshold, i.e., data with low confidence, to obtain filtered label data F(M(U));

[0043] Step 5, obtain data b*F(M(U)) by assigning weight b to the filtered labels;

[0044] Step 6: Mix b*F(M(U)) and labeled data, use the mixed data to enhance training M0, and obtain a new model M';

[0045] Step 7: Set M=M' and return to step 2 for a new round of training. The training ends when the model converges or the number of cycles reaches a threshold.

[0046] The technical effect of the present invention is that the front-end processing of the present invention adopts speech enhancement based on dual adversarial network (DAN) and endpoint detection model based on dual attention in time domain and frequency domain, which ensures the extraction of relatively pure speech signals in the presence of background noise and continuous long audio. At the same time, the back-end processing adopts the Transformer speech recognition model based on convolution enhancement: Conformer, which further improves the accuracy and stability of non-streaming speech recognition. The present invention is trained on the public data set AISHELL-1 and self-collected teaching video data, and compared with three existing models CNN-input, ACS and CNN+BLSTM+CTC, which shows that the method proposed in this article has high accuracy and good performance.

[0047] The present invention will be further described below in conjunction with the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the speech recognition execution flow of the present invention;

[0049] Figure 2 Schematic diagram of the Conformer model structure in the present invention. DETAILED DESCRIPTION

[0050] This embodiment is based on the fact that the audio in the teaching video is usually continuous long audio, especially in the case of continuous long audio in the background of non-stationary noise. This embodiment uses the CLDNN (convolution, long short-term memory, deep neural network) model, and the input is a 40-dimensional log Mel filter bank (log Mel-filterbanks), with acoustic features of 25ms frame length and 10ms shift length, for voice activity detection. The specific model structure is CLDNN_64, and a new attention module is introduced to integrate the LSTM in the CLDNN model, so that the endpoint detection model can pay attention to both time domain and frequency domain information, thereby improving the overall performance of the model. The imbalance problem of speech and non-speech data in the training set is reduced by using focal loss.

[0051] The Attention module provided in this embodiment can obtain feature information in terms of time and frequency respectively. First, the hidden feature layer is pooled in three ways: maximum, average and standard deviation. Then the three feature vectors obtained are connected and convolved by a one-dimensional convolution layer. The output of the last convolution layer is expanded to the original hidden feature layer H and then activated by a sigmoid function. When obtaining time and frequency information, there are differences in the pooling direction and the kernel size of the convolution layer.

[0052] Through the above operations, we can obtain the time-related features: and H temp ,in They represent the maximum, mean, and standard deviation pooling results of the hidden feature map H along the frequency dimension, with dimensions of T×1, H temp f temp Output of. Frequency-dependent characteristics: and H freq ,in They represent the maximum, mean, and standard deviation pooling results of the feature map H along the time dimension, with a dimension of 1×D, H freq represents f freq The output of . Finally, it is merged with H by element-wise summation, and then the improved hidden feature map H' is obtained. The attention module is expressed as follows:

[0053]

[0054]

[0055]

[0056] Among them, f temp represents a one-dimensional convolution with 11 convolution kernels. It consists of 4 layers, with the number of filters being 3, 5, 5, and 1 respectively. σ represents the sigmoid function, represents the sum of elements. freq Represents and f temp Convolution operations with the same number of layers and kernels, but f freq It is a one-dimensional convolution with 21 convolution kernels. The sequence length T in the training process is fixed in advance to construct the mini-batch for training.

[0057] This embodiment also uses the focal loss function to reduce the inefficiency of training in an unbalanced classification environment, which is described as follows:

[0058] l FL (y t )=-(1-y t ) γ log(y t ) (4)

[0059] where γ is an adjustable parameter and y t Same as cross entropy loss. In operation, the modulation factor (1-y t ) γ Multiply by the cross entropy. When the difference between the target value and the predicted value increases, the modulation factor increases. When the difference decreases, the modulation factor decreases. Through this mechanism, it strengthens the correction of misclassified instances and reduces the bias of the dominant class.

[0060] Since there is definitely noise in the audio of the teaching video, this noise will undoubtedly interfere with the learning process of the model, and the audio in the teaching video is mono audio. In order to improve the noise robustness of speech recognition (ASR), the front-end processing often uses an enhancement method based on the Generative Adversarial Network (GAN). Similarly, this article also uses a double adversarial network (DAN) model for front-end processing.

[0061] The dual adversarial network is mainly composed of an enhanced model E, a generator G, and a discriminator D. The enhanced model E is a convolutional recurrent network (Convolution Recurrent Networkk), and the input is the 40-dimensional log Mel-filterbanks feature of noisy speech. The generator and discriminator use the DC-GAN architecture, and randomly intercept 40 frames of log Mel-filterbanks features from clean and enhanced speech, which are treated as fake samples and real samples respectively. The intercepted log Mel-filterbanks features of the discriminator need to be upsampled to 64*64 to match its structure. For the generator, the input is a 128-dimensional random vector that conforms to the Gaussian distribution, and the output is a 64*64 upsampled log Mel-filterbanks fragment.

[0062] In the adversarial enhancement process, the main task of the enhancement model E is to reconstruct the clean speech s from the noisy speech x, and the discriminator D is trained to distinguish the real clean speech s from the enhanced speech By performing adversarial training between E and D, the enhanced model attempts to deceive the discriminator by generating speech similar to the actual clean speech, and let the discriminator learn the maximum distance between the enhanced speech and the clean speech. At the same time, the functional mean square error f-MSE is used to replace the adversarial loss term and correct the loss function of E. The loss functions of D and E are as follows:

[0063]

[0064]

[0065] where λ is a hyperparameter used to balance the reconstruction and adversarial losses.

[0066] In the adversarial generation process, an additional generator G is added and trained against the discriminator. By performing adversarial training between G and D, the discriminator learns not only the difference between clean speech and enhanced speech, but also the distribution of clean speech. The loss functions of D and G are as follows:

[0067]

[0068]

[0069] where z is a d-dimensional random vector following a standard Gaussian distribution N(0, 1).

[0070] GANs are always plagued by instability during training. This example uses gradient penalty regularization and an adjusted update strategy to stabilize the training process. Regularization will penalize the discriminator if the gradient of the output relative to the input is too large:

[0071]

[0072]

[0073] where y is sampled from a convex combination of real clean speech and fake speech: ε is a scalar sampled from a uniform distribution. In DAN, the enhanced speech and the generated speech are all considered as pseudo samples: To further stabilize the training process, the model is updated using an adjusted update strategy. Specifically, the discriminator is updated five times, and then the enhanced model and generator are updated once.

[0074] By integrating the two adversarial processes and gradient penalty regularization, the loss function of the discriminator in DANs can be obtained as follows:

[0075]

[0076] where γ is a hyperparameter used for GP regularization.

[0077] After the first two steps, unlabeled speech samples can be extracted from the teaching video. The next step is to enter the back-end processing part of speech recognition. This embodiment uses Conformer to build the model, that is, Figure 2 Model shown.

[0078] Once the structure of the end-to-end model is determined, the performance of the final model depends largely on the data provided, and the data requirements of the end-to-end model are also greater than those of the traditional model. Although there are many public data sets in the field of speech recognition, if you want to study speech recognition in a specific field, you will find that data problems still exist. For example, in teaching video speech recognition, without data that meets specific scenarios, it is impossible to determine whether the model's performance in actual situations meets expectations.

[0079] In order to solve this problem, this embodiment uses Noisy Student Training to enable the model to perform semi-supervised learning. The process of Noisy Student Training is as follows:

[0080] 1. Use the SpecAugment method to train the Conformer network on the labeled dataset S and obtain the initial model M0, denoted as M;

[0081] 2. Shallow fusion of M and LM models

[22] And evaluate the performance;

[0082] 3. Use the fused model and unlabeled dataset U to label;

[0083] 4. Filter the data with lower confidence to obtain the filtered label data F(M(U));

[0084] 5. Balance F(M(U)) to get data b*F(M(U));

[0085] 6. Mix the data obtained in 5 with the labeled data and use speech enhancement to train a new model M';

[0086] 7. Make M = M' and jump to step 2 for a new round of training. The training ends when the model converges or the number of cycles reaches the threshold.

[0087] Figure 1 The proposed scheme in this paper is explained. After obtaining the original unlabeled teaching video data, unlabeled audio clips are obtained after VAD processing and DAN denoising. Then the unlabeled audio clips are used to train the model using the Noisy Student Training process.

[0088] The data sets used in this embodiment include the public data set AISHELL-1 (with labeled data) and the teaching video data set to verify the effect. The teaching video data set has been processed by the pre-trained speech activity detection model and speech enhancement model, and finally a total of 800 hours of training samples were obtained. 40-dimensional log Mel-filterbanks is used as the input feature of the model, the frame window range is 25 milliseconds, and the moving window between consecutive frames is 10 milliseconds. The decoding target vocabulary includes all 4334 characters appearing in AISHELL-1 (4328 Chinese characters and 6 special marks '', 'a', 'b', 'c', 'k', 't'). Since AISHELL-1 already contains commonly used Chinese characters, no target vocabulary is added on this basis.

[0089] In the setting of Noisy Student Training, this embodiment uses SpecAugment and language model LM, does not use any filtering and balancing operations, and does not use batch-wise mixing in the mixing operation. The settings of SpecAugment and LM are as follows:

[0090] 1. This embodiment uses SpecAugment with adaptive time mask size, two frequency masks with mask parameter (F) 27 and ten time masks with maximum time mask ratio (pS) 0.05, where the maximum size of the time mask is set to pS multiplied by the utterance length, and time warping is not used.

[0091] 2. This embodiment uses a 3-layer LSTM language model with a width of 4096, and is trained on the AISHELL-1 corpus and self-collected teaching-related corpus data.

[0092] The solution proposed in this embodiment is compared with three existing models: CNN-input, ACS, and CNN+BLSTM+CTC. The test set uses data manually labeled from teaching videos, which have never appeared in the training data. The three models are trained on AISHELL-1 data and mixed data of AISHELL-1 and teaching video data to show performance.

[0093] The comparison results are shown in Table 1, where the lowest WER is shown in bold. The proposed scheme achieves the best performance, and the model trained on mixed data reduces the word error rate by about 5% compared to the model trained on AISHELL-1 alone, and finally reaches a word error rate of 21.3% in the teaching video data.

[0094] Table 1 Comparison results with other models

[0095]

Claims

1. A comprehensive teaching video speech extraction method, characterized in that: The following steps are involved: Step 1: Recognize the speech in the video based on the trained CLDNN, i.e., convolutional-LSTM-fully connected neural network model. The CLDNN model is based on the CLDNN_64 model, and the original LSTM in CLDNN_64 is replaced by an attention module that can obtain feature information in both time and frequency. Step 2, removing noise from the speech extracted in step 1 based on the trained dual adversarial network to obtain denoised speech; In step 2, the dual adversarial network includes an enhanced model E, a generator G and a discriminator D; wherein the enhanced model E is a convolutional recurrent network, and both the generator and the discriminator are established based on the DC-GAN architecture, and the loss function L is defined for the enhanced model E E , define the loss function L for the discriminator D D(E) , that is, L D(E) Represents the method used to measure the difference between E(x) and s s represents the noise-free speech sequence, i.e., clean speech, and p clean Corresponding to the speech sequence generated by the enhanced model, s~p clean represents s obeys p clean The sequence distribution of x represents the noisy speech sequence, E(x) represents the speech sequence generated by the enhanced model, and p noisy Represents a noisy speech sequence, x~p noisy represents x obeys p noisy The sequence distribution of is the mean operation; (s, x) represents the joint distribution of s and x, and λ is a hyperparameter used to balance reconstruction and adversarial losses; Loss function L D(G) and the loss function L of G G They are: z is a d-dimensional random vector following a standard Gaussian distribution N(0, 1), p z represents the probability distribution of z, G(z) is the generated speech sequence, so represents a measure of the difference between G(z) and s; The dual adversarial network training process in step 2 includes: Adversarial enhancement: The enhanced model generates enhanced speech similar to the actual clean speech to deceive the discriminator, and allows the discriminator to learn the maximum distance between the enhanced speech and the clean speech; at the same time, the functional mean square error f-MSE is used to replace the adversarial loss term to correct the loss function of the enhanced model; Adversarial generation: The discriminator learns the difference between clean and enhanced speech, as well as the distribution of clean speech, and uses a positive loss function; At the same time, during the training process, gradient penalty regularization and adjustment of the update strategy are used to stabilize the training process: if the gradient of the output relative to the input is too large, the regularization will penalize the discriminator L D(GP) : where y is sampled from a convex combination of real clean speech and fake speech: is the gradient of y; ε is a scalar sampled from a uniform distribution; in the dual adversarial network, the enhanced speech and the generated speech are considered as pseudo samples During the training process, the model is updated using an adjusted update strategy: the discriminator is updated five times, and then the enhanced model and generator are updated once.

2. The method according to claim 1, characterized in that In the step 1, the attention module is used to perform pooling operations on the input acoustic features in the hidden feature layer in three ways: maximum, average and standard deviation, to obtain three feature vectors, and then connect the three feature vectors and convolve them by four layers of one-dimensional convolutional layers, and activate the output of the last convolutional layer by a sigmoid function to obtain time-related features and frequency-related features, and finally sum and merge with the hidden feature map H to obtain an improved hidden feature map H′. The expression of the attention module is: in, and ∈R T×1 , and represent the maximum, mean, and standard deviation pooling results of the hidden feature map H along the frequency dimension, with dimensions of T×1, H temp f temp Output: and ∈R 1×D , and represent the maximum, mean, and standard deviation pooling results of the feature map H along the time dimension, with a dimension of 1×D, H freq represents f freq Output of f temp represents 4 layers of one-dimensional convolution operations. The number of convolution kernels in each layer is 3, 5, 5, and 1 respectively, and the convolution kernel size is 11×11; σ represents the sigmoid function, ⊕ represents the element-wise summation; f freq Represents and f temp Convolution operations with the same number of layers and kernels, but f freq The convolution kernel size is 21×21.

3. The method according to claim 2, characterized in that The CLDNN model training process in step 1 includes: The acoustic feature sequence with a frame length of 25ms and a shift length of 10ms output by a 40-dimensional logarithmic Mel filter bank is input into the CLDNN model, and the length T of the acoustic feature sequence is a fixed value to construct a data set for training; at the same time, the focal loss function is used to adjust the balance of speech and non-speech data in the training set, where the focal loss function l FL (y t )for: l FL (and t )=-(1-y t ) γ log(y t ) Where γ is an adjustable parameter, and y represents the sequence label, y t is the label corresponding to the sequence at the current moment, and is the same as the cross entropy loss; during the execution, the modulation factor (1-y t ) γ The modulation factor is multiplied by the cross entropy; when the difference between the target value and the predicted value increases, the modulation factor increases; when the difference decreases, the modulation factor decreases, thereby strengthening the correction of misclassified instances and reducing the bias of the dominant class.

4. A comprehensive teaching video speech extraction text method, characterized in that: The denoised speech obtained based on the method described in any one of claims 1 to 3 comprises the following steps: The trained end-to-end speech recognition model is used for back-end processing, where the end-to-end speech recognition model is built based on Conformer. The denoised speech is input into the speech recognition model, and then the output result is subjected to natural language processing to obtain text content.

5. The method according to claim 4, characterized in that The training process of the end-to-end speech recognition model includes: Step 1: Use the SpecAugment method to train the Conformer network structure on the labeled dataset S to obtain the initial model M0, denoted as M; Step 2: shallowly fuse M and the language model LSTM to integrate and evaluate the performance; Step 3: Use the fused model to label the unlabeled dataset U; Step 4: Set a confidence threshold, and then remove data below the confidence threshold, i.e., data with low confidence, to obtain filtered label data F(M(U)); Step 5, obtain data b*F(M(U)) by assigning weight b to the filtered labels; Step 6: Mix b*F(M(U)) and labeled data, use the mixed data to enhance training M0, and obtain a new model M'; Step 7: Set M=M' and return to step 2 for a new round of training. The training ends when the model converges or the number of cycles reaches a threshold.

Images