Noise reduction method and device based on neural network model

Abstract

The invention discloses a noise reduction method based on a neural network model, and the neural network model comprises a first encoder used for extracting coding features, and a state evaluator used for evaluating the correlation degree between the coding features. The state extractor is used for extracting at least one coding feature and recombining the associated historical coding feature into a fusion feature according to a state evaluation result output by the state evaluator; the first decoder is used for extracting a decoding feature from the fusion feature; the noise reduction method comprises the steps of inputting data to be subjected to noise reduction into a trained neural network model to obtain the first covering code, and performing data shielding operation on the data to be subjected to noise reduction based on the first covering code to obtain noise reduction data. The first encoder and the state evaluator are trained by using sample data obtained by the trained network model with the attention mechanism, and low power consumption is taken into account while the noise reduction effect is kept.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and in particular, to a noise reduction method based on a neural network model. Background Technology

[0002] With the development of artificial intelligence technology, more and more AI technologies are being applied to information and signal processing. For example, neural network models are used to reduce noise in audio, image, and text data.

[0003] Since denoising based on neural network models requires a lot of computing power, it is difficult to maintain the denoising effect while keeping power consumption low. Summary of the Invention

[0004] This invention provides a noise reduction method based on a neural network model, which maintains the noise reduction effect while taking into account low power consumption.

[0005] The first aspect of this invention provides a noise reduction method based on a neural network model, wherein the neural network model includes:

[0006] The first encoder used to extract encoded features

[0007] A state estimator used to assess the degree of correlation between encoded features.

[0008] A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator.

[0009] The first decoder used to extract decoding features from the fused features, and

[0010] A first masking code generator is used to generate a first masking code for noise reduction based on decoding features.

[0011] The noise reduction method includes:

[0012] The data to be denoised is input into the trained neural network model to obtain the first masking code output by the neural network model.

[0013] Based on the first masking code, a data masking operation is performed on the data to be denoised to obtain the denoised data.

[0014] One implementation method involves inputting the data to be denoised into a trained neural network model to obtain a first masking code output by the neural network model, including:

[0015] The data to be denoised is input into the first encoder, the encoded features output by the first encoder are input into the state estimator, and the state evaluation result output by the state estimator and the encoded features output by the first encoder are input into the state extractor.

[0016] The fused features output by the state extractor are input into the first decoder, and the decoded features output by the first decoder are input into the first masking code generator to obtain the first masking code output by the first masking code generator.

[0017] The data masking operation based on the first masking code includes:

[0018] Multiply the first masking code with the data to be denoised.

[0019] In one implementation method, the neural network model is trained in the following manner:

[0020] First training sample data is obtained for training the first encoder and the state estimator. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features, wherein the sample state evaluation results are used to characterize the degree of correlation between the encoded features of the first sample features.

[0021] The first encoder and state estimator are trained using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator.

[0022] The first sample feature is input into the trained first encoder to obtain the sample encoded feature output by the trained first encoder. The sample encoded feature is then input into the trained state estimator to obtain the state evaluation result of the sample encoded feature output by the trained state estimator.

[0023] The sample encoding features and the state evaluation results of the sample encoding features are input into the state extractor to obtain the sample fusion features output by the state extractor. The sample fusion features and the sample mask codes corresponding to the first sample features of the sample fusion features are used as the second training sample data for training the first decoder and the first mask code generator.

[0024] The first decoder and the first mask generator are trained using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first mask generator.

[0025] in,

[0026] The evaluation result of the sample state in the first training sample data is obtained by inputting the first sample feature data into the trained network model with attention mechanism to obtain the correlation information between the first sample features.

[0027] In one implementation method, the network model with the attention mechanism is an attention network model.

[0028] The attention network model is trained in the following manner:

[0029] The features of the first sample are input into the attention network model to obtain the sample prediction results output by the attention network model.

[0030] Based on the loss function value between the sample prediction result and the sample masking code corresponding to the first sample feature, adjust the model parameters of the attention network model until the expected result is achieved.

[0031] In one embodiment, the attention network model includes: a second encoder, an attention correlator, a second decoder, and a second masking code generator connected in sequence.

[0032] The step of inputting the features of the first sample into the attention network model to obtain the prediction result output by the attention network model includes:

[0033] The first sample feature is input into the second encoder, the sample encoded feature output by the second encoder is input into the attention correlator, the sample attention correlator output by the attention correlator is input into the second decoder, and the sample decoded feature output by the second decoder is input into the second mask code generator to obtain the sample prediction result of the first sample feature output by the second mask code generator.

[0034] One implementation method involves obtaining the sample state evaluation result from the first training sample data by inputting the first sample feature data into a trained network model with an attention mechanism to obtain information on the degree of correlation between the first sample features, including:

[0035] The first sample feature data is input into the second encoder of the trained attention network model. The sample encoded features output by the second encoder are then input into the attention correlator, resulting in an attention matrix output by the attention correlator that characterizes the degree of correlation between sample encoded features. In this matrix, the value in any column of any row represents the degree of correlation between the encoded feature corresponding to that row and the encoded feature corresponding to that column.

[0036] The sum of the values ​​in each column of the attention matrix is ​​calculated to obtain the sample state evaluation result of the first sample feature corresponding to the encoded feature of each column.

[0037] In one implementation, the attention network model is a transform structure;

[0038] The first encoder is a first RRN structure, the first decoder is a second RRN structure, the state estimator is a first DNN structure, and the first masking code generator is a second DNN structure.

[0039] The step of training the first encoder and the state estimator using the first training sample data includes:

[0040] The first sample feature from the first training sample data is input into the first encoder, and the sample encoded feature output by the first encoder is input into the state evaluator.

[0041] Based on the loss function value between the state evaluation result output by the state estimator and the sample state evaluation result of the input first sample feature, adjust the parameters of the first encoder and the state estimator until the expected result is achieved.

[0042] One implementation method involves inputting the sample coding features and the state evaluation results of the sample coding features into a state extractor to obtain the sample fusion features output by the state extractor. This includes: sequentially extracting the input sample coding features according to the extraction principle of adding one previously unextracted coding feature to the previously extracted historical coding features for each extraction, until there are no previously unextracted sample coding features. The previously extracted historical coding features refer to all coding features extracted in the previous extraction process adjacent to the current extraction.

[0043] During each extraction, the current state evaluation result of the extracted sample coding features is updated with the set extraction strength as the step size to obtain the updated state evaluation result. The state evaluation results that meet the conditions are selected from the updated state evaluation results, and the sample coding features corresponding to the state evaluation results that meet the conditions are used as the extraction result of this extraction. The extraction results of this extraction are fused to obtain the sample fusion feature used to represent the fusion of the added sample coding features and its associated historical sample coding features.

[0044] One implementation method involves updating the current state evaluation result of the extracted sample encoding features with a set extraction strength as the step size, including:

[0045] For each sample coding feature in the extracted sample coding features,

[0046] Subtract the extraction strength from the current state evaluation result of the encoded feature of the sample to obtain the updated state evaluation result of the encoded feature of the sample.

[0047] In one implementation method, selecting the state assessment results that meet the conditions from the updated state assessment results includes:

[0048] For each sample coding feature in the extracted sample coding features,

[0049] Determine whether the updated state evaluation result of the encoded features of the sample is greater than or equal to the set state evaluation threshold.

[0050] If so, the evaluation result of the updated state of the encoded features of the sample is deemed to meet the conditions;

[0051] In one implementation method, the fusion feature is obtained in the following manner:

[0052] The state extractor extracts the input encoded features one by one according to the extraction processing method during training, and obtains the fused features used to represent the added encoded features and their associated historical encoded features at each extraction.

[0053] One implementation method, wherein training the first decoder and the first masking code generator using the second training sample data, includes:

[0054] The sample fusion features from the second training sample data are input into the first decoder, and the sample decoding features output by the first decoder are input into the first mask code generator.

[0055] Based on the loss function value between the mask code output by the first mask code generator and the sample mask code of the first sample feature corresponding to the fused feature of the input sample, the parameters of the first decoder and the first mask code generator are adjusted until the expected result is achieved.

[0056] In one implementation method, the first sample feature is obtained in the following manner:

[0057] Noise sample data and noise-free source sample data are collected, and the noise samples are fused with the source samples to obtain noise source sample data.

[0058] Extract features from the noise source sample data to obtain the first sample features;

[0059] The sample masking code is obtained in the following manner:

[0060] Extract features from the source sample data to obtain the features of the second sample.

[0061] Calculate the sample masking code based on the features of the first sample and the features of the second sample.

[0062] In one implementation, the source sample data is noise-free sample speech data, and the noise sample data is sample audio data including noise.

[0063] One implementation method, the fusion of noise samples and source samples, includes: superimposing a frame of sample speech data and a frame of sample audio data in terms of intensity to obtain sample noise speech data;

[0064] One implementation method includes extracting the features of the noise source sample data, which includes: extracting the frequency domain features of the sample noise speech to obtain the first sample frequency domain features;

[0065] One implementation method involves extracting the features of the source sample data, including: extracting the frequency domain features of the sample speech data to obtain the second sample frequency domain features;

[0066] One implementation method, wherein calculating the sample masking code based on the first sample features and the second sample features, includes:

[0067] Calculate the sum of squares of the real and imaginary parts of the frequency domain features of the first sample.

[0068] Calculate the sum of the product of the real parts of the frequency domain features of the first sample and the real parts of the frequency domain features of the second sample, and the sum of the product of the imaginary parts of the frequency domain features of the first sample and the imaginary parts of the frequency domain features of the second sample. Divide this sum by the sum of squares to obtain the real part masking code.

[0069] Calculate the difference between the product of the real part of the frequency domain features of the first sample and the imaginary part of the frequency domain features of the second sample, and the product of the imaginary part of the frequency domain features of the first sample and the real part of the frequency domain features of the second sample. Divide this difference by the sum of squares to obtain the imaginary part masking code.

[0070] The sample masking code is obtained by concatenating the real part masking code and the imaginary part masking code.

[0071] In one implementation, the first encoder and state estimator in the neural network model are trained using sample data that characterizes the degree of correlation between sample features obtained from the trained network model with attention mechanism. Using the trained first encoder and state estimator, sample data for training the first decoder and the first masking generator are obtained through the state extractor, so that the first decoder of the trained neural network model outputs decoded features that characterize the degree of correlation between the data to be denoised.

[0072] A second aspect of the present invention provides a training method for a neural network model for noise reduction, the neural network model comprising: a first encoder for extracting coded features, and...

[0073] A state evaluator used to assess the correlation between encoded features.

[0074] A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator.

[0075] The first decoder used to extract decoding features from the fused features, and

[0076] A first masking code generator is used to generate a first masking code for noise reduction based on decoding features.

[0077] Among them, the first encoder, state estimator, first decoder, and first mask code generator are neural networks to be trained.

[0078] The training method includes:

[0079] First training sample data is obtained for training the first encoder and the state estimator. This first training sample data includes: first sample features characterizing the features of sample data including noise, and sample state evaluation results of the first sample features, wherein the sample state evaluation results characterize the degree of correlation between the first sample features.

[0080] The first encoder and state estimator are trained using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator.

[0081] The first sample feature is input into the trained first encoder to obtain the sample encoded feature output by the trained first encoder. The sample encoded feature is then input into the trained state estimator to obtain the state evaluation result of the sample encoded feature output by the trained state estimator.

[0082] The sample encoding features and the state evaluation results of the sample encoding features are input into the state extractor to obtain the sample fusion features output by the state extractor. The sample fusion features and the sample mask codes corresponding to the first sample features of the sample fusion features are used as the second training sample data for training the first decoder and the first mask code generator.

[0083] The first decoder and the first mask generator are trained using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first mask generator.

[0084] in,

[0085] The sample state evaluation result in the first training sample data is obtained by inputting the first sample feature data into the sample attention association information obtained in the trained network model with attention mechanism.

[0086] A third aspect of the present invention provides a noise reduction device based on a neural network model, the noise reduction device comprising:

[0087] The noise reduction information acquisition module is used to input the data to be denoised into a trained neural network model and obtain the first masking code output by the neural network model for noise reduction.

[0088] The noise reduction module is used to perform data masking operations on the data to be denoised based on the first masking code to obtain the denoised data;

[0089] in,

[0090] Neural network models include:

[0091] The first encoder used to extract encoded features

[0092] A state evaluator used to assess the correlation between encoded features.

[0093] A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator.

[0094] The first decoder used to extract decoding features from the fused features, and

[0095] A first masking code generator for generating a first masking code for noise reduction based on decoding features.

[0096] A fourth aspect of the present invention provides a training apparatus for a neural network model for noise reduction, the neural network model comprising: a first encoder for extracting coded features, and...

[0097] A state evaluator used to assess the correlation between encoded features.

[0098] A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator.

[0099] The first decoder used to extract decoding features from the fused features, and

[0100] A first masking code generator is used to generate a first masking code for noise reduction based on decoding features.

[0101] Among them, the first encoder, state estimator, first decoder, and first mask code generator are neural networks to be trained.

[0102] The training device includes:

[0103] The first training sample data acquisition module is used to acquire first training sample data for training the first encoder and the state estimator. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features. The sample state evaluation results are used to characterize the evaluation results of the correlation between the first sample features.

[0104] The first training module is used to train the first encoder and the state estimator using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator.

[0105] The second training sample data acquisition module is used to input the first sample features into the trained first encoder to obtain the sample encoding features output by the trained first encoder; input the sample encoding features into the trained state evaluator to obtain the state evaluation result of the sample encoding features output by the trained state evaluator; input the sample encoding features and the state evaluation result of the sample encoding features into the state extractor to obtain the sample fusion features output by the state extractor; and use the sample fusion features and the sample mask code corresponding to the first sample features as the second training sample data for training the first decoder and the first mask code generator.

[0106] The second training module is used to train the first decoder and the first masking code generator using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first masking code generator.

[0107] This application provides a denoising method based on a neural network model. It utilizes sample data representing the correlation between sample features obtained from a trained network model with an attention mechanism to train the first encoder and state estimator in the neural network model. This improves the inductive ability of the first encoder and state estimator. Using the trained first encoder and state estimator, a state extractor obtains sample data for training the first decoder and the first masking code generator. The first decoder of the trained neural network model outputs decoded features representing the correlation between the data to be denoised. This facilitates the formation of an attention mechanism between the first decoder and the first masking code generator. By combining the advantages of a network model with an attention mechanism with the neural network model, and through the state extractor, historical features are correlated while reducing computational power, thus reducing power consumption while maintaining denoising effectiveness. Attached Figure Description

[0108] Figure 1 This is a schematic diagram of a neural network model structure used for noise reduction in an embodiment of this application.

[0109] Figure 2 This is a schematic diagram of the attention denoising network model structure and its training in an embodiment of this application.

[0110] Figure 3 This is a schematic diagram illustrating the acquisition of sample status assessment result data in this embodiment.

[0111] Figure 4 This is a schematic diagram of a process for training a neural network model for noise reduction according to an embodiment of this application.

[0112] Figure 5 This is a schematic diagram illustrating the training of a first encoder and a state estimator according to an embodiment of this application.

[0113] Figure 6 This is a schematic diagram illustrating the acquisition of a second training sample in an embodiment of this application.

[0114] Figure 7 This is a schematic diagram illustrating how a first decoder and a first masking code generator are trained using second training sample data, according to an embodiment of this application.

[0115] Figure 8 This is a schematic diagram of a noise reduction method based on a neural network model in this embodiment.

[0116] Figure 9 This is a schematic diagram of a process for the state extractor in this embodiment to extract encoded features to obtain fused features.

[0117] Figure 10 This is a schematic diagram of a noise reduction device based on a neural network model according to an embodiment of this application.

[0118] Figure 11 This is a schematic diagram of a training device for a neural network model used for noise reduction according to an embodiment of this application.

[0119] Figure 12 Another schematic diagram of the noise reduction device based on the neural network model and / or the training device for the neural network model used for noise reduction in this application embodiment. Detailed Implementation

[0120] To make the objectives, technical means, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings.

[0121] The applicant found that in neural network models, the attention structure is a very effective neuron with a strong ability to generalize relationships. For example, the attention structure is widely used in large language models to obtain the relationships between contextual information. However, the disadvantage is that it is computationally expensive because it requires both current and historical data for each calculation. This requires storing a large amount of historical data and a lot of computing power, making it difficult to reduce power consumption.

[0122] This application provides a noise reduction method based on a neural network model to achieve low-power noise reduction.

[0123] See Figure 1 As shown, Figure 1 This is a schematic diagram of a neural network model structure used for noise reduction in an embodiment of this application. The neural network model includes:

[0124] The first encoder used to extract encoded features

[0125] A state estimator is used to assess the correlation between extracted encoded features, where correlation characterizes the degree of relevance between encoded features.

[0126] A state extractor used to extract at least one encoded feature and reconstruct fused features from associated historical features based on the state evaluation results output by the state evaluator.

[0127] The first decoder is used to decode the fused features to obtain the decoded features, and

[0128] A first masking code generator for generating a first masking code for noise reduction based on decoding features.

[0129] The first encoder, state evaluator, state extractor, first decoder, and first mask code generator are connected in sequence, and the first encoder is also connected to the state extractor.

[0130] As an example, the first encoder is a first recurrent neural network (RNN), the state estimator is a first deep neural network (DNN), the first decoder is a second recurrent neural network, and the first masking code generator is a second deep neural network. The network structures of the first and second recurrent neural networks can be the same or different, and the network structures of the first and second deep neural networks can also be the same or different. The state extractor is a non-neural network structure and can be implemented by software code.

[0131] Using sample data representing the correlation between sample features obtained from a trained network model with an attention mechanism, the first encoder and state estimator in the neural network model are trained. Using the trained first encoder and state estimator, sample data for training the first decoder and the first mask code generator is obtained through extraction by the state extractor. This allows the first decoder of the trained neural network model to output decoded features representing the correlation between the data to be denoised. This, combined with the attention mechanism of the network model with an attention mechanism, gives the trained neural network model an attention mechanism, which helps improve the neural network model's ability to correlate the input data to be denoised, thereby improving the accuracy of the first mask code.

[0132] In the inference process using a neural network model, the neural network model extracts historical encoded features by using the state evaluation results output by the state evaluator to obtain fused features that represent the historical encoded features for each encoded feature. This allows historical data with a high degree of correlation to be abstracted into a vector, which participates in each calculation. It is equivalent to only needing the same historical data for calculation. In this way, only one historical vector needs to be stored, which not only has a very low computational load, but also has a certain ability to connect data. This reduces storage resources and improves noise reduction while also taking into account low power consumption.

[0133] As an example, the neural network model is trained in the following manner:

[0134] First training sample data for training a first encoder and a state estimator is obtained. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features. The sample state evaluation results are used to characterize the degree of correlation between the first sample features, wherein the degree of correlation is the correlation degree.

[0135] The first encoder and state estimator are trained using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator.

[0136] The first sample feature is input into the trained first encoder to obtain the sample encoded feature. The sample encoded feature is then input into the trained state estimator to obtain the state evaluation result of the sample encoded feature output by the trained state estimator.

[0137] The sample encoding features and the state evaluation results of the sample encoding features are input into the state extractor to obtain the sample fusion features. The sample fusion features and the sample mask codes corresponding to the first sample features of the sample fusion features are used as the second training sample data for training the first decoder and the first mask code generator.

[0138] The first decoder and the first mask generator are trained using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first mask generator.

[0139] in,

[0140] The evaluation result of the sample state in the first training sample data is obtained by inputting the first sample feature data into the trained network model with attention mechanism to obtain the correlation information between the first sample features.

[0141] Given that the trained neural network model has the ability to reduce noise through attention, and that the neural network model itself is a recurrent neural network structure, it can be called a recurrent attention-based noise reduction neural network model.

[0142] The data to be denoised is input into a trained neural network model to obtain the first masking code output by the neural network model. As an example, the data to be denoised includes at least one of image data, speech data, and text data.

[0143] Based on the first masking code, a data masking operation is performed on the data to be denoised to obtain the denoised data.

[0144] The denoising method based on the neural network model in this application extracts associated historical encoded features by a state extractor and reconstructs them into fused features, which reduces the storage space required in the denoising process and solves the problem of not being able to remember long sequences. By using sample data that characterizes the degree of association between sample features obtained by a trained network model with an attention mechanism, the first encoder and state estimator in the neural network model are trained, so that the neural network model forms an attention mechanism.

[0145] To facilitate understanding of the embodiments of this application, the following description uses the noise reduction processing of voice data as an example. It should be understood that this application is not limited to voice data, and is equally applicable to noise reduction of audio data, image data, and text data.

[0146] To train the neural network model used for noise reduction, an attention network model and a training dataset are constructed.

[0147] The training dataset was obtained in the following manner:

[0148] Several noise-free sample speech signals were collected to obtain sample speech data.

[0149] Several noisy sample audio signals were collected to obtain sample audio data.

[0150] Randomly select one frame of sample speech data and one frame of sample audio data, and fuse them, for example, by superimposing the signal strength, to obtain one frame of sample noisy speech data; for example, with n frames of source sample speech data and m frames of sample audio data, n×m frames of sample noisy speech data can be obtained.

[0151] For any frame of sample noisy speech data, frequency domain features are extracted from the sample noisy speech data to obtain the first sample frequency domain feature F1, and frequency domain features are extracted from the sample speech data corresponding to the sample noisy speech data to obtain the second sample frequency domain feature F2.

[0152] The sample frequency domain features are calculated according to the following formula to obtain the sample masking code label1. The first sample frequency domain feature F1, the second sample frequency domain feature F2, and the sample masking code label1 are used as a training sample.

[0153]

[0154] label1 = label real ||label imag

[0155] The sample frequency domain features consist of real and imaginary parts, F1 real Represents the real part of F1, F1 imagrepresents the imaginary part of F1, and || is the vector concatenation operation.

[0156] See Table 1 for the training samples.

[0157] Table 1

[0158] Training samples First sample frequency domain feature F1 Second sample frequency domain feature F2 Sample masking code label1 Training Sample 1 xx yy zz Training Sample 2 …

[0159] See Figure 2 As shown, Figure 2 This is a schematic diagram of the structure and training of an attention network model. It includes a second encoder, an attention correlator, a second decoder, and a second masking code generator connected in sequence. As an example, the attention network model structure is a transform structure. The attention network model is trained to enable it to reduce noise. For example, the frequency domain features F1 of the first sample in the training samples are input into the attention network model to obtain the sample prediction result pred1 output by the attention network model. This sample prediction result represents the masking code of the region of interest (noise). The loss function value is calculated by comparing pred1 with the sample masking code label1 in the training samples, for example, calculating the MSE loss function value. The model parameters are then adjusted based on the loss function value until the desired result is achieved.

[0160] See Figure 3 As shown, Figure 3 This is a schematic diagram illustrating the acquisition of sample state evaluation result data in this embodiment. Using a trained attention network model, first training sample data is obtained for training the first encoder and state estimator in the neural network model used for noise reduction. As an example, the first sample frequency domain features F1 of several frames are input into the second encoder of the trained attention network model, and the sample attention matrix output by the attention correlator is extracted. This sample attention matrix represents the degree of correlation between the input first sample frequency domain features F1. Specifically, any value in the attention matrix represents the degree of correlation between the encoded feature corresponding to the column of that value and the encoded feature corresponding to the row of that value. Since the sample encoded features correspond to the input first sample frequency domain features, the values ​​in the attention matrix also represent the degree of correlation between the corresponding sample frequency domain features.

[0161] For example, the first sample frequency domain features F1 of frames 1 to 5 are input sequentially, and the attention correlator outputs a 5-row, 5-column attention matrix. In this attention matrix, the value located in row i and column j represents the degree of correlation between the sample coding feature i of the first sample frequency domain feature F1 of frame i and the sample coding feature j of the first sample frequency domain feature F1 of frame j.

[0162] The sum of the values ​​in each column of the attention matrix yields the sample state evaluation result S1 for each column. This result characterizes the degree of correlation between the sample coding features of the first sample frequency domain feature F1 of each frame; a higher value indicates a higher degree of correlation. For example, the sum of the values ​​in the j-th column characterizes the degree of correlation between the sample coding features of the first sample frequency domain feature F1 of frame j and the sample coding features of the first sample frequency domain feature F1 of all frames.

[0163] The input first sample frequency domain feature F1 and its sample state evaluation result are used as the first training sample. See Table 2.

[0164] Table 2

[0165] First training sample data First sample frequency domain feature F1 Sample status assessment result S1 First training sample 1 First training sample 2 …

[0166] See Figure 4 As shown, Figure 4 This is a schematic flowchart illustrating a process for training a neural network model for noise reduction, as described in an embodiment of this application. The training method includes:

[0167] Step 401: Using the first training sample data, train the first encoder and state estimator in the neural network model used for noise reduction.

[0168] As an example, see Figure 5 As shown, Figure 5 This is a schematic diagram for training the first encoder and the state estimator. The first sample frequency domain feature F1 from the first training sample data is input into the first encoder. The sample encoded features output by the first encoder are input into the state estimator. The state estimator outputs the state evaluation result P1 of the sample encoded features. A loss function value, such as the MSE loss function value, is calculated between this state evaluation result P1 and the sample state evaluation result S1 from the first training sample data. The model parameters of the first encoder and the state estimator are adjusted based on the loss function value until the desired result is achieved.

[0169] Step 402: Obtain second training sample data for training the first decoder and the first mask generator.

[0170] As an example, the first sample frequency domain feature F1 of each frame is input into the trained first encoder and state evaluator to obtain the sample coding feature M of the first sample frequency domain feature F1 of each frame output by the first encoder and the state evaluation result S1' of the sample coding feature M of the first sample frequency domain feature F1 of each frame output by the state evaluator. The sample coding feature M and its state evaluation result S1' are then stored.

[0171] As an example, extraction processing methods include:

[0172] Following the extraction principle of adding one previously unextracted coding feature to the previous historical coding features for each extraction, the stored coding features are extracted one after another until there are no previously unextracted coding features. The previously extracted historical coding features are all the coding features extracted in the previous extraction that is adjacent to the current extraction.

[0173] During each extraction, the current state evaluation result of the extracted coded features is updated with a set extraction strength as the step size. For example, the extraction strength is decreased by the step size to obtain an updated state evaluation result. The extraction strength is used to characterize the degree of correlation between the added coded features that have not been extracted and the coded features extracted in previous extractions. For each updated state evaluation result, if the updated state evaluation result is greater than or equal to the set state evaluation threshold, the extracted coded features corresponding to the updated state evaluation result are used as the extraction result for this extraction. All extraction results obtained in this extraction are fused. For example, all extraction results obtained in this extraction are added together to obtain a fused feature. The fused feature characterizes the feature fusion of the added coded features that have not been extracted and their associated historical coded features.

[0174] Given the principle that each extraction adds one previously unextracted coding feature, assuming the number of coding features is T, then it is necessary to extract T times to complete the extraction. Therefore, the number of extractions is determined based on the number of coding features M, and this number of extractions is the same as the number of coding features.

[0175] See Figure 6 As shown, Figure 6 This is a schematic diagram for obtaining a second training sample. In the diagram, the sample to be extracted has 5 encoded features, so the extraction is performed 5 times, and the extraction strength is set to 0.2.

[0176] When the number of extractions is 1, sample encoding feature 1 is extracted, and its current state evaluation result is reduced by the extraction strength as the step size. The state evaluation result is updated to 1.1. It is determined whether the updated state evaluation result is greater than or equal to the state evaluation threshold. For example, the state evaluation threshold is 0. Since the updated state evaluation result is greater than the state evaluation threshold, the extracted sample encoding feature 1 is taken as the extraction result of this extraction. Since the extraction result of this extraction only contains sample encoding feature 1, sample encoding feature 1 is taken as the sample fusion feature M' of this extraction. The sample fusion feature M' and the sample mask code corresponding to the first sample frequency domain feature corresponding to sample encoding feature 1 are taken as the second training sample.

[0177] When the number of extractions is 2, sample coding features 1 and 2 are extracted. The current state evaluation results are reduced by the extraction strength as the step size. The state evaluation results of sample coding features 1 and 2 are updated. Since the updated state evaluation results of sample coding features 1 and 2 are greater than the state evaluation threshold, the extracted sample coding features 1 and 2 are used as the extraction results of this extraction. Sample coding features 1 and 2 are fused to obtain the sample fusion feature M' extracted in this extraction. The sample fusion feature M' and the sample mask code corresponding to the first sample frequency domain feature corresponding to sample coding feature 2 are used as the second training sample.

[0178] When the number of extractions is 3, sample coding features 1, 2, and 3 are extracted. The current state evaluation results are reduced by the extraction strength as the step size. The state evaluation results of sample coding features 1, 2, and 3 are updated. Since the updated state evaluation results of sample coding features 1 and 3 are greater than or equal to the state evaluation threshold, the extracted sample coding features 1 and 3 are used as the extraction results of this extraction. Sample coding features 1 and 3 are fused to obtain the sample fusion feature M' extracted this time. The sample fusion feature M' and the sample mask code corresponding to the first sample frequency domain feature corresponding to sample coding feature 3 are used as the second training sample.

[0179] When the number of extractions is 4, sample coding features 1, 2, 3 and 4 are extracted. The current state evaluation results are reduced by the extraction strength as the step size. The state evaluation results of sample coding features 1, 2, 3 and 4 are updated. Since the updated state evaluation results of sample coding features 1 and 4 are greater than or equal to the state evaluation threshold, the extracted sample coding features 1 and 4 are taken as the extraction results of this extraction. Sample coding features 1 and 4 are fused to obtain the sample fusion feature M' of this extraction. The sample fusion feature M' and the sample mask code corresponding to the first sample frequency domain feature corresponding to sample coding feature 4 are taken as the second training sample.

[0180] When the number of extractions is 5, sample coding features 1, 2, 3, 4 and 5 are extracted. The current state evaluation results are reduced by the extraction strength as the step size. The state evaluation results of sample coding features 1, 2, 3, 4 and 5 are updated. Since the updated state evaluation results of sample coding features 1, 4 and 5 are greater than the state evaluation threshold, the extracted sample coding features 1, 4 and 5 are taken as the extraction results of this extraction. Sample coding features 1, 4 and 5 are fused to obtain the sample fusion feature M' extracted in this extraction. The sample fusion feature M' and the sample mask code corresponding to the first sample frequency domain feature corresponding to sample coding feature 5 are taken as the second training sample.

[0181] See Table 3, which shows the second training sample data.

[0182] Table 3

[0183] Second training sample data Sample fusion feature M' Sample masking code label1 Second training sample 1 Second training sample 2 …

[0184] Step 403: Train the first decoder and the first mask code generator using the second training sample data.

[0185] See Figure 7 As shown, Figure 7 This is a schematic diagram illustrating the training of a first decoder and a first masking code generator using second training sample data. The sample fusion feature M' from the second training samples is input into the first decoder to obtain the decoded feature. This decoded feature is then input into the first masking code generator to obtain the masking code P2 output by the generator. Based on the masking code P2 and the sample masking code corresponding to the sample fusion feature M' input from the second training samples, a loss function value is calculated. The parameters of the first decoder and the first masking code generator are adjusted according to the loss function value until the desired result is achieved.

[0186] See Figure 8 As shown, Figure 8 This is a schematic diagram of a noise reduction method based on a neural network model in this embodiment. Frequency domain features are extracted from several frames of speech data to be denoised, resulting in frequency domain features for each frame. These features are then input into a first encoder. The encoded features of each frame output by the first encoder are input into a state evaluator, yielding a state evaluation result of the encoded features for each frame. The state evaluation result and the encoded features output by the first encoder are then input into a state extractor. The state extractor extracts the encoded features of each frame according to the extraction method used during training, resulting in a fused feature of the encoded features for each frame output by the state extractor. The fused features of each frame are then... The features are input to the first decoder to obtain the decoded features of each frame output by the first decoder. The decoded features of each frame are then input to the first masking code generator to obtain the first masking code for each frame output by the first masking code generator. Based on the first masking code of each frame, a masking operation is performed on the corresponding frequency domain features. For example, the first masking code of each frame is multiplied by the corresponding frequency domain feature to obtain the denoised frequency domain features. In other words, for each frame's first masking code, the first masking code of that frame is multiplied by the frequency domain feature of that frame to obtain the denoised frequency domain features of that frame. The denoised frequency domain features are then converted into time domain features to obtain the denoised speech data.

[0187] As an example, the encoding features of each frame output by the first encoder and the state evaluation results of the encoding features of each frame output by the state evaluator are stored in the state register. Each time the state extractor extracts, it reads the encoding features and their state evaluation results required for that extraction. When the state evaluation result is less than the set state evaluation threshold, the state evaluation result and its encoding features are deleted.

[0188] See Figure 9 As shown, Figure 9 This is a flowchart illustrating the process of extracting encoded features from a state extractor to obtain fused features. It should be understood that this process is also suitable for the state extractor during training. As an example, the state extractor performs the following steps:

[0189] Step 901: Select a previously unextracted coded feature from the coded features output by the first encoder, and add this previously unextracted coded feature to the current historical coded features.

[0190] Step 902: Update the state evaluation results of each extracted coded feature according to the extraction intensity.

[0191] Step 903: Determine whether the current updated status assessment results of each institute meet the set status assessment result threshold. For example, whether the current updated status assessment results of each institute are greater than or equal to the status assessment result threshold.

[0192] If so, then the fused features of the updated state evaluation results that are greater than or equal to the state evaluation result threshold are merged to obtain the fused features of the previously unextracted coded features, while the currently extracted coded features are retained as the current historical coded features.

[0193] Otherwise, delete the encoded features whose updated state evaluation results are less than the state evaluation result threshold to remove encoded features with low correlation, and retain the currently not deleted encoded features as current historical encoded features.

[0194] Step 904: Determine whether there are any unextracted coded features in the coded features output by the first encoder.

[0195] If so, return to step 901.

[0196] Otherwise, end this process.

[0197] See Figure 10 As shown, Figure 10 This is a schematic diagram of a noise reduction device based on a neural network model according to an embodiment of this application. The noise reduction device includes:

[0198] The noise reduction information acquisition module is used to input the data to be denoised into a trained neural network model and obtain the first masking code output by the neural network model for noise reduction.

[0199] The noise reduction module is used to perform data masking operations on the data to be denoised based on the first masking code, so as to obtain the denoised data.

[0200] See Figure 11 As shown, Figure 11 This is a schematic diagram of a training apparatus for a neural network model used for noise reduction, according to an embodiment of this application. The training apparatus includes:

[0201] The first training sample data acquisition module is used to acquire first training sample data for training the first encoder and the state estimator. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features. The sample state evaluation results are used to characterize the evaluation results of the correlation between the first sample features.

[0202] The first training module is used to train the first encoder and the state estimator using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator.

[0203] The second training sample data acquisition module is used to input the first sample features into the trained first encoder to obtain the sample encoding features output by the trained first encoder; input the sample encoding features into the trained state evaluator to obtain the state evaluation result of the sample encoding features output by the trained state evaluator; input the sample encoding features and the state evaluation result of the sample encoding features into the state extractor to obtain the sample fusion features output by the state extractor; and use the sample fusion features and the sample mask code corresponding to the first sample features as the second training sample data for training the first decoder and the first mask code generator.

[0204] The second training module is used to train the first decoder and the first masking code generator using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first masking code generator.

[0205] See Figure 12 As shown, Figure 12 This is another schematic diagram of a noise reduction device based on a neural network model and / or a training device for a neural network model used for noise reduction, according to embodiments of this application. The device includes a memory and a processor. The memory stores a computer program, and the processor is configured to execute the computer program to implement the steps of the noise reduction method based on a neural network model described in embodiments of this application and / or the step of training a neural network model for noise reduction.

[0206] The memory may include random access memory (RAM) or non-volatile memory (NVM), such as at least one disk storage device. Optionally, the memory may also be at least one storage device located remotely from the aforementioned processor.

[0207] The processors mentioned above can be general-purpose processors, including central processing units (CPUs), network processors (NPs), etc.; they can also be digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components.

[0208] This invention also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of a denoising method based on a neural network model and / or the steps of a training method for a denoising neural network model.

[0209] For the device / network-side equipment / storage medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and relevant parts can be referred to in the description of the method embodiments.

[0210] In this document, relational terms such as "first" and "second" are used merely to distinguish one entity or operation from another, without necessarily requiring or implying any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0211] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A noise reduction method based on a neural network model, characterized in that, The neural network model includes: The first encoder used to extract encoded features A state estimator used to assess the degree of correlation between encoded features. A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator. The first decoder used to extract decoding features from the fused features, and A first masking code generator is used to generate a first masking code for noise reduction based on decoding features. The noise reduction method includes: The data to be denoised is input into the trained neural network model to obtain the first masking code output by the neural network model. Based on the first masking code, a data masking operation is performed on the data to be denoised to obtain the denoised data.

2. The noise reduction method as described in claim 1, characterized in that, The step of inputting the data to be denoised into a trained neural network model to obtain the first masking code output by the neural network model includes: The data to be denoised is input into the first encoder, the encoded features output by the first encoder are input into the state estimator, and the state evaluation result output by the state estimator and the encoded features output by the first encoder are input into the state extractor. The fused features output by the state extractor are input into the first decoder, and the decoded features output by the first decoder are input into the first masking code generator to obtain the first masking code output by the first masking code generator. The data masking operation based on the first masking code includes: Multiply the first masking code with the data to be denoised.

3. The noise reduction method as described in claim 2, characterized in that, The first encoder and state estimator in the neural network model are trained using sample data that characterizes the degree of correlation between sample features obtained from the trained network model with attention mechanism. Using the trained first encoder and state estimator, sample data for training the first decoder and the first masking generator are obtained through the state extractor, so that the first decoder of the trained neural network model outputs decoding features that characterize the degree of correlation between the data to be denoised.

4. The noise reduction method as described in claim 3, characterized in that, The neural network model is trained in the following manner: First training sample data is obtained for training the first encoder and the state estimator. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features, wherein the sample state evaluation results are used to characterize the degree of correlation between the encoded features of the first sample features. The first encoder and state estimator are trained using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator. The first sample feature is input into the trained first encoder to obtain the sample encoded feature output by the trained first encoder. The sample encoded feature is then input into the trained state estimator to obtain the state evaluation result of the sample encoded feature output by the trained state estimator. The sample encoding features and the state evaluation results of the sample encoding features are input into the state extractor to obtain the sample fusion features output by the state extractor. The sample fusion features and the sample mask codes corresponding to the first sample features of the sample fusion features are used as the second training sample data for training the first decoder and the first mask code generator. The first decoder and the first mask generator are trained using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first mask generator. in, The evaluation result of the sample state in the first training sample data is obtained by inputting the first sample feature data into the trained network model with attention mechanism to obtain the correlation information between the first sample features.

5. The noise reduction method as described in claim 4, characterized in that, The network model with an attention mechanism is an attention network model. The attention network model is trained in the following manner: The features of the first sample are input into the attention network model to obtain the sample prediction results output by the attention network model. Based on the loss function value between the sample prediction result and the sample masking code corresponding to the first sample feature, adjust the model parameters of the attention network model until the expected result is achieved.

6. The noise reduction method as described in claim 5, characterized in that, The attention network model includes: a second encoder, an attention correlator, a second decoder, and a second masking code generator connected in sequence. The step of inputting the features of the first sample into the attention network model to obtain the prediction result output by the attention network model includes: The first sample feature is input into the second encoder, the sample encoded feature output by the second encoder is input into the attention correlator, the sample attention correlator output by the attention correlator is input into the second decoder, and the sample decoded feature output by the second decoder is input into the second mask code generator to obtain the sample prediction result of the first sample feature output by the second mask code generator.

7. The noise reduction method as described in claim 6, characterized in that, The sample state evaluation result in the first training sample data is obtained by inputting the first sample feature data into a trained network model with an attention mechanism to obtain information on the degree of correlation between the first sample features, including: The first sample feature data is input into the second encoder of the trained attention network model. The sample encoded features output by the second encoder are then input into the attention correlator, resulting in an attention matrix output by the attention correlator that characterizes the degree of correlation between sample encoded features. In this matrix, the value in any column of any row represents the degree of correlation between the encoded feature corresponding to that row and the encoded feature corresponding to that column. The sum of the values ​​in each column of the attention matrix is ​​calculated to obtain the sample state evaluation result of the first sample feature corresponding to the encoded feature of each column.

8. The noise reduction method as described in claim 7, characterized in that, The attention network model is a transform structure; The first encoder is a first recurrent neural network (RRN) structure, the first decoder is a second RRN structure, the state estimator is a first deep learning neural network (DNN) structure, and the first masking code generator is a second DNN structure. The step of training the first encoder and the state estimator using the first training sample data includes: The first sample feature from the first training sample data is input into the first encoder, and the sample encoded feature output by the first encoder is input into the state evaluator. Based on the loss function value between the state evaluation result output by the state estimator and the sample state evaluation result of the input first sample feature, adjust the parameters of the first encoder and the state estimator until the expected result is achieved.

9. The noise reduction method as described in claim 8, characterized in that, The process of inputting the sample encoding features and the state evaluation results of the sample encoding features into the state extractor to obtain the sample fusion features output by the state extractor includes: Following the principle of adding one previously unextracted coded feature to each subsequent extraction, the input sample coded features are extracted sequentially until no previously unextracted sample coded features remain. The previously extracted historical coded features refer to all coded features extracted in the previous extraction process adjacent to this current extraction. During each extraction, the current state evaluation result of the extracted sample coding features is updated with the set extraction strength as the step size to obtain the updated state evaluation result. The state evaluation results that meet the conditions are selected from the updated state evaluation results, and the sample coding features corresponding to the state evaluation results that meet the conditions are used as the extraction result of this extraction. The extraction results of this extraction are fused to obtain the sample fusion feature used to represent the fusion of the added sample coding features and its associated historical sample coding features.

10. The noise reduction method as described in claim 9, characterized in that, The current state evaluation result of the extracted sample encoded features is updated with the set extraction strength as the step size, including: For each sample coding feature in the extracted sample coding features, Subtract the extraction strength from the current state evaluation result of the encoded feature of the sample to obtain the updated state evaluation result of the encoded feature of the sample. The step of selecting the state assessment results that meet the criteria from the updated state assessment results includes: For each sample coding feature in the extracted sample coding features, Determine whether the updated state evaluation result of the encoded features of the sample is greater than or equal to the set state evaluation threshold. If so, the evaluation result of the updated state of the encoded features of the sample is deemed to meet the conditions; The fusion feature is obtained in the following manner: The state extractor extracts the input encoded features one by one according to the extraction processing method during training, and obtains the fused features used to represent the added encoded features and their associated historical encoded features at each extraction.

11. The noise reduction method as described in claim 10, characterized in that, The step of training the first decoder and the first masking code generator using the second training sample data includes: The sample fusion features from the second training sample data are input into the first decoder, and the sample decoding features output by the first decoder are input into the first mask code generator. Based on the loss function value between the mask code output by the first mask code generator and the sample mask code of the first sample feature corresponding to the fused feature of the input sample, the parameters of the first decoder and the first mask code generator are adjusted until the expected result is achieved.

12. The noise reduction method as described in claim 11, characterized in that, The first sample features are obtained in the following manner: Noise sample data and noise-free source sample data are collected, and the noise samples are fused with the source samples to obtain noise source sample data. Extract features from the noise source sample data to obtain the first sample features; The sample masking code is obtained in the following manner: Extract features from the source sample data to obtain the features of the second sample. Calculate the sample masking code based on the features of the first sample and the features of the second sample.

13. The noise reduction method as described in claim 12, characterized in that, The source sample data is noise-free sample speech data, and the noise sample data is sample audio data including noise. The fusion of noise samples and source samples includes: superimposing one frame of sample speech data and one frame of sample audio data in terms of intensity to obtain sample noise speech data. The extraction of features from noise source sample data includes: extracting frequency domain features from sample noise speech to obtain first sample frequency domain features; The extraction of features from source sample data includes: extracting frequency domain features from sample speech data to obtain second sample frequency domain features; The step of calculating the sample masking code based on the first sample features and the second sample features includes: Calculate the sum of squares of the real and imaginary parts of the frequency domain features of the first sample. Calculate the sum of the product of the real parts of the frequency domain features of the first sample and the real parts of the frequency domain features of the second sample, and the sum of the product of the imaginary parts of the frequency domain features of the first sample and the imaginary parts of the frequency domain features of the second sample. Divide this sum by the sum of squares to obtain the real part masking code. Calculate the difference between the product of the real part of the frequency domain features of the first sample and the imaginary part of the frequency domain features of the second sample, and the product of the imaginary part of the frequency domain features of the first sample and the real part of the frequency domain features of the second sample. Divide this difference by the sum of squares to obtain the imaginary part masking code. The sample masking code is obtained by concatenating the real part masking code and the imaginary part masking code.

14. A training method for a neural network model for noise reduction, characterized in that, The neural network model includes: a first encoder for extracting encoded features; A state evaluator used to assess the correlation between encoded features. A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator. The first decoder used to extract decoding features from the fused features, and A first masking code generator is used to generate a first masking code for noise reduction based on decoding features. Among them, the first encoder, state estimator, first decoder, and first mask code generator are neural networks to be trained. The training method includes: First training sample data is obtained for training the first encoder and the state estimator. This first training sample data includes: first sample features characterizing the features of sample data including noise, and sample state evaluation results of the first sample features, wherein the sample state evaluation results characterize the degree of correlation between the first sample features. The first encoder and state estimator are trained using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator. The first sample feature is input into the trained first encoder to obtain the sample encoded feature output by the trained first encoder. The sample encoded feature is then input into the trained state estimator to obtain the state evaluation result of the sample encoded feature output by the trained state estimator. The sample encoding features and the state evaluation results of the sample encoding features are input into the state extractor to obtain the sample fusion features output by the state extractor. The sample fusion features and the sample mask codes corresponding to the first sample features of the sample fusion features are used as the second training sample data for training the first decoder and the first mask code generator. The first decoder and the first mask generator are trained using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first mask generator. in, The sample state evaluation result in the first training sample data is obtained by inputting the first sample feature data into the sample attention association information obtained in the trained network model with attention mechanism.

15. A noise reduction device based on a neural network model, characterized in that, The noise reduction device includes: The noise reduction information acquisition module is used to input the data to be denoised into a trained neural network model and obtain the first masking code output by the neural network model for noise reduction. The noise reduction module is used to perform data masking operations on the data to be denoised based on the first masking code to obtain the denoised data; in, Neural network models include: The first encoder used to extract encoded features A state evaluator used to assess the correlation between encoded features. A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator. The first decoder used to extract decoding features from the fused features, and A first masking code generator for generating a first masking code for noise reduction based on decoding features.

16. A training device for a neural network model for noise reduction, characterized in that, The neural network model includes: a first encoder for extracting encoded features; A state evaluator used to assess the correlation between encoded features. A state extractor used to extract at least one encoded feature and reassemble associated historical encoded features into fused features based on the state evaluation results output by the state evaluator. The first decoder used to extract decoding features from the fused features, and A first masking code generator is used to generate a first masking code for noise reduction based on decoding features. Among them, the first encoder, state estimator, first decoder, and first mask code generator are neural networks to be trained. The training device includes: The first training sample data acquisition module is used to acquire first training sample data for training the first encoder and the state estimator. The first training sample data includes: first sample features for characterizing the features of sample data including noise, and sample state evaluation results of the first sample features. The sample state evaluation results are used to characterize the evaluation results of the correlation between the first sample features. The first training module is used to train the first encoder and the state estimator using the first training sample data until the expected results are achieved, thus obtaining the trained first encoder and state estimator. The second training sample data acquisition module is used to input the first sample features into the trained first encoder to obtain the sample encoding features output by the trained first encoder; input the sample encoding features into the trained state evaluator to obtain the state evaluation result of the sample encoding features output by the trained state evaluator; input the sample encoding features and the state evaluation result of the sample encoding features into the state extractor to obtain the sample fusion features output by the state extractor; and use the sample fusion features and the sample mask code corresponding to the first sample features as the second training sample data for training the first decoder and the first mask code generator. The second training module is used to train the first decoder and the first masking code generator using the second training sample data until the expected result is achieved, thus obtaining the trained first decoder and the first masking code generator.

Images