Classroom speech enhancement method based on convolution bidirectional Mama

Abstract

The invention relates to a classroom speech enhancement method based on convolution bidirectional Mama, and belongs to the technical field of speech signal processing of specific scenes. The method comprises the following steps: firstly, carrying out feature enhancement preprocessing of perceiving contrast stretching on a noiseless classroom audio, then inputting the noisy classroom audio into a speech enhancement model, and carrying out feature extraction on the noisy classroom audio by the model through an audio encoder and a convolution bidirectional Mama module; and then jointly decoding the voice features by using an amplitude mask decoder and a phase decoder, and finally improving the voice understandability in a classroom noise environment. According to the method, the voice signals in the complex noise classroom environment are denoised through the voice enhancement model, the requirement of a downstream voice task for further operation of classroom audio is met, and the method has great significance in development of the education field.

Description

Technical Field

[0001] The present invention relates to a classroom speech enhancement method based on convolution bidirectional Mamba, and belongs to the technical field of speech signal processing in specific scenarios. Background Art

[0002] The clarity and audibility of speech signals in classroom settings have a direct impact on teaching quality. Classroom audio is not only used for student review and teacher teaching effectiveness review, but also provides important data for automated analysis and processing in smart education systems. However, factors such as noise, echo, interference from multiple speakers, and varying recording equipment and distance in the classroom environment severely impact speech signal quality, resulting in suboptimal speech transmission. Traditional speech processing technologies often struggle to effectively eliminate background noise or maintain speech signal integrity in complex classroom environments, thus impacting the accuracy of subsequent speech recognition or other processing tasks. To address these issues, researchers have begun developing speech enhancement technologies to improve speech signal quality in noisy environments. Speech enhancement not only effectively removes noise but also preserves key speech features, thereby improving speech intelligibility in various applications. In particular, it provides higher-quality input for downstream tasks such as speech recognition and classroom analysis. This paper proposes a classroom speech enhancement method based on convolutional bidirectional Mamba, which addresses the difficulty of traditional speech enhancement technologies in effectively eliminating background noise and maintaining speech signal integrity in complex classroom environments. Summary of the Invention

[0003] The purpose of the present invention is to provide a classroom speech enhancement method based on convolutional bidirectional Mamba, aiming to solve the technical problem that traditional speech processing technology is difficult to effectively eliminate background noise or maintain the integrity of speech signals in complex classroom environments.

[0004] To achieve the above objectives, the technical solution of the present invention is: a classroom speech enhancement method based on convolutional bidirectional Mamba, which can perform speech enhancement tasks on classroom teaching speech containing different noises, thereby improving the problem of poor speech audibility in existing teaching scenarios. Compared with the speech enhancement method for classroom teaching with noise that mainly relies on time domain features, the present invention takes into account the importance of time domain and frequency domain features to speech signals, as well as the complexity of the classroom environment, including the influence of external interference factors such as background noise and recording quality. It uses perceptual contrast stretching technology to enhance the features of the input noise-free audio, ensuring that the key features of the noise-free speech signal are fully preserved.

[0005] This paper uses an encoding-decoding framework to achieve speech enhancement for teaching speech in noisy classrooms. The encoder extracts time-frequency features from the speech signal, while the convolutional bidirectional Mamba module further extracts sequence features in the time-frequency domain using a state-space model. The decoder consists of an amplitude mask decoder and a phase decoder, which process the amplitude and phase features of the speech signal, respectively. Joint decoding restores high-quality speech signals.

[0006] The key to this invention lies in further inputting the time-frequency domain features extracted by the encoder into a convolutional bidirectional Mamba module for deep feature extraction. Simultaneously, the amplitude and phase features are jointly decoded using an amplitude mask decoder and a phase decoder, respectively, to achieve speech enhancement for noisy teaching audio in classroom settings. This method not only effectively addresses the shortcomings of traditional methods in noisy classroom environments but also provides a more reliable speech enhancement solution for applications in smart classrooms and educational technology.

[0007] The specific steps are: Step 1: Perform short-time Fourier transform and amplitude feature extraction on the noise-free classroom audio. Then, perform perceptual contrast stretching on the noise-free classroom audio amplitude features. The model's prediction results are optimized by comparing them with the prediction results of the speech enhancement model to obtain feature-enhanced noise-free classroom audio amplitude features. Step 2: Perform short-time Fourier transform (SFT) and extract amplitude and phase features on the noisy classroom audio. The two are then concatenated to obtain the frequency domain information of the noisy classroom audio. This information is then encoded using a two-dimensional convolutional layer, a diffuse dense connection network, and a convolutional bidirectional Mamba module to obtain the encoded noisy classroom audio features. Step 3: Based on the obtained noisy classroom audio features, they are processed separately through the amplitude mask decoder and phase decoder to obtain the predicted noisy classroom audio amplitude features and phase features. The predicted noisy classroom audio amplitude features are then compared and optimized with the feature-enhanced noise-free classroom audio amplitude features to obtain the final enhanced amplitude features. The final enhanced amplitude features are combined with the predicted noisy classroom audio phase features, and the final enhanced classroom audio signal is reconstructed through inverse short-time Fourier transform.

[0008] The Step 1 is specifically as follows: Step 1.1: Perform short-time Fourier transform on the noise-free and noisy classroom audio to convert the time domain signal into the frequency domain signal; Step 1.2: Extract the noise-free audio amplitude features and the noisy audio amplitude features from the short-time Fourier transform results of the noise-free and noisy classroom audio respectively, and perform a perceptual contrast stretching operation on the noise-free audio amplitude features; Step 1.3: Compare the amplitude features predicted by the speech enhancement model with the amplitude features of the noise-free audio after perceptual contrast stretching to optimize the amplitude features predicted by the speech enhancement model. Step 1.4: Combine the enhanced amplitude features predicted by the optimized speech enhancement model with the original phase information, and reconstruct the feature-enhanced noisy classroom audio signal through inverse short-time Fourier transform to obtain a noise-free classroom audio signal.

[0009] The Step 2 is specifically as follows: Step 2.1: Input the frequency domain information containing noise Feature extraction is performed through a two-dimensional convolutional layer, combined with instance normalization to eliminate batch differences, and nonlinear transformation is performed through normalization operations and parameterized rectified linear unit (PreLU) functions to extract local time-frequency feature patterns of the audio; Step 2.2: Process the nonlinear transformation features through a diffuse dense connection network, and use a dense skip connection structure to fuse feature information at different levels to achieve the fusion and transmission of features at different levels; Step 2.3: The features processed by the DDC network are mapped to the feature space processed by the bidirectional time series modeling through the combination of the last layer of two-dimensional convolution and PreLU function to complete the feature encoding; Step 2.4: Input the encoded features into the convolutional bidirectional Mamba module. Bidirectional time series modeling is performed through the forward and reverse processing branches. The state-space model and gating mechanism are combined to capture the long-range dependencies of the sequence, resulting in the forward and reverse time series modeling results, respectively. Specifically, the present invention improves the traditional Mamba, which only uses a forward processing branch. The present invention proposes a forward and backward combination. Both processing branches contain a combination structure of linear projection, one-dimensional convolution, gating unit and state space model. Among them, the backward processing branch first flips the encoded features before processing. Step 2.5: The forward timing modeling result and the reverse timing modeling result processed by the flip operation are integrated through the feature summation fusion mechanism to output the final encoding result.

[0010] The Step 3 is specifically as follows: Step 3.1: The encoded features are divided into two branches: amplitude features and phase features. These branches are input into the amplitude mask decoder and phase decoder respectively. Both branches are first fused through a diffuse dense connection network to achieve targeted feature processing. Step 3.2: After the amplitude feature branch undergoes sub-pixel convolution and PreLU function activation, a learnable Sigmoid function is used to generate mask information to distinguish valid speech and noise components. Step 3.3: The phase feature branch uses a two-way parallel two-dimensional convolutional structure to predict the real and imaginary phase information. It then uses a diffuse dense connection network to fuse the features to obtain enhanced phase information, improving the accuracy of phase reconstruction. Step 3.4: The amplitude mask decoder multiplies the generated mask information with the noisy frequency amplitude to obtain the enhanced amplitude mask information, and combines it with the enhanced phase information obtained by the phase decoder to reconstruct the enhanced classroom audio signal.

[0011] The specific steps of Step 1.2 are as follows: A preset perception stretch coefficient array is created based on the sensitivity of the human ear to different frequency bands, with a focus on enhancing the main speech frequency band of 400Hz-4400Hz. The short-time Fourier transform amplitude spectrum of the input feature is multiplied by the perceptual stretch coefficient to achieve perceptual contrast stretching of the amplitude.

[0012] The Step 2.5 is specifically as follows: The output features of the forward processing branch and the output features of the reverse processing branch after time series flipping are spliced in high dimensions on the feature dimension axis to form an extended feature representation that integrates bidirectional time series information. The spliced extended features are then reduced in dimension through a parameterized linear projection layer to map them back to the input noisy frequency domain information. Matching original dimensional space.

[0013] The present invention uses the amplitude characteristics and phase characteristics of noisy classroom audio as input conditions and applies them to classroom speech enhancement tasks. Compared with traditional speech enhancement methods, the classroom speech enhancement method based on convolutional bidirectional Mamba first performs feature enhancement preprocessing of perceptual contrast stretching on the noise-free classroom audio, enhances the key feature information in the noise-free speech signal, and combines the convolutional bidirectional Mamba module to further extract the feature information of the noisy classroom audio for speech enhancement. Traditional speech enhancement methods cannot better handle interference factors such as noise and reverberation in the classroom environment. The convolutional bidirectional Mamba module and parallel decoder are used in this method to improve the clarity of the speech signal, making the final speech enhancement effect more outstanding.

[0014] The beneficial effects of the present invention are: combining perceptual contrast stretching with the convolution bidirectional Mamba module, it can effectively improve the clarity and comprehensibility of classroom speech, achieve classroom speech enhancement in complex noise environments, and improve the audibility of speech signals while meeting the requirements of denoising. By utilizing the model design method of the audio encoder and the convolution bidirectional Mamba module, the present invention can better extract the acoustic features of noisy frequencies, and improve the decoding effect of amplitude and phase information through a joint decoder, so that the enhanced speech signal is clearer and more natural. In addition, the present invention has a wide range of application value and is suitable for specific scenarios, such as traditional classrooms, online education platforms, lectures, etc. This technological innovation is expected to play an important role in the fields of educational technology, speech enhancement, etc., and has great research significance and commercial potential. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 It is an architectural diagram of the speech enhancement model of the present invention; Figure 2 is a flow chart of the perceptual contrast stretching feature enhancement of the present invention; Figure 3 is a diagram of the audio encoder architecture of the present invention; Figure 4 This is a diagram of the convolutional bidirectional Mamba module architecture of the present invention; Figure 5 is a diagram of the amplitude mask decoder architecture of the present invention; Figure 6 2 is a diagram of the phase decoder architecture of the present invention. DETAILED DESCRIPTION

[0016] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.

[0017] Example 1: A classroom speech enhancement method based on convolutional bidirectional Mamba, the specific steps are as follows: Step 1: Perform short-time Fourier transform and amplitude feature extraction on the noise-free classroom audio. Then, perform perceptual contrast stretching on the noise-free classroom audio amplitude features. The model's prediction results are optimized by comparing them with the prediction results of the speech enhancement model to obtain feature-enhanced noise-free classroom audio amplitude features.

[0018] Specifically, this step is the perceptual contrast stretching feature enhancement stage, specifically: Step 1.1: Divide the noise-free and noisy classroom audio into multiple overlapping short-time frames. Use a short-time Fourier transform with a 400-point window length and a 100-point frame shift. Then, window each frame with a Hamming window function to reduce spectral leakage. Each frame is then divided into an amplitude spectrum and a phase spectrum to facilitate subsequent processing of the amplitude and phase information.

[0019] Step 1.2: Extract amplitude features from the short-time Fourier transform of the noise-free audio and perform perceptual contrast stretching based on the human ear's sensitivity to different frequency bands. This feature enhancement method focuses on the main speech frequency band of 400-4400Hz, making the key features of the speech signal more prominent.

[0020] Furthermore, an array of perception stretching coefficients is first preset based on the auditory sensitivity of the human ear to different frequencies, where the coefficient of the 0-100Hz frequency band is 1.0, and then the coefficient of the 100-200Hz frequency band gradually increases to 1.07, followed by the coefficient of the 200-320Hz frequency band increasing to 1.18, and the coefficient of the 320-400Hz frequency band further increasing to 1.29. Finally, a maximum coefficient of 1.4 is applied to the main speech frequency band of 400-4400Hz. After that, as the frequency increases, the coefficient gradually decreases, dropping to 1.32 in the 4400-5200Hz frequency band, 1.24 in the 5200-6400Hz frequency band, 1.16 in the 6400-7600Hz frequency band, and finally to 1.08 in the 7600-8000Hz frequency band. The amplitude spectrum obtained by short-time Fourier transform of the input signal is then multiplied element-by-element by these preset perception stretching coefficients, thereby achieving nonlinear stretching enhancement based on auditory perception.

[0021] Step 1.3: Perform a natural logarithm plus one transformation on the noise-free audio amplitude features and the noise-containing audio amplitude features after perceptual contrast stretching. This nonlinear transformation compresses the dynamic range of the data, minimizing the difference in values between different frequency bands, which facilitates model training and feature extraction.

[0022] Step 1.4: Perform an inverse natural logarithm plus one transformation on the prediction output by the speech enhancement model and compare it with the reference features that have undergone perceptual contrast stretching. The loss function constructed in this way can effectively guide the model in learning the feature representation of clear speech.

[0023] Step 1.5: Combine the enhanced amplitude features predicted by the model with the retained original phase information and reconstruct the enhanced speech signal through an inverse short-time Fourier transform. This step converts the processed frequency domain features back to the time domain, resulting in the final enhanced classroom audio.

[0024] Step 2: Perform short-time Fourier transform (STFT) and extract amplitude and phase features on the noisy classroom audio. The two are then concatenated to obtain the frequency domain information of the noisy classroom audio. The frequency domain information of the noisy classroom audio is then encoded through a two-dimensional convolutional layer, a diffuse dense connection network, and a convolutional bidirectional Mamba module to obtain the encoded noisy classroom audio features.

[0025] Specifically, this step is the audio feature encoding stage, which is as follows: Step 2.1: The noisy input audio signal is passed through a two-dimensional convolutional layer for initial feature extraction. This is combined with instance normalization to eliminate differences between batches, and the PreLU function introduces adaptive nonlinear transformation capabilities. This combination effectively extracts local time-frequency patterns in the audio signal, laying the foundation for subsequent processing.

[0026] Step 2.2: Input the nonlinear transformation features into a diffuse densely connected network for deep processing. This network structure, through dense skip connections, can fully utilize the feature information of shallow and deep layers, enhance the expressive power of features, and alleviate the gradient vanishing problem.

[0027] Step 2.3: The features processed by the DDC network are transformed again through a two-dimensional convolutional layer and adaptively adjusted with the PreLU function. This step remaps the enhanced features into a feature space more suitable for bidirectional processing, preparing them for subsequent processing by the Mamba module.

[0028] Step 2.4: The encoded features are fed into a convolutional bidirectional Mamba module, which consists of two parallel processing branches. The forward branch directly processes the sequence information, while the reverse branch flips the sequence before processing. Both branches combine state-space models and gating mechanisms to capture long-range dependencies.

[0029] Specifically, the forward processing branch directly processes the input sequence through a combination of linear projection layers, one-dimensional convolutional layers, gating units, and state-space models. At the same time, the backward branch first flips the input sequence in the time dimension and then processes it through the same combination of modules, thereby achieving bidirectional modeling.

[0030] Step 2.5: Integrate the features of the two branches' processing results. First, flip the output of the reverse branch to align the sequence order. Then, concatenate the features of the two branches along the channel dimension. Finally, linearly project the fused features back to the original dimensional space. This bidirectional processing mechanism fully utilizes the contextual information of the sequence.

[0031] Step 3: Based on the obtained noisy classroom audio features, they are processed separately through the amplitude mask decoder and phase decoder to obtain the predicted noisy classroom audio amplitude features and phase features. The predicted noisy classroom audio amplitude features are then compared and optimized with the feature-enhanced noise-free classroom audio amplitude features to obtain the final enhanced amplitude features. The final enhanced amplitude features are combined with the predicted noisy classroom audio phase features, and the final enhanced classroom audio signal is reconstructed through inverse short-time Fourier transform.

[0032] Specifically, this step is the audio feature decoding stage, which is as follows: Step 3.1: The encoded features are split into two independent processing branches: amplitude and phase. The amplitude features are input to the amplitude mask decoder, and the phase features are input to the phase decoder. Both decoders first perform feature enhancement using a diffuse dense network. This parallel decoding architecture allows the two decoders to be optimized for different feature types.

[0033] Step 3.2: In the amplitude feature branch, the enhanced features are upsampled using sub-pixel convolution and a nonlinear transformation is introduced using the PreLU function. A sigmoid function with learnable parameters is then used to generate an amplitude mask. This adaptive mask generation mechanism enables more accurate discrimination between noise and valid speech signals.

[0034] Step 3.3: The phase feature branch uses a two-way parallel 2D convolutional architecture to predict the real and imaginary components of the phase information. The processed features are then fused again through a diffuse dense network. This design allows for more accurate phase reconstruction and improves the naturalness of speech reconstruction.

[0035] Step 3.4: The mask output by the amplitude mask decoder is multiplied by the original amplitude to produce an enhanced amplitude spectrum. The phase decoder outputs enhanced phase information. These two pieces of information are combined and reconstructed using the complex spectrum. Finally, the enhanced speech signal is obtained through an inverse short-time Fourier transform. This decoding method better preserves the harmonic structure of the speech.

[0036] This embodiment will be further described below with reference to the accompanying drawings.

[0037] Figure 1 The overall architecture of the speech enhancement model of the present invention is shown. The method is mainly divided into the audio feature encoding stage and the audio feature decoding stage. In the audio feature encoding stage, the main task is to obtain audio features. First, a short-time Fourier transform is performed on the noisy classroom audio to convert the time domain signal into frequency domain information, thereby obtaining the original phase spectrum. and amplitude spectrum , in order to further extract the amplitude spectrum The characteristics of the amplitude spectrum Perform absolute value calculation and nonlinear compression operation to obtain the transformed original amplitude spectrum Then the original phase spectrum and the original amplitude spectrum Combined into a whole frequency domain information through splicing operation , and the frequency domain information Through Step 2.1, Step 2.2, and Step 2.3 encoding, the preliminary features of the noisy classroom audio are extracted, and then the preliminary features are further extracted through Step 2.4 and Step 2.5. In the audio feature decoding stage, the main task is to decode the obtained audio high-level frequency domain features. First, the high-level frequency domain features are predicted through Step 3.1. The result of Step 3.2 prediction is the mask information. , then the mask information With the original amplitude spectrum Perform element-wise multiplication to obtain the enhanced amplitude spectrum , the result predicted by Step 3.3 is the enhanced phase spectrum , and finally the decompressed enhanced amplitude spectrum and enhanced phase spectrum The frequency domain information is converted back into a time domain signal through Step 3.4 to generate a preliminary enhanced classroom audio.

[0038] Figure 2 The feature enhancement process of the perceptual contrast stretching of the present invention is illustrated. First, before the speech enhancement model is trained, the noise-free classroom audio is converted from a time domain signal to frequency domain information through Step 1.1, and the amplitude information therein is extracted, and then the amplitude information of the noise-free classroom audio is subjected to Step 1.2 to highlight the noise-free amplitude part in the amplitude information. Then, the stretched amplitude feature is subjected to Step 1.3, and it is used as a reference target for the contrast loss function. When the speech enhancement model is trained, the noisy classroom audio is converted from a time domain signal to frequency domain information through a short-time Fourier transform to obtain amplitude information and phase information, and then the amplitude information is transformed by a natural logarithm plus one to reduce the range difference of the amplitude information. The transformed amplitude feature is input into the speech enhancement model for processing, and the result predicted by the model is compared with the reference target through Step 1.4 to calculate the contrast loss function. Finally, the optimized amplitude feature is combined with the original phase information, and the final enhanced classroom speech signal is obtained through Step 1.5.

[0039] Figure 3 The overall architecture of the audio encoder of the present invention is shown. The encoder adopts a multi-level processing structure, inputting the frequency domain information of the noisy classroom audio The signal first undergoes a first layer of two-dimensional convolution and normalization, combined with a PreLU nonlinear activation, to extract initial features from the input audio. Next, the signal passes through a diffuse densely connected network for feature enhancement and information transfer, which fully utilizes feature information at different scales and establishes long-range dependencies. Finally, the signal undergoes another two-dimensional convolution, normalization, and PreLU processing to complete the final feature extraction and transformation.

[0040] Figure 4 The overall architecture of the convolutional bidirectional Mamba module of the present invention is shown. The module adopts a bidirectional processing architecture, which includes two parallel processing branches, forward and reverse. In each processing branch, the input noisy classroom audio sequence information first undergoes feature transformation through a linear projection layer, and is then input into a gating unit and a one-dimensional convolution layer, respectively. Among them, the output of the one-dimensional convolution layer enters the state space model for time series modeling, while the output of the gating unit is fused with the features generated by the state space model, and is again subjected to feature transformation through a linear projection layer. The processing branches in both directions ultimately achieve bidirectional flow of information through a flipping operation, and perform feature fusion at the top, thereby obtaining output features that fully utilize the forward and reverse audio context information. This bidirectional processing structure can better capture the long-term dependencies of audio sequences, and combined with the state space model and the gating mechanism, it can effectively improve the modeling capability of time series patterns in audio signals.

[0041] Figure 5 The overall architecture of the amplitude mask decoder of the present invention is shown. First, the input noisy classroom audio amplitude features are processed by a diffuse dense connection network, which can make full use of the correlation between features at different levels and enhance the feature expression ability. Then, the processed features are combined with sub-pixel convolution and normalization layers, and nonlinear activation is performed using the PreLU function. This step can improve the feature expression ability and maintain the integrity of the information. Finally, the features are processed by a two-dimensional convolution layer and combined with a learnable Sigmoid function to generate the final mask information.

[0042] Figure 6 The overall architecture of the phase decoder of the present invention is shown. First, the input noisy classroom audio phase features are processed by the first diffuse dense connection network, which can effectively aggregate and transmit multi-scale feature information. Then, the processed features are passed through the sub-pixel convolution layer and combined with the normalization operation and PreLU function for nonlinear transformation to enhance the expressive ability of the features. Then, the features are input into two parallel two-dimensional convolution branches for further feature extraction. Finally, the features of the two branches are merged and optimized into the second diffuse dense connection network to obtain the final enhanced phase estimation result.

[0043] The effectiveness of the technical solution of the present invention is illustrated below by way of experiments based on the specific implementation records.

[0044] The performance of this method is compared with other 8 mainstream speech enhancement models on the Chinese classroom speech enhancement dataset. The results are shown in Table 1.

[0045] Table 1 Performance comparison results

[0046] Among them, PESQ stands for Perceptual Speech Quality, which is a recognized industry standard for audio quality. It takes into account characteristics such as audio clarity, call volume, background noise, and audio interference. The PESQ score ranges from -0.5 to 4.5, and the higher the score, the better the quality. The three indicators COVL, CSIG, and CBAK evaluate the quality of enhanced speech from three different perspectives, meaning the signal distortion score, background noise interference score, and overall speech quality score, respectively. These indicators are all based on the MOS (Mean Opinion Score) scoring system, and the score range is usually between 1 and 5 points. The higher the value, the better the speech quality.

[0047] The experimental results in Table 1 show that our method outperforms mainstream speech enhancement methods in all indicators. Compared with the baseline SEGAN, our method improves by 35.8%, 26.8%, 14%, and 32.8% in PESQ, CSIG, CBAK, and COVL, respectively.

[0048] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the scope of the present invention.

Claims

1. A classroom speech enhancement method based on convolutional bidirectional Mamba, characterized by: Step 1: Perform short-time Fourier transform and amplitude feature extraction on the noise-free classroom audio. Then, perform perceptual contrast stretching on the noise-free classroom audio amplitude features. The model's prediction results are optimized by comparing them with the prediction results of the speech enhancement model to obtain feature-enhanced noise-free classroom audio amplitude features. Step 2: Perform short-time Fourier transform (SFT) and extract amplitude and phase features on the noisy classroom audio. The two are then concatenated to obtain the frequency domain information of the noisy classroom audio. This information is then encoded using a two-dimensional convolutional layer, a diffuse dense connection network, and a convolutional bidirectional Mamba module to obtain the encoded noisy classroom audio features. Step 3: Based on the obtained noisy classroom audio features, they are processed separately through the amplitude mask decoder and phase decoder to obtain the predicted noisy classroom audio amplitude features and phase features. The predicted noisy classroom audio amplitude features are then compared and optimized with the feature-enhanced noise-free classroom audio amplitude features to obtain the final enhanced amplitude features. The final enhanced amplitude features are combined with the predicted noisy classroom audio phase features, and the final enhanced classroom audio signal is reconstructed through inverse short-time Fourier transform.

2. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 1 is characterized in that, The Step 1 is specifically as follows: Step 1.1: Perform short-time Fourier transform on the noise-free and noisy classroom audio to convert the time domain signal into the frequency domain signal; Step 1.2: Extract the noise-free audio amplitude features and the noisy audio amplitude features from the short-time Fourier transform results of the noise-free and noisy classroom audio respectively, and perform a perceptual contrast stretching operation on the noise-free audio amplitude features; Step 1.3: Compare the amplitude features predicted by the speech enhancement model with the amplitude features of the noise-free audio after perceptual contrast stretching to optimize the amplitude features predicted by the speech enhancement model. Step 1.4: Combine the enhanced amplitude features predicted by the optimized speech enhancement model with the original phase information, and reconstruct the feature-enhanced noisy classroom audio signal through inverse short-time Fourier transform to obtain a noise-free classroom audio signal.

3. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 1 is characterized in that, The Step 2 is specifically as follows: Step 2.1: Input the frequency domain information containing noise Feature extraction is performed through a two-dimensional convolutional layer, and nonlinear transformation is performed in combination with normalization operations and parameterized rectified linear unit functions; Step 2.2: Process the nonlinear transformation features through a diffuse dense connection network to achieve the fusion and transmission of features at different levels; Step 2.3: The features processed by the DDN are mapped to the feature space processed by the bidirectional time series modeling through a combination of the last layer of two-dimensional convolution and parameterized rectified linear unit function to complete feature encoding; Step 2.4: Input the encoded features into the convolutional bidirectional Mamba module, and perform bidirectional time series modeling through the forward and reverse processing branches to obtain the forward time series modeling results and the reverse time series modeling results respectively; Step 2.5: The forward timing modeling result and the reverse timing modeling result processed by the flip operation are integrated through the feature addition fusion mechanism to output the final encoding result.

4. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 1 is characterized in that, The Step 3 is specifically as follows: Step 3.1: The encoded features are divided into two branches: amplitude features and phase features. They are input into the amplitude mask decoder and phase decoder respectively. Both branches are first fused through the diffuse dense connection network. Step 3.2: After the amplitude feature branch undergoes sub-pixel convolution and activation with a parameterized rectified linear unit function, a learnable sigmoid function is used to generate mask information. Step 3.3: The phase feature branch uses a two-way parallel 2D convolutional structure to predict the real and imaginary parts of the phase information. Then, the diffuse dense connection network is used again for feature fusion to obtain the enhanced phase information. Step 3.4: The amplitude mask decoder multiplies the generated mask information with the noisy frequency amplitude to obtain the enhanced amplitude mask information, and combines it with the enhanced phase information obtained by the phase decoder to reconstruct the enhanced classroom audio signal.

5. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 2 is characterized in that, The specific steps of Step 1.2 are as follows: A preset perception stretch coefficient array is created based on the sensitivity of the human ear to different frequency bands, with a focus on enhancing the main speech frequency band of 400Hz-4400Hz. The short-time Fourier transform amplitude spectrum of the input feature is multiplied by the perceptual stretch coefficient to achieve perceptual contrast stretching of the amplitude.

6. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 3 is characterized in that, The reverse processing branch first flips the encoded features before processing.

7. The classroom speech enhancement method based on convolutional bidirectional Mamba according to claim 3 is characterized in that, The Step 2.5 is specifically as follows: The output features of the forward processing branch and the output features of the reverse processing branch after time series flipping are spliced in high dimensions on the feature dimension axis to form an extended feature representation that integrates bidirectional time series information. The spliced extended features are then reduced in dimension through a parameterized linear projection layer to map them back to the input noisy frequency domain information. Matching original dimensional space.

Images