Cabin voice noise reduction method, device and equipment and storage medium

Abstract

The invention provides a cockpit voice noise reduction method and device, equipment and a storage medium, and the method comprises the steps: obtaining an initial noisy voice signal in a cockpit, carrying out the frequency spectrum feature extraction, obtaining a full-band frequency spectrum feature, carrying out the equivalent rectangular bandwidth conversion of the full-band frequency spectrum feature, carrying out the low-frequency part interception, obtaining a sub-band feature and a low-frequency feature, and carrying out the noise reduction of the cockpit voice. Respectively performing interactive enhancement processing on the sub-band features and the low-frequency features to obtain sub-band coding features and low-frequency coding features, fusing the sub-band coding features and the low-frequency coding features to generate fused spectrum features, and respectively inputting the fused spectrum features to a sub-band feature decoder and a low-frequency feature decoder to obtain a sub-band mask and a low-frequency mask; determining a noise reduction spectrum feature based on the sub-band mask, the low-frequency mask and the full-band spectrum feature; according to the method and the device, the sub-band features and the low-frequency features in the full-band spectrum features are respectively processed, and the noise suppression is realized by adopting the interactive enhancement processing and the dual-decoder structure, so that the low-frequency speech features are reserved while the full-band noise is suppressed, and the speech definition and naturalness can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of cockpit noise reduction, and in particular to a cockpit voice noise reduction method, device, equipment and storage medium. BACKGROUND

[0002] In a vehicle-mounted voice communication and interaction system, voice quality is easily affected by complex driving environments. Mechanical noises such as wind noise, road noise and engine noise generated during vehicle operation, as well as environmental disturbances such as street noise and horn sounds from the outside, combined with background noise such as conversations and entertainment system sounds inside the vehicle, together form a multi-source mixed high-noise acoustic scene, which not only significantly reduces the intelligibility of voice signals, but also hinders voice recognition and voice communication and other vehicle-mounted voice applications. To solve the application problems caused by the above problems, noise suppression technology has become a key component in vehicle-mounted audio preprocessing systems, and the goal is to extract as pure as possible human voice from noisy voice signals to improve voice quality and system robustness. Common noise reduction schemes include traditional signal processing methods based on spectral subtraction and Wiener filtering, as well as end-to-end voice enhancement models based on deep learning.

[0003] However, the above noise reduction methods on the one hand need to rely on prior assumptions of noise statistical characteristics, and are difficult to adapt to dynamically changing driving environments; on the other hand, existing deep learning models often use a single feature processing method, which cannot achieve comprehensive and collaborative processing, resulting in excessive suppression of useful components in the voice while removing background noise. This not only causes voice distortion and sound quality degradation, but also may introduce artificial effects such as musical noise, which seriously affects the naturalness and clarity of the voice and restricts the user experience of the vehicle-mounted voice system. SUMMARY

[0004] The purpose of the embodiments of the present application is to provide a cockpit voice noise reduction method, device, equipment and storage medium to solve the above technical problems.

[0005] The application provides a cockpit voice noise reduction method, which comprises: acquiring an initial noisy voice signal in a cockpit, and performing spectral feature extraction on the initial noisy voice signal to obtain a full-band spectral feature; performing equivalent rectangular bandwidth conversion on the full-band spectral feature to obtain a sub-band feature, and performing low-frequency part interception on the full-band spectral feature to obtain a low-frequency feature, wherein the low-frequency part interception comprises intercepting spectral features lower than a preset low-frequency threshold in the full-band spectral feature; respectively performing interactive enhancement processing on the sub-band feature and the low-frequency feature to obtain a sub-band encoded feature and a low-frequency encoded feature, and fusing the sub-band encoded feature and the low-frequency encoded feature to generate a fused spectral feature; inputting the fused spectral feature into a sub-band feature decoder and a low-frequency feature decoder respectively to obtain a sub-band mask and a low-frequency mask, so as to determine a noise reduction spectral feature based on the sub-band mask, the low-frequency mask and the full-band spectral feature.

[0006] In an embodiment of the application, the interactive enhancement processing of the sub-band feature and the low-frequency feature respectively comprises: inputting the sub-band feature and the low-frequency feature into a sub-band feature encoder, performing attention calculation on the low-frequency feature as a query, the sub-band feature as a key and a value, obtaining a sub-band encoding weight, and performing feature reconstruction on the sub-band feature by using the sub-band encoding weight to obtain a sub-band encoded feature; inputting the sub-band feature and the low-frequency feature into a low-frequency feature encoder, performing attention calculation on the sub-band feature as a query, the low-frequency feature as a key and a value, obtaining a low-frequency encoding weight, and performing feature reconstruction on the low-frequency feature by using the low-frequency encoding weight to obtain a low-frequency encoded feature.

[0007] In an embodiment of the application, the fusion of the sub-band encoded feature and the low-frequency encoded feature to generate a fused spectral feature comprises: splicing the sub-band encoded feature and the low-frequency encoded feature to form a spliced encoded feature; inputting the spliced encoded feature into a preset neural network to obtain an encoding fusion weight, wherein the preset neural network is used for weight distribution for feature fusion of the sub-band encoded feature and the low-frequency encoded feature; and performing weighted summation on the sub-band encoded feature and the low-frequency encoded feature according to the encoding fusion weight to obtain a fused spectral feature.

[0008] In an embodiment of the present application, inputting the fusion spectrum feature into the sub-band feature decoder and the low-frequency feature decoder respectively to obtain the sub-band mask and the low-frequency mask includes: inputting the fusion spectrum feature into the sub-band feature decoder for decoding processing, outputting a sub-band mask corresponding to the full-band spectrum feature, and the sub-band mask is used for noise suppression on the full-band part of the full-band spectrum feature; inputting the fusion spectrum feature into the low-frequency feature decoder for decoding processing, outputting a low-frequency mask corresponding to the low-frequency part of the full-band spectrum feature, and the low-frequency mask is used for noise suppression on the low-frequency part of the full-band spectrum feature.

[0009] In an embodiment of the present application, determining the noise reduction spectrum feature based on the sub-band mask, the low-frequency mask and the full-band spectrum feature includes: multiplying the sub-band mask and the full-band spectrum feature to obtain a sub-band noise reduction result; multiplying the low-frequency mask and the low-frequency part of the full-band spectrum feature to obtain a low-frequency noise reduction result; and fusing the sub-band noise reduction result and the low-frequency noise reduction result to obtain the noise reduction spectrum feature.

[0010] In an embodiment of the present application, fusing the sub-band noise reduction result and the low-frequency noise reduction result to obtain the noise reduction spectrum feature includes: performing confidence prediction on the sub-band noise reduction result to obtain a sub-band confidence score, and performing confidence prediction on the low-frequency noise reduction result to obtain a low-frequency confidence score; performing normalization processing on the sub-band confidence score and the low-frequency confidence score respectively to obtain a sub-band noise reduction weight and a low-frequency noise reduction weight, and performing weighted summation on the sub-band noise reduction result and the low-frequency noise reduction result based on the sub-band noise reduction weight and the low-frequency noise reduction weight respectively to obtain the noise reduction spectrum feature.

[0011] In an embodiment of the present application, after determining the noise reduction spectrum feature based on the sub-band mask, the low-frequency mask and the full-band spectrum feature, the cabin voice noise reduction method further includes: performing inverse Fourier transform on the noise reduction spectrum feature to obtain a to-be-processed time domain signal, and performing overlap-add processing on the to-be-processed time domain signal to obtain a time domain voice signal.

[0012] This application embodiment also provides a cockpit voice noise reduction device, which includes: a signal feature extraction module, used to acquire an initial noisy voice signal in the cockpit, and extract spectral features from the initial noisy voice signal to obtain full-band spectral features; perform equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncate the low-frequency portion of the full-band spectral features to obtain low-frequency features, wherein the low-frequency truncation includes truncating spectral features below a preset low-frequency threshold from the full-band spectral features; a spectral feature noise reduction module, used to perform interactive enhancement processing on the sub-band features and the low-frequency features respectively to obtain sub-band coded features and low-frequency coded features, and fuse the sub-band coded features and low-frequency coded features to generate fused spectral features; input the fused spectral features to the sub-band feature decoder and the low-frequency feature decoder respectively to obtain a sub-band mask and a low-frequency mask, so as to determine the noise reduction spectral features based on the sub-band mask, the low-frequency mask and the full-band spectral features.

[0013] This application also provides an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by the one or more processors, cause the electronic device to implement the cockpit voice noise reduction method as described in any of the above embodiments.

[0014] This application also provides a computer-readable storage medium storing computer-readable instructions, which, when executed by a computer's processor, cause the computer to perform the cockpit voice noise reduction method as described in any of the above embodiments.

[0015] The beneficial effects of this invention are as follows: This application provides a cockpit speech noise reduction method, apparatus, device, and storage medium. It acquires an initial noisy speech signal from within the cockpit, extracts spectral features from the initial noisy speech signal to obtain full-band spectral features, performs an equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncates the low-frequency portion of the full-band spectral features to obtain low-frequency features. It then performs interactive enhancement processing on the sub-band features and low-frequency features to obtain sub-band encoded features and low-frequency encoded features, and fuses the sub-band encoded features and low-frequency encoded features to generate fused spectral features. These fused spectral features are then input to a sub-band feature decoder and a low-frequency feature decoder to obtain a sub-band mask and a low-frequency mask, respectively. Noise reduction spectral features are then determined based on the sub-band mask, low-frequency mask, and full-band spectral features. This application achieves noise suppression by separately processing the sub-band features and low-frequency features within the full-band spectral features and employing interactive enhancement processing and a dual-decoder structure. This effectively preserves low-frequency speech features while suppressing full-band noise, thereby improving speech clarity and naturalness.

[0016] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0017] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0018] Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating a cockpit voice noise reduction method in an exemplary embodiment of this application;

[0020] Figure 3 This is a flowchart illustrating a specific cockpit voice noise reduction method as an exemplary embodiment of this application;

[0021] Figure 4 This is an exemplary embodiment of the present application illustrating a specific ERB feature encoder execution flowchart;

[0022] Figure 5 This is an exemplary embodiment of the present application illustrating a specific ERB feature decoder execution flowchart;

[0023] Figure 6 This is an exemplary embodiment of the present application illustrating a specific low-frequency feature decoder execution flowchart;

[0024] Figure 7 This is a schematic diagram of a cockpit voice noise reduction device shown in an exemplary embodiment of this application;

[0025] Figure 8 This is a schematic diagram of the structure of a computer system for an electronic device, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0026] The embodiments of this application will be described below with reference to the accompanying drawings and specific examples. Those skilled in the art can easily understand other advantages and effects of this application from the content disclosed in this specification. This application can also be implemented or applied through other different specific embodiments, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of this application. It should be understood that the preferred embodiments are only for illustrating this application and are not intended to limit the scope of protection of this application.

[0027] It should be noted that the illustrations provided in the following embodiments are only schematic representations of the basic concept of this application. Therefore, the illustrations only show the components related to this application and are not drawn according to the number, shape and size of the components in actual implementation. In actual implementation, the form, quantity and proportion of each component can be arbitrarily changed, and the layout of the components may also be more complex.

[0028] In the following description, numerous details are explored to provide a more thorough explanation of embodiments of the present application. However, it will be apparent to those skilled in the art that embodiments of the present application may be practiced without these specific details. In other embodiments, well-known structures and devices are shown in block diagram form rather than in detail to avoid obscuring embodiments of the present application.

[0029] The term "and / or" used in this application describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. The character " / " generally indicates that the related objects before and after it are in an "or" relationship.

[0030] Please see Figure 1 , Figure 1 This is a schematic diagram illustrating an exemplary system architecture as shown in an exemplary embodiment of this application.

[0031] Reference Figure 1 As shown, the system architecture may include a vehicle 110 and a computer device 120. The computer device 120 acquires the initial noisy speech signal from the cabin of the vehicle 110, extracts spectral features from the initial noisy speech signal to obtain full-band spectral features, performs equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncates the low-frequency portion of the full-band spectral features to obtain low-frequency features. The sub-band features and low-frequency features are then interactively enhanced to obtain sub-band coded features and low-frequency coded features, respectively. These sub-band coded features and low-frequency coded features are then fused to generate fused spectral features. The fused spectral features are input to the sub-band feature decoder and the low-frequency feature decoder to obtain a sub-band mask and a low-frequency mask, respectively, to determine the noise reduction spectral features based on the sub-band mask, the low-frequency mask, and the full-band spectral features. The aforementioned vehicle 110 includes a cockpit with voice functionality, which includes at least a microphone to collect noisy speech within the cockpit; the aforementioned computer equipment 120 refers to a computing power support terminal device used to carry out the program implementation environment for the cockpit speech noise reduction method, including but not limited to tablet devices, microcomputers, embedded computers, and cloud servers.

[0032] In a schematic manner, computer device 120 acquires the initial noisy speech signal from the cabin of vehicle 110, extracts spectral features from the initial noisy speech signal to obtain full-band spectral features, performs equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncates the low-frequency portion of the full-band spectral features to obtain low-frequency features. Interactive enhancement processing is performed on the sub-band features and low-frequency features respectively to obtain sub-band coded features and low-frequency coded features. The sub-band coded features and low-frequency coded features are then fused to generate fused spectral features. The fused spectral features are input to the sub-band feature decoder and the low-frequency feature decoder respectively to obtain the sub-band mask and the low-frequency mask. The noise reduction spectral features are then determined based on the sub-band mask, the low-frequency mask, and the full-band spectral features. This application achieves noise suppression by separately processing the sub-band features and low-frequency features in the full-band spectral features and employing interactive enhancement processing and a dual-decoder structure. While suppressing full-band noise, it effectively preserves low-frequency speech features, thereby improving speech clarity and naturalness.

[0033] Figure 2 This is a flowchart illustrating an exemplary embodiment of the present application of a cockpit voice noise reduction method, which can... Figure 1 It can be executed in the implementation environment described above, but it can also be implemented in other implementation environments. No specific limitations are imposed on the aforementioned implementation environments here. (See also...) Figure 2 As shown, the flowchart of this cockpit voice noise reduction method includes at least steps S210 to S240, which are described in detail below:

[0034] In step S210, the initial noisy speech signal in the cockpit is acquired, and the spectral features of the initial noisy speech signal are extracted to obtain the full-band spectral features.

[0035] In one embodiment of this application, spectral feature extraction of the initial noisy speech signal includes extracting the complete frequency domain of the speech signal through time-frequency transformation, including but not limited to using short-time Fourier transform or Mel filter banks. The initial noisy speech signal is a sound signal collected from inside the vehicle cabin, containing the sound of people speaking inside the cabin and various environmental noises, such as the static sound of doors opening and closing, and the dynamic sound of tire noise and wind noise. The final form is noisy speech after the mixture of various sounds. For example, if the initial noisy speech signal has 512 sampling points, after undergoing a 512-point FFT (Fast Fourier Transform), a spectral feature of length 257 will be obtained.

[0036] In step S220, the full-band spectrum features are converted to equivalent rectangular bandwidth to obtain sub-band features, and the low-frequency part of the full-band spectrum features is truncated to obtain low-frequency features.

[0037] In one embodiment of this application, the aforementioned low-frequency portion truncation includes truncating spectral features below a preset low-frequency threshold from the full-band spectral features.

[0038] In one embodiment of this application, the above-mentioned equivalent rectangular bandwidth conversion includes dividing the spectrum into sub-bands that conform to the characteristics of human hearing. In some feasible environments, the conversion can be performed by an ERB (Equivalent Rectangular Bandwidth) filter bank. The equivalent rectangular bandwidth conversion can reflect the differences in the sensitivity of the auditory system to noise in different frequency bands.

[0039] In a specific embodiment of this application, during the process of performing equivalent rectangular bandwidth conversion on full-band spectral features, the full-band spectral features are converted into 32-dimensional sub-band features. It is important to note that the width of the equivalent rectangular bandwidth feature needs to be predefined before calculating the sub-band features. For example, if the width of the sub-band feature ERB is W = [w1, w2, w3, ..., wn], where n = 32, then the calculation of the equivalent rectangular bandwidth feature conversion includes:

[0040] ERB(1)=(spec[1]+…+spec[w1]) / w1;

[0041] ERB(2)=(spec[w1+1]+spec[w1+2]+…+spec[w1+w2]) / w2;

[0042] …

[0043] Here, spec refers to the full-band spectral characteristics.

[0044] In a specific embodiment of this application, during the process of low-frequency truncation of full-band spectral features, in some feasible environments, when the number of sampling points of the initial noisy speech signal is 512, the full-band spectral features include 257 frequency points, and the low-frequency truncation can take the first 48 dimensions of the 257 frequency points.

[0045] In step S230, the sub-band features and low-frequency features are interactively enhanced to obtain sub-band coded features and low-frequency coded features, and the sub-band coded features and low-frequency coded features are fused to generate fused spectral features.

[0046] In one embodiment of this application, subband features and low-frequency features are input into a subband feature encoder. Attention is calculated using low-frequency features as queries and subband features as keys and values ​​to obtain subband coding weights. Subband features are then reconstructed using these subband coding weights to obtain subband coding features.

[0047] In one embodiment of this application, the subband feature encoder includes a neural network module for processing subband features after equivalent rectangular bandwidth transformation, including but not limited to a multilayer perceptron or a convolutional neural network. It establishes the correlation between subband features and low-frequency features through an attention mechanism, including modifying the subband features by adding low-frequency feature information to the subband feature information. The aforementioned attention calculation assigns weights based on the similarity between features to capture potential correlations between cross-frequency band features. Feature reconstruction refers to weighted fusion of the original features based on attention weights, including fusion through matrix multiplication, to strengthen the parts highly correlated with the subband features.

[0048] Specifically, in the subband feature encoder, low-frequency features are used as query vectors and key vectors of subband features for similarity matching to generate subband encoding weights that reflect the degree of association between the two. The value vectors of subband features are then weighted and summed based on the subband encoding weights to retain the subband features while introducing global information from low-frequency features.

[0049] In one embodiment of this application, subband features and low-frequency features are input into a low-frequency feature encoder. Attention is calculated using subband features as queries and low-frequency features as keys and values ​​to obtain low-frequency coding weights. Low-frequency features are then reconstructed using these low-frequency coding weights to obtain low-frequency coding features.

[0050] In one embodiment of this application, the low-frequency feature encoder includes a neural network module for processing truncated low-frequency features. In some embodiments, it adopts a structure symmetrical to the sub-band feature encoder. Feature reconstruction refers to weighted fusion of the original features based on attention weights, including fusion through matrix multiplication, to strengthen the parts that are highly correlated with the low-frequency features.

[0051] Specifically, in the low-frequency feature encoder, the sub-band feature is used as a query vector to match the key vector of the low-frequency feature, generating a low-frequency encoding weight, and reconstructing the low-frequency feature based on the low-frequency encoding weight, so that the low-frequency feature can integrate the detailed information in the sub-band feature.

[0052] In one embodiment of this application, a bidirectional attention mechanism enables sub-band features and low-frequency features to complement each other across frequency bands during the encoding process, thereby enhancing the representational capabilities of their respective features. Specifically, this approach can enhance the noise resistance of low-frequency features while preserving high-frequency speech details, reduce speech distortion caused by fragmented frequency band feature processing, improve the intelligibility and naturalness of speech signals in complex noise scenarios, and suppress noise interference on specific frequency band features through cross-band feature interaction, thereby enhancing the coherence of speech components across different frequency bands.

[0053] In one embodiment of this application, fusing subband coding features and low-frequency coding features includes concatenating the subband coding features and low-frequency coding features to form concatenated coding features, inputting the concatenated coding features into a preset neural network for weight allocation to obtain coding fusion weights, the preset neural network being used for weight allocation of feature fusion of subband coding features and low-frequency coding features, and performing weighted summation of subband coding features and low-frequency coding features according to the coding fusion weights to obtain fused spectral features.

[0054] In one embodiment of this application, concatenating subband coding features with low-frequency coding features includes forming a composite feature vector by connecting subband coding features and low-frequency coding features along the feature dimension. The process can be based on a fully connected layer or an attention mechanism to achieve channel-dimensional concatenation, so as to preserve the high-frequency details of the subband features and the structural information of the low-frequency features.

[0055] In one embodiment of this application, the aforementioned preset neural network refers to a deep learning model used for feature fusion, which can dynamically allocate feature weights through the adaptive learning capability of the neural network. In a specific embodiment, it can be characterized as follows:

[0056] weight=torch.sigmoid(MLP(torch.cat([new_erb_feat,new_df_feat],dim=-1)))

[0057] Wherein, new_erb_feat represents the subband encoded feature output by the ERB encoder; new_df_feat represents the low-frequency encoded feature output by the low-frequency encoder; torch.cat(...) represents splicing the subband encoded feature and the low-frequency encoded feature; MLP(...) represents a deep learning model including a small multilayer perceptron, used to learn and assign weights to the two features; torch.sigmoid(...) represents compressing the output of the deep learning model to the [0,1] interval, finally obtaining a floating-point number weight.

[0058] If the weight is close to 1, it means that the subband coding features of the current frame are more important, such as high frequencies needing to be suppressed in noisy environments; if the weight is close to 0, it means that low frequency coding features are more critical.

[0059] During the fusion process, the encoding fusion weights are proportional parameters reflecting the importance of subbands and low-frequency features. They are obtained by normalizing the neural network output using the Softmax function. The feature fusion process can be characterized as follows:

[0060] output=weight*new_erb_feat+(1-weight)*new_df_feat

[0061] It is represented as a linear combination of sub-band coding features and low-frequency coding features according to weights, which is used to retain the auditory optimization characteristics of sub-band coding features, while injecting the temporal dynamic information of low-frequency coding features to avoid the rigidity problem of fixed ratio fusion.

[0062] In one embodiment of this application, during the feature fusion stage, the interactively enhanced sub-band coding features and low-frequency coding features are first concatenated along the channel dimension to form a concatenated coding feature containing full-band information. This concatenated coding feature is then fed into a pre-defined neural network model, which analyzes the correlation between the sub-band features and low-frequency features in the time-frequency domain through multi-layer nonlinear transformations, outputting fusion weight coefficients. Finally, the sub-band coding features and low-frequency coding features are fused according to dynamically allocated weights using a weighted summation method, generating a fused spectral feature that retains both high-frequency noise suppression capabilities and low-frequency voice protection characteristics. This process effectively balances the needs of high-frequency noise suppression and low-frequency voice protection, achieving more accurate frequency band feature fusion in complex vehicle noise scenarios. It avoids speech distortion or noise residue problems caused by traditional fixed fusion strategies, significantly improving the clarity and naturalness of denoised speech.

[0063] In step S240, the fused spectral features are input to the sub-band feature decoder and the low-frequency feature decoder respectively to obtain the sub-band mask and the low-frequency mask, so as to determine the noise reduction spectral features based on the sub-band mask, the low-frequency mask and the full-band spectral features.

[0064] In one embodiment of this application, the fused spectral features are input into a sub-band feature decoder for decoding, and a sub-band mask corresponding to the full-band spectral features is output. The sub-band mask is used to suppress noise in the full-band portion of the full-band spectral features.

[0065] The subband feature decoder is a spectrum decoding module built on a fully connected network or convolutional neural network. Its input is the fused spectral features, and its output is a mask coefficient matrix covering the entire frequency band. The subband mask is a two-dimensional matrix generated by the subband feature decoder, where each element corresponds to the gain coefficient at each frequency point of the full-band spectrum, used to attenuate noise components in the frequency domain. In some specific implementations, the subband feature decoder can decode the fused spectral features into a 257-dimensional mask.

[0066] In one embodiment of this application, the fused spectral features are input into a low-frequency feature decoder for decoding, and a low-frequency mask corresponding to the low-frequency part of the full-band spectral features is output. The low-frequency mask is used to suppress noise in the low-frequency part of the full-band spectral features.

[0067] The low-frequency feature decoder refers to a decoding module built on a neural network structure based on bandpass filtering constraints. Its output is a mask coefficient that operates only on the low-frequency band. Specifically, it refers to a one-dimensional vector generated by the low-frequency feature decoder, where the vector elements correspond to the gain coefficients of each frequency point in the low-frequency band, used to selectively suppress low-frequency noise. In some specific implementation environments, the low-frequency feature decoder can decode the fused spectral features into a 48-dimensional low-frequency mask.

[0068] Specifically, the fused spectral features are simultaneously input into two independently operating decoder modules. The sub-band feature decoder maps the fused features to a full-band mask through multi-layer nonlinear transformations. This mask is then multiplied point-by-point with the original noisy speech spectrum in the frequency domain, achieving global suppression of broadband noise. Meanwhile, the low-frequency feature decoder, during decoding, uses a frequency band constraint mechanism to reconstruct features only in low-frequency regions below a preset threshold, generating a mask that operates only on the low-frequency band. This mask is then multiplied with the low-frequency portion of the original spectrum, focusing on eliminating low-frequency interference. This approach effectively addresses the mixed interference characteristics of broadband and low-frequency noise in the vehicle cabin environment, maintaining high-frequency speech clarity while avoiding distortion from low-frequency speech formants, significantly improving the naturalness and intelligibility of the denoised speech.

[0069] In one embodiment of this application, when determining the noise reduction spectrum features based on the sub-band mask, low-frequency mask, and full-band spectrum features, the sub-band mask is multiplied by the full-band spectrum features to obtain the sub-band noise reduction result, and the low-frequency mask is multiplied by the low-frequency part of the full-band spectrum features to obtain the low-frequency noise reduction result. Then, the sub-band noise reduction result and the low-frequency noise reduction result are fused to obtain the noise reduction spectrum features.

[0070] The multiplication process involves performing a point-by-point multiplication operation between the mask matrix and the spectral feature matrix, specifically through matrix element multiplication. Specifically, after obtaining the sub-band mask and the low-frequency mask, the sub-band mask is first applied to the full-band spectral feature, suppressing noise components in the mid-to-high frequency regions through point-by-point multiplication while preserving the main body of the speech signal. The low-frequency mask is then applied to a pre-trimmed low-frequency portion of the full-band spectral feature, suppressing low-frequency noise such as engine vibration through the same operation. Finally, the processed sub-band noise reduction result and the low-frequency noise reduction result are fused in the frequency domain to form a noise-reduced spectral feature that balances noise suppression across the entire frequency band with speech integrity.

[0071] The process of fusing subband denoising results with low-frequency denoising results to obtain denoised spectral features includes: predicting the confidence level of subband denoising results to obtain subband confidence scores; predicting the confidence level of low-frequency denoising results to obtain low-frequency confidence scores; normalizing the subband confidence scores and low-frequency confidence scores to obtain subband denoising weights and low-frequency denoising weights; and then weighting and summing the subband denoising results and low-frequency denoising results based on the subband denoising weights and low-frequency denoising weights to obtain denoised spectral features.

[0072] In one embodiment of this application, confidence prediction includes evaluating the reliability of the denoising result through a computational model. This can be achieved by probabilistically predicting the degree of speech component retention in the denoising result based on a neural network model. Normalization refers to converting the confidence score into weight coefficients using a softmax function, which can dynamically allocate the fusion ratio between sub-band denoising results and low-frequency denoising results. Specifically, the confidence prediction module generates reliability indices for both types of results. This can be achieved by performing feature analysis on the denoising result using an evaluation network constructed with, but not limited to, convolutional layers and fully connected layers. Then, normalization converts the confidence score into weight values ​​with a sum of 1. The sub-band denoising result is then multiplied by the sub-band denoising weight, and the low-frequency denoising result is multiplied by the low-frequency denoising weight. These are then summed to form a fused denoised spectral feature.

[0073] In one embodiment of this application, after determining the noise reduction spectral features based on the sub-band mask, low-frequency mask and full-band spectral features, the noise reduction spectral features are subjected to inverse Fourier transform to obtain the time-domain signal to be processed, and the time-domain signal to be processed is subjected to overlapping and addition processing to obtain the time-domain speech signal.

[0074] In one embodiment of this application, the inverse Fourier transform refers to the process of converting the denoised spectral features represented in the frequency domain into a time domain signal, including the inverse fast Fourier transform algorithm, which can restore the denoised spectral features after frequency domain denoising to a playable time domain waveform; the overlapping addition processing refers to the operation of smoothing the inter-frame superposition of the time domain signals processed by frame segmentation. In the specific implementation process, adjacent frame signals are weighted and superimposed by a window function with a fixed step size to eliminate the signal discontinuity caused by frame segmentation and restore the complete time domain speech signal.

[0075] Figure 3This is a flowchart illustrating a specific cockpit voice noise reduction method according to an exemplary embodiment of this application. It should be noted that in the following specific embodiments, the noisy speech spectral characteristics are consistent with the full-band spectral characteristics in the above embodiments, the ERP noise reduction result is consistent with the sub-band noise reduction result in the above embodiments, the ERP feature encoder is consistent with the sub-band feature encoder in the above embodiments, the low-frequency noise reduction result is consistent with the low-frequency noise reduction result in the above embodiments, and the final noise reduction result is consistent with the noise reduction spectral characteristics in the above embodiments. (Refer to...) Figure 3 As shown in the flowchart of this specific cockpit speech noise reduction method, the noisy speech spectral features are subjected to ERP feature extraction to obtain sub-band features, and low-frequency features are extracted to obtain low-frequency features. The extracted sub-band features and low-frequency features are then input into the ERP feature encoder and low-frequency feature encoder, respectively. The ERP feature encoder outputs the sub-band encoded features, and the low-frequency feature encoder outputs the low-frequency encoded features. Figure 4 This is an exemplary embodiment of the present application illustrating a specific ERB feature encoder execution flowchart, with reference to... Figure 4 As shown, after receiving ERB feature input, the ERB feature encoder performs feature interaction enhancement through three Conv convolutional layers, and performs weighted calculation and reconstruction of features based on liner linear layers and sequence processing by GRU gated recurrent units to output sub-band encoded features. Similarly, the low-frequency feature encoder also includes Conv convolutional layers, liner linear layers and GRU gated recurrent units, but the low-frequency feature encoder has fewer Conv convolutional layers than the ERB feature encoder.

[0076] In one specific embodiment of this application, sub-band coding features and low-frequency coding features are input into a feature fusion module to generate fused spectral features. These fused spectral features are then input into the ERP feature decoder and the low-frequency feature decoder, respectively. The ERP feature decoder outputs the ERP noise reduction result. Figure 5 This is an exemplary embodiment of the present application illustrating a specific ERB feature decoder execution flowchart, with reference to... Figure 5 As shown, after the ERB feature decoder receives the merged features (fused spectral features), it undergoes a nonlinear transformation through three layers of Tconv transposed convolutions, and then a Conv convolutional layer maps the fused features into a full-band mask for output; the low-frequency feature decoder outputs the low-frequency noise reduction result, specifically, Figure 6 This is an exemplary embodiment of the present application illustrating a specific low-frequency feature decoder execution flowchart, with reference to... Figure 6As shown, after the low-frequency feature decoder receives the merged features (fused spectral features), the features are transformed by two Conv convolutional layers, then processed by a sequence based on a GRU gated recurrent unit, and finally linearly processed by a liner linear layer to output the low-frequency noise reduction result. The ERP noise reduction result and the low-frequency noise reduction result are then fused to obtain the final noise reduction result.

[0077] This application provides a cockpit speech noise reduction method, apparatus, device, and storage medium. It acquires an initial noisy speech signal from within the cockpit, extracts spectral features from the initial noisy speech signal to obtain full-band spectral features, performs an equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncates the low-frequency portion of the full-band spectral features to obtain low-frequency features. It then performs interactive enhancement processing on the sub-band features and low-frequency features to obtain sub-band coded features and low-frequency coded features, respectively. Finally, it fuses the sub-band coded features and low-frequency coded features to generate fused spectral features. These fused spectral features are input to the sub-band feature decoder and the low-frequency feature decoder to obtain a sub-band mask and a low-frequency mask, respectively. Noise reduction spectral features are then determined based on the sub-band mask, low-frequency mask, and full-band spectral features. This application achieves noise suppression by separately processing the sub-band features and low-frequency features within the full-band spectral features and employing interactive enhancement processing and a dual-decoder structure. This effectively preserves low-frequency speech features while suppressing full-band noise, thereby improving speech clarity and naturalness.

[0078] The following describes an embodiment of the apparatus described in this application, which can be used to execute the cockpit voice noise reduction method described in the above embodiments of this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the cockpit voice noise reduction method described above in this application.

[0079] Figure 7 This is a schematic diagram illustrating a cockpit voice noise reduction device according to an exemplary embodiment of this application. The device can be applied to... Figure 2 The method implementation process shown can be based on the device Figure 1 The implementation environment shown can be applied to other exemplary implementation environments and specifically configured in other devices. This embodiment does not limit the implementation environment to which the device is applicable.

[0080] like Figure 7 As shown, the exemplary cockpit voice noise reduction device includes: a signal feature extraction module 701 and a spectrum feature noise reduction module 702.

[0081] The signal feature extraction module 701 is used to acquire the initial noisy speech signal in the cockpit and extract the spectral features of the initial noisy speech signal to obtain full-band spectral features; perform equivalent rectangular bandwidth transformation on the full-band spectral features to obtain sub-band features, and truncate the low-frequency part of the full-band spectral features to obtain low-frequency features. The low-frequency part truncation includes truncating the spectral features below a preset low-frequency threshold in the full-band spectral features; the spectral feature denoising module 702 is used to perform interactive enhancement processing on the sub-band features and low-frequency features respectively to obtain sub-band coded features and low-frequency coded features, and fuse the sub-band coded features and low-frequency coded features to generate fused spectral features; input the fused spectral features to the sub-band feature decoder and low-frequency feature decoder respectively to obtain sub-band mask and low-frequency mask, and determine the denoised spectral features based on the sub-band mask, low-frequency mask and full-band spectral features.

[0082] Embodiments of this application also provide an electronic device, including: one or more processors; and a storage device for storing one or more programs, which, when executed by one or more processors, cause the electronic device to implement the cockpit voice noise reduction method provided in the above embodiments.

[0083] Figure 8 This is a schematic diagram illustrating the structure of a computer system for an electronic device, as shown in an exemplary embodiment of this application. It should be noted that... Figure 8 The computer system 800 of the electronic device shown is merely an example and should not impose any limitation on the functionality and scope of use of the embodiments of this application.

[0084] like Figure 8 As shown, the computer system 800 includes a Central Processing Unit (CPU) 801, which can perform various appropriate actions and processes, such as executing the methods described in the above embodiments, based on a program stored in Read-Only Memory (ROM) 802 or a program loaded from storage into Random Access Memory (RAM) 803. The RAM 803 also stores various programs and data required for system operation. The CPU 801, ROM 802, and RAM 803 are interconnected via a bus. An I / O interface 805 is also connected to the bus 804, where the I / O interface 805 refers to an input / output interface.

[0085] The following components are connected to I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and speakers, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN (Local Area Network) card, modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on drive 810 as needed so that computer programs read from it can be installed into storage section 808 as needed.

[0086] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by central processing unit (CPU) 801, it performs various functions defined in the system of this application.

[0087] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media can also be any computer-readable medium other than computer-readable storage media, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0088] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0089] In the corresponding figures of the above embodiments, connecting lines can represent the connection relationship between various components, indicating more constitutive signal paths and / or one or more ends of some lines having arrows to indicate the main information flow direction. Connecting lines are an identifier and are not a limitation on the scheme itself, but rather the use of these lines in conjunction with one or more exemplary embodiments helps to more easily connect circuits or logic units. Any signal represented (determined by design requirements or preferences) can actually include one or more signals that can be transmitted in any direction and can be implemented in any suitable type of signal scheme.

[0090] The units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these units do not necessarily limit the specific unit itself.

[0091] Another aspect of this application provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the method as described above. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not assembled into the electronic device.

[0092] Embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements a cockpit voice noise reduction method as described in any of the above embodiments.

[0093] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to the embodiments of this application, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0094] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, touch terminal, or network device, etc.) to execute the method according to the embodiments of this application.

[0095] This application can be used in a wide range of general-purpose or special-purpose computing system environments or configurations. For example: personal computers, server computers, handheld or portable devices, tablet devices, multiprocessor systems, microprocessor-based systems, set-top boxes, programmable consumer electronics devices, network PCs, minicomputers, mainframe computers, distributed computing environments including any of the above systems or devices, etc.

[0096] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the embodiments disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein.

[0097] The above embodiments are merely illustrative of the principles and effects of this application and are not intended to limit this application. Any person skilled in the art can modify or alter the above embodiments without departing from the spirit and scope of this invention. Therefore, all equivalent modifications or alterations made by those skilled in the art without departing from the spirit and technical concept disclosed in this application should still be covered by the claims of this application.

Claims

1. A cockpit voice noise reduction method, characterized in that, The cockpit voice noise reduction method comprises: An initial noisy voice signal in the cockpit is acquired, and spectral feature extraction is performed on the initial noisy voice signal to obtain full-band spectral features; The full-band spectral features are subjected to equivalent rectangular bandwidth conversion to obtain sub-band features, and low-frequency part interception is performed on the full-band spectral features to obtain low-frequency features, the low-frequency part interception comprising intercepting spectral features lower than a preset low-frequency threshold in the full-band spectral features; The sub-band features and the low-frequency features are respectively subjected to interactive enhancement processing to obtain sub-band encoded features and low-frequency encoded features, and the sub-band encoded features and the low-frequency encoded features are fused to generate fused spectral features; The fused spectral features are respectively input into a sub-band feature decoder and a low-frequency feature decoder to obtain sub-band masks and low-frequency masks, so as to determine noise reduction spectral features based on the sub-band masks, the low-frequency masks and the full-band spectral features.

2. The cockpit speech noise reduction method of claim 1, wherein, The interactive enhancement processing of the sub-band features and the low-frequency features comprises: The sub-band features and the low-frequency features are input into a sub-band feature encoder, the low-frequency features are taken as queries, the sub-band features are taken as keys and values for attention calculation to obtain sub-band encoding weights, and the sub-band features are reconstructed by using the sub-band encoding weights to obtain sub-band encoded features; The sub-band features and the low-frequency features are input into a low-frequency feature encoder, the sub-band features are taken as queries, the low-frequency features are taken as keys and values for attention calculation to obtain low-frequency encoding weights, and the low-frequency features are reconstructed by using the low-frequency encoding weights to obtain low-frequency encoded features.

3. The cabin voice noise reduction method of claim 2, wherein, The fusion of the sub-band encoded features and the low-frequency encoded features to generate fused spectral features comprises: The sub-band encoded features and the low-frequency encoded features are spliced to form spliced encoded features; The spliced encoded features are input into a preset neural network for weight distribution to obtain encoding fusion weights, the preset neural network being used for weight distribution for feature fusion of the sub-band encoded features and the low-frequency encoded features; The sub-band encoded features and the low-frequency encoded features are weighted and summed according to the encoding fusion weights to obtain fused spectral features.

4. The cockpit speech noise reduction method of claim 1, wherein, The input of the fused spectral features into the sub-band feature decoder and the low-frequency feature decoder to obtain sub-band masks and low-frequency masks comprises: The fused spectral features are input into the sub-band feature decoder for decoding processing to output sub-band masks corresponding to the full-band spectral features, the sub-band masks being used for noise suppression on a full-band part of the full-band spectral features; The fused spectral features are input into the low-frequency feature decoder for decoding processing to output low-frequency masks corresponding to a low-frequency part of the full-band spectral features, the low-frequency masks being used for noise suppression on the low-frequency part of the full-band spectral features.

5. The cabin voice noise reduction method of claim 4, wherein, The determination of noise reduction spectral features based on the sub-band masks, the low-frequency masks and the full-band spectral features comprises: The sub-band masks are multiplied by the full-band spectral features to obtain sub-band noise reduction results; The low-frequency masks are multiplied by a low-frequency part of the full-band spectral features to obtain low-frequency noise reduction results; and The full-band spectral features are multiplied by the sub-band noise reduction results and the low-frequency noise reduction results to obtain noise reduction spectral features. The sub-band noise reduction result and the low-frequency noise reduction result are fused to obtain a noise reduction spectrum feature.

6. The cabin voice noise reduction method of claim 5, wherein, The sub-band noise reduction result and the low-frequency noise reduction result are fused to obtain a noise reduction spectrum feature. The sub-band noise reduction result is subjected to confidence prediction to obtain a sub-band confidence score, and the low-frequency noise reduction result is subjected to confidence prediction to obtain a low-frequency confidence score. The sub-band confidence score and the low-frequency confidence score are respectively subjected to normalization processing to obtain a sub-band noise reduction weight and a low-frequency noise reduction weight, and the sub-band noise reduction result and the low-frequency noise reduction result are respectively subjected to weighted summation based on the sub-band noise reduction weight and the low-frequency noise reduction weight to obtain a noise reduction spectrum feature.

7. The cockpit speech noise reduction method according to any one of claims 1-6, characterized in that, After the noise reduction spectrum feature is determined based on the sub-band mask, the low-frequency mask and the full-band spectrum feature, the cabin voice noise reduction method further comprises: The noise reduction spectrum feature is subjected to inverse Fourier transform to obtain a to-be-processed time domain signal, and the to-be-processed time domain signal is subjected to overlap-add processing to obtain a time domain voice signal.

8. A cockpit voice noise reduction device, characterized by, The cabin voice noise reduction device comprises: A signal feature extraction module is configured to acquire an initial noisy voice signal in a cabin, and extract a spectrum feature from the initial noisy voice signal to obtain a full-band spectrum feature; convert the full-band spectrum feature into a sub-band feature through equivalent rectangular bandwidth conversion, and extract a low-frequency feature from the full-band spectrum feature, wherein the low-frequency feature extraction comprises extracting spectrum features lower than a preset low-frequency threshold from the full-band spectrum feature; A spectrum feature noise reduction module is configured to respectively perform interactive enhancement processing on the sub-band feature and the low-frequency feature to obtain a sub-band encoded feature and a low-frequency encoded feature, fuse the sub-band encoded feature and the low-frequency encoded feature to generate a fused spectrum feature, and input the fused spectrum feature into a sub-band feature decoder and a low-frequency feature decoder to obtain a sub-band mask and a low-frequency mask, so as to determine a noise reduction spectrum feature based on the sub-band mask, the low-frequency mask and the full-band spectrum feature.

9. An electronic device, comprising: The device comprises a processor, a memory and a communication bus; the communication bus is configured to connect the processor and the memory; the processor is configured to execute a computer program stored in the memory to implement the cabin voice noise reduction method according to any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The device comprises a processor, a memory and a communication bus; the communication bus is configured to connect the processor and the memory; the processor is configured to execute a computer program stored in the memory to implement the cabin voice noise reduction method according to any one of claims 1-7. The device comprises a processor, a memory and a communication bus; the communication bus is configured to connect the processor and the memory; the processor is configured to execute a computer program stored in the memory to implement the cabin voice noise reduction method according to any one of claims 1-7.

Images