A noise suppression method and system for a microphone

Abstract

The application discloses a noise suppression method and system for a microphone, and belongs to the technical field of audio signal processing. The method comprises the following steps: a main audio processing device acquires an original audio stream to obtain complex spectrum data of a current frame; real-time running state data of the audio processing device is synchronously collected; a task allocation strategy is dynamically generated; a subtask processing result is generated, and data exchange is performed; a complete full-band gain matrix for the current frame is constructed; and a time-domain audio signal after noise reduction is reconstructed through inverse transformation. The noise reduction task is intelligently decomposed and dynamically allocated to the heterogeneous computing units of the main and auxiliary devices through a cooperative scheduling mechanism for cooperative processing, so that the technical effect of realizing high-quality and low-delay real-time voice noise reduction on a resource-limited mobile terminal is achieved, and the problems of processing delay, asynchronous state updating, reduced noise reduction quality and incoherent voice caused by insufficient terminal computing power in the prior art are solved.

Description

Technical Field

[0001] This invention relates to the field of audio signal processing technology, and in particular to a noise suppression method and system for a microphone. Background Technology

[0002] With the widespread adoption of mobile communications, smart wearable devices, and remote work, the demands for high-quality real-time voice communication are increasing, especially the need for clear calls in noisy environments. To improve voice clarity, real-time noise suppression technology is widely used due to its superior performance.

[0003] In existing technologies, noise reduction is typically achieved by deploying complex models containing convolutional and recurrent neural networks on terminal devices to perform end-to-end noise reduction on audio captured by microphones. Such systems heavily rely on the computing power of the terminal devices themselves. However, in practical applications, processing terminals are often low-power devices such as smartwatches and portable voice recorders, with limited hardware computing power. Complex neural network models are prone to insufficient computing power during loading and real-time processing, leading to increased latency in audio frame processing. This prevents the hidden states recording historical information within the neural network from being updated in a timely manner, causing discrepancies between current noise judgments and historical context. Simultaneously, the forced reduction in computational accuracy to maintain real-time performance weakens the extraction and suppression of noise features. Therefore, achieving high-quality, low-latency real-time noise suppression with limited hardware resources has become a critical technical bottleneck that urgently needs to be overcome in this field. Summary of the Invention

[0004] This application provides a noise suppression method and system for microphones, which solves the problems of delayed neural network model processing, asynchronous state updates, and decreased noise reduction quality caused by limited computing power of terminal devices in the prior art, and realizes high-quality, low-latency real-time noise suppression on resource-constrained devices.

[0005] This application provides a noise suppression method for a microphone, comprising: a main audio processing device acquiring an original audio stream, obtaining complex spectrum data of the current frame, and simultaneously separating spectrum amplitude information and phase information;

[0006] The noise suppression task is logically deconstructed, and real-time operating status data of the audio processing equipment is collected simultaneously.

[0007] Based on the real-time running status data and the spectrum amplitude information of the current frame, a task allocation strategy is dynamically generated.

[0008] According to the received task allocation strategy, execute the assigned subtasks, generate subtask processing results, and exchange data.

[0009] Receive and aggregate the subtask processing results to construct a complete full-band gain matrix for the current frame;

[0010] The complete full-band gain matrix is ​​combined with the separated phase information to perform weighted correction on the complex spectrum data, and then reconstructed into a noise-reduced time-domain audio signal through inverse transformation.

[0011] Furthermore, the steps of logically deconstructing the noise suppression task and simultaneously acquiring real-time operating status data of the audio processing equipment include:

[0012] Analyze the data flow graph of the current noise suppression model and define the computationally intensive convolutional coding layer in the noise suppression model as a feature extraction subtask;

[0013] Recurrent neural network layers with temporal memory dependency are defined as context fusion subtasks;

[0014] Define the fully connected decoding layer with fixed parameters as a gain generation subtask and establish the input-output dependency topology between the subtasks;

[0015] Read the memory bandwidth usage and computing load of the main processor and dedicated signal processor in the main audio processing device, and send a status query request to the auxiliary audio processing device;

[0016] In response to the request, the auxiliary audio processing device packages and feeds back the remaining battery percentage of its internal coprocessor, the received signal strength indication of the current wireless channel, and the signal-to-noise ratio to the collaborative scheduling manager.

[0017] The collaborative scheduling manager normalizes the collected real-time running status data to construct a multi-dimensional real-time state tensor.

[0018] Furthermore, the step of dynamically generating a task allocation strategy based on the real-time operating status data and the spectral amplitude information of the current frame includes:

[0019] Read the real-time state tensor, and when it is determined that cross-device collaboration is not required at the current moment and the main audio processing device is in a non-saturated load state, activate the heterogeneous acceleration mode within the device.

[0020] In the heterogeneous acceleration mode within the device, the feature extraction subtask with high computational load and high parallelism is forcibly mapped to the first dedicated signal processor of the main audio processing device for execution, and the convolution operation is accelerated by the vector instruction set of the dedicated hardware.

[0021] Context fusion subtasks, which have extremely high requirements for temporal continuity and are difficult to parallelize, are locked into execution in the main processor of the main audio processing device;

[0022] The gain generation subtask, which has relatively simple computational logic but a high frequency domain dimension, is assigned to the low-power coprocessor unit within the main audio processing device.

[0023] Furthermore, the generated task allocation strategy also includes an alternating frame processing mode for cross-device collaboration, with specific steps including:

[0024] When the computing load of the main audio processing device exceeds the preset congestion threshold and the wireless link quality of the auxiliary audio processing device is stable, the collaborative scheduling manager generates a cross-device alternating frame processing strategy.

[0025] This strategy defines the main audio processing device as responsible for processing audio frames with odd-numbered sequence numbers in the time series, and the auxiliary audio processing device as responsible for processing audio frames with even-numbered sequence numbers.

[0026] After completing the context fusion subtask of odd-numbered frames, the main audio processing device encapsulates them into a synchronization data packet and sends it to the auxiliary audio processing device via a wireless link.

[0027] After receiving the synchronization data packet, the auxiliary audio processing device parses it into the initial historical state required for even-numbered frame processing and inputs it into its locally running context fusion subtask.

[0028] Furthermore, the generated task allocation strategy also includes a frequency band division processing mode for cross-device collaboration, with specific steps including:

[0029] By analyzing the energy distribution characteristics of the current frame's spectral amplitude, a frequency band division processing strategy is generated when steady-state background noise is detected in the low-frequency band and sudden transient impact noise is detected in the high-frequency band.

[0030] This strategy divides the complete full-band spectrum data into low-frequency sub-band sets and high-frequency sub-band sets in the frequency domain;

[0031] The strategy instructs the main audio processing device to focus on processing the low-frequency subband set, performing feature extraction and gain calculation for steady-state noise suppression;

[0032] At the same time, the auxiliary audio processing equipment is instructed to focus on processing the high-frequency subband set and perform rapid attenuation calculations for high-frequency impulse noise;

[0033] The main audio processing device sends the raw high-frequency spectrum data to the auxiliary audio processing device via a wireless link, and receives the local gain matrix for the high-frequency part only from the auxiliary audio processing device.

[0034] Furthermore, the steps of executing the assigned subtasks, generating subtask processing results, and exchanging data include:

[0035] By real-time statistics of the execution time and memory throughput of the current operator;

[0036] If the auxiliary audio processing device detects a sudden increase in packet loss rate of its wireless transmission module or a drop in battery voltage to the low power warning line during the execution of the assigned subtask, the computing unit will immediately suspend the transmission of the current non-critical data and generate an asynchronous abnormal event interrupt signal.

[0037] The interrupt signal carries the location information of the intermediate data breakpoint that has been processed and is sent to the main audio processing device's collaborative scheduling manager with the highest priority.

[0038] Furthermore, the steps of receiving and aggregating the subtask processing results to construct a complete full-band gain matrix for the current frame include:

[0039] When running in alternating frame processing mode, the main audio processing device sets up a jitter buffer queue;

[0040] The integration node extracts the gain matrix from the buffer queue in the natural growth order of the frame number. If the even-numbered frame gain matrix returned by the auxiliary audio processing device arrives later than the odd-numbered frame of the main device, the interpolation compensation logic is started to synthesize a predicted even-numbered frame gain matrix to fill the gap. After the actual data arrives, a smooth correction is performed in the next processing cycle.

[0041] When running in frequency band division processing mode, the integration node performs frequency domain splicing operation, concatenating the low-frequency sub-band gain vector calculated by the master device with the high-frequency sub-band gain vector returned by the auxiliary device on the frequency domain axis;

[0042] At the cascaded boundary frequency point, the spectral discontinuity step effect caused by differences in the processing logic of different devices is eliminated.

[0043] Furthermore, the steps of weighting and correcting the complex spectral data and reconstructing it into a denoised time-domain audio signal through inverse transform include:

[0044] The main audio processing device calls the complex multiplication operator to multiply each real gain value in the fused full-band gain matrix point by point to obtain the corresponding frequency point of the complex spectrum data of the current frame, and obtain the noise-reduced complex spectrum.

[0045] Perform an inverse short-time Fourier transform on the denoised complex spectrum to map the frequency domain data back to the time domain, and obtain the time domain waveform data of the current frame;

[0046] The overlapping addition method is performed to accumulate the time-domain waveform of the current frame with the data of the previous frame in the overlapping area, and the amplitude is corrected according to the normalization coefficient of the window function.

[0047] The resulting continuous time-domain audio signal stream is directly transmitted to the speaker via the I2S interface.

[0048] This application provides a noise suppression system for a microphone, which implements a noise suppression method for a microphone, including:

[0049] Data acquisition module, data synchronization module, strategy generation module, result generation module, matrix construction module, signal noise reduction module;

[0050] The data acquisition module is used by the main audio processing device to acquire the original audio stream, obtain the complex spectrum data of the current frame, and separate the spectrum amplitude information and phase information.

[0051] The data synchronization module is used to logically deconstruct the noise suppression task and synchronously collect real-time operating status data of the audio processing device.

[0052] The strategy generation module is used to dynamically generate a task allocation strategy based on the real-time running status data and the spectrum amplitude information of the current frame.

[0053] The result generation module is used to execute the assigned sub-tasks according to the received task allocation strategy, generate sub-task processing results, and exchange data.

[0054] The matrix construction module is used to receive and aggregate the subtask processing results to construct a complete full-band gain matrix for the current frame;

[0055] The signal noise reduction module is used to combine the complete full-band gain matrix with the separated phase information, perform weighted correction on the complex spectrum data, and reconstruct the noise-reduced time-domain audio signal through inverse transformation.

[0056] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0057] By dynamically allocating noise reduction computation tasks based on the real-time status of the device and utilizing the various computing resources of the main and auxiliary devices, the limitations of insufficient computing power of a single device are overcome, ensuring the real-time performance of audio processing and avoiding task backlog. Furthermore, through heterogeneous acceleration within the device, subtasks with different characteristics, such as feature extraction and context fusion, are precisely assigned to the most suitable dedicated processors for execution, avoiding waiting caused by hardware mismatch and resolving the accumulation of processing latency and inter-frame blocking.

[0058] Furthermore, by leveraging cross-device collaborative modes such as alternating frame processing and frequency band allocation, and combining state synchronization, data interpolation, and edge smoothing mechanisms, this method ensures the temporal coherence and frequency smoothness of the speech context while utilizing auxiliary devices to share the computational load. Ultimately, this method achieves smooth and natural output speech while reducing the load on a single device, eliminating latency, stuttering, and abrupt changes in sound quality, thus improving the quality of real-time voice communication and user experience in resource-constrained environments. Attached Figure Description

[0059] Figure 1 A flowchart illustrating a noise suppression method for a microphone provided in an embodiment of this application;

[0060] Figure 2 This is a schematic diagram of a noise suppression system for a microphone provided in an embodiment of this application. Detailed Implementation

[0061] This application provides a noise suppression method and system for microphones, which solves the problem that low-power terminal devices in the prior art cannot run complex noise reduction neural networks with high quality and low latency due to insufficient computing power. By using a collaborative scheduling manager to dynamically decompose computing tasks according to real-time status and allocate them to heterogeneous computing units inside and outside the device for collaborative execution, the method improves the suppression effect of complex mixed noise and voice fidelity while ensuring real-time performance.

[0062] To better understand the above technical solutions, the following will provide a detailed explanation of the technical solutions in conjunction with the accompanying drawings and specific implementation methods.

[0063] like Figure 1 The diagram shown is a flowchart of a noise suppression method for a microphone provided in an embodiment of this application. The method is applied in a noise suppression system for a microphone and includes the following steps: the main audio processing device acquires the original audio stream through the audio acquisition interface, performs time-frequency conversion processing on it to obtain the complex spectrum data of the current frame, and separates the spectrum amplitude information and phase information.

[0064] The collaborative scheduling manager logically deconstructs the noise suppression task into feature extraction subtasks, context fusion subtasks, and gain generation subtasks, and synchronously collects real-time operating status data of the audio processing devices, which include main audio processing devices and auxiliary audio processing devices.

[0065] The primary audio processing device can be a smartphone, smart headphones, conference host, or other main recording device. The secondary audio processing device can be a smartwatch, wireless voice recorder, second mobile phone, or IoT microphone module paired with the primary device, possessing computing and communication capabilities.

[0066] Based on the real-time running status data and the spectrum amplitude information of the current frame, the collaborative scheduling manager dynamically generates a task allocation strategy, which instructs the above sub-tasks to be allocated to the computing units of the main audio processing device or the auxiliary audio processing device for execution.

[0067] Each computing unit executes the assigned subtasks in parallel or serially according to the received task allocation strategy. During the execution, it generates subtask processing results, including intermediate feature vectors, hidden state vectors, or local gain matrices, and exchanges data across devices or units through communication links.

[0068] The main audio processing device acts as an integration node, receiving and aggregating the subtask processing results from various computing units, performing time-series alignment and numerical fusion on data from different sources, and constructing a complete full-band gain matrix for the current frame.

[0069] The complete full-band gain matrix is ​​combined with the separated phase information to perform weighted correction on the complex spectrum data, and then reconstructed into a noise-reduced time-domain audio signal through inverse transformation.

[0070] Furthermore, the steps of logically deconstructing the noise suppression task and simultaneously acquiring real-time operating status data of the audio processing equipment include:

[0071] The data flow graph of the current noise suppression model is parsed by the collaborative scheduling manager, and the computationally intensive convolutional coding layer in the noise suppression model is defined as a feature extraction subtask.

[0072] Recurrent neural network layers with temporal memory dependency are defined as context fusion subtasks;

[0073] Define the fully connected decoding layer with fixed parameters as a gain generation subtask and establish the input-output dependency topology between the subtasks;

[0074] Based on this, the collaborative scheduling manager starts a status monitoring thread, reads the memory bandwidth usage and computing load of the main processor and dedicated signal processor in the main audio processing device through the hardware abstraction layer interface, and sends a status query request to the auxiliary audio processing device.

[0075] In response to the request, the auxiliary audio processing device packages and feeds back the remaining battery percentage of its internal coprocessor, the received signal strength indication of the current wireless channel, and the signal-to-noise ratio to the collaborative scheduling manager.

[0076] The collaborative scheduling manager normalizes the collected real-time running status data to construct a multi-dimensional real-time state tensor. This state tensor clearly represents the available margin of the current system in terms of computing power and communication bandwidth, providing a quantitative decision-making basis for accurately mapping the three deconstructed subtasks to specific physical computing units and ensuring the matching between task division and physical resource capabilities.

[0077] Furthermore, the step of dynamically generating a task allocation strategy based on the real-time operating status data and the spectral amplitude information of the current frame includes:

[0078] Coordinated Scheduling Manager Read

[0079] The real-time state tensor activates the intra-device heterogeneous acceleration mode when it is determined that cross-device collaboration is not required at the current moment and the main audio processing device is in a non-saturated load state.

[0080] The determination that cross-device collaboration is not required at the current moment and that the main audio processing device is under unsaturated load is based on a comparison of preset thresholds for multiple indicators in the real-time state tensor. For example, the main device's CPU / dedicated processor load rate is below 70%, memory bandwidth usage is below 50%, and the auxiliary device's wireless link signal-to-noise ratio is below 10dB or battery level is below 20%. When both the main device load is light and the auxiliary device is unreliable, it is determined that cross-device collaboration is not required.

[0081] In the heterogeneous acceleration mode within the device, instructions are generated by the scheduler to force the feature extraction subtasks with high computational load and high parallelism to run in the first dedicated signal processor of the main audio processing device, and the convolution operation is accelerated by the vector instruction set of the dedicated hardware.

[0082] The judgment of high computational complexity and high parallelism is mainly based on the computational characteristics of neural network layers. Convolutional layers involve sliding window multiplication and addition operations between the input feature map and multiple convolutional kernels. The computational complexity is proportional to the input size and the number of kernels. Moreover, the operations of each convolutional kernel and each spatial location are independent of each other, making them naturally suitable for simultaneous execution on multiple parallel computing units of GPU / DSP.

[0083] The context fusion subtask, which has extremely high requirements for temporal continuity and is difficult to parallelize, is locked in the main processor of the main audio processing device for execution. The high clock speed of the main processor is used to quickly update the hidden state of the recurrent neural network, preventing timing jitter caused by thread switching.

[0084] The requirement for extremely high temporal continuity and the difficulty in parallelization are primarily due to the structural characteristics of recurrent neural networks (RNN / LSTM / GRU). Their current output and hidden state strictly depend on the hidden state of the previous time step, forming a chain of sequential dependencies. This inherent sequentiality prevents computations at different time steps from being performed simultaneously; forcibly parallelizing them would compromise the logical correctness of state propagation.

[0085] The gain generation subtask, which has relatively simple computational logic but a high frequency domain dimension, is assigned to the low-power coprocessor unit within the main audio processing device.

[0086] The judgment that the computational logic is relatively simple but has a high dimensionality in the frequency domain is mainly based on the operational nature of fully connected layers. Its computation is a fixed matrix-vector multiplication, involving a large number of (frequency points × neurons) multiplication and accumulation operations, but without complex branching or data reorganization, resulting in a regular and simple computational pattern. Its "high dimensionality" is reflected in the typically large length of the input / output vectors (i.e., the number of frequency points).

[0087] The allocation strategy is issued in the form of instruction packets, which contain memory address pointers and execution priority parameters for each subtask. This ensures that within a single device, computing resources of different natures can process neural network layers that are compatible with their architectural features, thereby eliminating single-frame processing latency caused by computing power mismatch at the hardware level.

[0088] Furthermore, the generated task allocation strategy also includes an alternating frame processing mode for cross-device collaboration, with specific steps including:

[0089] When the real-time status tensor of the system shows that the computing load of the main audio processing device exceeds the preset congestion threshold, and the wireless link quality of the auxiliary audio processing device is stable, the collaborative scheduling manager generates a cross-device alternating frame processing strategy.

[0090] Computing load is read in real time through performance interfaces provided by the operating system kernel (such as / proc / stat and / proc / meminfo in Linux) or hardware performance counters (PMU). Specific metrics include: CPU / processor core utilization percentage, cache hit rate at each level, memory controller bandwidth utilization, and task queue length, etc.

[0091] This strategy defines the main audio processing device as responsible for processing audio frames with odd-numbered sequence numbers in the time series, and the auxiliary audio processing device as responsible for processing audio frames with even-numbered sequence numbers.

[0092] To maintain the contextual coherence of the recurrent neural network in the time dimension, a hidden state vector exchange protocol is embedded in the strategy.

[0093] According to the protocol, after completing the context fusion subtask of odd-numbered frames, the main audio processing device does not directly discard the current hidden state vector, but encapsulates it into a synchronization data packet and sends it to the auxiliary audio processing device via a wireless link.

[0094] After receiving the synchronization data packet, the auxiliary audio processing device parses it into the initial historical state required for even-numbered frame processing and inputs it into its locally running context fusion subtask, thereby continuing the feature information of the previous frame.

[0095] Conversely, after the auxiliary audio processing device finishes processing even-numbered frames, it also sends the updated hidden state vector back to the main audio processing device.

[0096] This strategy ensures that, under a distributed parallel computing architecture, although adjacent frames are physically processed by different devices, the memory units of the noise suppression model are always continuously updated logically, thus avoiding speech discontinuity caused by the break in information between frames.

[0097] Furthermore, the generated task allocation strategy also includes a frequency band division processing mode for cross-device collaboration, with specific steps including:

[0098] By analyzing the energy distribution characteristics of the current frame's spectrum amplitude through the collaborative scheduling manager, a frequency band division processing strategy is generated when steady-state background noise is detected in the low-frequency band and sudden transient impact noise is detected in the high-frequency band.

[0099] The main basis for judging the presence of steady-state background noise and sudden transient impulse noise is the time-frequency statistical characteristics of the spectrum. Steady-state noise: Over multiple consecutive frames, the energy values ​​at each frequency point have small variances and change slowly. Transient impulse noise: Within a very short time (1-2 frames), the energy in a specific frequency band (usually high frequency) rises sharply (exceeding several times the standard deviation of the historical mean), and then decays rapidly.

[0100] This strategy divides the complete full-band spectrum data into low-frequency sub-band sets and high-frequency sub-band sets in the frequency domain;

[0101] The strategy instructs the main audio processing device to focus on processing the low-frequency subband set, performing feature extraction and gain calculation for steady-state noise suppression to ensure the clarity of the underlying audio segments;

[0102] Meanwhile, the auxiliary audio processing equipment is instructed to focus on processing the high-frequency subband set, using its idle computing power to run a highly sensitive transient noise detection model and perform rapid attenuation calculations for high-frequency impulse noise;

[0103] The construction process of the transient noise detection model is as follows:

[0104] Using an audio dataset labeled with transient noise, extract the Mel frequency cepstral coefficients (MFCC) or logarithmic spectral features of the frame spectrum as input, train a small neural network (such as a 3-layer fully connected network) or support vector machine (SVM), and output the probability of "the presence of transient noise".

[0105] In this mode, the master and slave devices no longer exchange hidden state vectors, but instead exchange their respective calculated local spectral gain data;

[0106] The main audio processing device sends the raw high-frequency spectrum data to the auxiliary audio processing device via a wireless link, and receives the local gain matrix for the high-frequency part only from the auxiliary audio processing device.

[0107] The original high-frequency spectrum data refers to the complete complex spectrum obtained, which, after undergoing a frequency domain segmentation operation of "dividing into a set of low-frequency sub-bands and a set of high-frequency sub-bands", separates the complex spectrum points (including real and imaginary parts) corresponding to the high-frequency sub-band portion.

[0108] This mode replaces temporal frame-level decomposition with spatial frequency domain decomposition, effectively reducing the computational burden on a single device when processing wideband high sampling rate audio, while applying differentiated suppression strategies for the noise characteristics of different frequency bands.

[0109] Furthermore, the steps of executing the assigned subtasks, generating subtask processing results, and exchanging data include:

[0110] When each computing unit loads and executes the assigned subtask, it uses a built-in performance monitoring counter to count the execution time and memory throughput of the current operator in real time.

[0111] If the auxiliary audio processing device detects a sudden increase in packet loss rate of its wireless transmission module or a drop in battery voltage to the low power warning line during the execution of the assigned subtask, the computing unit will immediately suspend the transmission of the current non-critical data and generate an asynchronous abnormal event interrupt signal.

[0112] Determining a sudden increase in packet loss rate or a drop in battery voltage to the low battery warning threshold for the wireless transmission module primarily relies on real-time monitoring data from the device hardware. The wireless module driver layer calculates the ACK acknowledgment failure rate or CRC error rate within a given period. A sudden increase is defined as a packet loss rate jumping from <1% to >10% within a short period (e.g., 100ms). The power management chip (PMIC) continuously monitors the battery voltage, and triggers a low battery warning when the voltage falls below the safe shutdown threshold set by the device manufacturer (e.g., 3.3V).

[0113] The interrupt signal carries the location information of the intermediate data breakpoint that has been processed and is sent to the main audio processing device's collaborative scheduling manager with the highest priority.

[0114] During normal execution, data exchange between the main and auxiliary devices follows a strict time slot pairing principle. That is, the main audio processing device starts a timeout timer while sending data. If no processing result feedback is received from the auxiliary audio processing device within the preset time window, or if the above-mentioned asynchronous abnormal event interruption signal is received, the flow control mechanism is immediately triggered to temporarily deprive the auxiliary audio processing device of its computing privileges. The remaining unfinished subtasks are forcibly recycled to the local queue of the main audio processing device for downgrade processing. This sacrifices some noise reduction accuracy to ensure the real-time continuity of the audio stream and prevent audio stuttering caused by communication blockage.

[0115] Furthermore, the steps of receiving and aggregating the subtask processing results to construct a complete full-band gain matrix for the current frame include:

[0116] When running in alternating frame processing mode, the main audio processing device sets up a jitter buffer queue with dynamic depth to temporarily store the odd-numbered frame gain matrix that has been processed locally and the even-numbered frame gain matrix received from the auxiliary audio processing device.

[0117] The integration node extracts the gain matrix from the buffer queue in the natural growth order of the frame number. If the even-numbered frame gain matrix returned by the auxiliary audio processing device arrives later than the odd-numbered frame of the main device due to network jitter, the interpolation compensation logic is started. The gain matrix of the adjacent main device frames is weighted and averaged to synthesize a predicted even-numbered frame gain matrix to fill the gap. After the actual data arrives, a smooth correction is performed in the next processing cycle.

[0118] When running in frequency band division processing mode, the integration node performs frequency domain splicing operation, concatenating the low-frequency sub-band gain vector calculated by the master device with the high-frequency sub-band gain vector returned by the auxiliary device on the frequency domain axis;

[0119] At the cascaded boundary frequency points, edge smoothing filtering is performed. Through a transition window function with a preset bandwidth, the gain values ​​on both sides of the boundary are weighted and mixed to eliminate the spectral discontinuity step effect caused by the difference in processing logic of different devices, thereby constructing a smooth and complete full-band gain matrix.

[0120] Furthermore, the steps of weighting and correcting the complex spectral data and reconstructing it into a denoised time-domain audio signal through inverse transform include:

[0121] The main audio processing device calls the complex multiplication operator to multiply each real gain value in the fused full-band gain matrix point by point to obtain the corresponding frequency point of the complex spectrum data of the current frame, and obtain the noise-reduced complex spectrum.

[0122] During this process, the phase information of the original signal is kept unmodified in order to preserve the spatial sense and timbre characteristics of the speech to the greatest extent.

[0123] Perform an inverse short-time Fourier transform on the denoised complex spectrum to map the frequency domain data back to the time domain, and obtain the time domain waveform data of the current frame;

[0124] Because windowing is introduced during frame processing, there is an overlapping area between adjacent frames. Therefore, it is necessary to perform an overlap addition method to accumulate the time domain waveform of the current frame with the data of the previous frame in the overlapping area, and perform amplitude correction according to the normalization coefficient of the window function.

[0125] The final generated continuous time-domain audio signal stream is directly transmitted to the speaker through the I2S interface. This step ensures that after distributed and complex frequency domain processing and splicing, the final restored sound signal still maintains the continuity and smoothness of the waveform in the time domain, eliminating traces of artificial synthesis and abrupt changes in the listening experience.

[0126] like Figure 2 The diagram shown is a structural schematic of a noise suppression system for a microphone provided in an embodiment of this application. The noise suppression system for a microphone provided in an embodiment of this application includes: a data acquisition module, a data synchronization module, a strategy generation module, a result generation module, a matrix construction module, and a signal noise reduction module.

[0127] The data acquisition module is used by the main audio processing device to acquire the original audio stream, obtain the complex spectrum data of the current frame, and separate the spectrum amplitude information and phase information.

[0128] The data synchronization module is used to logically deconstruct the noise suppression task and synchronously collect real-time operating status data of the audio processing device.

[0129] The strategy generation module is used to dynamically generate a task allocation strategy based on the real-time running status data and the spectrum amplitude information of the current frame.

[0130] The result generation module is used to execute the assigned sub-tasks according to the received task allocation strategy, generate sub-task processing results, and exchange data.

[0131] The matrix construction module is used to receive and aggregate the subtask processing results to construct a complete full-band gain matrix for the current frame;

[0132] The signal noise reduction module is used to combine the complete full-band gain matrix with the separated phase information, perform weighted correction on the complex spectrum data, and reconstruct the noise-reduced time-domain audio signal through inverse transformation.

[0133] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0134] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented using software, the above embodiments can be implemented, in whole or in part, in the form of a computer program product.

[0135] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0136] In addition, the functional modules in the various embodiments of this application can be integrated into one processing module, or each module can exist physically separately, or two or more modules can be integrated into one module.

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0138] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A noise suppression method for a microphone, characterized in that, Includes the following steps: The main audio processing device acquires the raw audio stream, obtains the complex spectrum data of the current frame, and separates the spectrum amplitude information and phase information. The noise suppression task is logically deconstructed, and real-time operating status data of the audio processing equipment is collected simultaneously. The audio processing equipment includes the main audio processing equipment and the auxiliary audio processing equipment. Based on the real-time running status data and the spectrum amplitude information of the current frame, a task allocation strategy is dynamically generated. According to the received task allocation strategy, execute the assigned subtasks, generate subtask processing results, and exchange data. Receive and aggregate the subtask processing results to construct a complete full-band gain matrix for the current frame; The complete full-band gain matrix is ​​combined with the separated phase information to perform weighted correction on the complex spectrum data, and then reconstructed into a noise-reduced time-domain audio signal through inverse transformation.

2. The noise suppression method for a microphone as described in claim 1, characterized in that, The steps for logically deconstructing the noise suppression task and simultaneously acquiring real-time operational status data of the audio processing equipment include: Analyze the data flow graph of the current noise suppression model and define the computationally intensive convolutional coding layer in the noise suppression model as a feature extraction subtask; Recurrent neural network layers with temporal memory dependency are defined as context fusion subtasks; Define the fully connected decoding layer with fixed parameters as a gain generation subtask and establish the input-output dependency topology between the subtasks; Read the memory bandwidth usage and computing load of the main processor and dedicated signal processor in the main audio processing device, and send a status query request to the auxiliary audio processing device; In response to the request, the auxiliary audio processing device packages and feeds back the remaining battery percentage of its internal coprocessor, the received signal strength indication of the current wireless channel, and the signal-to-noise ratio to the collaborative scheduling manager. The collaborative scheduling manager normalizes the collected load data, power data, and communication quality data to construct a multi-dimensional real-time state tensor.

3. The noise suppression method for a microphone as described in claim 1, characterized in that, The steps for dynamically generating a task allocation strategy based on the real-time operating status data and the spectral amplitude information of the current frame include: Read the real-time state tensor, and when it is determined that cross-device collaboration is not required at the current moment and the main audio processing device is in a non-saturated load state, activate the heterogeneous acceleration mode within the device. In the heterogeneous acceleration mode within the device, the feature extraction subtask with high computational load and high parallelism is forcibly mapped to the first dedicated signal processor of the main audio processing device for execution, and the convolution operation is accelerated by the vector instruction set of the dedicated hardware. Context fusion subtasks, which have extremely high requirements for temporal continuity and are difficult to parallelize, are locked into execution in the main processor of the main audio processing device; The gain generation subtask, which has relatively simple computational logic but a high frequency domain dimension, is assigned to the low-power coprocessor unit within the main audio processing device.

4. The noise suppression method for a microphone as described in claim 3, characterized in that, The generated task allocation strategy also includes an alternating frame processing mode for cross-device collaboration, with specific steps including: When the computing load of the main audio processing device exceeds the preset congestion threshold and the wireless link quality of the auxiliary audio processing device is stable, the collaborative scheduling manager generates a cross-device alternating frame processing strategy. This strategy defines the main audio processing device as responsible for processing audio frames with odd-numbered sequence numbers in the time series, and the auxiliary audio processing device as responsible for processing audio frames with even-numbered sequence numbers. After completing the context fusion subtask of odd-numbered frames, the main audio processing device encapsulates them into a synchronization data packet and sends it to the auxiliary audio processing device via a wireless link. After receiving the synchronization data packet, the auxiliary audio processing device parses it into the initial historical state required for even-numbered frame processing and inputs it into its locally running context fusion subtask.

5. The noise suppression method for a microphone as described in claim 3, characterized in that, The generated task allocation strategy also includes a frequency band division processing mode for cross-device collaboration, with specific steps including: By analyzing the energy distribution characteristics of the current frame's spectral amplitude, a frequency band division processing strategy is generated when steady-state background noise is detected in the low-frequency band and sudden transient impact noise is detected in the high-frequency band. This strategy divides the complete full-band spectrum data into low-frequency sub-band sets and high-frequency sub-band sets in the frequency domain; The strategy instructs the main audio processing device to focus on processing the low-frequency subband set, performing feature extraction and gain calculation for steady-state noise suppression; At the same time, the auxiliary audio processing equipment is instructed to focus on processing the high-frequency subband set and perform rapid attenuation calculations for high-frequency impulse noise; The main audio processing device sends the raw high-frequency spectrum data to the auxiliary audio processing device via a wireless link, and receives the local gain matrix for the high-frequency part only from the auxiliary audio processing device.

6. The noise suppression method for a microphone as described in claim 1, characterized in that, The steps of executing the assigned subtasks, generating subtask processing results, and exchanging data include: By real-time statistics of the execution time and memory throughput of the current operator; If the auxiliary audio processing device detects a sudden increase in packet loss rate of its wireless transmission module or a drop in battery voltage to the low power warning line during the execution of the assigned subtask, the computing unit will immediately suspend the transmission of the current non-critical data and generate an asynchronous abnormal event interrupt signal. The interrupt signal carries the location information of the intermediate data breakpoint that has been processed and is sent to the main audio processing device's collaborative scheduling manager with the highest priority.

7. The noise suppression method for a microphone as described in claim 1, characterized in that, The steps for receiving and aggregating the subtask processing results to construct a complete full-band gain matrix for the current frame include: When running in alternating frame processing mode, the main audio processing device sets up a jitter buffer queue; The integration node extracts the gain matrix from the buffer queue in the natural growth order of the frame number. If the even-numbered frame gain matrix returned by the auxiliary audio processing device arrives later than the odd-numbered frame of the main device, the interpolation compensation logic is started to synthesize a predicted even-numbered frame gain matrix to fill the gap. After the actual data arrives, a smooth correction is performed in the next processing cycle. When running in frequency band division processing mode, the integration node performs frequency domain splicing operation, concatenating the low-frequency sub-band gain vector calculated by the master device with the high-frequency sub-band gain vector returned by the auxiliary device on the frequency domain axis; At the cascaded boundary frequency point, the spectral discontinuity step effect caused by differences in the processing logic of different devices is eliminated.

8. The noise suppression method for a microphone as described in claim 1, characterized in that, The steps of weighting and correcting complex spectral data and reconstructing it into a denoised time-domain audio signal via inverse transform include: The main audio processing device calls the complex multiplication operator to multiply each real gain value in the fused full-band gain matrix point by point to obtain the corresponding frequency point of the complex spectrum data of the current frame, and obtain the noise-reduced complex spectrum. Perform an inverse short-time Fourier transform on the denoised complex spectrum to map the frequency domain data back to the time domain, and obtain the time domain waveform data of the current frame; The overlapping addition method is performed to accumulate the time-domain waveform of the current frame with the data of the previous frame in the overlapping area, and the amplitude is corrected according to the normalization coefficient of the window function. The resulting continuous time-domain audio signal stream is directly transmitted to the speaker via the I2S interface.

9. A noise suppression system for a microphone, used to implement the noise suppression method for a microphone according to any one of claims 1-8, characterized in that, include: Data acquisition module, data synchronization module, strategy generation module, result generation module, matrix construction module, signal noise reduction module; The data acquisition module is used by the main audio processing device to acquire the original audio stream, obtain the complex spectrum data of the current frame, and separate the spectrum amplitude information and phase information. The data synchronization module is used to logically deconstruct the noise suppression task and synchronously collect real-time operating status data of the audio processing device, which includes a main audio processing device and an auxiliary audio processing device. The strategy generation module is used to dynamically generate a task allocation strategy based on the real-time running status data and the spectrum amplitude information of the current frame. The result generation module is used to execute the assigned sub-tasks according to the received task allocation strategy, generate sub-task processing results, and exchange data. The matrix construction module is used to receive and aggregate the subtask processing results to construct a complete full-band gain matrix for the current frame; The signal noise reduction module is used to combine the complete full-band gain matrix with the separated phase information, perform weighted correction on the complex spectrum data, and reconstruct the noise-reduced time-domain audio signal through inverse transformation.

Images