Intelligent microphone noise suppression method and system

Through the pseudo noise annotation and integration of the labelless audio sample set and the target noise recognition network, the problem of noise suppression in the prior art is difficult to adapt to complex environments and high training costs, and efficient and accurate noise recognition network training is achieved.

CN118711606BActive Publication Date: 2024-11-19SHENZHEN BBS ELECTRONICS CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411185370.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-27
Publication Date
2024-11-19
Estimated Expiration
2044-08-27

AI Technical Summary

Technical Problem

The prior art relies on complex signal processing algorithms or manually designed filters in noise suppression, making it difficult to adapt to complex and variable noise environments. Training high-performance noise recognition networks requires a large amount of labeled audio data, which is expensive and inefficient.

Method used

By obtaining the label-free audio sample set, the target noise identification network is used to generate and integrate pseudo noise annotation results, and automatically label pseudo noise annotation results for label-free audio samples, omitting the label annotation process, reducing training consumption, and improving the network Adjust the optimization effect.

Benefits of technology

High-quality training of noise recognition networks is achieved in the absence of real tags, reducing training costs and time, and improving the accuracy of noise recognition and the adaptability of networks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118711606B_ABST
    Figure CN118711606B_ABST
Patent Text Reader

Abstract

The present application provides a method and system for suppressing noise in an intelligent microphone. When training a noise recognition network, unlabeled audio samples are loaded into a neural network for noise recognition. Pseudo-noise labeling results are automatically labeled for unlabeled audio samples based on the recognition results. Unlabeled audio samples and pseudo-noise labeling results are used as debugging samples, label annotations are omitted, and training samples containing annotation information are obtained at the same time to train the neural network, thereby reducing the training cost while ensuring the quality of network training. In addition, when constructing pseudo-noise labeling results, the original noise labeling results output by different noise recognition components in the neural network for the same audio sample are paired. The paired original noise labeling results correspond to the same noise type and are closer. After integrating the paired noise labeling results corresponding to the same original noise labeling result, accurate pseudo-noise labeling results can be obtained, which helps increase the effect of network calibration and optimization.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of data processing, and in particular, to a method and system for suppressing noise of an intelligent microphone. Background Art

[0002] With the continuous development of audio processing technology, smart microphones have been widely used in many fields such as teleconferencing, speech recognition, and audio recording. In these application scenarios, noise suppression technology has become a key factor in improving audio quality and enhancing user experience. However, traditional noise suppression methods often rely on complex signal processing algorithms or manually designed filters. These methods are powerless in dealing with complex and changeable noise environments and are difficult to adapt to the noise characteristics in different scenarios. In recent years, with the rise of machine learning, especially deep learning technology, noise recognition and suppression methods based on neural networks have gradually become a research hotspot. Neural networks, with their powerful feature extraction and pattern recognition capabilities, have shown great potential in the field of noise recognition. However, training a high-performance noise recognition network requires a large amount of labeled audio data as support, and obtaining these labeled data usually requires a lot of manpower and material resources, which is costly and inefficient. Summary of the invention

[0003] The purpose of the present invention is to provide a method and system for suppressing noise of an intelligent microphone. This application is implemented as follows:

[0004] In a first aspect, the present application provides a method for intelligent microphone noise suppression, the method comprising: obtaining microphone audio to be subjected to noise suppression; loading the microphone audio to be subjected to noise suppression into a target noise recognition network, and obtaining an audio noise annotation result set corresponding to the microphone audio to be subjected to noise suppression; performing noise suppression on the microphone audio to be subjected to noise suppression in combination with the audio noise annotation result set; wherein the target noise recognition network is calibrated by the following steps: obtaining a first audio sample set, the first audio sample set being an unlabeled audio sample set; loading each first audio sample in the first audio sample set into a noise recognition network to be calibrated, and obtaining an original noise annotation result set corresponding to each first audio sample; the original noise annotation result set corresponding to the first audio sample includes each noise recognition component in the noise recognition network to be calibrated The output original noise labeling result and the noise type corresponding to the original noise labeling result; for the same original noise labeling result set, according to the noise labeling result commonality coefficient between the original noise labeling results corresponding to the same noise type, the original noise labeling result output by the current noise recognition component is paired with the original noise labeling results output by each of the other noise recognition components to obtain the paired noise labeling result corresponding to the original noise labeling result output by the current noise recognition component; the original noise labeling result and the corresponding paired noise labeling results are integrated to obtain a pseudo noise labeling result, and according to the pseudo noise labeling results corresponding to the same first audio sample, a pseudo noise labeling result set corresponding to the first audio sample is obtained; according to the first audio sample set and the corresponding pseudo noise labeling result sets, the noise recognition network to be adjusted is adjusted and optimized to obtain a target noise recognition network.

[0005] In a second aspect, the present application provides a computer system comprising: one or more processors; a memory; one or more computer programs; wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, the method as described above is implemented.

[0006] The beneficial effects of the present application include at least: the intelligent microphone noise suppression method and system provided by the present application, when training a noise recognition network, first obtain a first audio sample set, the first audio sample set is an unlabeled audio sample set, each first audio sample in the first audio sample set is loaded into the noise recognition network to be adjusted, and the original noise labeling result set corresponding to each first audio sample is obtained, the original noise labeling result set corresponding to the first audio sample includes the original noise labeling results output by each noise recognition component in the noise recognition network to be adjusted and the noise type corresponding to the original noise labeling result, for the same original noise labeling result set, according to the original noise labeling corresponding to the same noise type The commonality coefficient of the noise labeling results between the results is calculated, and the original noise labeling results output by the current noise recognition component are paired with the original noise labeling results output by the remaining noise recognition components to obtain the paired noise labeling results corresponding to the original noise labeling results output by the current noise recognition component. The original noise labeling results and the corresponding paired noise labeling results are integrated to obtain the pseudo noise labeling results. According to the pseudo noise labeling results corresponding to the same first audio sample, a pseudo noise labeling result set corresponding to the first audio sample is obtained. According to the first audio sample set and the corresponding pseudo noise labeling result sets, the noise recognition network to be adjusted is adjusted and optimized to obtain the target noise recognition network. Based on this, the unlabeled audio samples are loaded into the neural network for noise recognition, and the pseudo noise labeling results are automatically annotated for the unlabeled audio samples according to the original noise labeling result set obtained by recognition. The unlabeled audio samples and the pseudo noise labeling results are used as debugging samples of the neural network, omitting the label annotation process, and obtaining the training samples containing annotation information for neural network training at the same time, while ensuring the quality of network training and reducing the training consumption. In addition, in the process of constructing the pseudo-noise labeling results, the original noise labeling results output by different noise recognition components in the neural network for the same audio sample are paired. The paired original noise labeling results correspond to the same noise type and are closer. After integrating the paired noise labeling results corresponding to the same original noise labeling result, accurate pseudo-noise labeling results can be obtained, which helps increase the effect of network tuning and optimization. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 This is a flow chart of an intelligent microphone noise suppression method provided in an embodiment of the present application.

[0008] Figure 2 It is a schematic diagram of the composition of a computer system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0009] The execution subject of the intelligent microphone noise suppression method in the embodiment of the present application is a computer system, including but not limited to a server, a personal computer, a laptop computer, a tablet computer, etc. The computer system can be operated alone to implement the present application, or it can be connected to a network and implement the present application through interactive operations with other computer systems in the network. Among them, the network where the computer system is located includes but is not limited to the Internet, a wide area network, a metropolitan area network, a local area network, a VPN network, etc.

[0010] The present application embodiment provides a method for suppressing noise of an intelligent microphone, which is applied to a computer system, such as Figure 1 As shown, the method includes two steps of neural network calibration and application, wherein the calibration step involves steps S10 to S50, and the application process involves steps S100 to S300. Specifically, in the application stage, it includes:

[0011] Step S100: Acquire microphone audio to be subjected to noise suppression.

[0012] Specifically, the computer system can obtain the microphone audio to be subjected to noise suppression in a variety of ways. For example, in a conference scenario, smart microphones are arranged in each corner of the conference room. These microphones are not only used to capture the speeches of the participants, but also responsible for collecting environmental noise. When the meeting starts, the computer system starts to automatically monitor and receive real-time audio streams from these smart microphones. These audio streams contain the content of the meeting, the discussion sounds of the participants, and possible background noise, such as the sound of air conditioning, traffic sounds outside the window, etc. To illustrate more specifically, a conference room environment can be imagined, in which four smart microphones are configured, located in the four corners of the room. At the beginning of the meeting, the speeches of the participants are converted into digital audio signals through the microphones and transmitted to the computer system via wired or wireless means. The computer system receives these audio data in real time, and the audio stream of each microphone is processed as an independent audio channel. After receiving the audio data, the computer system performs preliminary preprocessing on it, such as sampling rate conversion, format unification, etc., to ensure that all audio data meet the requirements of the subsequent processing module. For example, if the original sampling rate of the microphone is 48kHz, and the subsequent processing module requires 16kHz, the computer system will downsample to 16kHz. In addition, if the audio data contains multiple channels (such as the four microphone channels in the above conference scenario), the computer system will merge or separate these channels as needed. For example, it can choose to merge the four channels into a stereo channel so that they can be analyzed as a whole in subsequent processing; or, it can keep the channels separate so that the audio captured by each microphone can be analyzed independently, which is particularly useful when locating the source of noise.

[0013] Step S200: Loading the microphone audio to be subjected to noise suppression into a target noise recognition network to obtain an audio noise annotation result set corresponding to the microphone audio to be subjected to noise suppression.

[0014] In actual scenarios such as meetings, speeches, and remote calls, the computer system obtains the microphone audio to be noise suppressed through step S100. These audios may contain clear human voice signals and various types of background noise, such as environmental noise, computer system interference, etc. Next, in step S200, the computer system loads these raw audio data into the trained target noise recognition network.

[0015] The target noise recognition network is a model based on deep learning. It may be a combination of a convolutional neural network (CNN), a recurrent neural network (RNN) or its variants (such as LSTM, GRU), which is specially designed to process audio signals and identify the noise components therein. This network has learned the characteristic representations of various noise signals during the training process, so it can accurately distinguish noise from complex audio. After loading the audio data into the network, the data is converted into a format that the network can process, such as a time-frequency graph (spectrogram). Then, these feature maps are sent to the input layer of the network. After being processed step by step through each layer of the network, the audio noise annotation results are finally output.

[0016] The specific form of the annotation results can be flexibly designed according to application requirements. For example, in one implementation, the network may output a binary sequence of the same length as the input audio, where "1" indicates that the audio signal at that time point is identified as noise, and "0" indicates non-noise. In another implementation, the network may directly draw an annotation box on the original audio waveform or highlight the noise area, so that the user can intuitively see which parts are noise.

[0017] Step S300: performing noise suppression on the microphone audio to be subjected to noise suppression in combination with the audio noise annotation result set.

[0018] Specifically, the computer system analyzes the audio noise annotation result set, which indicates in detail which parts of the audio are noise, which parts are human voices or other important information. Based on these annotations, the computer system can adopt a variety of strategies to suppress noise.

[0019] One strategy is to use spectral subtraction, where the computer system estimates the power spectral density of the background noise and then subtracts the estimated value from the power spectrum of the original audio signal to achieve the purpose of suppressing the noise. Since step S200 has provided an accurate labeling of the noise, the computer system can estimate the background noise more accurately, thereby improving the effectiveness of spectral subtraction.

[0020] Another strategy is to restore the noise-contaminated signal based on the Wiener Filtering method under the minimum mean square error criterion. In the intelligent microphone noise suppression scenario, the computer system can dynamically adjust the parameters of the Wiener filter based on the noise labeling results to better adapt to different types of noise and signal characteristics.

[0021] In the embodiment of the present application, the target noise recognition network is calibrated by the following steps:

[0022] Step S10: Acquire a first audio sample set, where the first audio sample set is an unlabeled audio sample set.

[0023] Specifically, in the embodiment of the present application, the computer system obtains a large amount of audio data from various possible sources. These data may come from daily meeting recordings, telephone calls, public speeches, etc. These data do not need to be labeled in any form when collected, that is, they are "unlabeled". This is because the purpose of step S10 is to use these unlabeled audio data to automatically generate pseudo labels through subsequent calibration steps, thereby avoiding expensive and time-consuming manual labeling processes.

[0024] For example, a computer system can batch download all meeting recording files from the company's internal conference system for the past week. These files contain various meeting contents, as well as various background noises, such as keyboard tapping, paper shuffling, occasional conversations, etc. These recording files are considered part of the first audio sample set. In order to build a richer and more diverse first audio sample set, the computer system may also collect publicly available audio datasets from the Internet, such as Freesound, AudioSet, etc. These datasets contain tens of thousands of audio clips, covering a variety of sound types from natural environmental sounds to human activity sounds, and are very suitable for training and improving noise recognition networks.

[0025] After collecting enough audio data, the computer system preprocesses them to ensure that they meet the requirements of the subsequent tuning steps. The preprocessing steps may include format conversion, sampling rate unification, noise reduction, etc. to ensure that all audio samples have consistent format and quality before being input into the noise recognition network. Although the first set of audio samples is unlabeled, it does not mean that they are of no value for the tuning of the noise recognition network. On the contrary, through pseudo-label generation and iterative tuning in subsequent steps, these unlabeled audio samples will become an important resource for training and optimizing the noise recognition network.

[0026] Step S20: Load each first audio sample in the first audio sample set into the noise recognition network to be adjusted, and obtain the original noise labeling result set corresponding to each of the first audio samples; the original noise labeling result set corresponding to the first audio sample includes the original noise labeling results output by each noise recognition component in the noise recognition network to be adjusted and the noise types corresponding to the original noise labeling results.

[0027] Specifically, the noise recognition network to be tuned is a complex system that includes multiple noise recognition components, which may be based on different machine learning algorithms or neural network architectures, each responsible for identifying different types of noise. When the audio files in the first audio sample set are input into the network one by one, each noise recognition component will analyze it and try to mark the noise part in the audio and its corresponding noise type.

[0028] For example, suppose the noise recognition network to be tuned contains three noise recognition components: one for environmental noise (such as wind and rain), one for mechanical noise (such as air conditioner and printer), and one for human voice interference (such as background conversation). When a conference recording in the first audio sample set is loaded into the network, the three components will analyze the recording separately and output a copy of the original noise labeling result.

[0029] The raw noise labeling result may exist in many forms, but here we can assume that it is presented in the form of an array or vector, which contains the noise identification results for each time point in the audio. For example, the component for environmental noise may output a Boolean array with a length of the number of audio sample timestamps, and each element in the array indicates whether there is environmental noise at the corresponding time point (1 for existence, 0 for non-existence). Similarly, the other two components will also output noise labeling results in their respective fields. In addition to Boolean values, the raw noise labeling results may also contain information about the type of noise. For example, if a time point is marked as noise, the labeling result may also include an index or label indicating the type of noise. In this way, the computer system can know what type of noise each time point marked as noise belongs to.

[0030] By combining the outputs of all these components, we get the original noise annotation result set corresponding to the audio sample. This result set not only contains information about which parts of the audio are noise, but also further points out the specific types of these noises. This provides valuable data support for pseudo-label generation and network tuning in subsequent steps.

[0031] Step S30: For the same set of original noise labeling results, according to the noise labeling result commonality coefficient between the original noise labeling results corresponding to the same noise type, the original noise labeling results output by the current noise recognition component are paired with the original noise labeling results output by each of the remaining noise recognition components to obtain paired noise labeling results corresponding to the original noise labeling results output by the current noise recognition component.

[0032] Specifically, in an embodiment of the present application, when an audio file in the first audio sample set is loaded into the noise recognition network to be adjusted, multiple noise recognition components in the network will analyze the audio file respectively and output their respective original noise annotation result sets. These result sets contain information about which parts of the audio are noise and the types of noise. In step S30, the computer system focuses on the same original noise annotation result set, that is, the output results of multiple noise recognition components for the same audio sample. Then, for each type of noise, the computer system compares the recognition results of different components for the type of noise. In order to quantify the similarity between these recognition results, the computer system calculates the "noise annotation result commonality coefficient" between them. This commonality coefficient can be calculated in many ways, but the core idea is to evaluate the degree of overlap, shape similarity, or consistency response of two recognition results in space or time. For example, if two components both recognize a continuous noise in the audio, and the position and length of this noise on the time axis are roughly the same, then the commonality coefficient of the two recognition results will be higher.

[0033] Based on this commonality coefficient, the computer system matches the original noise labeling results output by the noise recognition component currently being processed (referred to as the "current component") with the results output by all other components in the network. The basis for pairing is the level of the commonality coefficient, that is, looking for the recognition results of other components that are most similar to the recognition results of the current component (the highest commonality coefficient).

[0034] After the pairing is completed, for each original noise labeling result output by the current component, the computer system will find a paired noise labeling result from other components. These paired results will serve as a supplement or verification of the output results of the current component, and together form a richer and more accurate noise labeling set. This set is called a paired noise labeling result.

[0035] Through the processing of step S30, the computer system not only improves the recognition accuracy of a single noise recognition component, but also promotes the collaborative work between different components, so that the noise recognition capability of the entire noise recognition network is significantly improved.

[0036] Step S40: Integrate the original noise labeling result and the corresponding paired noise labeling results to obtain a pseudo noise labeling result, and obtain a pseudo noise labeling result set corresponding to the first audio sample according to the pseudo noise labeling results corresponding to the same first audio sample.

[0037] In the embodiment of the present application, the computer system has obtained the original noise labeling result set of each first audio sample and the pairing information between these results through the previous steps. These original labeling results are independently generated by different components in the noise recognition network to be adjusted, and each component may identify noise based on its specific algorithm or neural network architecture. However, due to the differences and limitations between components, these original labeling results may be inconsistent or not completely accurate.

[0038] To overcome these problems, step S40 introduces an integration mechanism that aims to generate more reliable and consistent pseudo-noise annotation results by comprehensively considering the annotation opinions of multiple components. This integration process can adopt a variety of strategies. A simple and intuitive method is to calculate the coordinate mean of all paired noise annotation results as the coordinate of the pseudo-noise annotation result. Assuming that each noise annotation result can be represented as an interval segment on a timeline (start timestamp, end timestamp), then for each original noise annotation result, the computer system can first collect all the annotation results paired with it, and then calculate the mean of the start and end timestamps of these annotation results on the timeline to obtain a comprehensive pseudo-noise annotation interval.

[0039] Another more sophisticated method is to use a weighted summation approach. In this method, each paired noise annotation result is assigned a weight, which can be determined based on the credibility or importance of the annotation result. For example, if a component performs well in identifying a specific type of noise, then the annotation result of its output can be given a higher weight. When weighted summation is performed, the computer system multiplies the coordinates (or feature vectors) of each paired annotation result by its corresponding weight, and then adds all the weighted results. The final sum is the coordinates (or feature vectors) of the pseudo-noise annotation result.

[0040] On the basis of weighted summation, if each noise recognition component can output the probability of the annotation result (i.e., how likely the annotation result is correct), then the probability can be used as a direct basis for the weight. Specifically, the computer system multiplies the coordinates of each paired annotation result by its corresponding probability and then sums them. This method can more accurately reflect the credibility of different annotation results, thereby generating more accurate pseudo-noise annotations.

[0041] In addition to the above basic methods, more advanced strategies can also be used for integration, such as optimization algorithms based on graph theory, ensemble learning methods in machine learning, etc. These methods can consider more complex factors, such as the spatial relationship between annotation results, temporal continuity, and mutual influence between components, thereby generating higher quality pseudo-noise annotation results.

[0042] During the integration process, the computer system will also summarize all the pseudo noise labeling results of each first audio sample to form a complete pseudo noise labeling result set. This result set will be used as part of the training data for subsequent tuning and optimization of the noise recognition network.

[0043] Through the integration process of step S40, the computer system not only solves the inconsistency problem that may exist between the original noise labeling results, but also generates more accurate and reliable pseudo noise labeling results by comprehensively considering the opinions of multiple components. These pseudo labeling results, as pseudo labels, provide valuable training information for unlabeled audio samples, so that the noise recognition network can self-optimize and improve in the absence of real labels.

[0044] Step S50: According to the first audio sample set and the corresponding pseudo-noise labeling result sets, the noise recognition network to be calibrated is optimized to obtain a target noise recognition network.

[0045] In step S50, the computer system uses the first audio sample set and its corresponding pseudo-noise labeling result set to adjust and improve the noise recognition network to be calibrated through a series of complex optimization algorithms and training processes, thereby obtaining a target noise recognition network.

[0046] Specifically, in the embodiment of the present application, the computer system obtains a large number of unlabeled audio samples (first audio sample set) and the pseudo noise annotation result set corresponding to these samples through the previous steps. Although these pseudo annotation results are not accurately annotated by humans, they have become relatively reliable by integrating the output results of multiple noise recognition components, and can provide valuable information for network calibration.

[0047] Next, the computer system will start the tuning and optimization process. This process usually includes the following key steps: Define the cost function: The cost function is a function that measures the difference between the network's prediction results and the true (or pseudo) labeling results. In the absence of true labels, the computer system will use the pseudo-noise labeling results as a supervisory signal to define a suitable cost function to evaluate the network performance. For example, the cross-entropy cost function (Cross-Entropy Loss) can be used to measure the difference between the noise labeling probability distribution output by the network and the pseudo-labeling results. Backpropagation: After determining the cost function, the computer system will use the backpropagation algorithm to update the weights and bias parameters of the network. The backpropagation algorithm minimizes the cost function by calculating the gradient of the cost function with respect to each parameter and updating the parameter value in the opposite direction of the gradient. This process will be iterated many times until a certain stopping condition is met (such as the loss value is lower than a certain threshold, the number of iterations reaches the upper limit, etc.). Parameter update: In each iteration, the computer system updates the parameters of the network according to the gradient obtained by backpropagation. The specific method of parameter update depends on the selected optimization algorithm. Common algorithms include stochastic gradient descent (SGD), Adam, etc. Taking the Adam algorithm as an example, it combines the advantages of the momentum method and the RMSprop algorithm, and dynamically adjusts the learning rate of each parameter by calculating the first-order moment estimate and the second-order moment estimate of the gradient. Network evaluation: During the tuning process, the computer system regularly evaluates the performance of the tuned network on the validation set to monitor the tuning effect and prevent overfitting. Although pseudo-annotated results are used for evaluation at this stage, it can still provide useful feedback on network performance. Tuning stop: When the tuning process meets the stopping conditions (such as the performance on the validation set is no longer significantly improved, the number of iterations reaches the preset upper limit, etc.), the tuning process will terminate, and the network obtained at this time is the target noise recognition network.

[0048] As an implementation mode, the method further includes a pre-training process of a noise recognition network, which specifically includes the following steps:

[0049] Step S1: Acquire a second audio sample set, where the second audio sample set is a labeled audio sample set.

[0050] In step S1, the computer system obtains a second audio sample set. These sample sets not only provide necessary data support for the initial training of the network, but also ensure that the network can generate high-quality pseudo labels in the subsequent unsupervised learning stage. This application does not limit the channels and methods for audio acquisition, but it needs to be carried out under the permission of laws and regulations.

[0051] After obtaining the second audio sample set, the computer system preprocesses it to ensure the consistency of the data format, the uniformity of the sampling rate, and the accuracy of the noise labeling. Then, this data will be used in the pre-training phase of the noise recognition network, providing the network with the opportunity for supervised learning, thereby initially establishing the ability to recognize noise.

[0052] Step S2: Load each second audio sample in the second audio sample set into the noise recognition network to be pre-trained, and obtain the inference noise labeling result set corresponding to each second audio sample; the inference noise labeling result set includes the inference noise labeling result subset output by each noise recognition component in the noise recognition network to be pre-trained, and the inference noise labeling result subset includes the inference noise labeling result output by the same noise recognition component, and the noise type and probability corresponding to the inference noise labeling result.

[0053] In step S2, the computer system loads each audio sample in the second audio sample set into the noise recognition network to be pre-trained one by one. This network is a complex machine learning model, usually composed of multiple specially designed noise recognition components, each component is responsible for recognizing different types of noise. For example, one component may be specifically used to detect environmental noise (such as wind and rain), while another component focuses on recognizing noise generated by human activities (such as footsteps and door closing).

[0054] When an audio sample is input into the network, each noise recognition component analyzes and processes it independently. These components use their own internal algorithms or neural network structures to extract features from the audio signal and infer the existence, location, type and corresponding probability of noise based on these features.

[0055] Specifically, for each second audio sample, the noise recognition network to be pre-trained will output a reasoning noise labeling result set. This result set is a comprehensive output that contains the reasoning results of all noise recognition components in the network for the sample. Each component will contribute a reasoning noise labeling result subset, which lists in detail the labeling information of all noise detected in the sample by the component.

[0056] For example, suppose an audio sample contains environmental noise (such as wind) and man-made noise (such as keyboard tapping). The environmental noise recognition component in the network can output a subset of inference noise annotation results, which includes the location of the wind (such as timestamp range), type (environmental noise), and the probability that the component believes that the wind exists in the sample. Similarly, the man-made noise recognition component will also output a subset that details the relevant information about the keyboard tapping. Each subset in the inference noise annotation result set follows the same structure, that is, it includes the inference noise annotation results, the corresponding noise type, and the probability of the noise occurring. This structured output method makes the subsequent evaluation and optimization of network performance more direct and efficient.

[0057] Through step S2, the computer system not only obtains the inference noise labeling result set of each audio sample, but also can intuitively observe the performance differences of different noise recognition components when processing the same audio sample. This provides valuable data support for subsequent steps (such as calculating component cost values, updating network parameters, etc.), which helps to further improve the accuracy and robustness of the noise recognition network.

[0058] Step S3: According to the errors between each annotated noise labeling result set corresponding to the same second audio sample and each corresponding inference noise labeling result subset, obtain the component cost values ​​corresponding to the second audio sample on each noise recognition component; the component cost values ​​include the noise labeling result cost value, the noise type cost value and the probability cost value.

[0059] Specifically, for each second audio sample, the computer system obtains its annotated noise labeling result set, which records in detail the actual location, type and probability of the noise in the sample. Then, this real labeling is compared with the network's inference noise labeling result set for the sample. The inference noise labeling result set is independently output by each noise recognition component in the noise recognition network to be pre-trained, and each component contributes a subset of inference noise labeling results. During the comparison process, the computer system calculates the noise labeling result cost value, the noise type cost value and the probability cost value respectively, and these three together constitute the composition cost value.

[0060] The noise annotation cost is used to measure the difference between the noise position inferred by the network and the actual noise position. This is usually achieved by calculating the overlap of the annotation boxes (such as IoU, intersection over union) or position-based distance metrics. For example, if the network marks the position of a noise segment too far forward or too far back, the noise annotation cost will be high.

[0061] The noise type cost is used to evaluate the network's accuracy in identifying the noise type, and is generated when the network incorrectly identifies one noise as another. For example, if the noise marked in the annotation is the sound of keyboard tapping, but the network identifies it as the sound of paper shuffling, a high noise type cost will be calculated.

[0062] The probability cost value reflects the difference between the network's estimate of the probability of the noise occurring and the true probability. This is usually achieved by calculating some distance measure (such as KL divergence) between the predicted probability and the true probability. If the network overestimates or underestimates the probability of a certain noise occurring, the probability cost value will increase.

[0063] These cost values ​​are combined to form the corresponding component cost value of each second audio sample in each noise recognition component. This value not only reflects the performance of the network in all aspects when recognizing the sample, but also provides a clear direction for subsequent parameter updates.

[0064] It is worth noting that the calculation method of the composition cost value can be adjusted according to the specific application scenario and network structure. For example, in some cases, the accuracy of the noise location (i.e., the noise labeling result cost value) may be more concerned, while in other cases, the ability to identify the noise type (i.e., the noise type cost value) may be more important. In addition, as the network training deepens, the weights of these cost values ​​can also change to adapt to the continuous improvement of network performance.

[0065] In the embodiment of the present application, these composition cost values ​​are crucial for optimizing the noise recognition network. By continuously adjusting the network parameters to minimize these cost values, the computer system can gradually improve the network's noise recognition ability in complex environments, thereby providing strong support for noise suppression in practical applications.

[0066] Step S4: obtaining component cost values ​​corresponding to each noise recognition component according to the component cost values ​​corresponding to each second audio sample on the same noise recognition component.

[0067] Specifically, for the noise recognition network in pre-training, it contains multiple specially designed noise recognition components, each of which is responsible for recognizing a specific type of noise. In step S3, corresponding component cost values ​​have been calculated for each second audio sample and each noise recognition component, which comprehensively reflect the accuracy of the noise labeling results, noise types and probability estimates of the components when recognizing specific samples.

[0068] In step S4, the computer system traverses all second audio samples, and for each noise recognition component, accumulates or averages the component cost values ​​generated on all samples (depending on the aggregation strategy adopted). This aggregation method can eliminate the accidental influence of a single sample and more accurately reflect the overall performance of the component.

[0069] For example, suppose there are two noise recognition components in the network: component A specializes in recognizing environmental noise (such as wind and rain), while component B focuses on recognizing man-made noise (such as keyboard tapping and footsteps). In step S4, the computer system calculates the component cost values ​​of component A and component B on all second audio samples respectively.

[0070] For component A, the computer system accumulates or averages the component cost values ​​generated on all samples containing environmental noise. These component cost values ​​may include noise labeling result cost value (measuring the difference between the environmental noise position labeled by component A and the true position), noise type cost value (evaluating whether component A correctly identifies the noise as environmental noise) and probability cost value (comparing the environmental noise probability estimated by component A with the true probability).

[0071] Similarly, for component B, the computer system processes all samples containing artificial noise and calculates the corresponding component cost values. These cost values ​​will reflect the accuracy of component B in identifying artificial noise.

[0072] Through step S4, the computer system obtains the component cost value of each noise recognition component. These values ​​not only quantify the performance of the component, but also provide direction for subsequent network optimization. For example, if the component cost value of a component is abnormally high, it may mean that there is a problem with the algorithm or parameter setting of the component and further adjustment is required; on the contrary, if the performance of a component is excellent, then its weight can be appropriately increased or its parameters can be kept unchanged in subsequent training.

[0073] Step S5: updating the component internal configuration parameters of the corresponding noise recognition component in the noise recognition network to be pre-trained according to the component cost value, stopping when the first calibration stop requirement is met, and obtaining the noise recognition network to be calibrated.

[0074] Specifically, in the embodiment of the present application, the noise recognition network may include multiple components, each of which is responsible for identifying different types of noise. The optimization process generally includes the following key steps:

[0075] Parameter gradient calculation: The computer system uses the backpropagation algorithm to calculate the gradient of each component cost with respect to its internal configuration parameters. These internal configuration parameters may include weights, biases, etc., which together determine the output of the component. The gradient calculation is based on the chain rule and is achieved by backpropagating the error signal layer by layer.

[0076] Parameter update: After obtaining the gradient, the computer system uses an optimization algorithm (such as stochastic gradient descent SGD, Adam, etc.) to update the internal configuration parameters of the component. These algorithms adjust the value of the parameter based on the gradient information to reduce the cost of the component. For example, in SGD, the update of the parameter usually follows the following formula: ;in, represents the parameter value of step t, express (i.e., a function of component cost) The gradient of is the learning rate, which controls the step size of parameter updates. Iterative optimization: The above parameter update process is repeated multiple times, and each iteration uses a new training sample set (or batch sample set) to calculate the gradient and update the parameters. As the iterations proceed, the component cost value usually decreases gradually, indicating that the noise recognition performance is gradually improving.

[0077] Stop condition judgment: After each iteration, the computer system checks whether the preset stop condition is met. These conditions may include: the component cost is lower than a certain threshold, the performance on the validation set is no longer significantly improved, the preset number of iterations is reached, etc. If the stop condition is met, the optimization process ends, and the current network state is the noise recognition network to be adjusted.

[0078] Through the optimization process of step S5, the computer system can gradually adjust the parameters of each component in the noise recognition network to better adapt it to a specific noise recognition task. In the embodiment of the present application, this means that the network will be able to more accurately identify and suppress noise from different sources, thereby improving the clarity and intelligibility of the recording. As an implementation method, step S30, for the same original noise labeling result set, pairs the original noise labeling results output by the current noise recognition component with the original noise labeling results output by each of the remaining noise recognition components according to the commonality coefficient of the noise labeling results between the original noise labeling results corresponding to the same noise type, and obtains the paired noise labeling results corresponding to the original noise labeling results output by the current noise recognition component, which may specifically include:

[0079] Step S31: for the current original noise labeling result set, determining a first noise labeling result from the original noise labeling results output by the current noise recognition component, and determining a second noise labeling result from the original noise labeling results output by the remaining noise recognition components, wherein the first noise labeling result and the second noise labeling result correspond to the same noise type;

[0080] Step S32: obtaining a noise labeling result commonality coefficient between the first noise labeling result and the second noise labeling result according to the noise labeling result intersection range between the first noise labeling result and the second noise labeling result;

[0081] Step S33: determine that the second noise labeling result corresponding to the commonality coefficient of the noise labeling results greater than the preset commonality coefficient is the pending noise labeling result, and determine that the pending noise labeling result corresponding to the largest noise labeling result commonality coefficient is the paired noise labeling result corresponding to the first noise labeling result.

[0082] During the calibration of the noise recognition network, step S30 optimizes the network's noise labeling capability by comparing and analyzing the recognition results of different noise recognition components for the same noise type. This step calculates the commonality coefficient of the noise labeling results to achieve pairing of the labeling results of different components, thereby integrating more accurate and consistent noise labeling.

[0083] In step S31, the computer system selects a specific annotation result from the original noise annotation results output by the noise recognition component currently being analyzed (referred to as the "current component") for the currently processed original noise annotation result set as the first noise annotation result. At the same time, it also needs to find the annotation result with the same noise type as the first noise annotation result from the output results of all other noise recognition components as the second noise annotation result.

[0084] To explain more specifically, let's assume that the first noise labeling result output by component A is: Noise type: wind; Time range: [10 seconds, 15 seconds]; the computer system then finds a noise segment labeled as wind in the output of component B: Noise type: wind; Time range: [10.5 seconds, 14.8 seconds]; this labeling result is considered as one of the second noise labeling results corresponding to the first noise labeling result. The same process is repeated for the outputs of other components.

[0085] In step S32, the computer system quantifies the similarity or consistency between the first noise annotation result and the second noise annotation result. This is usually achieved by calculating the intersection range (i.e., overlapping range) between the two, and deriving the commonality coefficient of the noise annotation results based on this, which reflects the degree of proximity between the two annotation results in time and space. The intersection range refers to the overlapping part of the two annotation results on the time axis. If the two annotation results do not overlap at all, the intersection range is 0; if they overlap completely, the intersection range is equal to the length of the shorter annotation result. The commonality coefficient can be calculated based on the ratio of the intersection range to the longer of the two annotation results. This method is simple and intuitive, and can effectively reflect the consistency between the annotation results. Assume that the time range of the first noise annotation result is [ , ], the time range of the second noise labeling result is [ , ], then the intersection range Overlap and the commonality coefficient C can be calculated as follows: Overlap=max(0,min( , )−max( , ); C = ; In step S33, the computer system selects the paired noise annotation result that best matches the first noise annotation result from all the second noise annotation results according to the commonality coefficient calculated in step S32. This process usually includes two key steps: first, the second noise annotation results whose commonality coefficient is greater than the preset threshold are screened out as pending pairs; then, the one with the highest commonality coefficient is selected from these pending pairs as the final paired noise annotation result. The preset commonality coefficient is a threshold set according to the actual application scenario and network performance requirements. It is used to screen out pairs of annotation results that are similar enough and may represent the same noise source. The selection of the preset commonality coefficient requires comprehensive consideration of multiple factors, such as the diversity of noise types, the characteristics of audio signals, and the accuracy of network recognition.

[0086] Through the implementation of step S30, the computer system pairs and integrates the recognition results of the same noise type by different noise recognition components. This process not only improves the accuracy and consistency of noise labeling, but also provides valuable data support for subsequent network adjustment and optimization.

[0087] As an implementation of step S50, that is, according to the first audio sample set and the corresponding pseudo noise labeling result sets, the proposed noise recognition network is adjusted and optimized to obtain the target noise recognition network, which may specifically include:

[0088] Step S51: Determine the target cost value corresponding to the target noise recognition component according to the error between the original noise labeling result subset and the corresponding pseudo noise labeling result set output by the same first audio sample based on the same target noise recognition component, and obtain the target cost value corresponding to each target noise recognition component.

[0089] In step S51, the computer system will calculate a quantized error value for each target noise recognition component by comparing the difference between the original noise labeling result subset output by the component and the corresponding pseudo noise labeling result set. This error value is then used as the target cost value in the optimization process.

[0090] During the tuning phase of the noise recognition network, the computer system has generated a set of pseudo-noise annotation results corresponding to the first audio sample set through unsupervised learning. Although these pseudo-labels are not as accurate as manual annotations, they provide valuable data support for further tuning of the network. Next, the system needs to evaluate the performance of each target noise recognition component (that is, the component in the network responsible for identifying a specific type of noise) in the current tuning state. For each target noise recognition component, the computer system performs the following operations:

[0091] The computer system selects a sample from the first audio sample set, and then, for the sample, the system obtains a subset of the original noise labeling results output by the target noise recognition component. This subset contains the component's recognition results for the noise in the current sample, which may be presented in the form of a timestamp, a noise type label, and a confidence (or probability). Next, the system extracts pseudo-labeling information corresponding to the current sample from the pseudo-noise labeling result set. These pseudo-labeling information also contain the temporal position, type, and possible confidence of the noise (although pseudo-labels usually do not provide precise confidence values, in some cases they may be obtained indirectly through certain mechanisms in the unsupervised learning process).

[0092] The system now needs to compare the difference between the subset of original annotation results output by the component and the pseudo-noise annotation results. This difference is quantified by calculating the error, and the way the error is calculated depends on the specific form of the annotation results and the goal of network calibration. For example, if the annotation results are presented in the form of timestamps, the error may be calculated based on the degree of overlap of the annotation time periods (such as intersection over union (IoU)); if the noise type label is also included, the consistency of type identification must also be considered. Assuming that the annotation results contain both timestamps and noise type labels, a possible error calculation formula is in the form of a weighted sum, where the time error and type error are assigned different weights: ;in, It can be some distance measure between the annotated time periods (such as 1 minus the intersection-over-union ratio), It can be an indicator function of the type identification error (correct identification is 0, incorrect identification is 1), is a hyperparameter used to balance the importance of time error and type error.

[0093] Finally, the calculated error value is used as the target cost value of the target noise recognition component on the current sample. This cost value reflects the performance of the component in the current tuning state and is the basis for subsequent parameter updates. The above process will be repeated for each target noise recognition component and each first audio sample to obtain the target cost value of each component on all samples. Then, these cost values ​​can be further summarized (such as averaging) to obtain the performance evaluation of the component on the entire data set.

[0094] Step S52: According to the target cost value corresponding to the target noise recognition component, the original component internal configuration parameters corresponding to the target noise recognition component in the noise recognition network to be adjusted are updated to obtain the component modified internal configuration parameters corresponding to the target noise recognition component.

[0095] In step S52, the computer system collects the target cost values ​​corresponding to each target noise identification component, which are calculated in step S51 by comparing the difference between the original noise labeling result subset output by the component and the pseudo noise labeling result set. The target cost value reflects the performance of the component in the current tuning state and is the direct basis for parameter adjustment.

[0096] Next, the computer system uses an optimization algorithm to update the original component internal configuration parameters of the target noise recognition component. These parameters may include weights and bias terms of the neural network, which together determine how the component processes the input audio signal and the output results. The choice of optimization algorithm depends on the specific application scenario and calibration objectives. Common algorithms include stochastic gradient descent (SGD), Adam, etc. Taking the Adam algorithm as an example, the algorithm combines the advantages of the momentum method and RMSprop, and dynamically adjusts the learning rate of each parameter by calculating the first-order moment estimate and second-order moment estimate of the gradient. In step S52, the computer system first calculates the gradient of each parameter based on the target cost value, that is, the partial derivative of the cost value with respect to the parameter. Then, the Adam algorithm update rule is used to adjust the value of each parameter in combination with the first-order moment and second-order moment of the gradient accumulated in the previous iteration. The specific update process can be expressed as: ; ; ; ; ; where mt and vt are the first-order moment and second-order moment estimates of the gradient, gt is the gradient of the current iteration, β1, β2 are the decay rates, and is the decay rate of the current iteration step, which is used to control the deviation correction of the moment estimate, η is the learning rate, ϵ is a very small number (such as 10−8) to prevent division by zero errors, θt is the current parameter value, and θt+1 is the updated parameter value.

[0097] In the embodiment of the present application, this means that the computer system dynamically adjusts the weights and bias items of its internal neural network according to the target cost value of each target noise recognition component, in order to reduce errors in subsequent iterations and improve the accuracy of noise recognition. By continuously repeating steps S51 to S54, the entire noise recognition network will gradually converge to a better state, thereby more effectively supporting the noise suppression function of the smart microphone. For example, if a component performs poorly in recognizing environmental noise (such as wind and rain), its corresponding target cost value will be relatively high. In step S52, the computer system adjusts the weight parameters related to the extraction and recognition of environmental noise features within the component based on this feedback, in order to improve its recognition performance in subsequent iterations. Through multiple iterative optimizations, the component will gradually learn to more accurately distinguish between environmental noise and other types of audio signals, thereby improving the robustness and accuracy of the entire noise recognition network.

[0098] Step S53: According to the component-corrected internal configuration parameters corresponding to each of the target noise recognition components, the original component-internal configuration parameters of the remaining noise recognition components in the noise recognition network to be calibrated, except for the target noise recognition component, are updated to obtain the component-corrected internal configuration parameters corresponding to the remaining noise recognition components.

[0099] In step S53, the computer system not only focuses on the target component to be directly optimized, but also takes into account the mutual influence of other components in the network, and maximizes the recognition ability of the overall network through collaborative adjustment. In the noise recognition network, different noise recognition components often undertake the task of identifying different types of noise, such as environmental noise (wind, rain), mechanical noise (air conditioner, printer), and human voice interference. Although these components have different functions, they do not exist in isolation, but are interrelated and influence each other. Therefore, in the adjustment process, it is not enough to optimize a single component to improve the performance of the entire network, and the synergy between components must be considered. Step S53 is designed based on this idea. In the previous steps, the computer system has corrected its internal configuration parameters (such as the weight and bias of the neural network) according to the target cost value of the target noise recognition component. Now, the system needs to use this corrected component information to guide the parameter adjustment of other components in the network. Specifically, the system first analyzes the parameter changes of the corrected components to understand how these changes affect the noise recognition performance of the components. Then, it compares these changes with the current status of other components in the network to find possible collaborative optimization space.

[0100] For example, suppose that a component A that specializes in identifying environmental noise has been optimized to significantly improve its ability to identify wind noise. At this point, the system can find that another component B (such as a component that identifies mechanical noise) that works with component A does not perform optimally when processing certain audio samples that contain a mixture of wind and mechanical noise. This may be because component B fails to fully utilize the improved wind recognition ability of component A when processing such complex audio. To solve this problem, the system infers the possible impact of component A's correction parameters on component B and adjusts the internal configuration parameters of component B accordingly. This adjustment may involve fine-tuning weights and bias terms so that component B can work better with component A when processing mixed noise. In practice, this collaborative adjustment may rely on some complex machine learning techniques, such as multi-task learning, transfer learning, or ensemble learning. These techniques allow the system to learn useful feature representations or knowledge from the optimized components and apply them to other components, thereby improving the performance of the entire network.

[0101] Step S54: Correct the internal configuration parameters of the components corresponding to each noise recognition component to obtain the target noise recognition network. S51 to S53 respectively complete the calculation of the target cost value, the correction of the parameters of a single component, and the coordinated adjustment between components. These steps work together to gradually reduce the error between the network prediction result and the expected annotation, and promote the continuous improvement of network performance. In step S54, the computer system has collected the corrected internal configuration parameters of all noise recognition components after multiple iterations of optimization. These parameters include but are not limited to the weights and bias items of the neural network, which jointly determine the processing method and output results of the components for the audio signal. In this step, the computer system reconstructs the noise recognition network based on these corrected parameters. Specifically, it loads the new parameters of each component into the corresponding network layer to replace the original parameter values. In this way, the entire network has the optimized recognition capability. After completing the parameter replacement, the computer system usually performs a comprehensive network verification to ensure that all components can work together and achieve the expected recognition effect. This verification process may include running the network on an independent test set to evaluate its noise recognition performance for unknown audio samples. If the verification results show that the network performance meets expectations, the noise recognition network is considered to be a tuned target network. This network not only has higher recognition accuracy and robustness, but also can better meet the actual needs of the intelligent microphone noise suppression system.

[0102] In the embodiment of the present application, the acquisition of the target noise recognition network is of great significance for improving the quality of voice communication. By accurately identifying and suppressing background noise, the system can provide users with a clearer and smoother voice communication experience. This not only helps to improve the user experience of application scenarios such as teleconferencing and online education, but also promotes the development and application of intelligent audio processing technology.

[0103] As an implementation manner, the step S51, determining the target cost value corresponding to the target noise recognition component according to the error between the original noise labeling result subset and the corresponding pseudo noise labeling result set output by the same first audio sample based on the same target noise recognition component, and obtaining the target cost value corresponding to each target noise recognition component, may specifically include:

[0104] Step S511: For the same original noise annotation result set, according to the noise annotation result commonality coefficient between the original noise annotation results corresponding to the same noise type, the original noise annotation results output by the current target noise recognition component are paired with the original noise annotation results output by each of the remaining target noise recognition components to obtain the pairing results corresponding to the original noise annotation results output by the current target noise recognition component. In step S511, the computer system focuses on the same original noise annotation result set, especially the annotation results corresponding to the same noise type. These results may come from different target noise recognition components, each of which attempts to annotate the same noise. Next, the system evaluates the similarity between these annotation results based on the noise annotation result commonality coefficient. The commonality coefficient is a quantitative indicator used to measure the similarity between two annotation results in space, time or features. In the embodiment of the present application, the commonality coefficient may be calculated based on a variety of factors, such as the timestamp overlap of the annotation results, the similarity of the frequency distribution, the Euclidean distance of the noise feature vector, etc. Specifically, if the two annotation results are highly overlapped on the time axis and the annotated noise features are also highly consistent, then their commonality coefficient will be high.

[0105] For each original noise labeling result output by the target noise recognition component currently being analyzed (referred to as "component A"), the system traverses the output results of all other target noise recognition components (such as component B, component C, etc.), and looks for the labeling result with the highest commonality coefficient with the labeling result of component A as the pairing object. Through this pairing process, the computer system can integrate the recognition results of different components for the same type of noise, providing richer and more accurate data support for subsequent performance evaluation and parameter optimization. This not only helps to improve the recognition ability of a single component, but also promotes the collaborative work between components, thereby enhancing the performance of the entire noise recognition network.

[0106] Step S512: for the current first audio sample, based on the original noise labeling result subset output by the current target noise recognition component, according to the error between the original noise labeling result in the original noise labeling result subset and the corresponding constrained noise labeling result, obtain the original component cost value corresponding to each original noise labeling result in the original noise labeling result subset; if the pairing result indicates that it is paired, then the constrained noise labeling result corresponding to the original noise labeling result is the paired noise labeling result; if the pairing result indicates that it is not paired, then the constrained noise labeling result corresponding to the original noise labeling result is the corresponding pseudo noise labeling result in the pseudo noise labeling result set.

[0107] Before step S512 is executed, the pairing work of step S511 has been completed, that is, the original noise annotation results of each component are paired according to the commonality coefficient of the noise annotation results. The result of this step provides a potential "paired noise annotation result" for each original noise annotation result of the current target noise recognition component (hereinafter referred to as the "target component"). However, not all original noise annotation results can be successfully paired. For those annotation results that cannot be paired, it is necessary to find the corresponding constrained noise annotation results from the pseudo noise annotation result set.

[0108] For example, for a first audio sample, the environmental noise recognition component outputs an original noise labeling result labeled as "wind sound". In step S511, the labeling result is successfully paired with a similar labeling result output by another component. However, the mechanical noise recognition component also outputs an original noise labeling result (assuming it is "motor sound") for the audio sample, but due to the lack of a pairing object with a sufficiently high commonality coefficient, the labeling result fails to be successfully paired. In this case, for the labeling result of the environmental noise recognition component, its constrained noise labeling result is the successfully paired paired noise labeling result; and for the labeling result of the mechanical noise recognition component, its constrained noise labeling result needs to be selected from the pseudo noise labeling result set. Specifically, the computer system searches for labeling results with similar timestamps and similar noise characteristics in the pseudo noise labeling result set as constrained noise labeling results. After determining the constrained noise labeling result corresponding to each original noise labeling result, the computer system calculates the error between them. This error reflects the accuracy of the target component in identifying the noise in the current audio sample and is an important indicator for evaluating the performance of the component. The error can be calculated in a variety of ways, depending on the form of the noise labeling result and the calibration target. For example:

[0109] Timestamp error: If the noise annotation results are expressed in the form of timestamps (i.e., the time points at which the noise starts and ends are marked), then the timestamp error is an intuitive evaluation indicator. The timestamp error can be obtained by calculating the difference between the start and end points of the annotation result time period and the start and end points of the actual noise time period. For example, the root mean square error (RMSE) or mean absolute error (MAE) can be used to measure this difference.

[0110] Noise type error: If the noise labeling result also includes information about the noise type, then the noise type error is also an important aspect of evaluating component performance. The noise type error can be obtained by comparing whether the predicted noise type is consistent with the actual noise type, and is usually measured using indicators such as classification accuracy or confusion matrix.

[0111] Feature vector error: In some advanced applications, the noise labeling result may be represented in the form of a feature vector, which contains multiple dimensions of the noise signal. In this case, the difference between the predicted feature vector and the actual noise feature vector can be calculated using metrics such as cosine similarity, Euclidean distance, or Mahalanobis distance.

[0112] Based on the calculated error value, the computer system can assign an original component cost value to each original noise annotation result. This cost value comprehensively reflects the accuracy of the annotation results in multiple dimensions such as time, type, and characteristics. In actual operation, different weights can be assigned to different types of errors according to the calibration goals to highlight certain aspects of performance evaluation. For example, in an embodiment of the present application, if the system pays special attention to the accurate annotation of the noise time period, then the weight of the timestamp error can be set relatively high; and if the system also needs to identify the type of noise, then the weight of the noise type error cannot be ignored.

[0113] Step S512 provides valuable performance evaluation data for the tuning process of the noise recognition network by calculating the error between the original noise labeling result and the constrained noise labeling result. These data not only help to understand the recognition accuracy of each component in the current tuning state, but also point out the direction for subsequent parameter optimization. Step S513: Determine the cost value importance coefficient corresponding to the original noise labeling result based on the pairing result corresponding to the original noise labeling result; the cost value importance coefficient corresponding to the pairing is greater than the cost value importance coefficient corresponding to the unpaired.

[0114] In step S512, the computer system has found the corresponding constrained noise annotation results for each original noise annotation result. These results may be paired noise annotation results obtained through successful pairing, or pseudo-annotation results selected from the pseudo-noise annotation result set. Since the constrained noise annotation results from these two sources may differ in credibility and accuracy, different levels of attention need to be given when evaluating the accuracy of the original noise annotation results. Step S513 is based on this consideration and introduces the cost value importance coefficient to achieve differentiated evaluation of different constrained noise annotation results. Specifically, for the successfully paired original noise annotation results, since their constrained noise annotation results have a higher credibility (because the pairing is based on the accurate calculation of the commonality coefficient of the noise annotation results), the cost value importance coefficients corresponding to these original noise annotation results will be set relatively high. On the contrary, for those original noise annotation results that failed to pair successfully and can only rely on pseudo-noise annotation results as constraints, their corresponding cost value importance coefficients will be relatively low. When the original noise annotation result is successfully paired with the output result of a component, it indicates that the two annotation results have a high similarity in multiple dimensions (such as time, frequency, noise characteristics, etc.). Therefore, it can be considered that the constrained noise annotation result has a high reference value for the evaluation of the original noise annotation result. At this time, the cost value importance coefficient can be set to a value greater than 1 (such as 1.2, 1.5, etc.) to highlight the importance of successful pairing. When the original noise annotation result fails to be successfully paired, its constrained noise annotation result usually comes from the pseudo-noise annotation result set. Since the pseudo-annotation result may have a certain degree of uncertainty and error, it needs to be given relatively low attention when evaluating the original noise annotation result. At this time, the cost value importance coefficient can be set to a value less than or equal to 1 (such as 0.8, 0.5, etc.) to reflect the uncertainty of the pseudo-annotation result.

[0115] In this way, the computer system can more reasonably evaluate the performance of the target noise recognition component in subsequent steps. For the successfully paired annotation results, due to their higher cost value importance coefficient, their errors will have a greater weight in the overall performance evaluation; while for the unsuccessfully paired annotation results, although their errors will also be taken into account, due to their lower cost value importance coefficient, their impact on the overall performance evaluation will be relatively small. This weighted evaluation mechanism helps to improve the accuracy and effectiveness of the tuning process.

[0116] Step S514: According to the cost value importance coefficient corresponding to the original noise labeling result, the original component cost values ​​corresponding to each original noise labeling result in the original noise labeling result subset are weighted and summed to obtain the target component cost value corresponding to the current first audio sample on the current target noise recognition component.

[0117] In step S513, the computer system has assigned a cost value importance coefficient to each original noise annotation result. This coefficient reflects the importance of the annotation result in the overall performance evaluation, and its value depends on the pairing of the annotation results. Specifically, the cost value importance coefficient of the successfully paired annotation results is usually set higher because of their higher credibility; while the cost value importance coefficient of the unsuccessful paired annotation results is relatively low because they rely on pseudo-noise annotation results that may not be accurate.

[0118] In step S514, the computer system obtains a subset of original noise labeling results output by the target noise recognition component currently being analyzed for the current first audio sample, which subset includes multiple original noise labeling results and their corresponding original component cost values ​​(these cost values ​​are calculated in step S512).

[0119] Next, the system traverses this subset and multiplies the original component cost value corresponding to each original noise annotation result by its cost value importance coefficient to obtain the weighted cost value. This process is actually adjusting the original component cost value to reflect the difference in importance of different annotation results in the overall performance evaluation.

[0120] Finally, the system sums up all the weighted cost values, and the result is the target component cost value corresponding to the current first audio sample on the current target noise recognition component. This value comprehensively reflects the recognition performance of the component on the current audio sample and is an important basis for subsequent optimization and adjustment.

[0121] Through the weighted summation process of step S514, the computer system can more accurately evaluate the performance of the target noise recognition component on the current audio sample, providing strong data support for subsequent optimization adjustments. Step S515: According to the target component cost value corresponding to each first audio sample on the current target noise recognition component, obtain the target cost value corresponding to the current target noise recognition component.

[0122] In the previous steps, the computer system has calculated the target component cost value corresponding to the current target noise recognition component for each first audio sample. These values ​​quantify the noise recognition performance of the component when processing a single sample and are important indicators for evaluating the accuracy and stability of the component. However, the performance evaluation of a single sample is often accidental and insufficient to fully reflect the overall capabilities of the component. Therefore, step S515 obtains a more representative target cost value by summarizing the target component cost values ​​on multiple samples.

[0123] In actual operation, the computer system can use the average calculation method to complete this step. Specifically, it will traverse all the first audio samples used for adjustment, accumulate the target component cost value obtained by each sample on the target noise recognition component, and then divide it by the total number of samples to obtain the average target component cost value, that is, the target cost value.

[0124] Through the calculation of step S515, the computer system obtains a quantified and comparable target cost value, which can not only help developers understand the performance level of the target noise recognition component under the current adjustment state, but also guide the subsequent parameter optimization work. Developers can adjust parameters such as network structure and learning rate according to the size and change trend of the target cost value, in order to further improve the noise recognition accuracy and stability of the component.

[0125] As an implementation manner, the step S53, according to the component-corrected internal configuration parameters corresponding to each of the target noise recognition components, updates the original component internal configuration parameters of the remaining noise recognition components other than the target noise recognition component in the noise recognition network to be adjusted, and obtains the component-corrected internal configuration parameters corresponding to the remaining noise recognition components, which may specifically include:

[0126] Step S531: obtaining component importance coefficients corresponding to respective noise recognition components, wherein the component importance coefficient corresponding to the target noise recognition component is smaller than the component importance coefficients corresponding to the remaining noise recognition components;

[0127] Step S532: According to the component importance coefficients corresponding to the respective noise recognition components, the component modified internal configuration parameters corresponding to the respective target noise recognition components and the original component internal configuration parameters corresponding to the remaining noise recognition components are merged to obtain the component modified internal configuration parameters corresponding to the remaining noise recognition components.

[0128] In step S531, the noise recognition network is usually composed of multiple components, each of which is responsible for identifying different types of noise. The status and role of these components in the network are different. Some components may be more focused on identifying common noise types that have a greater impact on voice communication (such as environmental noise, mechanical noise, etc.), while some components may be responsible for identifying relatively rare or special noise types. Therefore, during the tuning process, different optimization weights need to be assigned according to the importance of the components and the degree of impact on the overall performance of the network.

[0129] The core task of step S531 is to obtain the component importance coefficient corresponding to each noise recognition component. This coefficient reflects the importance of the component in the network and is the basis for subsequent parameter fusion and optimization adjustment. The determination of the component importance coefficient is usually based on a comprehensive consideration of multiple factors, including but not limited to the importance of the noise type identified by the component, the performance of the component in the previous tuning process, and the contribution of the component to the overall network performance. In actual operation, the computer system can use a variety of methods to evaluate the importance of the component. For example, the importance level of the component can be determined by analyzing the frequency of occurrence and recognition difficulty of different noise types in a large number of audio samples; or by temporarily removing a component from the network and observing the change in overall performance to evaluate its importance. Regardless of the method used, the goal is to ensure that the distribution of component importance coefficients is both reasonable and accurate. It is worth noting that in step S531, the component importance coefficient corresponding to the target noise recognition component (i.e., the component currently being directly optimized) is usually set relatively low. This is because the parameters of the target component have been directly optimized through the previous steps (such as S51 and S52), and the current task is to extend these optimization results to other components in the network. Therefore, in the process of parameter fusion, more weight and attention should be given to non-target components.

[0130] In step S532, the computer system will fuse the optimized component modified internal configuration parameters of the target noise identification component with the original component internal configuration parameters of the non-target noise identification component according to the component importance coefficient obtained in step S531. The purpose of this process is to pass the optimization results of the target component to other components in the network to achieve collaborative optimization of the entire network.

[0131] For example, assume that the noise recognition network contains three components: component A (target component), component B, and component C. After optimization, component A obtains a set of new internal configuration parameters (such as weight matrix WA′ and bias vector bA′). Now, it is necessary to fuse these optimized parameters into components B and C. First, according to the component importance coefficients obtained in step S531, it is assumed that the importance coefficient of component B is αB, and the importance coefficient of component C is αC (and αB>αC). Then, in the fusion process, the optimized parameters of component A can be integrated into components B and C by weighted averaging. Specifically, for a certain weight matrix WB of component B, its fused new weight matrix WB′ can be expressed as: WB′=λWA′+(1−λ)WB; where λ is a fusion coefficient, which is calculated based on the component importance coefficient (for example, λ=αA+αBαA, but considering that the importance coefficient of target component A is low, the value of λ may need to be further adjusted in actual applications). Similarly, bias vectors and other types of parameters can also be fused. For component C, the fusion process is similar to that of component B, but the fusion coefficient λ is adjusted according to the importance coefficient of component C. It should be noted that since the importance of component C may be lower, its fused parameters may retain more characteristics of the original parameters.

[0132] Through the parameter fusion process of step S532, the computer system transmits the optimization results of the target noise recognition component to other components in the network.

[0133] As another implementation of step S50, that is, according to the first audio sample set and the corresponding pseudo noise labeling result sets, the proposed noise recognition network is adjusted and optimized to obtain the target noise recognition network, which may specifically include:

[0134] Step S501: updating the internal configuration parameters of the proposed noise recognition network according to the first audio sample set and the corresponding pseudo-noise labeling result sets, to obtain a temporary noise recognition network;

[0135] Step S502: taking the temporary noise recognition network as the noise recognition network to be adjusted, jumping to the step of obtaining the first audio sample set for iteration, stopping when the second adjustment stop requirement is met, and obtaining the target noise recognition network.

[0136] In step S501, the computer system uses the unlabeled first audio sample set and its corresponding pseudo noise annotation result set as training data to perform preliminary calibration on the noise recognition network to be calibrated. Although the pseudo noise annotation result set is not as accurate as manual annotation, it is sufficient to provide valuable supervisory signals in the early stage of network calibration to guide the network to learn basic noise recognition patterns.

[0137] After obtaining the temporary noise recognition network, step S502 uses it as the noise recognition network to be adjusted, and returns to the starting point of step S501 to start a new round of iterative adjustment. This process will be repeated until the preset second adjustment stop requirement is met, and finally the target noise recognition network is generated.

[0138] Iteration initialization: Before each iteration, the temporary noise recognition network replaces the previous round of noise recognition network to be tuned and becomes a new starting point. At the same time, hyperparameters such as learning rate and batch size can be adjusted as needed to optimize the training process. In order to improve the generalization ability of the network, data enhancement techniques can be introduced during the iteration process, such as adding noise and adjusting the audio speed. In addition, dynamic sample selection strategies can be adopted, such as giving priority to samples that are difficult for the training network to recognize, to accelerate the convergence process. As the network performance improves, the accuracy of the pseudo-noise labeling results will gradually improve. Therefore, in each iteration, the pseudo-noise labeling result set can be dynamically adjusted according to the prediction results of the current network to make the training data closer to the current state of the network. The second tuning stop requirement may include multiple indicators, such as the training loss value drops below a preset threshold, the performance on the validation set is no longer significantly improved, and the number of iterations reaches a preset upper limit. When one of these conditions is met, the iteration process will stop, and the current temporary noise recognition network will be regarded as the target noise recognition network.

[0139] As an implementation manner, the step S501, based on the first audio sample set and the corresponding pseudo noise labeling result sets, updates the internal configuration parameters of the noise recognition network to be adjusted to obtain a temporary noise recognition network, which may specifically include:

[0140] Step S5011: Acquire a third audio sample set, wherein the third audio sample set is a labeled audio sample set;

[0141] Step S5012: According to the first audio sample set and the corresponding pseudo-noise labeling result sets, and the third audio sample set, the internal configuration parameters of the noise recognition network to be adjusted are updated to obtain a temporary noise recognition network. In step S5011, the computer system obtains a third audio sample set containing labeled audio samples. These sample sets usually come from pre-collected and manually labeled audio data. They not only contain various types of noise signals, but also clearly mark the type, location and possible attributes of the noise (such as the intensity and duration of the noise). These label information provides accurate supervision signals for network training, which helps guide the network to learn the correct noise recognition mode.

[0142] After obtaining the first audio sample set, the corresponding pseudo noise labeling result set and the third audio sample set with labels, the computer system enters step S5012 and uses these data to update the internal configuration parameters of the noise recognition network to be adjusted. This process usually involves multiple links such as data preprocessing, network forward propagation, loss calculation, back propagation and parameter update.

[0143] Before network training, all audio samples need to be preprocessed. This includes steps such as sampling rate unification, noise removal, and feature extraction. For the first audio sample set, since they are unlabeled, pseudo labels need to be generated through a pseudo-noise labeling algorithm; for the third audio sample set, its original label information is directly used.

[0144] When extracting features, audio features can include Mel-frequency cepstral coefficients (MFCC), linear prediction cepstral coefficients (LPCC), and spectral features, etc. These features can capture key information in audio signals and provide strong support for subsequent network training.

[0145] After the preprocessing is completed, the computer system inputs the processed audio features into the noise recognition network to be adjusted. The network may be a deep neural network composed of multiple convolutional layers, recurrent layers (such as LSTM, GRU) and fully connected layers. After receiving the input features, the network processes them layer by layer through forward propagation, and finally outputs the predicted noise labeling results. In order to evaluate the accuracy of the prediction results, the computer system calculates the loss value between the predicted results and the true (or false) labeling results. Commonly used cost functions include cross-entropy loss (Cross-Entropy Loss), mean squared error loss (Mean SquaredError, MSE), etc. For noise recognition tasks, since the output is usually a discrete label (such as whether noise exists, noise type, etc.), cross-entropy loss is a more appropriate choice. Assuming that the output layer of the network is a softmax layer, which is used to predict the probability of noise existence at each time step, the calculation formula of the cross-entropy loss can be expressed as: ; where N is the number of samples, C is the number of categories (which can be regarded as the number of noise types plus one type of non-noise), is the true label (0 or 1) of the i-th sample for category c, is the probability that the i-th sample belongs to category c predicted by the network.

[0146] After obtaining the loss value, the computer system uses the back-propagation algorithm to calculate the gradient of the loss value with respect to the internal configuration parameters of the network. These parameters include the weights and biases of each layer of the network. By back-propagating the gradient information layer by layer through the chain rule, the gradient value of each parameter can be calculated. Subsequently, the network parameters are updated according to the gradient information using an optimization algorithm (such as SGD, Adam, etc.). The choice of optimization algorithm depends on the requirements of the specific task and the characteristics of the network structure. In the noise recognition task, the Adam algorithm can be used as an optimization algorithm due to its advantages in adaptive learning rate and momentum term. The specific formula for parameter update (taking the Adam algorithm as an example) can refer to the introduction in the aforementioned step S52.

[0147] In the process of parameter updating, the computer system cleverly combines the first audio sample set and its pseudo-noise annotation result set with the third audio sample set. Since the third audio sample set is labeled, it can be directly used for supervised training; while the first audio sample set is unlabeled, but its pseudo-noise annotation result set provides additional supervisory signals for network training. In order to fully utilize the advantages of these two sample sets, the following strategies can be adopted:

[0148] Phased training: First, the network is pre-trained using the third audio sample set to give it a certain noise recognition capability. Then the first audio sample set and its pseudo-noise labeled result set are added to the training process for further fine-tuning. This approach ensures that the network learns the correct noise recognition pattern at the initial stage, and then the generalization ability of the network is enhanced with the assistance of unlabeled data.

[0149] Mixed sampling: Samples from both the first audio sample set and the third audio sample set are sampled for training in each training batch. For samples from the first audio sample set, pseudo-noise labeling results are used as supervisory signals; for samples from the third audio sample set, their original label information is used directly. This method ensures that the network receives supervisory signals from both labeled and unlabeled data during training, which helps improve the overall performance of the network.

[0150] Dynamic weight adjustment: Dynamically adjust the weights of the two sample sets during the training process based on the network's performance on the validation set. For example, if the network's performance on the validation set improves slowly or overfitting occurs, the weight of the first audio sample set during the training process can be appropriately increased; conversely, if the network performance is relatively stable and no overfitting occurs, the weight of the first audio sample set can be gradually reduced to avoid interference with the supervisory signal of the third audio sample set.

[0151] After multiple iterations of training, the computer system determines whether to stop training based on the network's performance on the validation set. If the network performance has reached the preset threshold or is no longer significantly improved, training is stopped and the current network state is saved as a temporary noise recognition network. This network already has a certain noise recognition capability and will serve as the basis for further optimization and evaluation in subsequent steps.

[0152] As an implementation manner, the step S5011, obtaining the third audio sample set, may include:

[0153] Step S50111: Acquire a fourth audio sample set, wherein the fourth audio sample set is an unlabeled audio sample set.

[0154] Step S50111 plays a fundamental and key role in the technical process of intelligent microphone noise suppression. It involves the collection and organization of unlabeled audio data, providing the necessary raw materials for subsequent steps. The following is a detailed explanation of this step, combined with specific application scenarios.

[0155] In the embodiment of the present application, high-quality audio data is an indispensable resource for training and optimizing the noise recognition network. However, in practical applications, a large amount of audio data with accurate noise annotations is often difficult to obtain because the manual annotation process is time-consuming and laborious. Therefore, the computer system usually needs to collect unlabeled audio sample sets from various channels as the basis for subsequent processing. Step S50111 is the starting point of this process, which requires the computer system to obtain a fourth audio sample set containing unlabeled audio samples. The "unlabeled" here means that these audio samples are not accompanied by any annotation information about the location, type or intensity of the noise when they are collected. Despite this, these unlabeled samples still contain rich noise and speech signals, which are important materials for subsequent noise identification, annotation and network training.

[0156] Step S50112: Load each fourth audio sample in the fourth audio sample set into the noise recognition network to be calibrated, and obtain an original noise labeling result set corresponding to each of the fourth audio samples; the original noise labeling result set corresponding to the fourth audio sample includes an original noise labeling result subset output by each noise recognition component in the noise recognition network to be calibrated, and the original noise labeling result subset includes an original noise labeling result output by the same noise recognition component and a probability corresponding to the original noise labeling result.

[0157] In step S50112, the computer system loads each unlabeled audio sample in the fourth audio sample set into the noise recognition network to be adjusted one by one. This network is a complex system composed of multiple noise recognition components, each of which is responsible for identifying a specific type of noise or noise characteristics. When the audio sample is input into the network, each component will independently analyze the sample and output its own original noise labeling result. When these unlabeled audio samples are loaded into the noise recognition network to be adjusted, each noise recognition component in the network will start working. For example, the network may include a component that specializes in identifying environmental noise (such as wind and rain), a component that identifies mechanical noise (such as air conditioning and printer), and a component that identifies human voices (especially non-target human voices, i.e. background conversations).

[0158] For each fourth audio sample, these components output their own subset of the original noise annotation results. Each subset records in detail the component's recognition of the noise in the sample, including the location of the noise (such as the timestamp range), the type, and the corresponding probability. The probability value reflects the component's confidence in the noise annotation result, which is a value between 0 and 1. The closer it is to 1, the more confident the component is in the accuracy of its annotation.

[0159] These subsets of original noise annotation results together constitute the original noise annotation result set corresponding to the fourth audio sample. This set not only contains the recognition results of multiple components for the same audio sample, but also provides a probability evaluation of these results, providing valuable data support for pseudo-label generation, uncertainty measurement and sample screening in subsequent steps.

[0160] By executing step S50112, the computer system converts the unlabeled fourth audio sample into a data set containing preliminary noise labeling results. Although these data sets have not been manually verified or fine-tuned, they have laid a solid foundation for subsequent pseudo-label generation, network training and optimization.

[0161] Step S50113: perform uncertainty measurement on the probabilities corresponding to the respective original noise annotation results in the same subset of original noise annotation results to obtain the inference clutter corresponding to the respective subset of original noise annotation results, integrate the inference clutter corresponding to the same fourth audio sample to obtain the integrated clutter corresponding to the respective fourth audio samples.

[0162] In step S50113, the computer system evaluates the stability and consistency of the labeling results of each noise identification component by performing uncertainty measurement on the probability values ​​in the original noise labeling results, thereby providing strong support for subsequent data screening and labeling work.

[0163] In step S50112, the computer system has generated a corresponding original noise annotation result set for each sample in the fourth audio sample set, and these result sets include the original noise annotation result subsets output by multiple noise recognition components. Each subset not only indicates the type and location of the noise, but also gives a corresponding probability value, reflecting the confidence of the component in the annotation result.

[0164] However, since different components may be based on different algorithms or model structures, their recognition results for the same noise may be different, and this difference reflects the uncertainty of the labeling result to a certain extent. In order to quantify this uncertainty, the computer system executes step S50113.

[0165] In this step, the computer system focuses on each original noise annotation result in the same subset of original noise annotation results. For the annotation results output by each component and its corresponding probability value, the system uses an uncertainty measurement method to evaluate its stability. This measurement method may be based on the concept of entropy in information theory, or it may use other statistical indicators that can reflect the degree of data dispersion.

[0166] Assume that entropy is used as the uncertainty metric. For a subset of the original noise annotation results output by a component, its entropy value can be calculated using the following formula: ;in, is the probability value corresponding to the i-th annotation result in the subset, and n is the total number of annotation results in the subset. The larger the entropy value H, the higher the uncertainty of the annotation result; conversely, the smaller the entropy value, the more stable and reliable the annotation result. By calculating the entropy value of each subset of the original noise annotation results, the computer system obtains the inference chaos (i.e., the uncertainty measurement result) of each component for the annotation result of the same audio sample. Then, the system integrates these inference chaos to evaluate the uncertainty level of the entire fourth audio sample.

[0167] The calculation method of integrated clutter may vary depending on the application scenario. A simple and intuitive method is to take the average or weighted average of the inference clutter of all components. However, in some cases, in order to more accurately reflect the overall uncertainty of the sample, the system can adopt a more complex fusion strategy, such as considering the mutual influence between components or using advanced machine learning models for prediction. Regardless of the integration method used, the final integrated clutter reflects the overall uncertainty level of the fourth audio sample in the noise labeling process. This indicator is of great significance for subsequent data screening and labeling work. By setting appropriate thresholds or sorting strategies, the computer system can give priority to samples with lower integrated clutter, because the labeling results of these samples are relatively more reliable and stable.

[0168] In the embodiment of the present application, the execution of step S50113 helps to improve the accuracy and efficiency of noise labeling. By quantifying the uncertainty of the labeling results and screening out high-quality samples for further processing or labeling, the computer system can more effectively use limited resources to optimize the performance of the noise recognition network. This not only helps to improve the noise suppression ability of the smart microphone in complex environments, but also provides users with a clearer and purer voice communication experience.

[0169] Step S50114: performing clutter segmentation according to the integrated clutter corresponding to each fourth audio sample to obtain a plurality of clutter value ranges, and determining a target value range in the plurality of clutter value ranges.

[0170] In step S50113, the computer system has calculated the integrated clutter for each fourth audio sample, which reflects the overall uncertainty level of the sample in the noise labeling process. The higher the integrated clutter, the more unstable the noise labeling result of the sample is, which may contain more mislabeling or inconsistency; conversely, the lower the integrated clutter, the more reliable the labeling result is.

[0171] In step S50114, the computer system divides the integrated chaos into different value ranges to form multiple chaos levels. The purpose of this process is to classify samples according to their uncertainty levels, so as to facilitate the selection of appropriate sample sets for further processing according to different requirements.

[0172] There are many ways to segment the clutter, such as equal-width segmentation, equal-frequency segmentation, or adaptive segmentation based on clustering algorithms. Equal-width segmentation is to divide the range of the integrated clutter into several intervals; equal-frequency segmentation is to ensure that the number of samples in each interval is roughly equal. However, in practical applications, since the distribution of the integrated clutter may not be uniform, it is more common to use adaptive segmentation methods, such as those based on K-means clustering or DBSCAN algorithms, to determine the segmentation points according to the actual distribution of the data.

[0173] After completing the clutter segmentation, the computer system determines one or more target value ranges from multiple clutter value ranges based on specific application scenarios and requirements. These target value ranges usually correspond to intervals with lower integrated clutter (i.e., less uncertainty), because samples within such intervals are more likely to contain accurate and consistent noise labeling results, which are more valuable for the training and optimization of the noise recognition network.

[0174] In the embodiment of the present application, it is assumed that a noise recognition network is being optimized for a teleconference system. In order to improve the generalization ability and recognition accuracy of the network, a training set containing high-quality labeled data needs to be constructed. However, since it is costly to directly obtain a large number of labeled audio samples, some samples are selected from the fourth unlabeled audio sample set for manual labeling.

[0175] In this process, step S50114 is particularly important. By calculating the integrated chaos of each sample and dividing it, we can quickly identify those samples with relatively stable labeling results (i.e., samples with low integrated chaos). Then, we can give priority to manual labeling of these samples to ensure the quality of the training set.

[0176] Specifically, the computer system can classify the integrated clutter into three levels: "low", "medium" and "high". After determining the target value range (such as the interval corresponding to the "low" level), it will randomly select a portion of the samples within the range as the target audio samples to be annotated. These samples will then be sent to a professional annotation team for manual noise annotation, and finally generate a third set of labeled audio samples for training the noise recognition network.

[0177] In this way, the computer system not only improves the efficiency of data screening, but also ensures the labeling quality of the selected samples, providing strong support for subsequent noise recognition network training.

[0178] Step S50115: determining a target audio sample to be annotated from the fourth audio sample whose integrated clutter is within the target value range, performing noise annotation on the target audio sample, and obtaining an annotated noise labeling result set corresponding to the target audio sample.

[0179] In the previous step, the computer system has determined the target value range by calculating the integrated clutter and dividing the clutter. The fourth audio samples within this range have relatively low integrated clutter, which means that their noise labeling results are relatively stable and reliable, and are suitable as candidate samples for manual annotation.

[0180] In step S50115, the primary task of the computer system is to further screen out the target audio samples to be annotated from these candidate samples. The screening process may be based on a variety of factors, such as the representativeness of the samples, the diversity of noise types, and the consideration of annotation costs. In order to balance the quality and cost of annotation, the system can select target samples by random sampling or a strategy based on a certain heuristic rule (such as preferentially selecting samples containing a specific type of noise). Once the target audio samples are determined, the computer system will send these samples to a professional noise annotation team for manual annotation. The annotation team is usually composed of experts with audio processing experience. They use professional audio editing software and annotation tools to listen carefully to each sample and mark the noise part therein. The annotation results usually include information such as the type of noise, start time, end time, etc., which will be organized into an annotated noise annotation result set. In the embodiment of the present application, the annotated noise annotation result set is crucial for the training of the noise recognition network. They provide the network with clear instructions on which parts are noise and which parts are non-noise, so that the network can learn effective noise recognition patterns. By continuously optimizing these annotation results and combining them with unlabeled audio samples for semi-supervised or self-supervised learning, the computer system can gradually improve the performance and robustness of the noise recognition network.

[0181] It is worth noting that although the manual annotation process can generate high-quality labeled data, its cost is relatively high. Therefore, in practical applications, pseudo-noise labeling algorithms and manual annotation can be combined to balance cost and performance. Pseudo-noise labeling algorithms can automatically generate pseudo-labels for a large number of unlabeled audio samples. Although these pseudo-labels may have certain errors, they provide rich supervisory signals for network training. Manual annotation is mainly used to correct errors in pseudo-labels and improve labeling accuracy, thereby ensuring that the network can learn the correct noise recognition pattern.

[0182] In summary, step S50115 provides valuable supervisory signals for the training of the noise recognition network by screening out high-quality fourth audio samples for artificial noise annotation. This process not only improves the accuracy and reliability of the labeled data, but also lays a solid foundation for subsequent network optimization and performance improvement.

[0183] Step S50116: Obtain a third audio sample set according to each target audio sample and the corresponding annotation noise labeling result set.

[0184] In the embodiment of the present application, the third audio sample set is the cornerstone of the noise recognition network learning. It contains multiple carefully selected and accurately annotated target audio samples, each of which is accompanied by detailed noise annotation information, such as the type, location, intensity, etc. of the noise. This information is invaluable to the network because it provides clear supervision signals for the network to guide the network on how to distinguish between noise and pure speech.

[0185] In step S50116, the computer system summarizes all manually annotated target audio samples and their corresponding annotated noise labeling result sets. These labeling result sets are generated by a professional audio processing team to ensure the accuracy and reliability of the labeling. Then, the system organizes these samples and labeling results in a certain format to form a third audio sample set. This sample set has significant advantages over the previous unlabeled audio sample set (such as the fourth audio sample set). It not only contains a rich variety of noise types and scene changes, but also ensures the accuracy of noise labeling in each sample through manual annotation. This makes the third audio sample set an ideal choice for training a noise recognition network, because it can guide the network to learn the correct noise recognition mode, thereby improving the noise suppression effect of the system in practical applications. In the process of constructing the third audio sample set, the computer system may also need to perform some additional data preprocessing work. For example, in order to ensure data consistency during training, the system can perform operations such as resampling and normalization on the samples; in order to enhance the generalization ability of the network, the system can also enhance the samples, such as adding noise, changing the volume, etc. Although these preprocessing steps do not directly fall within the scope of step S50116, they are equally important for generating a high-quality third audio sample set.

[0186] Finally, the carefully screened and annotated third audio sample set will be used in the noise recognition network training process. In this process, the network system will continuously improve its recognition ability by learning the noise characteristics in these samples. As the training progresses, the performance of the network will gradually improve, and it will eventually be able to effectively suppress various noise signals captured by smart microphones in practical applications, providing users with a clearer and purer voice communication experience.

[0187] As an implementation manner, the step S200, loading the microphone audio to be noise suppressed into a target noise recognition network, and obtaining an audio noise annotation result set corresponding to the microphone audio to be noise suppressed, may specifically include: loading the microphone audio to be noise suppressed into several target noise recognition networks, and obtaining several audio noise annotation result sets corresponding to the microphone audio to be noise suppressed, different target noise recognition networks correspond to different candidate noise type sets, and a target noise recognition network is configured to determine the target noise type corresponding to the microphone audio to be noise suppressed in the corresponding candidate noise type set.

[0188] Step S200 inputs the audio data to be processed into multiple target noise recognition networks in parallel to achieve accurate labeling of different types of noise.

[0189] During the specific implementation, the computer system prepares several fully trained target noise recognition networks. These networks are each focused on identifying different types of noise, that is, they correspond to different candidate noise type sets. For example, network A is specifically used to identify environmental noise, and its candidate noise type set includes natural sounds such as wind and rain; network B focuses on the recognition of mechanical noise, and its candidate noise type set includes sounds generated by mechanical operation such as air conditioners and printers; and network C is configured to identify human voice interference, especially background conversations of non-target speakers.

[0190] When a microphone audio to be noise suppressed is captured, the computer system does not send it to a single noise recognition network for processing, but loads the audio into all the target noise recognition networks mentioned above in parallel. Each network will analyze the audio independently and find and determine the type of noise in the audio from the corresponding candidate noise type set based on its internally trained parameters and model structure.

[0191] As the processing progresses, each target noise recognition network will output an audio noise annotation result set. These result sets not only contain the location information of the noise (such as the timestamp range), but also clearly indicate the type of noise. Because different networks focus on different types of noise, these result sets will be differentiated in content, and together constitute a comprehensive noise annotation of the original audio.

[0192] For example, suppose the original audio contains both wind and background conversation. Then, network A, which focuses on environmental noise recognition, will output a labeling result set indicating the presence of wind in a certain period of time in the audio; while network C, which focuses on human voice interference recognition, will output another labeling result set, marking the presence of background conversation and its time range. Together, these two result sets will provide important basis for subsequent noise suppression processing.

[0193] Through this parallel processing method, the computer system not only improves the efficiency of noise recognition, but also enhances the accuracy and robustness of recognition. Because each network is optimized and trained for a specific type of noise, they are able to perform at their best in their respective fields. At the same time, due to the parallel work of multiple networks, the system can quickly complete a comprehensive analysis of complex audio signals, making real-time noise suppression possible.

[0194] The present application embodiment provides a computer system, such as Figure 2 As shown, the computer system 100 includes: a processor 101 and a memory 103. The processor 101 and the memory 103 are connected, such as through a bus 102. Optionally, the computer system 100 may also include a transceiver 104. It should be noted that in actual applications, the transceiver 104 is not limited to one, and the structure of the computer system 100 does not constitute a limitation on the embodiments of the present application.

[0195] An embodiment of the present application provides a computer system. The computer system in the embodiment of the present application includes: one or more processors; a memory; and one or more computer programs, wherein the one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more programs are executed by the processor, the above method is implemented.

Claims

1. A method for suppressing noise of an intelligent microphone, characterized in that: The method comprises: Acquire the microphone audio to be subjected to noise suppression; Loading the microphone audio to be subjected to noise suppression into a target noise recognition network to obtain an audio noise annotation result set corresponding to the microphone audio to be subjected to noise suppression; Performing noise suppression on the microphone audio to be subjected to noise suppression in combination with the audio noise annotation result set; The target noise recognition network is calibrated by the following steps: Acquire a first audio sample set, where the first audio sample set is an unlabeled audio sample set; Loading each first audio sample in the first audio sample set into the noise recognition network to be adjusted, and obtaining an original noise labeling result set corresponding to each first audio sample; the original noise labeling result set corresponding to the first audio sample includes the original noise labeling results output by each noise recognition component in the noise recognition network to be adjusted and the noise types corresponding to the original noise labeling results; For the same original noise labeling result set, according to the noise labeling result commonality coefficient between the original noise labeling results corresponding to the same noise type, the original noise labeling result output by the current noise recognition component is paired with the original noise labeling results output by each of the other noise recognition components to obtain the paired noise labeling result corresponding to the original noise labeling result output by the current noise recognition component; Integrate the original noise labeling result and the corresponding paired noise labeling results to obtain a pseudo noise labeling result, and obtain a pseudo noise labeling result set corresponding to the first audio sample according to the pseudo noise labeling results corresponding to the same first audio sample; According to the first audio sample set and the corresponding pseudo noise labeling result sets, the noise recognition network to be calibrated is optimized to obtain a target noise recognition network; Among them, for the same original noise labeling result set, according to the noise labeling result commonality coefficient between the original noise labeling results corresponding to the same noise type, the original noise labeling results output by the current noise recognition component are paired with the original noise labeling results output by each of the other noise recognition components to obtain the paired noise labeling results corresponding to the original noise labeling results output by the current noise recognition component, including: For the current original noise labeling result set, determine a first noise labeling result from the original noise labeling results output by the current noise recognition component, and determine a second noise labeling result from the original noise labeling results output by the remaining noise recognition components, wherein the first noise labeling result and the second noise labeling result correspond to the same noise type; Obtaining a noise labeling result commonality coefficient between the first noise labeling result and the second noise labeling result according to a noise labeling result intersection range between the first noise labeling result and the second noise labeling result; Determine that the second noise labeling result corresponding to the commonality coefficient of the noise labeling results greater than the preset commonality coefficient is the pending noise labeling result, and determine that the pending noise labeling result corresponding to the maximum noise labeling result commonality coefficient is the paired noise labeling result corresponding to the first noise labeling result.

2. The method according to claim 1, characterized in that The method further comprises: Acquire a second audio sample set, where the second audio sample set is a labeled audio sample set; Loading each second audio sample in the second audio sample set into the noise recognition network to be pre-trained, and obtaining a corresponding inference noise labeling result set for each second audio sample; the inference noise labeling result set includes a subset of inference noise labeling results output by each noise recognition component in the noise recognition network to be pre-trained, and the subset of inference noise labeling results includes an inference noise labeling result output by the same noise recognition component, and a noise type and probability corresponding to the inference noise labeling result; According to the errors between each annotated noise labeling result set corresponding to the same second audio sample and each corresponding subset of inference noise labeling results, obtaining the component cost values ​​corresponding to each noise recognition component of the second audio sample; the component cost values ​​include the noise labeling result cost value, the noise type cost value and the probability cost value; Obtain component cost values ​​corresponding to each noise recognition component according to component cost values ​​corresponding to each second audio sample on the same noise recognition component; The internal configuration parameters of the components corresponding to the noise recognition components in the noise recognition network to be pre-trained are updated according to the component cost values, and the process stops when the first calibration stop requirement is met to obtain the noise recognition network to be calibrated.

3. The method according to claim 1, characterized in that The step of optimizing the noise recognition network to be calibrated according to the first audio sample set and the corresponding pseudo noise labeling result sets to obtain a target noise recognition network includes: Determine a target cost value corresponding to the target noise recognition component according to an error between an original noise labeling result subset and a corresponding pseudo noise labeling result set outputted by the same first audio sample based on the same target noise recognition component, and obtain a target cost value corresponding to each target noise recognition component; According to the target cost value corresponding to the target noise recognition component, the original component internal configuration parameters corresponding to the target noise recognition component in the noise recognition network to be adjusted are updated to obtain the component modified internal configuration parameters corresponding to the target noise recognition component; According to the component-corrected internal configuration parameters corresponding to each of the target noise recognition components, the original component-internal configuration parameters of the remaining noise recognition components other than the target noise recognition component in the noise recognition network to be adjusted are updated to obtain the component-corrected internal configuration parameters corresponding to the remaining noise recognition components; The internal configuration parameters are corrected according to the components corresponding to each noise recognition component to obtain the target noise recognition network.

4. The method according to claim 3, characterized in that The step of determining the target cost value corresponding to the target noise recognition component based on the error between the original noise labeling result subset and the corresponding pseudo noise labeling result set outputted by the same first audio sample based on the same target noise recognition component, and obtaining the target cost value corresponding to each target noise recognition component, comprises: For the same original noise labeling result set, according to the noise labeling result commonality coefficient between the original noise labeling results corresponding to the same noise type, the original noise labeling results output by the current target noise recognition component are paired with the original noise labeling results output by each of the other target noise recognition components to obtain the paired results corresponding to the original noise labeling results output by the current target noise recognition component; For the current first audio sample, based on the original noise labeling result subset output by the current target noise recognition component, according to the error between the original noise labeling result in the original noise labeling result subset and the corresponding constrained noise labeling result, obtain the original component cost value corresponding to each original noise labeling result in the original noise labeling result subset; if the pairing result indicates that it is paired, then the constrained noise labeling result corresponding to the original noise labeling result is the paired noise labeling result; if the pairing result indicates that it is not paired, then the constrained noise labeling result corresponding to the original noise labeling result is the corresponding pseudo noise labeling result in the pseudo noise labeling result set; Determine the importance coefficient of the cost value corresponding to the original noise labeling result according to the pairing result corresponding to the original noise labeling result; the importance coefficient of the cost value corresponding to the pairing is greater than the importance coefficient of the cost value corresponding to the unpairing; According to the cost value importance coefficient corresponding to the original noise labeling result, weighted sum of the original component cost values ​​corresponding to each original noise labeling result in the original noise labeling result subset is performed to obtain the target component cost value corresponding to the current first audio sample on the current target noise recognition component; Obtaining a target cost value corresponding to the current target noise recognition component according to the target component cost value corresponding to each first audio sample on the current target noise recognition component; The method of updating the original component internal configuration parameters of the remaining noise recognition components other than the target noise recognition component in the noise recognition network to be adjusted according to the component corrected internal configuration parameters corresponding to each of the target noise recognition components, and obtaining the component corrected internal configuration parameters corresponding to the remaining noise recognition components, includes: Obtaining component importance coefficients corresponding to respective noise identification components, wherein the component importance coefficient corresponding to the target noise identification component is less than the component importance coefficients corresponding to the remaining noise identification components; According to the component importance coefficients corresponding to the respective noise identification components, the component modified internal configuration parameters corresponding to the respective target noise identification components and the original component internal configuration parameters corresponding to the remaining noise identification components are fused to obtain the component modified internal configuration parameters corresponding to the remaining noise identification components.

5. The method according to any one of claims 1 to 4, characterized in that: The step of optimizing the noise recognition network to be calibrated according to the first audio sample set and the corresponding pseudo noise labeling result sets to obtain a target noise recognition network includes: According to the first audio sample set and the corresponding pseudo-noise labeling result sets, updating the internal configuration parameters of the noise recognition network to be adjusted to obtain a temporary noise recognition network; The temporary noise recognition network is used as the noise recognition network to be adjusted, and the process jumps to the step of obtaining the first audio sample set for iteration, and stops when the second adjustment stop requirement is met to obtain the target noise recognition network.

6. The method according to claim 5, characterized in that The updating of the internal configuration parameters of the proposed noise recognition network according to the first audio sample set and the corresponding pseudo noise labeling result sets to obtain a temporary noise recognition network includes: Acquire a third audio sample set, wherein the third audio sample set is a labeled audio sample set; According to the first audio sample set and the corresponding pseudo noise labeling result sets, and the third audio sample set, the internal configuration parameters of the proposed noise recognition network are updated to obtain a temporary noise recognition network.

7. The method according to claim 6, characterized in that The obtaining of the third audio sample set comprises: Acquire a fourth audio sample set, wherein the fourth audio sample set is an unlabeled audio sample set; Loading each fourth audio sample in the fourth audio sample set into the noise recognition network to be adjusted, and obtaining an original noise labeling result set corresponding to each fourth audio sample; the original noise labeling result set corresponding to the fourth audio sample includes an original noise labeling result subset output by each noise recognition component in the noise recognition network to be adjusted, and the original noise labeling result subset includes an original noise labeling result output by the same noise recognition component and a probability corresponding to the original noise labeling result; Measuring the uncertainty of the probabilities corresponding to the original noise annotation results in the same original noise annotation result subset to obtain the inference clutter corresponding to each original noise annotation result subset, and integrating the inference clutter corresponding to the same fourth audio sample to obtain the integrated clutter corresponding to each fourth audio sample; Performing clutter segmentation according to the integrated clutter corresponding to each fourth audio sample to obtain a plurality of clutter value ranges, and determining a target value range in the plurality of clutter value ranges; Determine a target audio sample to be annotated from the fourth audio sample whose integrated clutter is within the target value range, perform noise annotation on the target audio sample, and obtain an annotated noise labeling result set corresponding to the target audio sample; A third audio sample set is obtained according to each target audio sample and the corresponding annotated noise labeling result set.

8. The method according to claim 1, characterized in that The step of loading the microphone audio to be subjected to noise suppression into a target noise recognition network to obtain an audio noise annotation result set corresponding to the microphone audio to be subjected to noise suppression includes: loading the microphone audio to be subjected to noise suppression into several target noise recognition networks to obtain several audio noise annotation result sets corresponding to the microphone audio to be subjected to noise suppression, different target noise recognition networks correspond to different candidate noise type sets, and one target noise recognition network is configured to determine a target noise type corresponding to the microphone audio to be subjected to noise suppression in the corresponding candidate noise type set.

9. A computer system, characterized in that: include: one or more processors; Memory; one or more computer programs; The one or more computer programs are stored in the memory and configured to be executed by the one or more processors, and when the one or more computer programs are executed by the processors, the method according to any one of claims 1 to 8 is implemented.