Training method of audio noise detection model, and audio noise detection method and device

Abstract

The invention provides a training method of an audio noise detection model. The method comprises the following steps: acquiring multiple pieces of audio data; performing sound feature extraction on the audio data to obtain frequency domain features, short-time noise energy and an A-weighted sound pressure level of the audio data; fusing the frequency domain feature, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data; and training an original self-supervised learning model based on the comprehensive feature vector to obtain an audio noise detection model. The audio noise detection model obtained by applying the method can automatically detect the audio and judge whether noise exists in the audio, and has high detection efficiency and accuracy.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, and in particular to a training method for an audio noise detection model, an audio noise detection method and a device. Background Art

[0002] During the manufacturing process of computer hardware, there may be electromagnetic interference between the audio signal transmission line and the wiring such as the power supply unit cable. This interference will cause noise in the audio output, that is, background noise, which will affect the audio quality and the user experience. To address this problem, the current detection method is to first manually insert the headphones into the device, and then use human hearing to determine whether there is background noise in the audio output. Or use professional equipment to simulate the human ear for testing. Human hearing relies on subjective factors, and the detection accuracy and efficiency are easily affected. Professional equipment is expensive and requires professional technicians to operate. Summary of the invention

[0003] The present disclosure provides a training method for an audio noise detection model, an audio noise detection method and a device, so as to at least solve the above technical problems existing in the prior art.

[0004] According to a first aspect of the present disclosure, a training method for an audio noise detection model is provided, the method comprising: acquiring multiple audio data; performing sound feature extraction on the audio data to obtain frequency domain features, short-time noise energy, and A-weighted sound pressure level of the audio data; fusing the frequency domain features, the short-time noise energy, and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data; and training an original self-supervised learning model based on the comprehensive feature vector to obtain an audio noise detection model.

[0005] In one possible implementation, sound features are extracted from the audio data to obtain frequency domain features of the audio data, including: performing frequency domain conversion on the audio data to obtain a frequency domain signal; and determining the frequency domain features of the audio data based on energy distribution of the frequency domain signal.

[0006] In one possible implementation, sound features are extracted from the audio data to obtain short-time noise energy of the audio data, including: segmenting the audio data to obtain multiple short-time frames; and determining the short-time noise energy of the audio data based on the mean square value of each short-time frame.

[0007] In one possible implementation, performing sound feature extraction on the audio data to obtain the A-weighted sound pressure level of the audio data includes: performing weighted processing on the audio data to obtain weighted audio data; determining a root mean square value of the weighted audio data; and performing sound pressure level extraction on the weighted audio data based on the root mean square value to obtain the A-weighted sound pressure level of the audio data.

[0008] In one possible implementation manner, the fusing the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data includes: determining weighting coefficients corresponding to the frequency domain features, the short-time noise energy and the A-weighted sound pressure level respectively; and based on the weighting coefficients, performing weighted fusion of the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data.

[0009] In one possible implementation manner, the original self-supervised learning model is trained based on the comprehensive feature vector to obtain the audio noise detection model, including: fine-tuning the original self-supervised learning model based on the comprehensive feature vector to obtain the fine-tuned original self-supervised learning model; adjusting the learning rate of the fine-tuned original self-supervised learning model to obtain the audio noise detection model.

[0010] In one possible implementation, the multiple audio data include virtual negative sample data, and obtaining the multiple audio data includes: obtaining real sample data; performing data enhancement on the real negative sample data in the real sample data to obtain virtual negative sample data; and obtaining multiple audio data based on the real sample data and the virtual negative sample data.

[0011] According to a second aspect of the present disclosure, an audio noise detection method is provided, the method comprising: obtaining audio data to be detected; inputting the audio data to be detected into an audio noise detection model to obtain a detection result, the audio noise detection model being trained by the method described in the above embodiment.

[0012] According to a third aspect of the present disclosure, an audio noise detection device is provided, comprising: an audio acquisition module for acquiring audio data to be detected; an audio detection module for inputting the audio data to be detected into an audio noise detection model to obtain a detection result, wherein the audio noise detection model is trained by the method described in the above embodiment.

[0013] The present invention discloses a training method for an audio noise detection model. The method first obtains a plurality of audio data. Then, sound features are extracted from the audio data to obtain frequency domain features, short-time noise energy, and A-weighted sound pressure level of the audio data. The three are then fused to obtain a comprehensive feature vector corresponding to the audio data. Finally, the original self-supervised learning model is trained based on the comprehensive feature vector to obtain an audio noise detection model. The present invention extracts multi-dimensional features to form a comprehensive feature vector, so that the model can capture the characteristics of the sound more comprehensively, thereby improving the accuracy of noise detection. By adopting a self-supervised learning model, unlabeled audio data can be used for training, thereby reducing the cost and difficulty of data standards.

[0014] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0016] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0017] Figure 1 A schematic diagram of the implementation process of a training method for an audio noise detection model according to an embodiment of the present disclosure is shown;

[0018] Figure 2 A schematic diagram of an application scenario of an embodiment of the present disclosure is shown;

[0019] Figure 3 A schematic diagram of spectrum energy distribution according to an embodiment of the present disclosure is shown;

[0020] Figure 4 A schematic diagram of RMS data distribution according to an embodiment of the present disclosure is shown;

[0021] Figure 5 A schematic diagram of A-weighted sound pressure level data distribution according to an embodiment of the present disclosure is shown;

[0022] Figure 6 A schematic diagram showing a detection result of an embodiment of the present disclosure Figure 1 ;

[0023] Figure 7 A schematic diagram showing a detection result of an embodiment of the present disclosure Figure 2 ;

[0024] Figure 8 A schematic diagram of the structure of an audio noise detection device according to an embodiment of the present disclosure is shown;

[0025] Fig. 9 A schematic diagram showing the composition structure of a training device for an audio noise detection model according to an embodiment of the present disclosure is shown;

[0026] Fig.10 A schematic diagram of the structure of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0027] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0028] Figure 1 A schematic diagram of the implementation process of a training method for an audio noise detection model according to an embodiment of the present disclosure is shown. Figure 1 As shown, according to a first aspect of an embodiment of the present disclosure, a training method for an audio noise detection model is provided, and the method includes:

[0029] Step 101, obtaining multiple audio data.

[0030] If the device under test has two jacks, microphone and headphone, such as Figure 2 As shown, a male-to-male audio cable can be used to insert into the two jacks to obtain the audio data of the device under test. If there is only one jack on the device under test, the two ends of the male-to-male audio cable can be first inserted into the two female interfaces of the one-to-two audio cable, and then the male connector of the one-to-two audio cable is inserted into the jack of the device under test to form a closed loop to obtain the audio data of the device under test. The collected audio data is then classified and labeled. As shown in Table 1 below, Table 1 lists audio data 1-audio data 8 obtained from devices under test 1, 2, 3, and 4, as well as the labels corresponding to each audio data, including failed (noise exists) and passed (no noise exists):

[0031]

[0032]

[0033] Table 1

[0034] This method only requires one male-to-male audio cable and one split-to-two audio cable to complete the audio data collection, which greatly reduces the equipment cost and reduces the complexity of operation and maintenance. In addition, this solution uses audio signal re-recording to obtain audio data, which does not involve external environmental interference, greatly reducing the traditional solution's dependence on the silent room. It has strong flexibility and scalability, and can adapt to the audio noise detection needs in different scenarios.

[0035] In one possible implementation, in order to reduce the cost of obtaining audio data, audio data related to electromagnetic interference can be directly obtained from audio websites such as Freesound.io.

[0036] In one embodiment, after the audio data is obtained, the audio data is first preprocessed. The preprocessing mainly includes cutting the audio data, removing unnecessary parts, and retaining only the core areas useful for analysis. For example, the silent part, the beginning, and the ending part are removed. Removing the silent part can make the processed audio data more compact and efficient. Removing the beginning and the ending part can avoid the impact of the non-steady-state signal generated when the recording device (such as the CODEC) is powered on and off on the audio data. Make the preprocessed audio data more reliable and accurate. At the same time, improve processing efficiency and reduce computing resource consumption.

[0037] Step 102: extract sound features from the audio data to obtain frequency domain features, short-time noise energy, and A-weighted sound pressure level of the audio data.

[0038] Frequency characteristics refer to the manifestation of audio signals in the frequency domain, reflecting the energy distribution of audio signals at different frequencies. Under normal circumstances, the spectral energy distribution of audio signals will show a specific pattern, reflecting its inherent frequency changes. When there is noise in the audio signal, additional energy is usually introduced in certain specific frequency bands, resulting in abnormal spectrum. Therefore, by analyzing the changes in the spectral energy distribution, these abnormal energies caused by noise can be detected, and then it can be determined whether the audio data is noisy. Specifically, the audio data can be converted from the time domain to the frequency domain through Fourier transform. Then, the converted frequency domain signal is subjected to spectral analysis to obtain frequency domain features such as spectrum, spectral centroid, and spectral roll-off.

[0039] Short-term noise energy refers to the measure of the noise signal energy in each frame after the audio signal is processed by frame division and windowing. It can be obtained by calculating the sum of squares (or average square value) of signal samples in each frame.

[0040] The A-weighted sound pressure level not only takes into account the size of the sound pressure level, but also simulates the difference in sensitivity of the human ear to sounds of different frequencies by introducing an A-weighted filter, which is closer to the actual perception of the human ear. Specifically, the audio signal can be decomposed and reconstructed using time-frequency analysis methods such as wavelet transformation and short-time Fourier transform to obtain detailed information of the audio signal in the time domain and frequency domain, and then the A-weighted sound pressure level can be calculated in combination with the A-weighted filter.

[0041] Step 103: Fusing the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data.

[0042] The frequency domain features, short-term noise energy and A-weighted sound pressure level extracted in step 102 are fused to form a comprehensive feature vector. The comprehensive feature vector can reflect various characteristics of the audio data, including its frequency distribution, energy distribution and subjectively perceived noise level.

[0043] Step 104: training the original self-supervised learning model based on the comprehensive feature vector to obtain an audio noise detection model.

[0044] First, the obtained comprehensive feature vector can be divided into a training set and a test set. The training set is used to train the model, and the test set is used to test the model. The division ratio can be set according to the actual situation, for example, 70% is used for the training set and 30% is used for the test set. The training set is used as input to train the original self-supervised learning model, and the trained model is the audio noise detection model.

[0045] The audio noise detection model trained by this method can realize the automated detection process without manually configuring a professional test environment or relying on manual auditory judgment, eliminating the interference of human factors and improving the reliability and consistency of detection. It also supports simultaneous detection of multiple machines, improves the detection efficiency of the production line, and reduces the defect rate caused by missed detection. By extracting multi-dimensional features such as frequency domain features, short-term noise energy, and A-weighted sound pressure level of audio data and fusing them into a comprehensive feature vector, the model can capture the characteristics of the sound more comprehensively, thereby improving the accuracy of noise detection. Using a self-supervised learning model, unlabeled audio data can be used for training, thereby reducing the cost and difficulty of data standards.

[0046] In one embodiment of the present disclosure, the multiple audio data include virtual negative sample data, and the obtaining of the multiple audio data includes: obtaining real sample data; performing data enhancement on the real negative sample data in the real sample data to obtain virtual negative sample data; and obtaining multiple audio data based on the real sample data and the virtual negative sample data.

[0047] The real sample data is the real audio data obtained from the device or website, including real positive sample data and real negative sample data. Real negative sample data refers to audio data containing background noise. Generally speaking, the number of real negative sample data is much less than the number of real positive sample data. In order to avoid the model being biased towards the majority class (i.e., real positive sample data) during training, in this embodiment, data enhancement processing is performed on the real negative sample data to balance the category distribution in the audio data set. Specifically, the number of real negative sample data can be expanded by methods such as generative adversarial networks and traditional audio synthesis technology.

[0048] Taking the WaveNet generative adversarial network as an example, a specific explanation is given. First, a generator is trained. The generator uses the real negative sample data as the starting point and produces virtual negative sample data through the WaveNet model. Then, both the real negative sample data and the virtual negative sample data are input into the discriminator, and the discriminator is trained to correctly distinguish between the real negative sample data and the virtual negative sample data. The generator adjusts its parameters according to the feedback of the discriminator. Through the adversarial training between the generator and the discriminator, the generator generates more realistic virtual negative sample data. The discriminator continuously improves its recognition ability. As the adversarial training proceeds, the generator can eventually generate high-quality virtual negative sample data, which are highly similar to the real negative sample data. These virtual negative sample data are combined with the original real sample data (including real positive sample data and real negative sample data) to construct a larger and more diverse audio data set. It can not only enhance the training basis of the model, but also improve the generalization ability and practical application effect of the audio noise detection model.

[0049] In one embodiment of the present disclosure, sound features are extracted from the audio data to obtain frequency domain features of the audio data, which can be specifically achieved through the following technical means: performing frequency domain conversion on the audio data to obtain a frequency domain signal; and determining the frequency domain features of the audio data based on the energy distribution of the frequency domain signal.

[0050] Specifically, Fast Fourier Transform (FFT) can be used to perform frequency domain conversion on audio data, converting the audio signal that changes continuously on the time axis to the frequency axis to obtain its distribution at each frequency point, that is, the frequency domain signal. Then, based on the energy distribution of the frequency domain signal, the energy intensity at different frequency bands is calculated. The energy intensity information reflects the characteristics of the audio signal at different frequencies, that is, the frequency domain characteristics of the audio data. Figure 3 As shown, it is the energy distribution of the frequency domain signal (frequency domain characteristics).

[0051] In one embodiment of the present disclosure, sound features are extracted from the audio data to obtain the short-time noise energy of the audio data, which can be specifically achieved through the following technical means: segmenting the audio data to obtain multiple short-time frames; and determining the short-time noise energy of the audio data based on the mean square value of each short-time frame.

[0052] First, the audio data is divided into multiple continuous short-time frames, each of which contains a sample of the audio data in a short period of time. Then, based on the sample values ​​in each short-time frame, its mean square value is calculated, that is, the average value of the sum of squares of all sample values. The mean square value reflects the overall energy level of the audio data in the short-time frame. Therefore, by calculating the mean square value of each short-time frame, the short-time noise energy of the audio data at different time points can be determined.

[0053] Specifically, the mean square value of the i-th frame, that is, the short-time noise energy, can be calculated by the following formula (1):

[0054]

[0055] Among them, E(i) represents the short-term noise energy of the i-th frame, y i (n) represents the value of the nth sample in the i-th frame, and L represents the frame length.

[0056] Compared with other complex audio feature extraction methods, Short-time Noise Energy (SNE) does not require tedious frequency conversion or complex data transformation. It only evaluates the short-time energy of noise by dividing the audio data into multiple short-time frames and calculating the sum of squares (i.e., mean square value) of the sample values ​​in each frame. Not only is it simple to implement, but it can also accurately capture the dynamic changes of noise levels, while saving computing resources and time, which can improve the efficiency of noise detection.

[0057] In one embodiment of the present disclosure, sound feature extraction is performed on the audio data to obtain the A-weighted sound pressure level of the audio data, which can be specifically achieved through the following technical means: weighted processing is performed on the audio data to obtain weighted audio data; a root mean square value of the weighted audio data is determined; and based on the root mean square value, sound pressure level extraction is performed on the weighted audio data to obtain the A-weighted sound pressure level of the audio data.

[0058] First, based on the simulation of the human ear's sensitivity to sounds of different frequencies, the original audio data is weighted so that the weighted audio data can better reflect the volume actually felt by the human ear. Then the root mean square (RMS) of the weighted audio data is calculated. RMS can reflect the strength of the audio signal. The higher the RMS value, the greater the power of the audio signal, that is, the louder the volume. Finally, based on the RMS, the sound pressure level is extracted to obtain the A-weighted sound pressure level.

[0059] Specifically, the original audio data is processed by an A-weighted filter. The A-weighted filter performs weighted processing on each frequency component in the audio data according to the difference in sensitivity of the human ear to sounds of different frequencies, so that signals of some frequency bands are attenuated and signals of some frequency bands are enhanced. This can be achieved specifically by the following formula (2):

[0060]

[0061] Among them, f represents the frequency, f 1 =20.6Hz, f 2 =107.7Hz, f 3 =737.9Hz, f 4 =12194Hz respectively represent the key frequency points used in the filter, A 1000 is a normalization constant used to ensure that the gain of the A-weighted filter is 0 dB at 1000 Hz.

[0062] The RMS value of the weighted audio data is calculated using the following formula (3):

[0063]

[0064] Where N represents the number of samples in the signal, x i Characterize the amplitude of each sample.

[0065] For example, the RMS value shown in Table 2 below is obtained, and the corresponding data distribution diagram is shown in Table 2. Figure 4 As shown:

[0066]

[0067]

[0068] Finally, based on the obtained RMS value, the sound pressure level (SPL) of the weighted audio data is extracted to obtain the A-weighted sound pressure level. The calculation formula is as follows:

[0069]

[0070] Among them, p 0 is the reference sound pressure, which is about 20 microPa; Characterizes the RMS sound pressure.

[0071] For example, the A-weighted sound pressure values ​​shown in Table 3 below are obtained, and the corresponding data distribution diagram is shown in Table 3 below. Figure 5 As shown:

[0072] Data name RMS Audio data 1 24.9225784 Audio data 2 16.9166285 Audio data 3 25.0050051 Audio Data 4 36.1534022 Audio data 5 -3.2294921 Audio Data 6 -3.1685647 Audio Data 7 5.0164123 Audio data 8 -4.4388447

[0073] Table 3

[0074] In one embodiment of the present disclosure, the frequency domain features, the short-time noise energy and the A-weighted sound pressure level are fused to obtain a comprehensive feature vector corresponding to the audio data, including the following steps: determining weighting coefficients corresponding to the frequency domain features, the short-time noise energy and the A-weighted sound pressure level respectively; based on the weighting coefficients, performing weighted fusion of the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data.

[0075] The best weighting coefficients can be determined by conducting experiments, using grid search, etc. In one embodiment, the best weighting coefficients are: w1 = 0.3 (short-term noise energy), w2 = 0.4 (A-weighted sound pressure level), w3 = 0.3 (frequency domain characteristics).

[0076] Then, based on the weight coefficients corresponding to each eigenvalue, a weighted sum is performed to form a comprehensive eigenvector. Specifically, it can be obtained by the following formula (5):

[0077] Comprehensive feature vector = w1*feature 1+w2*feature 2+…+wn*feature n(5)

[0078] Among them, w1, w2...wn represent the weight coefficients of each feature, and feature 1, feature 2...feature n are the extracted features.

[0079] In one embodiment of the present disclosure, the original self-supervised learning model is trained based on the comprehensive feature vector to obtain the audio noise detection model, including: fine-tuning the original self-supervised learning model based on the comprehensive feature vector to obtain the fine-tuned original self-supervised learning model; adjusting the learning rate of the fine-tuned original self-supervised learning model to obtain the audio noise detection model.

[0080] Specifically, the original self-supervised learning model is an audio representation learning model based on self-supervised learning, such as the Wav2Vec 2.0 model, the HuBER model, etc. Taking the Wav2Vec 2.0 model as an example, the pre-trained Wav2Vec 2.0 model is first loaded using the PyTorch framework as a starting point. This pre-trained model performs self-supervised learning on a wide range of audio data and can capture common features in audio signals. In order to adapt the model to a specific audio noise detection task, the pre-trained model is fine-tuned using a comprehensive feature vector. The fine-tuning process includes freezing some layers of the model, which are usually responsible for extracting common audio features, and keeping their parameters unchanged at the beginning of training to retain useful information in the training model. Then, through a phased unfreezing strategy, the other layers of the model are gradually adjusted so that the model can learn and adapt to the data distribution and characteristics of the audio noise detection task.

[0081] In the fine-tuning process, in order to optimize the training efficiency and model performance, an adaptive learning rate adjustment strategy, such as the Aadm optimizer, is used. The model dynamically adjusts the learning rate during the training process, thereby accelerating convergence in the early stages of training and making stable adjustments when approaching the optimal solution to avoid overfitting or falling into a local optimum. Finally, a fine-tuned and optimized audio noise detection model is obtained.

[0082] In one embodiment of the present disclosure, after obtaining a trained audio noise detection model, a portion of the audio data set is used as a test set to evaluate the performance of the model. Specifically, the accuracy, precision, recall and F1 score (the harmonic mean of precision and recall) of the model can be calculated.

[0083] Among them, the precision rate is the ratio of the number of samples correctly predicted by the model to the total number of samples, and its calculation formula is:

[0084] The precision rate is the ratio of the number of samples predicted by the model as positive samples and correctly predicted to the number of samples predicted by the model as positive samples. It reflects the reliability of the model's prediction of positive samples, and its calculation formula is:

[0085] The recall rate is the ratio of the number of samples predicted by the model as positive samples and correctly predicted to all actual positive samples. It reflects the ability of the model to find positive samples, and its calculation formula is:

[0086] The F1 score is the harmonic mean of precision and recall, which is used to strike a balance between precision and recall. The calculation formula is as follows:

[0087] Among them, “True Positives (TP)” is the number of positive samples correctly predicted by the model, “False Positives (FP)” is the number of positive samples incorrectly predicted by the model, and “False Negatives (FN)” is the number of negative samples incorrectly predicted by the model.

[0088] By obtaining the above indicators, we can get the performance evaluation information of the model. For example, a high accuracy rate indicates that the model has strong overall prediction ability, a high precision rate indicates that the model has high reliability in predicting positive samples, a high recall rate indicates that the model can find most of the positive samples, and a high F1 score means that the model has achieved a good balance between precision and recall.

[0089] In summary, the training method of the audio noise detection model disclosed in the present invention has the following advantages:

[0090] 1. This solution innovatively adopts a multi-source audio feature fusion strategy, integrating frequency domain features, short-term noise energy, and A-weighted sound pressure level features through a weighted average method. It enriches the feature dimensions, enhances the model's adaptability to complex audio environments, and can accurately capture and evaluate the dynamic changes of background noise at different time frequencies and times, providing a more comprehensive and accurate information basis for subsequent background noise detection.

[0091] 2. Make full use of existing audio data resources, such as the electromagnetic interference related data set in Freesouund.io, to effectively reduce the cost of data acquisition. At the same time, by producing adversarial networks for data enhancement, the diversity of training samples is significantly expanded, and the robustness of the model is improved. In addition, with the help of the trained deep learning model, combined with transfer learning and fine-tuning technology, the model can quickly adapt to specific audio interference detection, further improving the model training efficiency and model performance.

[0092] 3. This solution introduces a self-supervised learning mechanism. Through self-learning of unlabeled data, the model automatically discovers the inherent rules and structures in the audio signal, thereby continuously optimizing its own performance without supervision. This not only reduces the cost and difficulty of data labeling, but also improves the generalization ability and adaptability of the model.

[0093] 4. In terms of hardware requirements, this solution only requires simple audio cables (such as male-to-male audio cables and one-to-two audio cables), which greatly reduces the cost of equipment. At the same time, due to its simplicity and easy access, it also reduces the complexity of operation and maintenance, making this solution have a wider application prospect and market competitiveness.

[0094] 5. For noisy and diverse sound environments such as factories, this solution uses audio signal playback to obtain audio data. This method does not rely on external environmental conditions. It effectively reduces the traditional solution's dependence on silent rooms. At the same time, combined with technical means such as multi-source audio feature fusion and self-supervised learning, this solution can better adapt to complex and changing audio environments and achieve efficient and accurate noise detection and processing.

[0095] According to another aspect of the present disclosure, an audio noise detection method is provided, the method comprising: acquiring audio data to be detected; inputting the audio data to be detected into an audio noise detection model to obtain a detection result, the audio noise detection model being trained by the method described in the above embodiment.

[0096] Deploy the trained audio noise detection model to the device or detection device under test. Use a male-to-male audio cable, or a one-to-two audio cable to insert it into the device under test to form a closed loop. Then obtain the audio data to be detected in real time and input it into the deployed detection model for detection. The detection result of whether the audio data includes background noise or not is obtained, and the result is displayed. If the detection result shows that the background noise is included, the production line personnel are prompted to check the equipment and automatically record the background noise data. For example, refer to Figure 6 As shown in the figure, the detection result in the dotted box is "Test FAIL", indicating that the audio data to be detected contains background noise. Figure 7 As shown in the figure, the detection result in the dotted box is "TestPASS", indicating that the audio data to be detected does not include background noise and passes the detection.

[0097] Compared with the traditional manual testing method, this solution realizes an automatic and objective testing process. There is no need to manually configure the professional equipment test environment or rely on auditory judgment. This avoids interference and errors caused by human factors. This automated testing method not only improves the reliability and consistency of testing, but also significantly improves production efficiency and reduces the defect rate through simultaneous testing of multiple machines.

[0098] According to another aspect of the present disclosure, an audio noise detection device is provided, wherein the device is equipped with an audio noise detection model, such as Figure 8 As shown, the device comprises:

[0099] The audio acquisition module 801 is used to acquire the audio data to be detected;

[0100] The audio detection module 802 is used to input the audio data to be detected into an audio noise detection model to obtain a detection result. The audio noise detection model is trained by the method described in the above embodiment.

[0101] According to another aspect of the present disclosure, a training device for an audio noise detection model is provided, such as Fig. 9 As shown, the device comprises:

[0102] A data acquisition module 901 is used to acquire multiple audio data;

[0103] A feature extraction module 902 is used to extract sound features from the audio data to obtain frequency domain features, short-time noise energy, and A-weighted sound pressure level of the audio data;

[0104] A feature fusion module 903 is used to fuse the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data;

[0105] The model training module 904 is used to train the original self-supervised learning model based on the comprehensive feature vector to obtain an audio noise detection model.

[0106] In one embodiment of the present disclosure, the feature extraction module 902 is further configured to perform frequency domain conversion on the audio data to obtain a frequency domain signal; and determine the frequency domain features of the audio data based on the energy distribution of the frequency domain signal.

[0107] In one embodiment of the present disclosure, the feature extraction module 902 is further configured to segment the audio data to obtain a plurality of short-time frames; and determine the short-time noise energy of the audio data based on the mean square value of each short-time frame.

[0108] In one embodiment of the present disclosure, the feature extraction module 902 is further used to perform weighted processing on the audio data to obtain weighted audio data; determine the root mean square value of the weighted audio data; and based on the root mean square value, perform sound pressure level extraction on the weighted audio data to obtain the A-weighted sound pressure level of the audio data.

[0109] In one embodiment of the present disclosure, the feature fusion module 903 is also used to determine the weighting coefficients corresponding to the frequency domain features, the short-time noise energy and the A-weighted sound pressure level respectively; based on the weighting coefficients, the frequency domain features, the short-time noise energy and the A-weighted sound pressure level are weightedly fused to obtain a comprehensive feature vector corresponding to the audio data.

[0110] In one embodiment of the present disclosure, the model training module 904 is also used to fine-tune the original self-supervised learning model based on the comprehensive feature vector to obtain the fine-tuned original self-supervised learning model; and adjust the learning rate of the fine-tuned original self-supervised learning model to obtain an audio noise detection model.

[0111] In one embodiment of the present disclosure, the multiple audio data include virtual negative sample data, and the data acquisition module 901 is also used to acquire real sample data; perform data enhancement on the real negative sample data in the real sample data to obtain virtual negative sample data; and obtain multiple audio data based on the real sample data and the virtual negative sample data.

[0112] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0113] Fig.10 A schematic block diagram of an example electronic device 1000 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0114] like Fig.10 As shown, the device 1000 includes a computing unit 1001, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1002 or a computer program loaded from a storage unit 1008 into a random access memory (RAM) 1003. In the RAM 1003, various programs and data required for the operation of the device 1000 can also be stored. The computing unit 1001, the ROM 1002, and the RAM 1003 are connected to each other via a bus 1004. An input / output (I / O) interface 1005 is also connected to the bus 1004.

[0115] A number of components in the device 1000 are connected to the I / O interface 1005, including: an input unit 1006, such as a keyboard, a mouse, etc.; an output unit 1007, such as various types of displays, speakers, etc.; a storage unit 1008, such as a disk, an optical disk, etc.; and a communication unit 1009, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1009 allows the device 1000 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0116] The computing unit 1001 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1001 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1001 performs the various methods and processes described above, such as a training method for an audio noise detection model. For example, in some embodiments, a training method for an audio noise detection model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1008. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 1000 via the ROM 1002 and / or the communication unit 1009. When the computer program is loaded into the RAM 1003 and executed by the computing unit 1001, one or more steps of the training method for an audio noise detection model described above may be performed. Alternatively, in other embodiments, the computing unit 1001 may be configured to execute a training method for an audio noise detection model in any other appropriate manner (eg, by means of firmware).

[0117] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0119] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0120] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0121] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0122] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0123] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of this disclosure can be achieved, and this document is not limited here.

[0124] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0125] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A training method for an audio noise detection model, characterized in that: The method comprises: Get multiple audio data; Extracting sound features from the audio data to obtain frequency domain features, short-time noise energy, and A-weighted sound pressure level of the audio data; fusing the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data; The original self-supervised learning model is trained based on the comprehensive feature vector to obtain an audio noise detection model.

2. The method according to claim 1, characterized in that: Extracting sound features from the audio data to obtain frequency domain features of the audio data includes: Performing frequency domain conversion on the audio data to obtain a frequency domain signal; Based on the energy distribution of the frequency domain signal, the frequency domain characteristics of the audio data are determined.

3. The method according to claim 1, characterized in that Extracting sound features from the audio data to obtain short-term noise energy of the audio data includes: Segmenting the audio data to obtain a plurality of short-time frames; Based on the mean square value of each short-time frame, the short-time noise energy of the audio data is determined.

4. The method according to claim 1, characterized in that: Extracting sound features from the audio data to obtain an A-weighted sound pressure level of the audio data includes: Performing weighted processing on the audio data to obtain weighted audio data; Determining a root mean square value of the weighted audio data; Based on the RMS value, sound pressure level extraction is performed on the weighted audio data to obtain an A-weighted sound pressure level of the audio data.

5. The method according to claim 1, characterized in that The fusing the frequency domain features, the short-time noise energy and the A-weighted sound pressure level to obtain a comprehensive feature vector corresponding to the audio data includes: Determine the weighting coefficients corresponding to the frequency domain characteristics, short-time noise energy and A-weighted sound pressure level; Based on the weighting coefficients, the frequency domain features, short-time noise energy and A-weighted sound pressure level are weightedly fused to obtain a comprehensive feature vector corresponding to the audio data.

6. The method according to claim 1, characterized in that The method of training the original self-supervised learning model based on the comprehensive feature vector to obtain an audio noise detection model includes: Fine-tuning the original self-supervised learning model based on the comprehensive feature vector to obtain a fine-tuned original self-supervised learning model; The learning rate of the fine-tuned original self-supervised learning model is adjusted to obtain an audio noise detection model.

7. The method according to claim 1, characterized in that The multiple audio data include virtual negative sample data, and the acquiring the multiple audio data includes: Get real sample data; Performing data enhancement on real negative sample data in the real sample data to obtain virtual negative sample data; Based on the real sample data and the virtual negative sample data, a plurality of audio data are obtained.

8. An audio noise detection method, characterized in that: The method comprises: Get the audio data to be detected; The audio data to be detected is input into an audio noise detection model to obtain a detection result, wherein the audio noise detection model is trained by the method described in any one of claims 1 to 7.

9. An audio noise detection device, characterized in that: The device is deployed with an audio noise detection model, and the device comprises: An audio acquisition module, used to acquire audio data to be detected; An audio detection module is used to input the audio data to be detected into an audio noise detection model to obtain a detection result, wherein the audio noise detection model is trained by the method described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to make a computer execute the method according to any one of claims 1-7.

Images