A method and system for audio noise reduction based on deep learning

Abstract

The present invention relates to the technical field of audio noise reduction, and in particular to an audio noise reduction method and system based on deep learning. The method comprises the following steps: obtaining original audio and sound device operation data associated with the original audio; performing invalid audio elimination on the original audio to generate valid audio; performing audio time alignment on the valid audio to generate preprocessed standard audio; performing audio output mode detection on the preprocessed standard audio and the sound device associated with the original audio to generate single-device output audio and multi-device output audio; performing audio period regularity segmentation on the single-device output audio to generate single-device short-period output spectrum data and single-device long-period output spectrum data. The present invention improves the accuracy and real-time performance of the noise reduction effect by eliminating invalid audio, time alignment, layered noise extraction and abnormal noise identification.

Description

Technical Field

[0001] The present invention relates to the technical field of audio noise reduction, and in particular to an audio noise reduction method and system based on deep learning. Background Art

[0002] Initially, audio noise reduction mainly relied on traditional signal processing methods, such as spectral subtraction and Wiener filtering. These methods reduce noise components by analyzing the spectrum of audio signals, but the effect is limited, especially in complex background noise environments. With the rise of deep learning, audio noise reduction methods based on deep neural networks (DNNs) have begun to make breakthroughs. In 2014, deep learning was widely used in speech recognition and sound separation tasks, and deep neural networks demonstrated strong capabilities in feature learning. Early audio noise reduction methods used convolutional neural networks (CNNs) and recurrent neural networks (RNNs). By learning time series audio signals, effective noise features were automatically extracted, significantly improving the noise reduction effect. In recent years, with the continuous optimization of model structure and the improvement of computing power, deep learning technology has become the mainstream method for audio noise reduction, especially in real-time noise reduction and low-latency audio processing. However, the existing technologies usually deal with the noise of a single device, ignoring the interference in a multi-device environment. At the same time, the existing deep learning noise reduction models usually process the overall noise in a unified manner, lacking the detailed identification and processing of different noise types, resulting in low accuracy and real-time performance of the noise reduction effect. Summary of the invention

[0003] Based on this, it is necessary to provide an audio noise reduction method and system based on deep learning to solve at least one of the above technical problems.

[0004] To achieve the above object, a method for audio noise reduction based on deep learning is provided, the method comprising the following steps:

[0005] Step S1: obtaining original audio and sound device operation data associated with the original audio; removing invalid audio from the original audio to generate valid audio; performing audio time alignment on the valid audio to generate pre-processed standard audio;

[0006] Step S2: Perform audio output mode detection on the preprocessed standard audio and the sound-generating device associated with the original audio to generate single-device output audio and multi-device output audio; perform audio period regularity segmentation on the single-device output audio to generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise;

[0007] Step S3: extract the ground loop noise of the device cluster from the audio output of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data;

[0008] Step S4: Based on the abnormal noise pattern recognition data, the power spectrum of the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise is subtracted in each time frame to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

[0009] The present invention provides additional contextual information for noise analysis by combining audio and device operation data, which can help analyze the impact of device status on audio output, specific device failures, unstable performance, etc., thereby providing valuable reference for subsequent noise reduction. By removing invalid parts (such as background noise, silent segments, etc.) in the original audio, the interference of irrelevant information is reduced, and the effectiveness and computational efficiency of the subsequent audio noise reduction algorithm are ensured. It enables the system to focus more on the effective audio part that needs noise reduction, thereby improving the noise reduction accuracy. Audio time alignment eliminates the inconsistency caused by time deviation by ensuring the synchronization of multiple audio signals, and avoids inaccurate noise reduction results caused by time dislocation, which is particularly important for processing audio that sounds from multiple devices at the same time, and helps to provide more accurate input for subsequent noise separation and analysis. By detecting the audio output modes of single devices and multiple devices, it helps to classify and process noise according to the number and configuration of devices. Specifically, in a multi-device environment, the noise generated by each device can be targeted and processed without confusing it, which provides a flexible strategy for subsequent noise removal. This step makes noise extraction more refined by dividing the audio signal into short-period and long-period segments. Short-period spectrum data focuses on fast-changing noise components, while long-period spectrum data focuses on slowly changing environmental noise. After segmentation, noise in different frequency ranges can be more accurately identified and eliminated, improving the noise reduction effect. By extracting noise from data of different periods in layers, it is possible to effectively distinguish short-term motion noise (such as mechanical vibration noise) from long-term environmental noise (such as background noise from electrical appliances, air conditioners, etc.), remove different types of noise in a targeted manner, avoid false noise reduction, and improve audio clarity. In a multi-device environment, interference and interaction between different devices produce cluster noise. This step helps identify noise caused by improper device configuration or location by extracting device cluster noise, and avoids multi-device interference factors that are not considered when a single device is noise-reduced. Through abnormal noise pattern recognition, accurate identification can be performed for device failures, unexpected interference, or other atypical noise. After identifying these abnormal noise patterns, the system can avoid treating them as regular noise, thereby reducing false deletion or erroneous processing during the noise reduction process. Accurate identification of abnormal noise improves the robustness and noise reduction effect of the system, especially for complex or irregular noise environments. By subtracting the power spectrum of the noise on each time frame, short-term motion noise, environmental interference noise, and cluster noise can be accurately removed. The subtraction process is based on noise pattern recognition data to ensure that each type of noise is processed in a targeted manner. This frame-by-frame processing method effectively improves the accuracy of noise reduction, retains the original details of the audio, and avoids excessive loss of sound quality. This step reconstructs the effective audio power spectrum after noise reduction into an audio signal through an inverse Fourier transform, making the noise-reduced audio closer to the quality of the original audio. The inverse Fourier transform ensures that noise removal in the frequency domain can be effectively restored to the time domain audio, thereby generating a clear and realistic audio signal.This step ensures the naturalness and audibility of the audio after noise reduction, and avoids common noise reduction side effects such as "metallic sound" or "distortion". Therefore, the present invention improves the accuracy and real-time performance of noise reduction effect through invalid audio elimination, time alignment, layered noise extraction and abnormal noise identification.

[0010] Preferably, step S1 comprises the following steps:

[0011] Step S11: obtaining original audio and sound device operation data associated with the original audio;

[0012] Step S12: performing audio sampling rate conversion on the original audio to generate sampling converted audio;

[0013] Step S13: performing audio endpoint detection on the sampled converted audio to obtain an audio start point and an audio end point; based on the audio start point and the audio end point, performing invalid audio elimination on the sampled converted audio to generate valid audio;

[0014] Step S14: Divide the effective audio into audio short-time frames to generate an audio short-time frame set; and use the audio short-time frame set to perform audio time alignment on the effective audio, thereby generating pre-processed standard audio.

[0015] The present invention can significantly improve the efficiency of the subsequent processing stage and avoid unnecessary noise and irrelevant information interference by performing steps such as sampling rate conversion, endpoint detection and invalid audio elimination on the original audio. The operation of audio short-time frame segmentation and time alignment helps to ensure the consistency of audio data on the time axis, which is convenient for further analysis and processing, such as speech recognition, sentiment analysis and other tasks. Invalid audio elimination can remove unnecessary background noise or silent segments, ensure the quality of audio data, and make subsequent signal processing more accurate. The pre-processed standard audio provides a standardized basic data set for subsequent feature extraction, classification, enhancement and other processing steps, which improves the reliability and accuracy of the overall processing flow. By eliminating invalid or erroneous audio data, the robustness of the system under different conditions can be improved, ensuring that the processing results are not interfered by factors such as environmental noise and equipment errors. The pre-processing process is applicable to data of various sound-generating devices, whether it is a voice acquisition device, an ambient sound acquisition device or other audio recording device, and the audio data can be optimized through this process to provide consistent and high-quality input data for different application scenarios.

[0016] Preferably, step S2 comprises the following steps:

[0017] Step S21: performing audio output mode detection on the preprocessed standard audio and the sound device associated with the original audio, and when the number of the preprocessed standard audio and the sound device associated with the original audio is equal to 1, marking the preprocessed standard audio as single-device output audio;

[0018] Step S22: when the number of sound-emitting devices associated with the preprocessed standard audio and the original audio is greater than 1, the preprocessed standard audio is marked as multi-device output audio; audio frequency domain conversion is performed on the single-device output audio to generate single-device output audio spectrum data;

[0019] Step S23: segmenting the audio periodic regularity of the single device output audio spectrum data based on a preset periodic threshold value to generate single device short-period output spectrum data and single device long-period output spectrum data; using the sound device operation data associated with the original audio to perform audio device short-term frequency vibration noise analysis on the single device short-period output spectrum data to generate single device short-term motion frequency vibration noise;

[0020] Step S24: performing long-period environmental interference noise analysis on the long-period output spectrum data of the single device to generate long-period environmental interference noise of the single device.

[0021] The present invention can automatically identify the device type (single device output or multi-device output) of the audio source by detecting the audio output mode of the pre-processed standard audio and the sound-generating device, providing a basis for subsequent noise analysis and processing. After the single device output audio is converted in the frequency domain, the generated spectrum data is convenient for further analysis of its frequency characteristics. By segmenting the short-period spectrum data and the long-period spectrum data, short-term frequency vibration noise and long-term environmental interference noise can be processed separately to optimize the quality of the audio signal. By using the operating data of the sound-generating device to perform frequency vibration noise analysis on the short-period spectrum data, the noise caused by internal problems or movement of the device can be identified and isolated, the clarity of the audio signal can be improved, and interference can be reduced. Performing environmental interference noise analysis on the long-period spectrum data helps to identify long-standing environmental noise, such as background sound, wind noise, etc., and then take effective noise reduction measures to improve the audio quality. Through accurate analysis of different types of noise (short-term motion noise and long-term environmental noise), useful audio information can be more effectively extracted, and the accuracy and robustness of audio processing can be improved, especially in complex recording environments. This method is not only applicable to single-device environments, but can also distinguish and process audio in multi-device environments, making it possible to more accurately identify the audio features of different devices in complex audio acquisition scenarios, and facilitating multi-source audio fusion and processing. By combining the operating data of the sound-generating device for noise analysis, the system can adapt to different devices and environmental conditions, optimize audio output and processing strategies, and thus improve the stability and performance of the entire system.

[0022] Preferably, step S23 includes the following steps:

[0023] Step S231: extracting audio spectrum peak information from the single device output audio spectrum data to obtain the single device output audio spectrum peak data; discretizing the single device output audio spectrum peak data to generate a single device output audio spectrum peak discrete graph;

[0024] Step S232: performing height matching on the audio spectrum peak discrete points in the single device output audio spectrum peak discrete graph, mapping the audio spectrum peak discrete points at the same height to the single device output audio spectrum peak to separate the related spectrum curve segments, and obtaining the single device output same frequency curve;

[0025] Step S233: Calculate the spectrum period of the single device output same-frequency curve to obtain the same-frequency curve period; perform period threshold judgment on the same-frequency curve period based on a preset period threshold, and when the preset period threshold is greater than or equal to the same-frequency curve period, mark the corresponding single device output same-frequency curve as single device short-period output spectrum data;

[0026] Step S234: When the preset period threshold is less than the period of the same-frequency curve, the corresponding single-device output same-frequency curve is marked as a single-device long-period output spectrum data; the single-device short-period output spectrum data is analyzed for audio device short-term frequency vibration noise using the sound device operation data associated with the original audio to generate a single-device short-term motion frequency vibration noise.

[0027] The present invention can effectively capture important spectral features in audio by extracting and discretizing the peak information of the audio spectrum. This process helps to identify the key frequency components in the audio signal and provide accurate spectrum data for subsequent noise analysis and processing. By generating a spectrum peak discrete graph, the spectrum characteristics of the audio signal can be presented more clearly, helping analysts or automated systems to quickly identify and process peaks of different frequencies, and improving the processing efficiency of the audio signal. By highly matching the discrete points of the audio spectrum peak and separating the same-frequency curve, the audio signals of different frequency components can be effectively distinguished. This process helps to accurately extract the signals of the same frequency band, laying the foundation for subsequent spectrum period analysis and noise analysis. By calculating the period of the same-frequency curve and distinguishing the period threshold, the parts of the audio signal with short-period and long-period characteristics can be marked as short-period output spectrum data and long-period output spectrum data respectively. This classification helps to further identify noise and signal components from different sources and improve the accuracy of audio processing. Frequency-vibration noise analysis of short-period spectrum data can distinguish noise caused by short-term factors such as equipment movement and vibration, which helps to improve the clarity of the audio signal and avoid error interference caused by equipment characteristics. By separating and analyzing short-period spectrum data from long-period spectrum data, we can better eliminate equipment noise and environmental interference, and improve the quality and robustness of audio signals, especially in complex recording or monitoring environments. Whether the audio signal is short-term noise caused by equipment vibration or long-term noise caused by environmental changes, this process can accurately identify and classify it, thereby improving the system's adaptability to different audio sources.

[0028] Preferably, performing audio device short-term frequency vibration noise analysis on single device short-term output spectrum data using sound device operation data associated with the original audio includes:

[0029] Performing equipment operation vibration analysis on the sound-generating equipment operation data associated with the original audio to generate the sound-generating equipment operation vibration data; converting the sound-generating equipment operation vibration data into a vibration spectrum to generate a sound-generating equipment operation vibration spectrum diagram; and matching the suspicious spectrum segments of the single-equipment short-period output spectrum data based on the sound-generating equipment operation vibration spectrum diagram to generate the equipment operation suspicious noise spectrum segment;

[0030] Perform Hilbert transform on the suspicious noise spectrum of the equipment operation to generate a suspicious envelope signal of the equipment operation; perform instantaneous amplitude calculation on the suspicious envelope signal of the equipment operation to obtain the signal strength of the suspicious noise of the equipment operation, wherein the instantaneous amplitude calculation is as follows:

[0031] ;

[0032] In the formula, Represented as the equipment operation noise signal at time The instantaneous amplitude of Expressed as time The original equipment operating noise signal, Expressed as time The signal after Hilbert transformation, Expressed as time The original device runs the function of Hilbert transform of the noise signal;

[0033] The noise intensity average is calculated for the suspicious noise signal intensity of the equipment operation, thereby generating the average short-term operation noise intensity of a single device. The short-term frequency vibration noise of the audio device is screened for the suspicious noise spectrum segment of the equipment operation using the average short-term operation noise intensity of a single device, thereby generating the short-term motion frequency vibration noise of a single device.

[0034] The present invention can effectively identify short-term frequency vibration noise caused by equipment vibration by combining the operation data of the sound-generating device with the audio spectrum data. Through vibration analysis and spectrum conversion, the system can accurately match the suspicious noise frequency band in the audio spectrum and identify the noise source related to the equipment operation. The envelope signal of the equipment operation noise spectrum is generated by Hilbert transform, and the instantaneous amplitude of the equipment operation noise can be extracted from the spectrum. This operation can help further analyze the noise signal intensity and fluctuation, so as to capture the short-term frequency vibration noise more finely. Through instantaneous amplitude calculation, the intensity change of the equipment operation noise can be accurately captured, and the intensity of the short-term operation noise can be obtained by mean calculation. This process can reflect the long-term trend and stability of the noise intensity, which is helpful to identify abnormal or sudden noise events. The suspicious noise spectrum segment is screened by using the mean value of the short-term operation noise intensity of a single device, which helps to filter out noise interference caused by non-equipment vibration. Through this refined screening, the noise signal related to the equipment operation can be retained more accurately, and the system's responsiveness to the noise source can be improved. By combining the equipment's operating data with the noise analysis of the audio signal, it is possible to monitor the equipment's operating status in real time, promptly discover potential frequency vibration noise problems, provide data support for equipment maintenance, fault diagnosis and optimization, and reduce the occurrence of equipment failures. Through the linkage analysis of equipment operation vibration data and audio spectrum, the source of the noise source can be accurately identified and located, especially in complex environments, and the equipment vibration noise and other noise sources can be effectively distinguished, thereby improving the performance of the noise elimination algorithm. This analysis method can significantly improve the quality of the audio signal and reduce the impact of short-term frequency vibration noise caused by the equipment on the audio quality. Provide clearer and more accurate signal input for subsequent audio signal processing (such as speech recognition, noise suppression, etc.).

[0035] Preferably, step S24 includes the following steps:

[0036] Step S241: performing spectrum periodicity detection on the single-device long-period output spectrum data to generate single-device spectrum periodicity detection data; performing power spectrum density analysis on the single-device long-period output spectrum data based on the single-device spectrum periodicity detection data to generate a single-device long-period power spectrum density graph;

[0037] Step S242: performing spectrum energy distribution analysis on the long-period power spectrum density diagram of a single device to generate spectrum energy distribution data; performing suspicious spectrum feature extraction on the long-period output spectrum data of the single device through the spectrum energy distribution data to obtain suspicious spectrum feature data;

[0038] Step S243: Pair the suspicious spectrum feature data with a preset environmental interference feature library, and screen the single-device long-period output spectrum data for environmental interference noise based on the result of the spectrum feature pairing, thereby generating single-device long-period environmental interference noise.

[0039] The present invention can effectively identify the periodic components in the audio signal by detecting the spectral periodicity regularity of the long-period output spectrum data of a single device, and reveal the persistent noise caused by the long-term operation of the device or environmental factors. This process can help accurately distinguish the normal operation of the device from potential interference signals, and provide more detailed signal analysis. Power spectrum density analysis helps to reveal the energy distribution of the signal in different frequency bands, thereby generating a long-period power spectrum density diagram. Through this figure, the energy changes and frequency components during the long-period operation of the device can be clearly seen, which helps to identify the long-standing noise source and provide data support for the subsequent spectrum energy distribution analysis. The spectrum energy distribution analysis of the power spectrum density diagram can reveal the energy concentration in different frequency bands. This analysis helps to identify potential noise features, especially long-period environmental interference noise. Accurate energy distribution data provides a strong basis to support subsequent noise feature extraction and interference screening. By extracting suspicious spectrum features through spectrum energy distribution data, interference noise features can be automatically identified from long-period spectrum data. By matching these features with the environmental interference feature library, the system can automatically identify and screen out specific environmental noise sources, reduce manual intervention, and improve the level of automation. By matching spectral features and screening interference noise, environmental interference noise (such as wind noise, traffic noise, etc.) can be effectively removed, and the normal signal components of the device can be retained. This process significantly improves the clarity and accuracy of the signal and reduces the impact of external environmental factors on signal quality. After removing environmental interference noise, the generated audio signal is clearer and unnecessary noise pollution is reduced, which is crucial for audio processing tasks (such as speech recognition, voiceprint recognition, etc.) and helps to improve the accuracy and effect of subsequent processing algorithms. By analyzing the relationship between the spectral features and environmental interference noise during device operation, it can provide real-time feedback on the device's operating status, help monitor device performance, troubleshoot potential sources of failure, and provide data support for device maintenance and optimization. By automatically matching and screening noise features, the system can efficiently adapt to and handle different types of interference in a changing environment, especially in an environment with severe noise pollution, to ensure the stability of device operation and the robustness of audio processing.

[0040] Preferably, step S3 comprises the following steps:

[0041] Step S31: performing device cluster ground loop influence analysis on the multi-device output audio to generate the device cluster ground loop influence frequency; performing device cluster noise extraction on the multi-device output audio according to the device cluster ground loop influence frequency to obtain the multi-device cluster noise;

[0042] Step S32: integrating data features of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise, and multi-device cluster noise to generate abnormal noise feature data;

[0043] Step S33: Divide the abnormal noise feature data into data sets to generate a model training set and a model test set; use a deep residual network algorithm to perform model training on the model training set to generate an audio noise recognition pre-model; perform model optimization iteration on the audio noise recognition pre-model through the model test set to generate an audio noise recognition model;

[0044] Step S34: performing abnormal noise pattern recognition on the preprocessed standard audio based on the audio noise recognition model to generate abnormal noise pattern recognition data.

[0045] The present invention can identify mutual interference or noise superposition effects between devices by performing device cluster ground loop influence analysis on the output audio of multiple devices. This analysis helps to accurately identify frequency interference caused by ground loop effects in device clusters, thereby avoiding the influence of these interferences on the overall audio signal. Based on the analysis of ground loop influence frequency, cluster noise can be extracted from the output audio of multiple devices. This operation can effectively identify and remove the synthetic noise generated when multiple devices are running simultaneously, ensuring that the quality of the audio signal will not be affected by cluster noise during system processing. By integrating the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device and the cluster noise of multiple devices, abnormal noise feature data is generated. This integration can effectively combine different types of noise information, provide a more comprehensive and accurate basis for noise identification, and improve the accuracy and robustness of noise detection. By dividing the abnormal noise feature data into a training set and a test set, and optimizing the model training using a deep residual network (ResNet) algorithm, the accuracy of audio noise recognition can be improved. The deep residual network has significant advantages in processing complex noise pattern recognition tasks, can effectively avoid the overfitting problem in model training, and ensure good generalization ability for unknown data. By using the trained audio noise recognition model to identify abnormal noise patterns on the pre-processed standard audio, the system can automatically detect abnormal noise patterns in the audio signal. This process improves the automation level of the system, reduces the need for manual intervention, and enhances the speed and accuracy of noise recognition. The generation and optimization of the audio noise recognition model enables the system to better cope with various complex noise environments, whether it is equipment noise, environmental noise or cluster noise. By accurately identifying abnormal noise patterns, the system can maintain high robustness in a changing environment, ensuring the stability and reliability of audio signal processing.

[0046] Preferably, performing device cluster ground loop impact analysis on multiple device output audios includes:

[0047] Perform device crosstalk on audio data output by multiple devices to generate audio crosstalk data of multiple devices; perform ground impedance measurement on audio crosstalk data of multiple devices to generate impedance characteristic data;

[0048] Perform loop current and ground noise correlation analysis on impedance characteristic data to generate cluster current noise interference data; perform electromagnetic interference frequency response calculation on cluster current noise interference data to generate equipment cluster electromagnetic interference response characteristic data;

[0049] The distortion and phase shift of the electromagnetic interference response characteristic data of the equipment cluster are evaluated to generate the quality parameter data of the equipment cluster; the loop stability and frequency characteristics of the equipment cluster quality parameter data are extracted to obtain the ground loop influence frequency of the equipment cluster.

[0050] The present invention can identify and quantify the mutual interference between devices, especially the crosstalk phenomenon generated when multiple devices are running at the same time, by performing device crosstalk analysis on the audio data output by multiple devices. Effectively identify the source and influence of crosstalk, avoid the negative impact of crosstalk on the quality of audio signals, and ensure the clarity and accuracy of audio signals. Through ground impedance measurement, impedance characteristic data can be generated to provide basic data for subsequent noise interference analysis. This operation helps to identify the electrical performance of device grounding and the poor grounding phenomenon that affects audio signals, thereby providing accurate electrical parameter support for system optimization. Through the correlation analysis of loop current and ground noise, the noise interference caused by the ground loop can be identified. This analysis helps to accurately locate the noise source and reveal the specific impact of these noises on the quality of audio signals, thereby providing a basis for subsequent noise suppression and interference source elimination. Through the calculation of electromagnetic interference frequency response, the electromagnetic interference characteristics of the device cluster in different frequency bands can be quantified. This process can reveal the frequency response mode generated by the device cluster when working, and provide systematic spectrum data support for subsequent noise identification and suppression. By evaluating the distortion and phase shift of the electromagnetic interference response characteristic data, we can gain a deep understanding of the signal distortion and phase shift generated by the device cluster during operation. This evaluation provides valuable data for further optimizing device performance and reducing audio signal distortion, ensuring the quality and stability of the audio signal during output. By extracting the loop stability and frequency characteristics of the device cluster quality parameter data, we can gain a deep understanding of the stability of the electrical circuit and the operating frequency characteristics. This operation can reveal the stability of the device cluster under different operating conditions, help optimize the collaborative work between devices, and reduce interference caused by frequency instability.

[0051] Preferably, step S4 comprises the following steps:

[0052] Step S41: performing noise power spectrum conversion on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the cluster noise of multiple devices according to the abnormal noise pattern recognition data to generate an abnormal noise power spectrum;

[0053] Step S42: Based on the abnormal noise power spectrum, the power spectrum of the original audio at each time frame is subtracted to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed by inverse Fourier transform to obtain a noise-reduced audio signal.

[0054] The present invention can accurately extract the power spectrum characteristics of various types of noise by converting the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the noise of a multi-device cluster, which provides a key data basis for subsequent noise suppression and ensures the effective identification and classification of noise. By subtracting the abnormal noise power spectrum from the power spectrum of the original audio in each time frame, the abnormal noise components in the audio can be effectively removed, thereby generating an effective audio power spectrum after noise reduction. This step can significantly improve the clarity and quality of the audio signal, making the audio signal more real and accurate. The inverse Fourier transform performs audio reconstruction on the effective audio power spectrum after noise reduction to ensure that the audio signal after noise reduction is restored to a state close to the original audio, while removing the interference of noise. This noise reduction process helps to enhance the audibility of the audio, remove background noise, frequency vibration noise and environmental interference, and make the audio content clearer and easier to understand. This process is not only effective in single-device audio processing, but also can process audio interference noise in a multi-device cluster. Through noise reduction, the audio signal can better reflect the actual output of the device, reduce the crosstalk and environmental noise caused by the device cluster, and ensure that the audio quality in a multi-device environment is improved. The denoised audio signal is more suitable for subsequent audio analysis and recognition, improving the accuracy of tasks such as noise source identification and speech recognition. After removing the interference noise, the useful information in the audio signal is retained, making subsequent processing more accurate and efficient. Through efficient noise power spectrum conversion and noise reduction processing, the negative impact of environmental noise (such as short-term frequency vibration noise, long-period environmental interference noise, etc.) on the audio signal is reduced.

[0055] In this specification, a deep learning-based audio noise reduction system is provided, which is used to execute the above-mentioned deep learning-based audio noise reduction method. The deep learning-based audio noise reduction system includes:

[0056] The audio preprocessing module is used to obtain the original audio and the sound device operation data associated with the original audio; remove invalid audio from the original audio to generate valid audio; perform audio time alignment on the valid audio to generate preprocessed standard audio;

[0057] The single-device noise extraction module is used to detect the audio output mode of the pre-processed standard audio and the sound-generating device associated with the original audio, and generate single-device output audio and multi-device output audio; segment the single-device output audio according to the audio period regularity, and generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise;

[0058] The multi-device noise extraction module is used to extract the ground loop noise of the device cluster from the output audio of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data;

[0059] The audio reconstruction module is used to subtract the power spectrum of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise and multi-device cluster noise in each time frame according to the abnormal noise pattern recognition data to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

[0060] The beneficial effect of the present invention is that the quality and consistency of the processed standard audio data are ensured by eliminating invalid audio and aligning audio time, which provides clear and accurate input for subsequent noise extraction, pattern recognition and reconstruction, and reduces errors and interference in the processing process. The single-device noise extraction module and the multi-device noise extraction module effectively extract various noise sources from the audio, including short-term motion frequency vibration noise, long-period environmental interference noise and cluster ground loop noise. The extraction and classification of these noises ensure the accuracy and clarity of the audio signal. Through the abnormal noise pattern recognition module, multiple noise patterns can be efficiently identified and distinguished to ensure that the noise components that affect the quality in the audio can be accurately found. This method improves the system's processing ability for complex noise patterns and ensures accurate analysis and improvement of audio signals. The audio reconstruction module achieves efficient audio noise reduction through power spectrum subtraction and inverse Fourier transform. This process effectively removes abnormal noise sources, restores the original quality of the audio signal, significantly improves the clarity of the signal, and makes the audio data more real and reliable. The denoised audio signal removes noise while retaining the original content, optimizing the audibility of the audio, and is suitable for various scenarios that require clear audio, such as speech recognition, hearing equipment, broadcasting, and conference systems. The system is able to handle complex audio environments where multiple devices are working simultaneously, including noise interference in device clusters, which makes it highly adaptable in multi-device systems and can improve the audio quality of multi-device collaboration, and is particularly suitable for scenarios such as industry, transportation, and communications. Through accurate noise pattern recognition and noise reduction processing, the system can effectively resist interference from factors such as equipment, environment, and ground loops, improving the stability and performance of the system in a noisy environment. Therefore, the present invention improves the accuracy and real-time performance of the noise reduction effect through invalid audio elimination, time alignment, layered noise extraction, and abnormal noise identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 A schematic diagram of the steps of an audio noise reduction method based on deep learning;

[0062] Figure 2 for Figure 1 Detailed implementation steps of step S2 in the flowchart;

[0063] Figure 3 for Figure 1 Detailed implementation steps of step S3 in FIG.

[0064] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0065] The technical method of the present invention is described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by technicians in this field without creative work are within the scope of protection of the present invention.

[0066] In addition, the accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. The functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor methods and / or microcontroller methods.

[0067] It should be understood that although the terms "first", "second", etc. may be used herein to describe various units, these units should not be limited by these terms. These terms are used only to distinguish one unit from another. For example, without departing from the scope of the exemplary embodiments, a first unit may be referred to as a second unit, and similarly, a second unit may be referred to as a first unit, and the term "and / or" used herein includes any and all combinations of one or more of the listed associated items.

[0068] To achieve this, please refer to Figures 1 to 3 , an audio noise reduction method based on deep learning, the method comprising the following steps:

[0069] Step S1: obtaining original audio and sound device operation data associated with the original audio; removing invalid audio from the original audio to generate valid audio; performing audio time alignment on the valid audio to generate pre-processed standard audio;

[0070] Step S2: Perform audio output mode detection on the preprocessed standard audio and the sound-generating device associated with the original audio to generate single-device output audio and multi-device output audio; perform audio period regularity segmentation on the single-device output audio to generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise;

[0071] Step S3: extract the ground loop noise of the device cluster from the audio output of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data;

[0072] Step S4: Based on the abnormal noise pattern recognition data, the power spectrum of the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise is subtracted in each time frame to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

[0073] The present invention provides additional contextual information for noise analysis by combining audio and device operation data, which can help analyze the impact of device status on audio output, specific device failures, unstable performance, etc., thereby providing valuable reference for subsequent noise reduction. By removing invalid parts (such as background noise, silent segments, etc.) in the original audio, the interference of irrelevant information is reduced, and the effectiveness and computational efficiency of the subsequent audio noise reduction algorithm are ensured. It enables the system to focus more on the effective audio part that needs noise reduction, thereby improving the noise reduction accuracy. Audio time alignment eliminates the inconsistency caused by time deviation by ensuring the synchronization of multiple audio signals, and avoids inaccurate noise reduction results caused by time dislocation, which is particularly important for processing audio that sounds from multiple devices at the same time, and helps to provide more accurate input for subsequent noise separation and analysis. By detecting the audio output modes of single devices and multiple devices, it helps to classify and process noise according to the number and configuration of devices. Specifically, in a multi-device environment, the noise generated by each device can be targeted and processed without confusing it, which provides a flexible strategy for subsequent noise removal. This step makes noise extraction more refined by dividing the audio signal into short-period and long-period segments. Short-period spectrum data focuses on fast-changing noise components, while long-period spectrum data focuses on slowly changing environmental noise. After segmentation, noise in different frequency ranges can be more accurately identified and eliminated, improving the noise reduction effect. By extracting noise from data of different periods in layers, it is possible to effectively distinguish short-term motion noise (such as mechanical vibration noise) from long-term environmental noise (such as background noise from electrical appliances, air conditioners, etc.), remove different types of noise in a targeted manner, avoid false noise reduction, and improve audio clarity. In a multi-device environment, interference and interaction between different devices produce cluster noise. This step helps identify noise caused by improper device configuration or location by extracting device cluster noise, and avoids multi-device interference factors that are not considered when a single device is noise-reduced. Through abnormal noise pattern recognition, accurate identification can be performed for device failures, unexpected interference, or other atypical noise. After identifying these abnormal noise patterns, the system can avoid treating them as regular noise, thereby reducing false deletion or erroneous processing during the noise reduction process. Accurate identification of abnormal noise improves the robustness and noise reduction effect of the system, especially for complex or irregular noise environments. By subtracting the power spectrum of the noise on each time frame, short-term motion noise, environmental interference noise, and cluster noise can be accurately removed. The subtraction process is based on noise pattern recognition data to ensure that each type of noise is processed in a targeted manner. This frame-by-frame processing method effectively improves the accuracy of noise reduction, retains the original details of the audio, and avoids excessive loss of sound quality. This step reconstructs the effective audio power spectrum after noise reduction into an audio signal through an inverse Fourier transform, making the noise-reduced audio closer to the quality of the original audio. The inverse Fourier transform ensures that noise removal in the frequency domain can be effectively restored to the time domain audio, thereby generating a clear and realistic audio signal.This step ensures the naturalness and audibility of the audio after noise reduction, and avoids common noise reduction side effects such as "metallic sound" or "distortion". Therefore, the present invention improves the accuracy and real-time performance of noise reduction effect through invalid audio elimination, time alignment, layered noise extraction and abnormal noise identification.

[0074] In the embodiment of the present invention, reference Figure 1 FIG. 1 is a schematic diagram of a step flow of an audio noise reduction method based on deep learning of the present invention. In this example, the audio noise reduction method based on deep learning includes the following steps:

[0075] Step S1: obtaining original audio and sound device operation data associated with the original audio; removing invalid audio from the original audio to generate valid audio; performing audio time alignment on the valid audio to generate pre-processed standard audio;

[0076] In an embodiment of the present invention, if the audio file is stored in a local hard disk, flash memory or other media in advance, the audio file is found under the specified path using the file reading interface provided by the operating system according to the file system rules. Common audio file formats such as WAV, MP3, FLAC, etc., use corresponding audio decoding libraries for different formats. Specifically, when reading a WAV file, use Python's wave module to parse the file header according to the format specification, obtain parameters such as audio sampling rate, number of channels, and quantization bit number, and then read the audio data part; when reading MP3, use third-party libraries such as pydub to decode first. If it is obtained from a real-time audio stream, such as microphone input or network audio stream, for the microphone, use the operating system audio interface (CoreAudioAPI under Windows, ALSA of Linux, etc.) to initialize the audio acquisition device, set the sampling parameters, and start to continuously receive audio data; for network audio streams, according to the transmission protocol (such as HTTP Live Streaming, RTMP), use the network library (such as Python's socket, requests combined with a specific protocol parsing library) to connect to the audio source server, and download and decode audio data fragments in real time. If it is a sound device with its own operating system or smart module, such as a smart speaker or Bluetooth headset, use the developer interface or system management tool provided by the device. Taking a smart speaker as an example, through the SDK that comes with the speaker, send a query command to obtain the device power, Bluetooth connection status, audio decoding chip temperature, and speaker diaphragm status data. These data are usually returned in JSON or XML format, and are parsed and stored in a local data structure. For the case of adding sensors to ordinary sound devices, such as adding vibration sensors and temperature sensors to traditional speakers. Through the microcontroller (such as Arduino) connected to the sensor, use its serial communication, SPI communication and other protocols to collect sensor data at a set frequency and then transmit it to the main processing device. To calculate the short-term energy of audio, first divide the audio signal into frames, set the duration of each frame to 10-30 milliseconds, and adjacent frames can have a certain overlap (such as 5-15 milliseconds). For each frame of audio data, calculate the square sum to get the energy value. Set a higher energy threshold. When the energy of multiple consecutive frames (such as 3-5 frames) exceeds the threshold, it is determined as the audio start point. Similarly, set a lower energy threshold. When the energy of multiple consecutive frames is lower than it, it is determined to be the audio end point. Count the number of times the audio signal crosses the zero level, and process it in frames, and calculate the zero-crossing rate for each frame. At the beginning of the audio, the zero-crossing rate will change significantly. Combined with the energy method, the start and end points are comprehensively determined. For example, the zero-crossing rate near the starting point rises rapidly from a low-frequency fluctuation state, and with the breakthrough of the energy threshold, it can be more accurately located. Based on the determined audio start and end points, use array indexing (programming language level, such as Python's list slicing operation) to remove the audio data segments before the start point and after the end point, leaving the middle part, and then generate valid audio.Divide the effective audio into multiple short-time frames according to a fixed duration (such as 20 milliseconds), with adjacent frames overlapping by about 10 milliseconds. This division can reduce the loss of audio information caused by framing. You can use a loop combined with an array index to intercept fixed-length data segments from the beginning of the audio data in sequence and store them in a new frame set data structure. Pick out a reference frame from these short-time frames, generally a frame located in the middle of the audio with relatively strong energy. Calculate the energy of each frame, traverse the frame set to find a frame that meets the conditions as a standard. Calculate the deviation of the remaining frames from the reference frame on the time axis, and by translating the audio data, that is, adjusting the starting position of each frame in the time domain, all short-time frames are accurately aligned in the time dimension, and finally generate preprocessed standard audio.

[0077] Step S2: Perform audio output mode detection on the preprocessed standard audio and the sound-generating device associated with the original audio to generate single-device output audio and multi-device output audio; perform audio period regularity segmentation on the single-device output audio to generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise;

[0078] In an embodiment of the present invention, a data structure, specifically a dictionary or a list, is constructed to store the preprocessed standard audio and the sound device information associated therewith. From the device management system or the association table constructed when the sound device operation data is obtained before, all the sound device identifiers corresponding to the original audio are sorted out and stored one by one with the standard audio. Traverse this data structure and use counter variables to count the number of standard audio and the number of sound devices respectively. Specifically, in Python, two integer variables audio_count and device_count can be defined. Each time a group of audio-device pairs is traversed, the corresponding counter is increased by 1. If audio_count and device_count are both equal to 1, a "single device output audio" mark is added to the metadata part of the audio data; if the number is greater than 1, it is marked as "multiple device output audio". Subsequent processing procedures can quickly distinguish the audio type based on this mark and carry out targeted processing. For audio data marked as single device output audio, use fast Fourier transform (FFT) to convert it from time domain to frequency domain. In Python, with the help of numpy.fft.fft function, first fill the audio time domain data with zeros so that its length meets the optimal FFT calculation requirements (usually an integer power of 2), and then perform the transformation to obtain the single-device output audio spectrum data, which is presented in a complex array and contains amplitude and phase information. Traverse the spectrum data, find the spectrum peak point, that is, the point whose amplitude is greater than the amplitude of the adjacent frequency point, extract its frequency and amplitude information, and generate the single-device output audio spectrum peak data. Set the discretization parameters, divide the amplitude value range into multiple intervals, map the amplitude of each peak point to the corresponding interval, and generate a single-device output audio spectrum peak discrete graph. Specifically, the amplitude range of 0-100dB is divided into an interval of 10dB. The spectrum peak points in the same height interval (discrete point set) are traced back to the original spectrum data, and the corresponding curve segments are marked to obtain the single-device output same-frequency curve. That is, frequency points with similar amplitude characteristics are associated to divide different frequency component curves. For each single-device output same-frequency curve, the autocorrelation function (ACF) is used to calculate its spectrum period. Through the scipy.signal.correlate function, given a delay range, find the delay value corresponding to the first peak of the autocorrelation value (except zero delay), which is the curve period. Preset the period threshold, which is set based on the characteristics of the sound device and the previous experimental data. Compare the same frequency curve period with the threshold. If it is less than or equal to the threshold, it is marked as a single device short-period output spectrum data; if it is greater than the threshold, it is marked as a single device long-period output spectrum data. Extract key parameters such as device vibration frequency and motor speed stability from the sound device operation data associated with the original audio.Perform correlation analysis on these parameters and the short-period output spectrum data of a single device. For example, use the Pearson correlation coefficient calculation method to find the frequency bands where the fluctuation of the operating data is highly correlated with the short-period fluctuation of the audio spectrum. After filtering, feature extraction and other processing, generate the short-term motion frequency vibration noise of a single device. For the long-period output spectrum data of a single device, a machine learning method can be used to pre-train a long short-term memory network (LSTM) model. Input long-period spectrum data, the model learns the long-period spectrum change law, and outputs the estimated result of the environmental interference noise. You can also use traditional adaptive filtering technology to adaptively adjust the filter parameters according to the long-term stability characteristics of the audio spectrum, filter out the spectrum components belonging to environmental interference, and extract the long-period environmental interference noise of a single device.

[0079] Step S3: extract the ground loop noise of the device cluster from the audio output of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data;

[0080] In the embodiment of the present invention, the physical connection information of the sound device cluster corresponding to the multi-device output audio is collected, including detailed information such as the cable connection method between the devices, the direction of the grounding line, and the plug and socket specifications. Use drawing software or special circuit design tools to draw the electrical connection topology of the device cluster to clarify the position of each device in the cluster and the grounding relationship. Check the placement of the equipment on the spot, consider the electromagnetic interference between the equipment, the specific equipment spacing, whether there are metal shielding objects and other factors, because unreasonable placement aggravates the ground loop problem. Use a professional ground impedance measuring instrument, such as a ground resistance tester, connect the probe to the ground terminal of the device cluster, measure the ground impedance at different frequency points according to the standard measurement process, record the change of impedance value with frequency, and generate ground impedance data. According to the measured ground impedance data, use circuit simulation software (such as LTspice, Multisim, etc.) to build a ground loop equivalent circuit model of the device cluster. Set the actual power supply, resistance, capacitance, inductance and other component parameters in the model, and simulate the current flow and voltage distribution in the ground loop under different audio signal excitations through simulation operations to find the frequency point where noise interference occurs. According to the key interference frequency points determined by the above simulation analysis, design targeted filters, specifically bandpass filters or notch filters. In the digital audio processing environment, use digital filter design algorithms, such as the finite impulse response (FIR) filter design method, to determine the filter coefficients. The multi-device output audio is passed through the designed filter to filter out other frequency components, retaining only the noise signal related to the ground loop of the device cluster, thereby obtaining the multi-device cluster noise. For the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise, extract their features respectively. For short-term motion frequency vibration noise, extract features such as vibration frequency range and frequency vibration amplitude change rate; for long-period environmental interference noise, extract the spectrum shape and energy distribution characteristics of environmental noise; for multi-device cluster noise, extract the frequency components and phase relationship characteristics unique to cluster noise. Integrate the extracted noise features into a unified data structure, and specifically construct a multi-dimensional feature vector, with each dimension corresponding to a key feature of a noise, to facilitate subsequent pattern recognition processing. Machine learning models such as deep neural networks (DNNs) and support vector machines (SVMs) can be used for abnormal noise pattern recognition. If DNN is selected, initialize the network structure first, determine the number of neurons in the input layer, hidden layer, and output layer, and configure appropriate activation functions (such as ReLU) and loss functions (such as cross entropy loss). Use known noise sample data (with correct noise pattern labels) to train the model, and continuously adjust the network weight parameters through the back propagation algorithm so that the model can accurately identify the feature combination of different noise patterns. Input the integrated noise feature data into the trained model or recognition system based on preset rules.If it is based on rules, the discrimination thresholds of different noise patterns are set, and if the frequency vibration amplitude within a specific frequency range exceeds a certain value, it is judged as a short-term motion frequency vibration noise pattern; if it is based on a machine learning model, the model directly outputs the predicted noise pattern results, and finally generates abnormal noise pattern recognition data.

[0081] Step S4: Based on the abnormal noise pattern recognition data, the power spectrum of the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise is subtracted in each time frame to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

[0082] In an embodiment of the present invention, by processing the time domain signal of a single-device short-term motion frequency vibration noise in the frame, the duration of each frame can be set according to the audio characteristics, such as 20-30 milliseconds. For each frame, Fourier transform (FFT) is used to convert it to the frequency domain. In Python, this operation is performed with the help of numpy.fft.fft function to obtain a complex number representation in the frequency domain. After that, the square of the modulus value of the complex number is calculated to obtain the power spectrum of the frame noise. Repeat the same process for single-device long-period environmental interference noise and multi-device cluster noise to obtain their respective power spectra. The abnormal noise pattern recognition data indicates the existence and characteristics of different types of noise in each time frame. Based on this information, it is determined which time frames are affected by which noise, and the corresponding noise power spectrum needs to participate in subsequent operations. The original audio is also processed by frame to obtain the original audio power spectrum of each time frame. For each time frame affected by noise, the original audio power spectrum is subtracted from the corresponding single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise, and multi-device cluster noise power spectrum in turn according to the guidance of the abnormal noise pattern recognition data. Specifically, in Python, assume that P_original is the power spectrum of a frame of the original audio, P_short is the power spectrum of the frame of short-term motion frequency noise of a single device, P_long is the power spectrum of the frame of long-period environmental interference noise of a single device, and P_cluster is the power spectrum of the frame of multi-device cluster noise. The power spectrum after subtraction P_subtracted = P_original-P_short-P_long-P_cluster. To avoid negative power values, a threshold is set. When the calculated result is less than 0, the power value is set to 0 to generate the effective audio power spectrum after noise reduction. The effective audio power spectrum after noise reduction is real and non-negative. To perform an inverse Fourier transform (IFFT), it must first be converted into a complex form. A simple approach is to assume that the phase is 0, that is, to generate a complex array whose real part is the power spectrum value and whose imaginary part is 0. Use the numpy.fft.ifft function to perform an inverse Fourier transform on the complex array after supplementing the phase information, and convert the frequency domain data back to the time domain. This function will output a complex result containing real and imaginary parts. We usually only take the real part as the reconstructed audio time domain data, because the imaginary part should theoretically be 0 (based on the previously assumed zero phase). The time domain data of each time frame obtained by the inverse Fourier transform are spliced ​​together in the original order of the audio, and the redundant information added during the frame processing is removed to finally obtain the denoised audio signal. Specifically, if there are overlapping parts during frame processing, the overlapping areas need to be properly processed to ensure the continuity and integrity of the audio signal.

[0083] Preferably, step S1 comprises the following steps:

[0084] Step S11: obtaining original audio and sound device operation data associated with the original audio;

[0085] Step S12: performing audio sampling rate conversion on the original audio to generate sampling converted audio;

[0086] Step S13: performing audio endpoint detection on the sampled converted audio to obtain an audio start point and an audio end point; based on the audio start point and the audio end point, performing invalid audio elimination on the sampled converted audio to generate valid audio;

[0087] Step S14: Divide the effective audio into audio short-time frames to generate an audio short-time frame set; and use the audio short-time frame set to perform audio time alignment on the effective audio, thereby generating pre-processed standard audio.

[0088] In an embodiment of the present invention, audio data is read from a variety of storage media or real-time audio streams. Specifically, if the audio processing of a smart speaker is to be performed, the original audio file that has just been recorded can be directly obtained through the built-in audio acquisition module interface of the speaker, and the file format is common WAV, MP3, etc. The corresponding audio decoding library is used to decode the audio file into the original audio data stream to prepare for subsequent processing. For sound-emitting devices associated with audio, operation data is collected with the help of a device management system or a sensor built into the device. For example, for a Bluetooth headset, the current power of the headset, Bluetooth connection stability parameters, temperature of the audio decoding chip and other information are read through specific instructions in the Bluetooth communication protocol. These operation data will establish a mapping relationship with the audio data and enter the subsequent processing flow together to provide auxiliary information for audio quality problem troubleshooting or personalized audio optimization. According to the requirements of the subsequent audio processing algorithm and the target application scenario, a suitable target sampling rate is pre-set. Specifically, if the subsequent processing involves some high-precision speech recognition algorithms, the target sampling rate is set to 16kHz to ensure that the audio signal has sufficient resolution in the frequency domain. Use professional audio processing libraries, such as the sampling rate conversion function provided by Librosa or FFmpeg. Convert the sampling rate of the original audio to the target sampling rate. During the conversion process, use the resampling algorithm to perform reasonable interpolation or extraction operations on the original audio data points to generate sampled converted audio. Specifically, the original audio sampling rate is 44.1kHz. After algorithm processing, the output is 16kHz sampled converted audio to adapt it to the subsequent processing flow. Use endpoint detection algorithms based on energy, zero-crossing rate and other features. First calculate the audio short-time energy and set an energy threshold. When the audio short-time energy exceeds the threshold, the audio start point is preliminarily determined; similarly, when the audio short-time energy continues to be lower than another lower threshold for a period of time, it is identified as the audio end point. For the zero-crossing rate, a similar threshold comparison strategy is also used to assist endpoint determination, and the results of the two are combined to determine the precise audio start point and audio end point. According to the obtained audio start point and audio end point, the sampled converted audio is trimmed. Discard the audio clips before the start point and after the end point. These clips contain invalid audio such as environmental noise, noise from device startup or shutdown, and only retain the valid middle part to generate valid audio. According to a fixed duration, specifically every 20-30 milliseconds as a frame, divide the valid audio into multiple short-time audio frames, and all these frames constitute an audio short-time frame set. When dividing the frames, appropriate overlapping parts can be set between adjacent frames, generally overlapping by 10-15 milliseconds. This can reduce the loss of audio information caused by dividing the frames and improve the accuracy of subsequent processing. Select a key frame in the audio short-time frame set as the reference frame, usually the frame with the strongest energy or located in the middle of the audio.Calculate the deviation between other frames and the reference frame on the time axis, and align all short-term audio frames on the time dimension by translating the audio data. Finally, generate pre-processed standard audio to ensure the synchronization and coordination of various parts of the audio in subsequent complex processing.

[0089] As an example of the present invention, refer to Figure 2 As shown, in this example, step S2 includes:

[0090] Step S21: performing audio output mode detection on the preprocessed standard audio and the sound device associated with the original audio, and when the number of the preprocessed standard audio and the sound device associated with the original audio is equal to 1, marking the preprocessed standard audio as single-device output audio;

[0091] Step S22: when the number of sound-emitting devices associated with the preprocessed standard audio and the original audio is greater than 1, the preprocessed standard audio is marked as multi-device output audio; audio frequency domain conversion is performed on the single-device output audio to generate single-device output audio spectrum data;

[0092] Step S23: segmenting the audio periodic regularity of the single device output audio spectrum data based on a preset periodic threshold value to generate single device short-period output spectrum data and single device long-period output spectrum data; using the sound device operation data associated with the original audio to perform audio device short-term frequency vibration noise analysis on the single device short-period output spectrum data to generate single device short-term motion frequency vibration noise;

[0093] Step S24: performing long-period environmental interference noise analysis on the long-period output spectrum data of the single device to generate long-period environmental interference noise of the single device.

[0094] In an embodiment of the present invention, a data structure is constructed to store the preprocessed standard audio and the sound device information associated therewith. The data structure can be a list or a dictionary, in which the key-value pairs correspond to the index and device identification of the audio data respectively. A list of all sound devices associated with the original audio is obtained from the device management module of the system, and the list is used together with the preprocessed standard audio as the detection object. The data structure constructed above is counted to count the number of preprocessed standard audio and the corresponding number of sound devices. A simple counter variable is used to traverse the data structure, and each time an audio-device pair is encountered, the corresponding counter is increased by 1. If the number finally counted is equal to 1, a "single device output audio" mark is added to the metadata part of the audio data; the mark can be used for the subsequent process to quickly identify the audio type and facilitate targeted processing. The counting result in step S21 is used. If the number is greater than 1, "multiple device output audio" is marked in the corresponding audio data metadata. For audio data marked as single device output audio, Fourier transform (FFT) is used to perform audio frequency domain conversion. Select a suitable FFT algorithm implementation library, specifically the NumPy library in Python. First, fill the audio time domain data with zeros to make its length meet the optimal calculation requirements of the fast Fourier transform, which is usually an integer power of 2. Then call the numpy.fft.fft function to convert the single-device output audio in the time domain into a frequency domain representation, and generate the single-device output audio spectrum data. The spectrum data is presented in the form of a complex array and contains the amplitude and phase information of each frequency component of the audio. Pre-set the period threshold, which can be obtained by statistically analyzing the audio data of a large number of similar sound-generating devices. Specifically, after preliminary experiments, the audio period threshold of a certain type of speaker is determined to be 50ms. Traverse the single-device output audio spectrum data, based on the frequency axis, and compare the energy fluctuation period of adjacent frequency bands with the preset threshold according to the periodic changes of the spectrum characteristics. When the period is less than the threshold, the data of the corresponding frequency band is divided into the single-device short-period output spectrum data; otherwise, it is divided into the single-device long-period output spectrum data. Extract the key parameters of the sound-generating device operation data associated with the original audio, such as the speed stability data of the device motor, the vibration sensor feedback information, etc. For the short-period output spectrum data of a single device, the correlation analysis of these operating data and audio spectrum data is performed. The Pearson correlation coefficient calculation method can be used to find the frequency bands where the fluctuations of the operating data are highly correlated with the short-period fluctuations of the audio spectrum. After filtering, feature extraction and other processing, noise data that can characterize the short-term frequency vibration of the device is generated, that is, the short-term motion frequency vibration noise of a single device. For the long-period output spectrum data of a single device, the noise estimation technology of machine learning or signal processing is used.A noise estimation model based on a long short-term memory network (LSTM) can be pre-trained, and long-period spectrum data can be input. After learning the long-period spectrum change rules, the model outputs the estimation result of the environmental interference noise and generates the long-period environmental interference noise of a single device. In addition, the traditional adaptive filtering method can also be used to adaptively adjust the filter parameters according to the long-term stability characteristics of the audio spectrum, filter out the spectrum components belonging to environmental interference, and thus extract the corresponding noise data.

[0095] Preferably, step S23 includes the following steps:

[0096] Step S231: extracting audio spectrum peak information from the single device output audio spectrum data to obtain the single device output audio spectrum peak data; discretizing the single device output audio spectrum peak data to generate a single device output audio spectrum peak discrete graph;

[0097] Step S232: performing height matching on the audio spectrum peak discrete points in the single device output audio spectrum peak discrete graph, mapping the audio spectrum peak discrete points at the same height to the single device output audio spectrum peak to separate the related spectrum curve segments, and obtaining the single device output same frequency curve;

[0098] Step S233: Calculate the spectrum period of the single device output same-frequency curve to obtain the same-frequency curve period; perform period threshold judgment on the same-frequency curve period based on a preset period threshold, and when the preset period threshold is greater than or equal to the same-frequency curve period, mark the corresponding single device output same-frequency curve as single device short-period output spectrum data;

[0099] Step S234: When the preset period threshold is less than the period of the same-frequency curve, the corresponding single-device output same-frequency curve is marked as a single-device long-period output spectrum data; the single-device short-period output spectrum data is analyzed for audio device short-term frequency vibration noise using the sound device operation data associated with the original audio to generate a single-device short-term motion frequency vibration noise.

[0100] In an embodiment of the present invention, by traversing the audio spectrum data output by a single device, the spectrum data is usually an array or matrix containing frequencies and corresponding amplitude values. For each frequency point, check the size relationship between its amplitude value and the amplitude value of the adjacent frequency point. If the amplitude value of a certain frequency point is greater than the amplitude value of the adjacent frequency points on both sides, then the frequency point is determined to be a spectrum peak point, and the frequency and amplitude information of these peak points are extracted, and the peak data of the single device output audio spectrum is obtained by summarizing. Specifically, the NumPy library is used in Python to process the spectrum data, and the index operation of the array is used to compare adjacent elements to quickly locate the peak point. Set appropriate discretization parameters, such as dividing the amplitude value range into several equally spaced intervals, and mapping the amplitude value of each spectrum peak point to the corresponding interval, while retaining its frequency information, so that each interval represents a peak point set of a certain amplitude range, thereby generating a single device output audio spectrum peak discrete map, which can be stored in a simple dictionary structure, with the key being the amplitude interval and the value being the set of frequency points falling within the interval. Traverse each amplitude interval in the single device output audio spectrum peak discrete map, that is, check each discrete point set. For all frequency points in the same amplitude range (discrete points of audio spectrum peaks), they have the same "height" feature. Using the frequency information corresponding to these discrete points, trace back to the original single-device output audio spectrum peak data, associate all spectrum peak points belonging to the same height range, mark the corresponding curve segments in the original spectrum data, and separate the related spectrum curve segments, so as to obtain the same-frequency curve output by a single device. These same-frequency curves reflect the law of the change of different frequency components in the audio over time, and the frequency points in the same curve have similar amplitudes. For each single-device output same-frequency curve, the autocorrelation function (ACF) method is used to calculate its spectrum period. The autocorrelation function can measure the similarity between a signal and itself at different time delays. For discrete audio spectrum data, given a time delay range, the autocorrelation value of the same-frequency curve at each delay value is calculated. The delay value corresponding to the first peak of the autocorrelation value (except for zero delay) is the estimated period of the same-frequency curve, and the same-frequency curve period is obtained. In Python, the scipy.signal.correlate function of the SciPy library can be used to conveniently implement autocorrelation calculations. The calculated period of the same frequency curve is compared with the preset period threshold, which is set based on the previous research on the audio characteristics of the sound-generating device and a large amount of experimental data. It distinguishes the short-period and long-period spectrum characteristics generated by the device. If the preset period threshold is greater than or equal to the period of the same frequency curve, it means that the audio frequency component corresponding to the same frequency curve changes rapidly, and it is marked as a single device short-period output spectrum data.On the contrary, when the preset period threshold is less than the period of the same-frequency curve, it means that the audio frequency component corresponding to the same-frequency curve changes slowly, and it is marked as a single-device long-period output spectrum data. Extract key parameters such as equipment vibration frequency and motor speed stability from the sound-emitting device operation data associated with the original audio. These parameters are jointly analyzed with the short-period output spectrum data of a single device. Specifically, a multivariate linear regression analysis is used, with the equipment operation parameters as independent variables and certain characteristic quantities in the short-period spectrum data (such as the peak amplitude change in a specific frequency band) as dependent variables to find the quantitative relationship between the two. Through this relationship, the abnormal part of the audio spectrum caused by the short-term frequency vibration of the equipment is identified, and after sorting and quantification, the short-term motion frequency vibration noise of a single device is generated.

[0101] Preferably, performing audio device short-term frequency vibration noise analysis on single device short-term output spectrum data using sound device operation data associated with the original audio includes:

[0102] Performing equipment operation vibration analysis on the sound-generating equipment operation data associated with the original audio to generate the sound-generating equipment operation vibration data; converting the sound-generating equipment operation vibration data into a vibration spectrum to generate a sound-generating equipment operation vibration spectrum diagram; and matching the suspicious spectrum segments of the single-equipment short-period output spectrum data based on the sound-generating equipment operation vibration spectrum diagram to generate the equipment operation suspicious noise spectrum segment;

[0103] Perform Hilbert transform on the suspicious noise spectrum of the equipment operation to generate a suspicious envelope signal of the equipment operation; perform instantaneous amplitude calculation on the suspicious envelope signal of the equipment operation to obtain the signal strength of the suspicious noise of the equipment operation, wherein the instantaneous amplitude calculation is as follows:

[0104] ;

[0105] In the formula, Represented as the equipment operation noise signal at time The instantaneous amplitude of Expressed as time The original equipment operating noise signal, Expressed as time The signal after Hilbert transformation, Expressed as time The original device runs the function of Hilbert transform of the noise signal;

[0106] The noise intensity average is calculated for the suspicious noise signal intensity of the equipment operation, thereby generating the average short-term operation noise intensity of a single device. The short-term frequency vibration noise of the audio device is screened for the suspicious noise spectrum segment of the equipment operation using the average short-term operation noise intensity of a single device, thereby generating the short-term motion frequency vibration noise of a single device.

[0107] In the embodiment of the present invention, various sensor data related to the vibration of the equipment are extracted from the operation data of the sound-generating equipment associated with the original audio, specifically the data of the accelerometer and the gyroscope. These original data contain noise and abnormal values, so filtering processing must be performed first. Commonly used filtering methods include low-pass filter, Kalman filter, etc. Taking the low-pass filter as an example, the cutoff frequency is set according to the vibration frequency characteristics of the equipment to filter out high-frequency noise interference, make the data smoother, and then generate the vibration data of the sound-generating equipment operation. The Fourier transform (FFT) is used to convert the vibration data of the sound-generating equipment operation from the time domain to the frequency domain to obtain the vibration spectrum of the sound-generating equipment operation. In actual operation, the numpy.fft.fft function of the NumPy library in Python is used to pre-process the time domain data by filling zeros, so that its length meets the requirements of the FFT algorithm, and then the transformation is performed to convert the relationship between the vibration amplitude of the time domain and the time change into the vibration amplitude distribution corresponding to different frequency components in the frequency domain. Compare the vibration spectrum of the sound-generating equipment operation with the short-period output spectrum data of a single device. By setting the matching threshold, the relative error threshold of the specific spectrum peak, the spectrum shape similarity threshold, etc., the key features such as the peak position and spectrum curve trend in the two spectrum graphs are compared one by one in the frequency dimension. When the similarity between the two in a specific frequency band exceeds the set threshold, the corresponding frequency band in the single device short-term output spectrum data is marked as the suspicious noise spectrum band of the device operation. The time domain signal corresponding to the suspicious noise spectrum band of the device operation is Hilbert transformed. In Python, the scipy.signal.hilbert function can be used to implement this transformation process. This function adds a 90° phase shift to the original signal to generate a suspicious envelope signal of the device operation, which helps to extract the envelope characteristics of the signal and capture the change law of the signal amplitude over time. According to the given formula A(t)=√(x(t)^2+x ̂(t)^2), x ̂(t)=H(x(t)), the instantaneous amplitude is calculated. In the code implementation, the suspicious envelope signal of the device operation and its corresponding original time domain signal are first obtained, and the formula is substituted into the calculation point by point. Specifically, in the Python environment, a loop is used to traverse the time series, and x(t) and x ̂(t) at each time point t are obtained respectively, and the square root operation is completed to obtain the suspicious noise signal strength of the device operation. The calculated suspicious noise signal strength of the device operation is averaged. In simple terms, the instantaneous amplitude values ​​of all time points are added together and then divided by the total number of time points to generate the short-term operation noise intensity mean of a single device. In Python, the numpy.mean function can be used to quickly implement this statistical calculation. Set an intensity threshold, which can be determined based on the noise intensity statistics under normal operation of the device.The average short-term operation noise intensity of a single device is compared with the intensity threshold. If the average exceeds the threshold, it means that the suspicious noise spectrum band of the corresponding device operation is indeed significantly affected by the short-term frequency vibration of the device. The frequency band is retained and the single device short-term motion frequency vibration noise is finally generated; otherwise, the frequency band is excluded.

[0108] Preferably, step S24 includes the following steps:

[0109] Step S241: performing spectrum periodicity detection on the single-device long-period output spectrum data to generate single-device spectrum periodicity detection data; performing power spectrum density analysis on the single-device long-period output spectrum data based on the single-device spectrum periodicity detection data to generate a single-device long-period power spectrum density graph;

[0110] Step S242: performing spectrum energy distribution analysis on the long-period power spectrum density diagram of a single device to generate spectrum energy distribution data; performing suspicious spectrum feature extraction on the long-period output spectrum data of the single device through the spectrum energy distribution data to obtain suspicious spectrum feature data;

[0111] Step S243: Pair the suspicious spectrum feature data with a preset environmental interference feature library, and screen the single-device long-period output spectrum data for environmental interference noise based on the result of the spectrum feature pairing, thereby generating single-device long-period environmental interference noise.

[0112] In an embodiment of the present invention, an autocorrelation function (ACF) is used to detect the periodicity of a spectrum of a single device outputting long-period spectrum data. The autocorrelation function can measure the degree of similarity between a signal and itself under different time delays. For discrete spectrum data, a suitable delay range is set, and the autocorrelation value corresponding to each delay value is calculated. The fluctuation of the autocorrelation value can reflect the periodic characteristics of the spectrum. These autocorrelation values ​​and the corresponding delay parameters are sorted out to generate the periodicity detection data of the spectrum of a single device. In Python, the autocorrelation calculation can be efficiently completed using the scipy.signal.correlate function. Based on the obtained periodicity detection data of the spectrum of a single device, a power spectrum density analysis is carried out. The power spectrum density (PSD) characterizes the distribution of signal power in the frequency domain. It is usually calculated by the square and normalization method of the fast Fourier transform (FFT). First, the long-period output spectrum data of a single device is subjected to an FFT transformation, and then the transformation result is squared, and finally divided by the data length to generate a long-period power spectrum density diagram of a single device, which intuitively presents the power distribution of each frequency band of the long-period audio spectrum. Observe the long-term power spectrum density diagram of a single device, divide it according to certain frequency intervals, and count the total energy in each frequency interval to generate spectrum energy distribution data. Specifically, you can set every 10Hz as an interval, traverse all frequency points in the power spectrum density diagram, and add up the power values ​​falling in the same interval. Use the spectrum energy distribution data to extract suspicious spectrum features from the long-term output spectrum data of a single device. Those frequency bands whose energy distribution is obviously different from the overall trend, and the specific parts where the energy suddenly jumps or drops, will be marked as suspicious spectrum feature data, involving some simple threshold judgments, such as setting a threshold for the energy change rate. When the energy change of a certain frequency band exceeds the threshold, it is judged to have suspicious features. The preset environmental interference feature library stores the spectrum features of various common environmental interferences. These features are collected, analyzed and extracted in advance by a large amount of audio data in different environments. Compare the suspicious spectrum feature data with each feature in the preset environmental interference feature library one by one. The comparison dimensions include frequency range, energy distribution mode, peak position, etc. Environmental interference noise is screened according to the results of spectrum feature pairing. If the suspicious spectrum feature data has a high degree of match with a feature in the environmental interference feature library and reaches a preset matching threshold (specifically, the matching degree exceeds 80%), then the corresponding single-device long-period output spectrum data is determined to belong to environmental interference noise, and these matching spectrum data are screened out to finally generate single-device long-period environmental interference noise.

[0113] As an example of the present invention, refer to Figure 3 As shown, in this example, step S3 includes:

[0114] Step S31: performing device cluster ground loop influence analysis on the multi-device output audio to generate the device cluster ground loop influence frequency; performing device cluster noise extraction on the multi-device output audio according to the device cluster ground loop influence frequency to obtain the multi-device cluster noise;

[0115] Step S32: Integrate data features of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise, and multi-device cluster noise to generate abnormal noise feature data;

[0116] Step S33: Divide the abnormal noise feature data into data sets to generate a model training set and a model test set; use a deep residual network algorithm to perform model training on the model training set to generate an audio noise recognition pre-model; perform model optimization iteration on the audio noise recognition pre-model through the model test set to generate an audio noise recognition model;

[0117] Step S34: performing abnormal noise pattern recognition on the preprocessed standard audio based on the audio noise recognition model to generate abnormal noise pattern recognition data.

[0118] In an embodiment of the present invention, by outputting audio for multiple devices, first collect electrical connection information related to these sound-generating devices, especially the part involving the ground loop. Analyze the grounding layout, line impedance and other parameters of the device cluster, use circuit simulation software or establish a mathematical model based on electromagnetic theory to simulate the flow of current in the ground loop. Since different ground loop settings will introduce interference of specific frequencies, the ground loop influence frequency of the device cluster is determined through this simulation analysis. Specifically, in a sound system composed of multiple speakers, improper ground connection between speakers produces low-frequency hum. After analyzing the line parameters and grounding topology, it is found that 50Hz or 60Hz (depending on the mains frequency in the area) and its harmonic frequencies are significantly affected. These frequencies are the ground loop influence frequencies of the device cluster. Using filtering technology, design corresponding bandpass or notch filters according to the determined ground loop influence frequencies of the device cluster. In digital audio processing, finite impulse response (FIR) filters or infinite impulse response (IIR) filters can be used. Pass the multi-device output audio through these filters to filter out other irrelevant frequency components, and only retain the audio signals related to the ground loop influence frequencies of the device cluster, thereby obtaining multi-device cluster noise. The data formats of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise, and multi-device cluster noise are processed in a unified manner to ensure that they can be subsequently jointly analyzed. Specifically, all different noise data are converted into a matrix form containing key features such as frequency, amplitude, and phase. Then, the feature dimensions are integrated one by one, such as merging the amplitude features of the three noises at the same frequency point into a new vector, and similarly integrating the phase features to generate abnormal noise feature data, which comprehensively covers the key features of noise from different sources and different characteristics. Usually, the abnormal noise feature data is divided into a model training set and a model test set according to a certain ratio, such as the common 80:20 or 70:30. Random sampling can be used to randomly extract the corresponding proportion of data from the entire abnormal noise feature data set using the random number generation function in the programming language to form a training set and a test set respectively. The model training is carried out using the deep residual network (ResNet) algorithm. Initialize the ResNet architecture, adjust the input layer size of the network according to the dimension of the abnormal noise feature data, and configure the appropriate number of hidden layers, number of neurons, and activation functions, such as the ReLU function. The model training set is input into the network. Based on the back propagation algorithm, the network learns the relationship between noise features and category labels (specific normal / abnormal), gradually adjusts the network weights, and generates an audio noise recognition pre-model. The audio noise recognition pre-model is evaluated using the model test set, and evaluation indicators such as accuracy, recall, and F1 value are calculated.According to the evaluation results, the network's hyperparameters, such as learning rate and regularization parameters, are adjusted, and training is performed again. This process is repeated until the model performance reaches a satisfactory stable state on the test set, thereby generating an audio noise recognition model. The preprocessed standard audio is converted into a feature representation consistent with that during training and input into the audio noise recognition model. Based on the learned noise feature pattern, the model identifies the input audio to determine whether there is abnormal noise and what kind of abnormal noise pattern it belongs to, and finally generates abnormal noise pattern recognition data, providing a basis for subsequent audio quality improvement, fault diagnosis and other applications.

[0119] Preferably, performing device cluster ground loop impact analysis on multiple device output audios includes:

[0120] Perform device crosstalk on audio data output by multiple devices to generate audio crosstalk data of multiple devices; perform ground impedance measurement on audio crosstalk data of multiple devices to generate impedance characteristic data;

[0121] Perform loop current and ground noise correlation analysis on impedance characteristic data to generate cluster current noise interference data; perform electromagnetic interference frequency response calculation on cluster current noise interference data to generate equipment cluster electromagnetic interference response characteristic data;

[0122] The distortion and phase shift of the electromagnetic interference response characteristic data of the equipment cluster are evaluated to generate the quality parameter data of the equipment cluster; the loop stability and frequency characteristics of the equipment cluster quality parameter data are extracted to obtain the ground loop influence frequency of the equipment cluster.

[0123] In an embodiment of the present invention, when multiple devices work at the same time, the audio signal will generate crosstalk between the devices through electromagnetic coupling, line capacitance, etc. First, an electromagnetic model of multi-device audio transmission is constructed, taking into account physical factors such as the distance between devices, placement angle, and layout of connecting cables. Electromagnetic simulation software (such as CST, ANSYS, etc.) or numerical solutions based on Maxwell's equations are used to simulate the propagation process of audio signals in this complex environment, thereby obtaining the audio components of mutual crosstalk between devices and generating multi-device audio crosstalk data. In actual test scenarios, high-precision audio acquisition equipment can also be used to simultaneously collect audio output from multiple devices, and separate the additional signal components introduced into the audio output of each device due to crosstalk through signal processing algorithms (such as correlation analysis, spectrum subtraction, etc.). For the devices involved in the multi-device audio crosstalk data, a professional impedance measuring instrument (such as an LCR tester) is used to connect the test probe to the ground terminal of the device, and the ground impedance of the device is measured point by point according to a predetermined measurement frequency range (usually covering the audio frequency band and its nearby low-frequency range, specifically 20Hz-2kHz). By collating the data of impedance value changes with frequency, impedance characteristic data are generated, which reflects the electrical characteristics of the equipment grounding loop at different frequencies. According to Kirchhoff's law, combined with impedance characteristic data, the current distribution in the grounding loop is analyzed. The circuit equation is constructed to solve the current magnitude and phase of each branch in the grounding loop under different audio signal excitation. At the same time, the audio sensor is used to collect ground noise signals in the environment around the equipment, and the loop current data and the ground noise signal are jointly analyzed in the time domain and frequency domain. Specifically, the correlation between the two in time is calculated through the cross-correlation function, and the corresponding relationship between the two in frequency is checked by spectrum analysis, the key frequency band and amplitude relationship of the ground noise caused by current fluctuations are found, and cluster current noise interference data is generated. Based on the cluster current noise interference data, the shielding and conduction effects of factors such as equipment casing and internal circuit layout on electromagnetic interference are considered, and the electromagnetic transmission line theory and antenna theory are used to model. Calculate the transmission loss, reflection coefficient and other parameters at different frequencies when electromagnetic interference is inside the equipment cluster and radiated externally, and obtain the electromagnetic interference response characteristic data of the equipment cluster. This process involves complex matrix operations. In terms of software implementation, use professional electromagnetic calculation libraries or write your own linear algebra-based algorithm code to simulate the response of electromagnetic interference with frequency changes. Perform an inverse Fourier transform on the electromagnetic interference response characteristic data of the equipment cluster to obtain the electromagnetic interference response signal in the time domain. Compare it with the original audio signal and calculate the distortion of the signal. The total harmonic distortion (THD) indicator can be used to decompose the signal into harmonic components through fast Fourier transform, and calculate the ratio of harmonic energy to fundamental wave energy.At the same time, measure the phase shift of the signal, count the phase change of the electromagnetic interference response signal relative to the original audio signal at different frequency points, summarize key parameters such as distortion and phase shift, and generate equipment cluster quality parameter data. Analyze the equipment cluster quality parameter data to check the changes in loop stability at different frequencies. Specifically, when the distortion suddenly increases and the phase shift fluctuates violently, the corresponding frequency is likely to be seriously affected by the ground loop problem. By traversing all frequency points, extract those frequency bands associated with loop instability, and finally determine the frequency affecting the ground loop of the equipment cluster.

[0124] Preferably, step S4 comprises the following steps:

[0125] Step S41: performing noise power spectrum conversion on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the cluster noise of multiple devices according to the abnormal noise pattern recognition data to generate an abnormal noise power spectrum;

[0126] Step S42: Based on the abnormal noise power spectrum, the power spectrum of the original audio at each time frame is subtracted to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed by inverse Fourier transform to obtain a noise-reduced audio signal.

[0127] In the embodiment of the present invention, for the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the cluster noise of multiple devices, these noise data need to be converted to the frequency domain by Fourier transform (FFT) in the time domain, so as to generate the corresponding power spectrum. Taking the short-term motion frequency vibration noise of a single device as an example, the noise signal in the time domain is first discrete Fourier transformed to obtain a complex number representation in the frequency domain. Then, the square of the modulus value of the complex number is calculated to obtain the power value corresponding to each frequency point, and then the power spectrum of the short-term motion frequency vibration noise of the single device is generated. According to the same method, the long-period environmental interference noise of a single device and the cluster noise of multiple devices are processed, and finally the abnormal noise power spectrum is obtained according to the abnormal noise pattern recognition data integration. In Python, the fast Fourier transform is realized with the help of numpy.fft.fft function, and then the power spectrum is obtained by simple mathematical operations. The original audio is often framed in the preprocessing stage, and the audio data of each time frame is obtained. For each time frame, the corresponding power spectrum is subtracted from the abnormal noise power spectrum to obtain the effective audio power spectrum after noise reduction. Then, the inverse Fourier transform (IFFT) is used to convert it from the frequency domain back to the time domain to achieve audio reconstruction. Since the power spectrum is real and non-negative, it must be converted to complex form before the inverse Fourier transform is performed to supplement the phase information (generally assuming the phase is 0). In Python, the numpy.fft.ifft function is used to perform the inverse Fourier transform. The transformed time domain data is sorted to finally obtain the denoised audio signal, which removes all kinds of abnormal noise components identified in the early stage.

Claims

1. An audio noise reduction method based on deep learning, characterized in that: The following steps are involved: Step S1: obtaining original audio and sound device operation data associated with the original audio; Invalid audio is removed from the original audio to generate valid audio; audio time alignment is performed on the valid audio to generate pre-processed standard audio; Step S2: Perform audio output mode detection on the preprocessed standard audio and the sound-generating device associated with the original audio to generate single-device output audio and multi-device output audio; segment the single-device output audio according to the audio periodic regularity to generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise; Step S2 includes the following steps: Step S21: performing audio output mode detection on the preprocessed standard audio and the sound device associated with the original audio, and when the number of the preprocessed standard audio and the sound device associated with the original audio is equal to 1, marking the preprocessed standard audio as single-device output audio; Step S22: when the number of sound-emitting devices associated with the preprocessed standard audio and the original audio is greater than 1, the preprocessed standard audio is marked as multi-device output audio; audio frequency domain conversion is performed on the single-device output audio to generate single-device output audio spectrum data; Step S23: segmenting the audio periodic regularity of the single device output audio spectrum data based on a preset periodic threshold value to generate single device short-period output spectrum data and single device long-period output spectrum data; using the sound device operation data associated with the original audio to perform audio device short-term frequency vibration noise analysis on the single device short-period output spectrum data to generate single device short-term motion frequency vibration noise; Step S24: performing long-period environmental interference noise analysis on the long-period output spectrum data of the single device to generate long-period environmental interference noise of the single device; Step S3: extract the ground loop noise of the device cluster from the audio output of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data; Step S3 includes the following steps: Step S31: performing device cluster ground loop influence analysis on the multi-device output audio to generate the device cluster ground loop influence frequency; performing device cluster noise extraction on the multi-device output audio according to the device cluster ground loop influence frequency to obtain the multi-device cluster noise; Step S32: integrating data features of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise, and multi-device cluster noise to generate abnormal noise feature data; Step S33: Divide the abnormal noise feature data into data sets to generate a model training set and a model test set; use a deep residual network algorithm to perform model training on the model training set to generate an audio noise recognition pre-model; perform model optimization iteration on the audio noise recognition pre-model through the model test set to generate an audio noise recognition model; Step S34: performing abnormal noise pattern recognition on the preprocessed standard audio based on the audio noise recognition model to generate abnormal noise pattern recognition data; Step S4: Based on the abnormal noise pattern recognition data, the power spectrum of the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise is subtracted in each time frame to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

2. The audio noise reduction method based on deep learning according to claim 1, characterized in that: Step S1 includes the following steps: Step S11: obtaining original audio and sound device operation data associated with the original audio; Step S12: performing audio sampling rate conversion on the original audio to generate sampling converted audio; Step S13: performing audio endpoint detection on the sampled converted audio to obtain an audio start point and an audio end point; based on the audio start point and the audio end point, performing invalid audio elimination on the sampled converted audio to generate valid audio; Step S14: Divide the effective audio into audio short-time frames to generate an audio short-time frame set; and use the audio short-time frame set to perform audio time alignment on the effective audio, thereby generating pre-processed standard audio.

3. The audio noise reduction method based on deep learning according to claim 1, characterized in that: Step S23 includes the following steps: Step S231: extracting audio spectrum peak information from the single device output audio spectrum data to obtain the single device output audio spectrum peak data; discretizing the single device output audio spectrum peak data to generate a single device output audio spectrum peak discrete graph; Step S232: performing height matching on the audio spectrum peak discrete points in the single device output audio spectrum peak discrete graph, mapping the audio spectrum peak discrete points at the same height to the single device output audio spectrum peak to separate the related spectrum curve segments, and obtaining the single device output same frequency curve; Step S233: Calculate the spectrum period of the single device output same-frequency curve to obtain the same-frequency curve period; perform period threshold judgment on the same-frequency curve period based on a preset period threshold, and when the preset period threshold is greater than or equal to the same-frequency curve period, mark the corresponding single device output same-frequency curve as single device short-period output spectrum data; Step S234: When the preset period threshold is less than the period of the same-frequency curve, the corresponding single-device output same-frequency curve is marked as a single-device long-period output spectrum data; the single-device short-period output spectrum data is analyzed for audio device short-term frequency vibration noise using the sound device operation data associated with the original audio to generate a single-device short-term motion frequency vibration noise.

4. The audio noise reduction method based on deep learning according to claim 3, characterized in that: Using the sound device operation data associated with the original audio to analyze the short-term frequency vibration noise of the audio device on the short-term output spectrum data of a single device includes: Performing equipment operation vibration analysis on the sound-generating equipment operation data associated with the original audio to generate the sound-generating equipment operation vibration data; converting the sound-generating equipment operation vibration data into a vibration spectrum to generate a sound-generating equipment operation vibration spectrum diagram; and matching the suspicious spectrum segments of the single-equipment short-period output spectrum data based on the sound-generating equipment operation vibration spectrum diagram to generate the equipment operation suspicious noise spectrum segment; Perform Hilbert transform on the suspicious noise spectrum of the equipment operation to generate a suspicious envelope signal of the equipment operation; perform instantaneous amplitude calculation on the suspicious envelope signal of the equipment operation to obtain the signal strength of the suspicious noise of the equipment operation, wherein the instantaneous amplitude calculation is as follows: ； In the formula, Represented as the equipment operation noise signal at time The instantaneous amplitude of Expressed as time The original equipment operating noise signal, Expressed as time The signal after Hilbert transformation, Expressed as time The original device runs the function of Hilbert transform of the noise signal; The noise intensity average is calculated for the suspicious noise signal intensity of the equipment operation, thereby generating the average short-term operation noise intensity of a single device. The short-term frequency vibration noise of the audio device is screened for the suspicious noise spectrum segment of the equipment operation using the average short-term operation noise intensity of a single device, thereby generating the short-term motion frequency vibration noise of a single device.

5. The audio noise reduction method based on deep learning according to claim 1, characterized in that: Step S24 includes the following steps: Step S241: performing spectrum periodicity detection on the single-device long-period output spectrum data to generate single-device spectrum periodicity detection data; performing power spectrum density analysis on the single-device long-period output spectrum data based on the single-device spectrum periodicity detection data to generate a single-device long-period power spectrum density graph; Step S242: performing spectrum energy distribution analysis on the long-period power spectrum density diagram of a single device to generate spectrum energy distribution data; performing suspicious spectrum feature extraction on the long-period output spectrum data of the single device through the spectrum energy distribution data to obtain suspicious spectrum feature data; Step S243: Pair the suspicious spectrum feature data with a preset environmental interference feature library, and screen the single-device long-period output spectrum data for environmental interference noise based on the result of the spectrum feature pairing, thereby generating single-device long-period environmental interference noise.

6. The audio noise reduction method based on deep learning according to claim 1, characterized in that: Analysis of the impact of device cluster ground loops on multi-device audio outputs includes: Perform device crosstalk on audio data output by multiple devices to generate audio crosstalk data of multiple devices; perform ground impedance measurement on audio crosstalk data of multiple devices to generate impedance characteristic data; Perform loop current and ground noise correlation analysis on impedance characteristic data to generate cluster current noise interference data; perform electromagnetic interference frequency response calculation on cluster current noise interference data to generate equipment cluster electromagnetic interference response characteristic data; The distortion and phase shift of the electromagnetic interference response characteristic data of the equipment cluster are evaluated to generate the quality parameter data of the equipment cluster; the loop stability and frequency characteristics of the equipment cluster quality parameter data are extracted to obtain the ground loop influence frequency of the equipment cluster.

7. The audio noise reduction method based on deep learning according to claim 1, characterized in that: Step S4 includes the following steps: Step S41: performing noise power spectrum conversion on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the cluster noise of multiple devices according to the abnormal noise pattern recognition data to generate an abnormal noise power spectrum; Step S42: Based on the abnormal noise power spectrum, the power spectrum of the original audio at each time frame is subtracted to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed by inverse Fourier transform to obtain a noise-reduced audio signal.

8. An audio noise reduction system based on deep learning, characterized in that: For executing the audio noise reduction method based on deep learning as claimed in claim 1, the audio noise reduction system based on deep learning comprises: The audio preprocessing module is used to obtain the original audio and the sound device operation data associated with the original audio; remove invalid audio from the original audio to generate valid audio; perform audio time alignment on the valid audio to generate preprocessed standard audio; The single-device noise extraction module is used to detect the audio output mode of the pre-processed standard audio and the sound-generating device associated with the original audio, and generate single-device output audio and multi-device output audio; segment the single-device output audio according to the audio period regularity, and generate single-device short-period output spectrum data and single-device long-period output spectrum data; perform hierarchical noise extraction on the single-device short-period output spectrum data and the single-device long-period output spectrum data to obtain single-device short-term motion frequency vibration noise and single-device long-period environmental interference noise; The multi-device noise extraction module is used to extract the ground loop noise of the device cluster from the output audio of multiple devices to obtain the multi-device cluster noise; perform abnormal noise pattern recognition on the short-term motion frequency vibration noise of a single device, the long-period environmental interference noise of a single device, and the multi-device cluster noise to generate abnormal noise pattern recognition data; The audio reconstruction module is used to subtract the power spectrum of single-device short-term motion frequency vibration noise, single-device long-period environmental interference noise and multi-device cluster noise in each time frame according to the abnormal noise pattern recognition data to generate an effective audio power spectrum after noise reduction; the effective audio power spectrum after noise reduction is reconstructed through inverse Fourier transform to obtain a noise-reduced audio signal.

Images