A Multi-Scene Sound Denoising Method Based on Operating System

Abstract

A multi-scenario audio denoising method based on an operating system includes: obtaining the audio application calling pulseaudio; parsing the application name of the audio application through the pulseaudio API; determining the denoising scenario corresponding to the application name through a denoising scenario setting service; determining the denoising strategy of the audio application based on the denoising scenario; determining the denoising parameters corresponding to the denoising scenario through a denoising parameter adjustment service; and denoising the audio stream of the audio application based on the denoising parameters and the denoising strategy. This invention can dynamically adjust the denoising parameters according to the denoising scenario, reducing resource consumption while improving the user's audio experience, providing the best denoising effect, and maintaining good compatibility with mainstream operating systems and applications.

Description

Technical Field

[0001] This invention relates to the field of audio denoising technology, specifically to a multi-scene sound denoising method based on an operating system. Background Technology

[0002] With the efforts of researchers and the expansion of market opportunities, domestically developed operating systems have gradually grown stronger and more mature. While making progress and breakthroughs, the continuous development of domestically produced sound cards has led to increasingly mature hardware support for various high-resolution audio formats. At the same time, technologies such as VR / AR have increased user demand for immersive audio.

[0003] Most mainstream desktop operating systems currently use pulseaudio by default as an intermediary to link audio device drivers and applications, thereby managing and processing audio streams. However, its audio noise reduction capabilities are limited to specific scenarios and cannot achieve ideal noise reduction results tailored to actual needs.

[0004] To achieve the desired noise reduction effect, there is an urgent need for a noise reduction method that can meet users' daily needs for automatic sound adjustment and noise reduction while reducing the unnecessary consumption of CPU resources, while ensuring good compatibility with mainstream operating systems and applications. Summary of the Invention

[0005] To overcome the above-mentioned shortcomings, this invention proposes a multi-scene audio denoising method based on an operating system, which solves the problems of limited audio denoising scenarios and lack of customizable adjustment.

[0006] This invention provides a multi-scene sound denoising method based on an operating system, comprising the following steps:

[0007] S1, obtain the audio application that calls pulseaudio, and parse the application name of the audio application through the pulseaudio API;

[0008] S2, the noise reduction scene corresponding to the application name is determined through the noise reduction scene setting service, and the noise reduction strategy of the audio application is determined according to the noise reduction scene; the noise reduction scene setting service is set in ukui-media;

[0009] S3 determines the denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service, and denoises the audio stream of the audio application according to the denoising parameters and denoising strategy; wherein, the denoising parameter adjustment service is set in ukui-media.

[0010] A further improvement of the present invention is that it further includes:

[0011] S0, Draw the denoising operation panel in the ukui-media denoising module. The denoising operation panel includes a denoising scene setting panel and a graphical operation panel for adjusting denoising parameters.

[0012] The noise reduction scene settings panel is used to display the preset noise reduction scene types and their corresponding noise reduction scene buttons. The noise reduction scene buttons are used to obtain the scene value of the audio application.

[0013] The graphical user interface for adjusting noise reduction parameters includes a noise reduction parameter slider, which is used to adjust and obtain the noise reduction parameter values ​​for audio applications.

[0014] A further improvement of the present invention is that the preset noise reduction scenarios include: real-time call scenarios, audio-visual playback scenarios, and game scenarios.

[0015] A further improvement of the present invention is that the specific process of determining the noise reduction scene corresponding to the application name through the noise reduction scene setting service is as follows:

[0016] Select the noise reduction scene button based on the application name to obtain the scene value;

[0017] In ukui-media, set up a denoising scene setting service and a denoising scene signal, which includes scene values; connect to the denoising scene setting service via QDBusConnection:sessionBus and register the denoising scene signal to the session bus;

[0018] Read the denoising scene signal from the session bus and call handle_scence. Determine the denoising scene corresponding to the application name based on the scene value in the denoising scene signal, that is, the denoising scene of the audio application.

[0019] A further improvement of the present invention is that the specific process of determining the denoising parameters corresponding to the denoising scene through the denoising parameter adjustment service is as follows:

[0020] The noise reduction parameter values ​​are obtained by sliding the noise reduction parameter slider according to the noise reduction scenario.

[0021] In ukui-media, set up a denoising parameter adjustment service and a denoising parameter signal, which includes denoising parameter values; connect to the denoising parameter adjustment service via QDBusConnection:sessionBus and register the denoising parameter signal to the session bus;

[0022] Read the denoising parameter signal from the session bus, and determine the denoising parameter corresponding to the denoising scenario based on the denoising parameter value in the denoising parameter signal.

[0023] A further improvement of the present invention is that the denoising strategy of the audio application is determined according to the denoising scenario. Specifically, when the denoising scenario is a real-time call scenario, the WebRTC is called to configure the AGC automatic gain and NS noise suppression modules, and the captured audio stream is denoised through the AGC automatic gain and NS noise suppression modules.

[0024] A further improvement of the present invention is that, in determining the denoising strategy for the audio application based on the denoising scenario, it further includes: when the denoising scenario is an audio-visual playback scenario, calling the Wavelet++ library to denoise the captured audio stream according to the wavelet transform method; wherein, the wavelet transform method includes wavelet decomposition, thresholding, and wavelet reconstruction.

[0025] A further improvement of the present invention is that, in determining the denoising strategy for the audio application based on the denoising scenario, it further includes: when the denoising scenario is a game scenario, implementing Wiener filtering through noise estimation and short-time Fourier transform to denoise the captured audio stream.

[0026] A further improvement of the present invention is that the audio stream of the audio application is denoised according to the denoising parameters and the denoising strategy, specifically as follows:

[0027] Capture undenoised audio streams from audio applications using pulseaudio;

[0028] Denoise the audio stream that has not been denoised by the audio application based on the denoising parameters and denoising strategy;

[0029] PulseAudio returns the denoised audio stream to the audio application.

[0030] The beneficial effects of this invention are:

[0031] By obtaining the name of the audio application using the audio device, a noise reduction scenario is set for each application name. Noise reduction parameters can be dynamically adjusted based on the scenario, reducing resource consumption while improving the user's audio experience and providing optimal noise reduction. Furthermore, by setting different noise reduction strategies for different audio applications and combining them with dynamically adjustable noise reduction parameters, appropriate noise reduction parameter settings can be applied to different scenarios, meeting users' daily sound noise reduction needs while reducing unnecessary CPU resource consumption. The method in this invention is designed based on a domestically developed operating system, ensuring good compatibility with mainstream operating systems and applications. Attached Figure Description

[0032] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a logical architecture diagram of an embodiment of the present invention;

[0034] Figure 2 This is a technical architecture diagram of an embodiment of the present invention;

[0035] Figure 3 This is a schematic diagram of a front-end interface according to an embodiment of the present invention;

[0036] Figure 4 This is a logical architecture diagram of setting a noise reduction scene based on the application name according to an embodiment of the present invention;

[0037] Figure 5 This is a logical architecture diagram of a graphical adjustment of noise reduction parameters according to an embodiment of the present invention. Detailed Implementation

[0038] To gain a better understanding of the technical solution and beneficial effects of the present invention, the technical solution of the present invention and its beneficial effects are described in detail below with reference to the accompanying drawings.

[0039] Definitions of abbreviations and key terms:

[0040] ukui-media: The audio settings component used by the Kirin V10 desktop system, which includes modules such as volume adjustment, sound device selection, and noise reduction.

[0041] PulseAudio is a cross-platform audio system composed of multiple modules that simplifies audio management in operating systems and provides various audio devices and protocols. PulseAudio is a fundamental and essential component in modern desktop and server operating systems.

[0042] Filtering: A technique in signal processing used to remove noise from a signal or extract useful signal components.

[0043] D-Bus: Connects various processes to form a message bus, enabling the sending and receiving of messages between processes on the bus.

[0044] Figure 1 This is a logical architecture diagram of an embodiment of the present invention, such as... Figure 1As shown, the present invention provides a multi-scenario sound denoising method based on an operating system. This method is implemented on a domestically produced terminal, equipped with a Lenovo UC02 sound card and the Galaxy Kylin V10 desktop system. After deploying the method of the present invention, Tencent Meeting, Kylin Video and Kylin Music are opened respectively. Corresponding denoising scenarios can be set respectively, and the best sound quality experience can be obtained by adjusting the denoising parameters, while not occupying too many system resources.

[0045] Figure 2 This is a technical architecture diagram of an embodiment of the present invention, such as... Figure 2 As shown, the audio settings components include ukui-media, kylin-music, and kylin-video. These components are centrally managed by pulseaudio, which is also initialized and protected by denoise-daemon. Audio signals are input from devices such as microphones, processed by the sound card, and then passed to the kernel for noise reduction. After processing, the audio stream is output through output devices such as headphones and speakers via the sound card.

[0046] Combination Figure 1 As shown, the noise reduction process of the present invention is as follows:

[0047] S1, Add a graphical user interface to the noise reduction module:

[0048] In the denoising module of ukui-media, create a denoising operation panel that includes a denoising scene settings panel and a graphical interface for adjusting denoising parameters. For example... Figure 3 As shown, Figure 3 In the front-end interface, the button section on the left is the noise reduction scene setting panel, and the slider section on the right is the graphical operation interface for adjusting noise reduction parameters. The noise reduction scene setting panel and the graphical operation interface for adjusting noise reduction parameters correspond one-to-one. The noise reduction scene setting panel displays the preset noise reduction scene types and their corresponding noise reduction scene buttons, which are used to obtain the scene values ​​for the audio application. The graphical operation interface for adjusting noise reduction parameters has a noise reduction parameter slider used to adjust and obtain the noise reduction parameter values ​​for the audio application. Each noise reduction scene has a corresponding noise reduction parameter slider to dynamically adjust the noise reduction parameter values. In this embodiment, the preset noise reduction scene types include: real-time call scene, audio / video playback scene, and game scene. When the smart noise reduction button in the operating system is turned off, the noise reduction operation panel is hidden; when the smart noise reduction button is turned on, the hidden noise reduction operation panel is displayed.

[0049] When an audio application calls PulseAudio to use an audio device, the noise reduction module uses the PulseAudio API to parse the name of the application currently using the audio device (i.e., the application name of the audio application) and displays that application name on the noise reduction panel. Figure 3The application name is located in the front-end interface shown.

[0050] S2, In this embodiment, the front-end interface and the back-end service communicate via the D-Bus service; therefore, the D-Bus service is used to set the denoising scene. The denoising scene setting service determines the denoising scene corresponding to the application name (corresponding to...). Figure 1 Step 2: Set the noise reduction scene according to the application name and Figure 4 Step 2: Set the noise reduction scene according to the application name. The process is as follows:

[0051] S201, please continue reading Figure 3 When an audio application is in use, its name will be displayed on the front-end interface. Users can select a noise reduction scene based on the application name. Clicking the corresponding noise reduction scene button will display the selected scene value for the current audio application. Since the noise reduction scene button is on the front-end interface, and service processes and other business logic need to be handled on the back-end, the following steps are designed to transmit the scene value of the front-end noise reduction scene button to the back-end service via the session bus for subsequent settings:

[0052] In the ukui-media denoising module, a custom denoising scene setting service D-Bus1 is defined: the object path is " / come / example / button", the interface name is "come.example.button", the denoising scene signal is scenariochange, and the denoising scene signal includes the scene value. In this embodiment, scenariochange is the scene value of the denoising scene button.

[0053] Connect to D-Bus1 via QDBusConnection:sessionBus and register the denoised scene signal to the session bus. Register the matching rules and read the scenechange in a loop.

[0054] The `handle_scence` function is called to determine and set the noise reduction scenario based on the `scencechange` value; that is, the noise reduction scenario for the audio application. A `scencechange` value of 0 corresponds to a real-time call scenario, a `scencechange` value of 1 corresponds to a video / audio playback scenario, and a `scencechange` value of 2 corresponds to a gaming scenario. Figure 4 Step 2.1: Customizing D-Bus service 1 by setting the noise reduction scene via handle_scence

[0055] After determining the noise reduction scenario for the audio application, the process of determining the noise reduction strategy for the audio application based on the noise reduction scenario is as follows:

[0056] S202, real-time calls have high requirements for audio real-time performance. When the denoising scenario is a real-time call scenario, a custom `meeting_denoise` function in `denoise-daemon` calls WebRTC to configure the AGC automatic gain control and NS noise suppression modules to denoise the captured audio stream (corresponding to...). Figure 4 Step 2.2: Noise reduction in real-time call scenarios using metting_denoise.

[0057] S203, the main purpose of audio / video playback is to ensure that the timbre and pitch of the music are not affected by noise. When the denoising scenario is an audio / video playback scenario, a custom `video_denoise` is defined in `denoise-daemon` to call the Wavelet++ library to denoise the captured audio stream using wavelet transform. The wavelet transform includes wavelet decomposition, thresholding, and wavelet reconstruction (corresponding to...). Figure 4 Step 2.3: Noise reduction in the music playback scene is achieved through video_denoise.

[0058] S204, game scene audio is highly interactive. When the denoising scene is a game scene, Wiener filtering denoising is achieved through noise estimation and short-time Fourier transform in the custom `game_denoise` function within `denoise-daemon`. Figure 4 Step 2.4: Noise reduction of the game scene using game_denoise.

[0059] It should be noted that the technical means in the denoising strategy of this embodiment (such as AGC automatic gain control) are existing technologies. Since the focus of this embodiment is on how to perform multi-scene sound denoising and dynamically adjust the denoising parameters, the principle of the technical means will not be described in detail.

[0060] S3, similar to S2, also uses the D-Bus service to transmit denoising parameters from the front-end interface to the back-end service. The denoising parameters are adjusted graphically (corresponding to...). Figure 1 Step 3: Graphically adjust noise reduction parameters and Figure 5 Step 3: Graphically adjust the noise reduction parameters. The process is as follows:

[0061] In S301, when the noise reduction scene button is clicked in S201, a noise reduction parameter slider will appear. By adjusting the noise reduction parameter slider, the noise reduction parameter values ​​of the audio application can be adjusted to different degrees. The process of transmitting the noise reduction parameter values ​​of the front-end interface to the back-end service through the session bus is as follows:

[0062] In the ukui-media denoising module, a custom denoising parameter adjustment service D-Bus2 is defined: the object path is " / come / example / adjust", the interface name is " / come / example / adjust", and a dictionary-type signal signal_data={adjust, value} is defined as the denoising parameter signal, which includes the denoising parameter value.

[0063] Connect to D-Bus2 via QDBusConnection:sessionBus and register the denoising parameter signal to the session bus. The registered matching rules loop through signal_data, using the denoising parameter values ​​in signal_data as the denoising parameters for the audio application. (Corresponding to...) Figure 5 Step 3.1: Customize D-Bus service 2 by adjusting the corresponding scene denoising parameters using denoise_adjust.

[0064] After determining the denoising parameters for the audio application, the denoising process for the audio stream of the audio application based on the denoising parameters and denoising strategy is as follows:

[0065] S302, when the denoising scenario is a real-time call scenario, the denoising parameters adjusted for the real-time call scenario in the front-end interface denoising operation panel are received through a custom `metting_adjust`, and the processed parameters are passed to `meeting_denoise` to implement denoising (i.e., the denoising strategy for the real-time call scenario in S202, which also corresponds to...). Figure 5 Step 3.2: Dynamic parameter tuning and noise reduction in real-time call scenarios using `metting_adjust`. The adjustable noise reduction parameters are as follows:

[0066] The target level adjustment range for AGC (Automatic Gain Control) is -20dBFs to 10dBFs, with a default value of -12dBFs; the compression gain adjustment range for AGC is 5dB to 20dB, with a default value of 12dB; the suppression levels for the NS (Noise Suppression) module include low, medium, and high, with a default value of low.

[0067] S303, when the denoising scenario is an audio-visual playback scenario, the denoising parameters adjusted for the audio-visual playback scenario in the front-end interface denoising operation panel are received through a custom `video_adjust`, and the processed parameters are passed to `video_denoise` to implement denoising (i.e., the denoising strategy for the audio-visual playback scenario in S203, which also corresponds to...). Figure 5 Step 3.3: Dynamic parameter tuning and noise reduction in the music playback scene using video_adjust. The adjustable noise reduction parameters are as follows:

[0068] The number of decomposition layers is 3 to 5, with a default of 4; the scale factor is 2 to 50, with a default of 25; and the threshold is 0.01 to 1, with a default of 0.5.

[0069] S304, when the denoising scene is a game scene, receives the denoising parameters adjusted for the game scene from the front-end interface's denoising operation panel through a custom `game_adjust`, and passes the processed parameters to `game_denoise` to implement denoising (i.e., the denoising strategy for the game scene in S204, which also corresponds to...). Figure 5 Step 3.4: Dynamic parameter tuning and noise reduction of the game scene using game_adjust. The adjustable noise reduction parameters are as follows:

[0070] The noise estimation window length is 100 to 200 ms, with a default value of 150 ms; the spectral smoothing coefficient is 0 to 1, with a default value of 0.5; the frame length is 5 ms to 90 ms, with a default value of 40 ms; and the frame shift is 25% to 75%, with a default value of 50%.

[0071] It should be noted that the "processing" mentioned in S302-S304 refers to the conventional processing methods for parameters.

[0072] In this embodiment, a custom denoise.service is implemented to enable denoise-dameon to start automatically at boot and run in the background, and the denoise-dameon daemon backend service is used for (corresponding) Figure 1 Step 4: denoise-dameon daemon Figure 4 The "denoise-dameon daemon" and Figure 5 The "denoise-daemon" in the context of pulseaudio initializes the pulseaudio context and main loop by calling the audio_stream_capture function. Figure 1 The "System Startup" option captures the undenoised audio stream from pulseaudio (corresponding to...). Figure 1 In the "Capture Audio Stream" section, steps S302-S304 are executed to denoise the undenoised audio stream and write the denoised audio streams from each denoising scenario back to pulseaudio for use by the audio application (corresponding to...). Figure 1 (referring to "returning the denoised audio stream" in the text).

[0073] The beneficial effects of this invention are as follows:

[0074] By obtaining the name of the audio application using the audio device, a noise reduction scenario is set for each application name. Noise reduction parameters can be dynamically adjusted based on the scenario, reducing resource consumption while improving the user's audio experience and providing optimal noise reduction. Furthermore, by setting different noise reduction strategies for different audio applications and combining them with dynamically adjustable noise reduction parameters, appropriate noise reduction parameter settings can be applied to different scenarios, meeting users' daily sound noise reduction needs while reducing unnecessary CPU resource consumption. The method in this invention is designed based on a domestically developed operating system, ensuring good compatibility with mainstream operating systems and applications.

[0075] Although the present invention has been described using the above preferred embodiments, it is not intended to limit the scope of protection of the present invention. Any changes and modifications made by those skilled in the art to the above embodiments without departing from the spirit and scope of the present invention shall still fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be defined by the claims.

Claims

1. A multi-scene sound denoising method based on an operating system, characterized in that, include: S0, Draw the denoising operation panel in the ukui-media denoising module. The denoising operation panel includes a denoising scene setting panel and a graphical operation panel for adjusting denoising parameters. The noise reduction scene settings panel is used to display the preset noise reduction scene types and their corresponding noise reduction scene buttons. The noise reduction scene buttons are used to obtain the scene value of the audio application. The graphical user interface for adjusting noise reduction parameters includes a noise reduction parameter slider, which is used to adjust and obtain the noise reduction parameter values ​​for audio applications. S1, obtain the audio application that calls pulseaudio, and parse the application name of the audio application through the pulseaudio API; S2, the noise reduction scene corresponding to the application name is determined through the noise reduction scene setting service, and the noise reduction strategy of the audio application is determined according to the noise reduction scene; the noise reduction scene setting service is set in ukui-media; S3 determines the denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service, and denoises the audio stream of the audio application according to the denoising parameters and denoising strategy; wherein, the denoising parameter adjustment service is set in ukui-media; The audio stream of the audio application is denoised according to the denoising parameters and denoising strategy, specifically as follows: Capture undenoised audio streams from audio applications using pulseaudio; Denoise the audio stream that has not been denoised by the audio application based on the denoising parameters and denoising strategy; The denoised audio stream is returned to the audio application via pulseaudio; The specific process of determining the noise reduction scene corresponding to the application name through the noise reduction scene setting service is as follows: Select the noise reduction scene button based on the application name to obtain the scene value; In ukui-media, set up a denoising scene setting service and a denoising scene signal, which includes scene values; connect to the denoising scene setting service via QDBusConnection:sessionBus and register the denoising scene signal to the session bus; Read the denoised scene signal from the session bus and call handle_scence to determine the denoised scene corresponding to the application name based on the scene value in the denoised scene signal; The specific process of determining the noise reduction parameters corresponding to the noise reduction scenario through the noise reduction parameter adjustment service is as follows: The noise reduction parameter values ​​are obtained by sliding the noise reduction parameter slider according to the noise reduction scenario. In ukui-media, set up a denoising parameter adjustment service and a denoising parameter signal, which includes denoising parameter values; connect to the denoising parameter adjustment service via QDBusConnection:sessionBus and register the denoising parameter signal to the session bus; Read the denoising parameter signal from the session bus, and determine the denoising parameter corresponding to the denoising scenario based on the denoising parameter value in the denoising parameter signal.

2. The multi-scene sound denoising method based on an operating system as described in claim 1, characterized in that, Preset noise reduction scene types include: Real-time calling scenarios, audio and video playback scenarios, and gaming scenarios.

3. The multi-scene sound denoising method based on an operating system as described in claim 1, characterized in that, The noise reduction strategy for the audio application is determined based on the noise reduction scenario, specifically as follows: When the noise reduction scenario is a real-time call scenario, the WebRTC is called to configure the AGC automatic gain and NS noise suppression modules, and the captured audio stream is denoised through the AGC automatic gain and NS noise suppression modules.

4. The multi-scene sound denoising method based on an operating system as described in claim 1, characterized in that, Determining the noise reduction strategy for audio applications based on the noise reduction scenario also includes: When the denoising scenario is an audio-visual playback scenario, the Wavelet++ library is called to denoise the captured audio stream using the wavelet transform method; the wavelet transform method includes wavelet decomposition, thresholding, and wavelet reconstruction.

5. The multi-scene sound denoising method based on an operating system as described in claim 1, characterized in that, Determining the noise reduction strategy for audio applications based on the noise reduction scenario also includes: When the denoising scenario is a game scenario, Wiener filtering is implemented through noise estimation and short-time Fourier transform to denoise the captured audio stream.

Images