Multi-scene sound denoising method based on operating system

Abstract

A multi-scene sound denoising method based on an operating system comprises the following steps: acquiring an audio application calling pulseaudio, and analyzing an application name of the audio application through api of the pulseaudio; determining a de-noising scene corresponding to the application name through a de-noising scene setting service, and determining a de-noising strategy of the audio application according to the de-noising scene; and determining a denoising parameter corresponding to the denoising scene through the denoising parameter adjustment service, and denoising the audio stream of the audio application according to the denoising parameter and the denoising strategy. The denoising parameters can be dynamically adjusted according to the denoising scene, resource consumption is reduced, the audio use experience of a user is improved, the user obtains the optimal denoising effect, and good compatibility with a mainstream operating system and an application program can be kept.

Description

Technical Field

[0001] The present invention relates to the technical field of audio denoising, and in particular to a multi-scene sound denoising method based on an operating system. Background Art

[0002] With the efforts of researchers and the expansion of market opportunities, domestic operating systems are gradually growing and maturing. Along with these breakthroughs, the continuous development of domestic sound cards has also led to increasingly mature hardware support for various high-resolution audio formats. At the same time, technologies like VR and AR are driving a growing demand for immersive audio.

[0003] Current mainstream desktop operating systems use PulseAudio by default as an intermediary linking audio device drivers and applications to manage and process audio streams. However, this approach only applies to a limited number of audio denoising scenarios and cannot achieve the desired denoising effect based on actual needs.

[0004] In order to achieve the ideal denoising effect, a denoising method is urgently needed that can meet the user's daily needs for automatic sound adjustment and noise reduction while reducing the ineffective loss of CPU resources while ensuring good compatibility with mainstream operating systems and applications. Summary of the Invention

[0005] In order to overcome the above-mentioned defects, the present invention proposes a multi-scene sound denoising method based on an operating system to solve the problems of a single audio denoising scene and the inability to customize the adjustment.

[0006] The present invention provides a multi-scene sound denoising method based on an operating system, comprising the following steps: S1, obtain the audio application that calls pulseaudio, and parse the application name of the audio application through the pulseaudio API; S2: Determine the denoising scene corresponding to the application name through the denoising scene setting service, and determine the denoising strategy of the audio application based on the denoising scene; wherein the denoising scene setting service is set in ukui-media; S3: Determine denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service, and denoise the audio stream of the audio application according to the denoising parameters and the denoising strategy; wherein the denoising parameter adjustment service is set in ukui-media.

[0007] A further improvement of the present invention is that it also includes: S0, draw the denoising operation panel in the denoising module of ukui-media, which includes the denoising scene setting panel and the denoising parameter adjustment graphical operation panel; The denoising scene setting panel is used to display the preset denoising scene types and their corresponding denoising scene buttons. The denoising scene buttons are used to obtain the scene values ​​of the audio application; The denoising parameter adjustment graphical operation interface is provided with a denoising parameter slider for adjusting and obtaining the denoising parameter value of the audio application.

[0008] A further improvement of the present invention is that the preset noise reduction scene types include: real-time call scene, audio and video playback scene and game scene.

[0009] A further improvement of the present invention is that the specific process of determining the denoising scene corresponding to the application name through the denoising scene setting service is as follows: Select the denoising scene button according to the application name to get the scene value; Set up the denoising scene setting service and denoising scene signal in ukui-media. The denoising scene signal includes the scene value. Connect to the denoising scene setting service through QDBusConnection:sessionBus and register the denoising scene signal to the session bus. Read the denoising scene signal from the session bus and call handle_scence. Determine the denoising scene corresponding to the application name based on the scene value in the denoising scene signal, that is, the denoising scene of the audio application.

[0010] A further improvement of the present invention is that the specific process of determining the denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service is as follows: Slide the denoising parameter slider according to the denoising scenario to obtain the denoising parameter value; Set up the denoising parameter adjustment service and denoising parameter signal in ukui-media. The denoising parameter signal includes the denoising parameter value. Connect to the denoising parameter adjustment service through QDBusConnection:sessionBus and register the denoising parameter signal to the session bus. A denoising parameter signal is read from the session bus, and a denoising parameter corresponding to the denoising scenario is determined according to the denoising parameter value in the denoising parameter signal.

[0011] A further improvement of the present invention is that the denoising strategy of the audio application is determined according to the denoising scenario. Specifically, when the denoising scenario is a real-time call scenario, webrtc is called to configure the AGC automatic gain and NS noise suppression modules, and the captured audio stream is denoised by the AGC automatic gain and NS noise suppression modules.

[0012] A further improvement of the present invention is that the denoising strategy of the audio application is determined according to the denoising scenario, and also includes: when the denoising scenario is an audio and video playback scenario, calling the Wavelet++ library to denoise the captured audio stream according to the wavelet transform method; wherein the wavelet transform method includes wavelet decomposition, threshold processing and wavelet reconstruction.

[0013] A further improvement of the present invention is that the denoising strategy of the audio application is determined according to the denoising scenario, and further includes: when the denoising scenario is a game scenario, Wiener filtering is implemented through noise estimation and short-time Fourier transform to denoise the captured audio stream.

[0014] A further improvement of the present invention is to denoise the audio stream of the audio application according to the denoising parameters and the denoising strategy, specifically: Capture the undenoised audio stream using pulseaudio; applying denoising to the undenoised audio stream according to the denoising parameters and the denoising strategy; The denoised audio stream is returned to the audio application through pulseaudio.

[0015] Beneficial effects of the present invention: By obtaining the name of the audio application using the audio device, setting the denoising scene for the application name, and dynamically adjusting the denoising parameters according to the denoising scene, resource consumption is reduced while improving the user's audio experience, allowing users to obtain the best denoising effect. In addition, by setting different denoising strategies for different audio applications, combined with dynamically adjustable denoising parameters, appropriate denoising parameters are set for the denoising scene, which can meet the user's daily sound noise reduction needs while reducing the ineffective loss of CPU resources. The method in the present invention is designed based on a domestic operating system and can maintain good compatibility with mainstream operating systems and applications. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the specific embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0017] Figure 1 A logical architecture diagram of an embodiment of the present invention; Figure 2 A technical architecture diagram of an embodiment of the present invention; Figure 3 A schematic diagram of a front-end interface according to an embodiment of the present invention; Figure 4This is a logical architecture diagram for setting a noise reduction scenario according to an application name according to an embodiment of the present invention; Figure 5 This is a logical architecture diagram for graphically adjusting denoising parameters according to an embodiment of the present invention. DETAILED DESCRIPTION

[0018] In order to have a further understanding of the technical solution and beneficial effects of the present invention, the technical solution and beneficial effects of the present invention are described in detail below with reference to the accompanying drawings.

[0019] Definitions of Abbreviations and Key Terms: ukui-media: The audio settings component used by the Kylin V10 desktop system, including modules such as volume adjustment, sound device selection, and noise reduction.

[0020] PulseAudio: is a cross-platform sound system composed of multiple modules that can simplify the audio management of the operating system and provide a variety of audio devices and protocols. PulseAudio is a basic and important component of modern desktop and server operating systems.

[0021] Filtering: A method of signal processing used to remove noise from a signal or extract useful signal components.

[0022] D-Bus: Connects processes to form a message bus, and implements the sending and receiving of messages between processes on the bus.

[0023] Figure 1 A logical architecture diagram of an embodiment of the present invention is shown in FIG. Figure 1 As shown, a multi-scene sound denoising method based on an operating system of the present invention is implemented on a domestic terminal, paired with a Lenovo UC02 sound card, and installed with the Kylin V10 desktop system. After deploying the method of the present invention, Tencent Meeting, Kylin Video and Kylin Music are opened respectively, and the corresponding denoising scenes can be set respectively. After adjusting the denoising parameters, the best sound quality experience is obtained without occupying too many system resources.

[0024] Figure 2 A technical architecture diagram of an embodiment of the present invention is shown in FIG. Figure 2 As shown, the system includes audio configuration components: ukui-media, kylin-music, and kylin-video. These components are centrally managed by pulseaudio, which is initialized and maintained by denoise-daemon. Audio signals are input from input devices such as microphones, processed by the sound card, and then passed to the kernel for denoising. After processing, the audio signal is output through the sound card to output devices such as headphones and speakers.

[0025] Combine Figure 1As shown, the denoising process of the present invention is as follows: S1, add a graphical operation interface in the denoising module: In the denoising module of ukui-media, a denoising operation panel is drawn, which includes a denoising scene setting panel and a denoising parameter adjustment graphical operation interface. Figure 3 As shown, Figure 3 In the figure, the button part on the left side of the front-end interface is the denoising scene setting panel, and the slider part on the right side is the denoising parameter adjustment graphical operation interface. The denoising scene setting panel corresponds to the denoising parameter adjustment graphical operation interface one by one. The denoising scene setting panel is used to display the preset denoising scene types and their corresponding denoising scene buttons. The denoising scene buttons are used to obtain the scene values ​​of audio applications; the denoising parameter adjustment graphical operation interface is provided with a denoising parameter slider for adjusting and obtaining the denoising parameter values ​​of audio applications; each denoising scene has a corresponding denoising parameter slider to dynamically adjust the denoising parameter values. In this embodiment, the preset denoising scene types include: real-time call scenes, audio and video playback scenes, and game scenes. When the smart denoising button in the operating system is turned off, the denoising operation panel is hidden. When the smart denoising button is turned on, the hidden denoising operation panel is displayed.

[0026] When an audio application calls pulseaudio to use the audio device, the denoising module calls the pulseaudio API to parse the application name currently using the audio device (that is, the application name of the audio application) and displays the application name on the denoising operation panel, such as Figure 3 The application name position in the front-end interface shown.

[0027] S2, in this embodiment, the front-end interface and the back-end service communicate through the D-Bus service, so the D-Bus service is used to set the denoising scene. The denoising scene corresponding to the application name is determined by the denoising scene setting service (corresponding to Figure 1 "Step 2: Set the noise reduction scene according to the application name" and Figure 4 "Step 2: Set the noise reduction scene according to the application name"), the process is as follows: S201, please continue to view Figure 3 When an audio application is in use, the application name will be displayed on the front-end interface. You can select the denoising scene type based on the application name. Clicking the corresponding denoising scene button will display the scene value selected by the current audio application. Since the denoising scene button is a front-end interface, and the service process and other business need to be handled on the back-end, the following steps are designed to transmit the scene value of the front-end denoising scene button to the back-end service through the session bus for subsequent settings, as follows: In the ukui-media denoising module, customize the denoising scene setting service D-Bus1: the object path is " / come / example / button", the interface name is "come.example.button", the denoising scene signal is scencechange, and the denoising scene signal includes the scene value. In this embodiment, scencechange is the scene value of the denoising scene button.

[0028] Connect to D-Bus1 through QDBusConnection:sessionBus and register the denoising scene signal to the session bus. Register the matching rules and read the scencechange in a loop.

[0029] Call handle_scence to determine and set the denoising scene according to the scencechange value, that is, the denoising scene of the audio application. When the scencechange value is 0, it corresponds to the real-time call scene, when the scencechange value is 1, it corresponds to the audio and video playback scene, and when the scencechange value is 2, it corresponds to the game scene. (Corresponding Figure 4 "Step 2.1: Customize D-Bus Service 1 to Set the Noise Reduction Scenario Using handle_scence") After determining the denoising scenario for an audio application, determine the denoising strategy for the audio application based on the denoising scenario as follows: S202, real-time calls have high requirements for the real-time performance of audio. When the denoising scenario is a real-time call scenario, customize meeting_denoise in denoise-daemon to call webrtc to configure AGC automatic gain and NS noise suppression module to denoise the captured audio stream (corresponding to Figure 4 "Step 2.2: Implement denoising with metting_denoise in real-time call scenarios").

[0030] S203, the main purpose of audio and video playback is to ensure that the timbre and pitch of the music are not disturbed by noise. When the denoising scenario is the audio and video playback scenario, the custom video_denoise in denoise-daemon calls the Wavelet++ library to denoise the captured audio stream according to the wavelet transform method; the wavelet transform method includes wavelet decomposition, threshold processing and wavelet reconstruction (corresponding to Figure 4 "Step 2.3: Denoising the music playback scene using video_denoise").

[0031] S204, the game scene audio is highly interactive. When the denoising scene is a game scene, customize game_denoise in denoise-daemon to implement Wiener filter denoising through noise estimation and short-time Fourier transform (corresponding to Figure 4 "Step 2.4: Denoise the game scene with game_denoise" in the tutorial).

[0032] It should be noted that the technical means (such as AGC automatic gain, etc.) in the denoising strategy in this embodiment are existing technologies. Since the focus of this embodiment is on how to perform multi-scene sound denoising and dynamic adjustment of denoising parameters, the principles of the technical means are not described in too much detail.

[0033] S3, similar to S2, also uses D-Bus service to transfer denoising parameters from the front-end interface to the back-end service. Graphically adjust denoising parameters (corresponding to Figure 1 "Step 3: Graphically Adjust Denoising Parameters" and Figure 5 (see "Step 3: Graphically Adjust Denoising Parameters" in the previous section). The process is as follows: In step S301, when the denoising scene button is clicked in step S201, a denoising parameter slider bar will appear for the corresponding denoising scene button. By adjusting the denoising parameter slider bar, the denoising parameter value of the audio application can be adjusted to varying degrees. The denoising parameter value on the front-end interface is transmitted to the back-end service via the session bus as follows: In the ukui-media denoising module, customize the denoising parameter adjustment service D-Bus2: the object path is " / come / example / adjust", the interface name is " / come / example / adjust", and the dictionary type signal signal_data = {adjust, value} is defined as the denoising parameter signal, which includes the denoising parameter value.

[0034] Connect to D-Bus2 via QDBusConnection:sessionBus and register the denoising parameter signal to the session bus. Register the matching rule to read signal_data in a loop and use the denoising parameter value in signal_data as the denoising parameter for the audio application. (Corresponding to Figure 5 "Step 3.1: Customize D-Bus Service 2 and adjust the denoising parameters for the corresponding scenario through denoise_adjust") After determining the denoising parameters for the audio application, the process of denoising the audio stream of the audio application based on the denoising parameters and denoising strategy is as follows: S302, when the denoising scenario is a real-time call scenario, the denoising parameters adjusted for the real-time call scenario in the denoising operation panel of the front-end interface are received through the custom metting_adjust, and the processed parameters are passed to meeting_denoise to implement denoising (that is, the denoising strategy of the real-time call scenario in S202 also corresponds to Figure 5 (See "Step 3.2: Implementing Dynamic Parameter Adjustment and Denoising in Real-Time Call Scenario" in the previous section.) The adjustable denoising parameters are as follows: The target level adjustment range of AGC automatic gain is -20dBFs to 10dBFs, and the default value is -12dBFs; the compression gain adjustment range of AGC automatic gain is 5dB to 20dB, and the default value is 12dB; the suppression level of the NS noise suppression module includes low, medium, and high, and the default value is low.

[0035] S303, when the denoising scene is the audio and video playback scene, the denoising parameters adjusted for the audio and video playback scene in the denoising operation panel of the front-end interface are received by customizing video_adjust, and the processed parameters are passed to video_denoise to implement denoising (that is, the denoising strategy for the audio and video playback scene in S203 also corresponds to Figure 5 (See "Step 3.3: Implement dynamic parameter denoising using video_adjust in the music playback scenario" in the previous section.) The adjustable denoising parameters are as follows: The number of decomposition levels is 3 to 5, and the default is 4; the scale factor is 2 to 50, and the default is 25; the threshold is 0.01 to 1, and the default threshold is 0.5.

[0036] S304, when the denoising scene is a game scene, the denoising parameters adjusted for the game scene in the denoising operation panel of the front-end interface are received through the custom game_adjust, and the processed parameters are passed to game_denoise to implement denoising (that is, the denoising strategy of the game scene in S204, also corresponding to Figure 5 (See "Step 3.4: Dynamically adjust denoising parameters in game scenes using game_adjust" in the previous section.) The adjustable denoising parameters are as follows: The window length of noise estimation is 100 to 200 ms, with a default value of 150 ms; the spectrum smoothing coefficient is 0 to 1, with a default value of 0.5; the frame length is 5 to 90 ms, with a default value of 40 ms; and the frame shift is 25% to 75%, with a default value of 50%.

[0037] It should be noted that the processing in the “processed parameters” mentioned in S302-S304 refers to conventional processing methods of the parameters.

[0038] In this embodiment, the custom denoise.service service implements denoise-dameon to start up and run in the background, and is performed by the denoise-dameon daemon backend service (corresponding to Figure 1 "Step 4: Denoise the dameon process" in Figure 4 The denoise-dameon daemon in Figure 5 In denoise-daemon, the pulseaudio context and main loop are initialized by calling the pulseaudio API through the audio_stream_capture function (corresponding to Figure 1 "System Startup" in ), capture the undenoised audio stream from pulseaudio (corresponding to Figure 1 ), execute S302-S304, denoise the undenoised audio stream, and write the denoised audio stream of each denoising scene back to pulseaudio for use by the audio application (corresponding to Figure 1 "Returning the Denoised Audio Stream" in [Note: The following sentences appear in the original text.]

[0039] The beneficial effects of the present invention are as follows: By obtaining the name of the audio application using the audio device, setting the denoising scene for the application name, and dynamically adjusting the denoising parameters according to the denoising scene, resource consumption is reduced while improving the user's audio experience, allowing users to obtain the best denoising effect. In addition, by setting different denoising strategies for different audio applications, combined with dynamically adjustable denoising parameters, appropriate denoising parameters are set for the denoising scene, which can meet the user's daily sound noise reduction needs while reducing the ineffective loss of CPU resources. The method in the present invention is designed based on a domestic operating system and can maintain good compatibility with mainstream operating systems and applications.

[0040] Although the present invention has been described using the above preferred embodiments, they are not intended to limit the scope of protection of the present invention. Any person skilled in the art may make various changes and modifications to the above embodiments without departing from the spirit and scope of the present invention. These changes and modifications are still within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be based on the definition of the claims.

Claims

1. A multi-scene sound denoising method based on an operating system, characterized in that: include: S1, obtain the audio application that calls pulseaudio, and parse the application name of the audio application through the pulseaudio API; S2: Determine the denoising scene corresponding to the application name through the denoising scene setting service, and determine the denoising strategy of the audio application based on the denoising scene; wherein the denoising scene setting service is set in ukui-media; S3: Determine the denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service, and denoise the audio stream of the audio application based on the denoising parameters and the denoising strategy; the denoising parameter adjustment service is set in ukui-media; Denoise the audio stream of the audio application according to the denoising parameters and denoising strategy, specifically: Capture the undenoised audio stream using pulseaudio; applying denoising to the undenoised audio stream according to the denoising parameters and the denoising strategy; The denoised audio stream is returned to the audio application through pulseaudio.

2. The multi-scene sound denoising method based on an operating system according to claim 1, characterized in that: Also includes: S0, draw the denoising operation panel in the denoising module of ukui-media, which includes the denoising scene setting panel and the denoising parameter adjustment graphical operation panel; The denoising scene setting panel is used to display the preset denoising scene types and their corresponding denoising scene buttons. The denoising scene buttons are used to obtain the scene values ​​of the audio application; The denoising parameter adjustment graphical operation interface is provided with a denoising parameter slider for adjusting and obtaining the denoising parameter value of the audio application.

3. The multi-scene sound denoising method based on an operating system according to claim 2, characterized in that: The preset denoising scene types include: Real-time call scenarios, audio and video playback scenarios, and game scenarios.

4. The multi-scene sound denoising method based on an operating system according to claim 2, characterized in that: The specific process of determining the denoising scene corresponding to the application name through the denoising scene setting service is as follows: Select the denoising scene button according to the application name to get the scene value; Set up the denoising scene setting service and denoising scene signal in ukui-media. The denoising scene signal includes the scene value. Connect to the denoising scene setting service through QDBusConnection:sessionBus and register the denoising scene signal to the session bus. Read the denoising scene signal from the session bus and call handle_scence to determine the denoising scene corresponding to the application name based on the scene value in the denoising scene signal.

5. The multi-scene sound denoising method based on an operating system according to claim 2, characterized in that: The specific process of determining the denoising parameters corresponding to the denoising scenario through the denoising parameter adjustment service is as follows: Slide the denoising parameter slider according to the denoising scenario to obtain the denoising parameter value; Set the denoising parameter adjustment service and denoising parameter signal in ukui-media. The denoising parameter signal includes the denoising parameter value. Connect to the denoising parameter adjustment service through QDBusConnection:sessionBus and register the denoising parameter signal to the session bus; A denoising parameter signal is read from the session bus, and a denoising parameter corresponding to the denoising scenario is determined according to the denoising parameter value in the denoising parameter signal.

6. The multi-scene sound denoising method based on an operating system according to claim 3, characterized in that: Determine the denoising strategy for the audio application based on the denoising scenario, specifically: When the denoising scenario is a real-time call scenario, call webrtc to configure the AGC automatic gain and NS noise suppression modules, and denoise the captured audio stream through the AGC automatic gain and NS noise suppression modules.

7. The multi-scene sound denoising method based on an operating system according to claim 3, characterized in that: Determine the denoising strategy for the audio application based on the denoising scenario, including: When the denoising scenario is an audio and video playback scenario, the Wavelet++ library is called to denoise the captured audio stream according to the wavelet transform method; the wavelet transform method includes wavelet decomposition, threshold processing and wavelet reconstruction.

8. The multi-scene sound denoising method based on an operating system according to claim 3, characterized in that: Determine the denoising strategy for the audio application based on the denoising scenario, including: When the denoising scenario is a game scenario, Wiener filtering is implemented through noise estimation and short-time Fourier transform to denoise the captured audio stream.

Images