Audio processing method, apparatus, device, medium, and program product

Abstract

The application provides an audio processing method, device, equipment, medium and program product; the method comprises: filtering the environment audio signal collected by a noise reduction device to obtain an effective audio signal and an environment reference signal; enhancing the effective audio signal and an input audio signal to obtain an enhanced audio signal; generating a counter signal based on the device feedback signal collected by the noise reduction device and the environment reference signal; and generating a playing signal of the noise reduction device according to the enhanced signal, the environment reference signal and the counter signal. Through the application, the active noise reduction process can reduce the damage to the effective audio signal and the input audio signal while actively reducing the environment reference signal, thereby comprehensively improving the effect of active noise reduction.

Description

Technical Field

[0001] This application relates to noise reduction technology, and more particularly to an audio processing method, apparatus, device, medium, and program product. Background Technology

[0002] Active noise cancellation is a key research area in audio processing. Its core objective is to suppress ambient reference signals by generating a signal whose sound wave is opposite to that of the ambient reference signal. This technology has profound implications for improving users' communication experience in noisy environments, enhancing speech clarity, and improving overall auditory comfort.

[0003] In related technologies, active noise cancellation technology suppresses ambient audio signals indiscriminately. However, when there are valid audio signals that need to be retained in the ambient audio signals, such as human voices or vehicle sounds, the playback effect of these valid audio signals is often damaged by active noise cancellation, making it impossible for users to clearly capture important information in the ambient audio signals. Summary of the Invention

[0004] This application provides an audio processing method, apparatus, device, medium, and program product that can reduce the damage to the effective audio signal and the input audio signal during the active noise reduction process while actively reducing noise on the environmental reference signal, thereby comprehensively improving the effect of active noise reduction.

[0005] The technical solution of this application embodiment is implemented as follows:

[0006] This application provides an audio processing method, the method comprising:

[0007] The ambient audio signal collected by the noise reduction device is filtered to obtain an effective audio signal and an environmental reference signal;

[0008] The effective audio signal and the input audio signal are enhanced to obtain an enhanced audio signal;

[0009] Based on the device feedback signal and the environmental reference signal collected by the noise reduction device, a cancellation signal is generated;

[0010] The playback signal of the noise reduction device is generated based on the enhanced signal, the environmental reference signal, and the cancellation signal.

[0011] This application provides an audio processing apparatus, including:

[0012] The filtering module is used to filter the ambient audio signals collected by the noise reduction device to obtain effective audio signals and ambient reference signals;

[0013] An enhancement module is used to enhance the effective audio signal and the input audio signal to obtain an enhanced audio signal;

[0014] An active noise reduction module is used to generate a cancellation signal based on the device feedback signal collected by the noise reduction device and the environmental reference signal;

[0015] The signal combination module is used to generate the playback signal of the noise reduction device based on the enhanced signal, the environmental reference signal and the cancellation signal.

[0016] The audio processing device further includes an audio filtering model training module, used to acquire valid audio signal samples, and fuse the valid audio signal samples with environmental reference signal samples to obtain environmental audio signal samples; filter the environmental audio signal samples using an audio filtering model to obtain predicted valid audio signals; determine the filtering loss based on the error between the predicted valid audio signals and the valid audio signal samples; and update the parameters of the audio filtering model based on the filtering loss to obtain an updated audio filtering model.

[0017] In the above scheme, the filtering module is further used to call the audio filtering model to filter the ambient audio signal to obtain the effective audio signal; and to obtain the difference between the ambient audio signal and the effective audio signal as the ambient reference signal.

[0018] The audio processing device further includes an audio enhancement model training module, used to acquire original audio signal samples, and perform quality weakening processing on the original audio signal samples to obtain weakened audio signal samples corresponding to the original audio signal samples; enhance the weakened audio samples through the audio enhancement model to obtain a predicted enhanced audio signal; determine the audio enhancement loss based on the difference between the predicted enhanced audio signal and the original audio signal samples; and update the parameters of the audio enhancement model based on the audio enhancement loss to obtain an updated audio enhancement model.

[0019] In the above scheme, the enhancement module is further configured to perform time-frequency conversion processing on the mixed audio signal to obtain a frequency-domain mixed audio signal; acquire a first audio signal below a first frequency in the frequency-domain mixed audio signal; perform enhancement processing on the first audio signal to obtain an enhanced first audio signal; fuse the enhanced first audio signal and the frequency-domain mixed audio signal to obtain an enhanced frequency-domain mixed audio signal; and perform inverse time-frequency conversion processing on the enhanced frequency-domain mixed audio signal to obtain the enhanced audio signal.

[0020] In the above scheme, the active noise reduction module is further configured to: acquire the error signal between the environmental reference signal and the device feedback signal; adjust the active noise reduction weights based on the error signal, the device feedback signal, and the step size parameter to obtain the adjusted active noise reduction weights; perform convolution processing on the environmental reference signal based on the adjusted active noise reduction weights to obtain the convolution-processed environmental reference signal; and perform a phase flip operation on the convolution-processed environmental reference signal to obtain the cancellation signal.

[0021] In the above scheme, the active noise reduction module is further configured to determine the weight change value based on the error signal, the device feedback signal and the step size parameter; and adjust the active noise reduction weight based on the weight change value to obtain the adjusted active noise reduction weight.

[0022] In the above scheme, the active noise reduction module is further configured to obtain the square of the amplitude of the device feedback signal; obtain the ratio of the device feedback signal to the square of the amplitude; and multiply the ratio, the error signal, and the step size parameter to obtain the weight change value.

[0023] In the above scheme, the active noise reduction module is further configured to perform time-frequency conversion processing on the convolutional environmental reference signal to obtain a frequency domain environmental reference signal; determine the phase value of the frequency domain environmental reference signal; adjust the phase value based on a preset coefficient to obtain an adjusted phase value; and perform time-frequency inverse conversion processing on the adjusted phase value to obtain the cancellation signal.

[0024] The audio processing device further includes a filter configuration module for obtaining a filter mode in response to a filter configuration operation of an object, wherein the filter mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; the step of filtering the environmental audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: filtering the environmental audio signal collected by the noise reduction device to obtain the environmental reference signal and an effective audio signal that satisfies the filter mode.

[0025] The audio processing device further includes an environmental information determination module, used to acquire environmental information of the object using the noise reduction device, and determine a filtering mode based on the environmental information, wherein the filtering mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; the step of filtering the environmental audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: filtering the environmental audio signal collected by the noise reduction device to obtain the environmental reference signal and an effective audio signal that satisfies the filtering mode.

[0026] In the above scheme, the signal combination module is further configured to acquire a first weighted signal corresponding to the enhanced signal, a second weighted signal corresponding to the canceled signal, and a third weighted signal corresponding to the environmental reference signal; and to superimpose the first weighted signal, the second weighted signal, and the third weighted signal to obtain the playback signal.

[0027] This application provides an electronic device, including:

[0028] Memory is used to store executable instructions or computer programs.

[0029] The processor, when executing computer-executable instructions or computer programs stored in the memory, implements the audio processing method provided in the embodiments of this application.

[0030] This application provides a computer-readable storage medium storing a computer program or computer-executable instructions for implementing the audio processing method provided in this application when executed by a processor.

[0031] This application provides a computer program product, including a computer program or computer executable instructions, which, when executed by a processor, implements the audio processing method provided in this application.

[0032] The embodiments of this application have the following beneficial effects:

[0033] The ambient audio signal collected by the noise reduction device is filtered to obtain an effective audio signal and an ambient reference signal. The effective audio signal is separated before active noise reduction to avoid it being denoised. The effective audio signal and the input audio signal are then enhanced to obtain an enhanced audio signal. By filtering the ambient audio signal to obtain the effective audio signal and the ambient reference signal, and enhancing both the effective and input audio signals, the degree of cancellation on these signals during noise reduction is reduced. A cancellation signal is generated based on the device feedback signal and the ambient reference signal collected by the noise reduction device. A playback signal for the noise reduction device is then generated based on the enhanced signal, the ambient reference signal, and the cancellation signal. By actively denoising the device feedback signal and the ambient reference signal, the resulting cancellation signal can cancel the ambient reference signal during playback, improving the effectiveness of active noise reduction. Furthermore, while actively denoising the ambient reference signal, the damage to the effective and input audio signals during active noise reduction is reduced, resulting in a comprehensive improvement in the effectiveness of active noise reduction. Attached Figure Description

[0034] Figure 1This is a schematic diagram of the audio processing system architecture provided in the embodiments of this application;

[0035] Figure 2 This is a schematic diagram of the structure of the audio processing device provided in the embodiments of this application;

[0036] Figures 3-10 This is a schematic flowchart of the audio processing method provided in the embodiments of this application;

[0037] Figure 11 This is a schematic diagram of the structure of a common noise reduction device provided in the embodiments of this application;

[0038] Figure 12 This is a schematic diagram illustrating an application scenario of the noise reduction device provided in the embodiments of this application;

[0039] Figure 13 This is a schematic diagram of the active noise reduction process provided in an embodiment of this application;

[0040] Figure 14 This is a schematic diagram of the structure of the encoding and decoding convolutional neural network provided in the embodiments of this application. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of this application clearer, the application will be further described in detail below with reference to the accompanying drawings. The described embodiments should not be regarded as limitations on this application. All other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0042] In the following description, references are made to “some embodiments,” which describe a subset of all possible embodiments. However, it is understood that “some embodiments” may be the same subset or different subsets of all possible embodiments and may be combined with each other without conflict.

[0043] In the following description, the terms "first, second, third" are used merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first, second, third" may be interchanged in a specific order or sequence where permitted, so that the embodiments of this application described herein can be implemented in an order other than that illustrated or described herein.

[0044] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0045] Unless otherwise defined, all technical and scientific terms used in the embodiments of this application have the same meaning as commonly understood by one of ordinary skill in the art. The terminology used in the embodiments of this application is for the purpose of describing the embodiments of this application only and is not intended to limit this application.

[0046] In the implementation of this application, the collection and processing of relevant data should strictly comply with the requirements of relevant laws and regulations, obtain the informed consent or separate consent of the personal information subject, and carry out subsequent data use and processing within the scope of laws and regulations and the authorization of the personal information subject.

[0047] Before providing a further detailed description of the embodiments of this application, the nouns and terms involved in the embodiments of this application will be explained, and the nouns and terms involved in the embodiments of this application shall be interpreted as follows.

[0048] (1) Active Noise Cancellation (ANC): This is a technology that cancels out noise by generating opposite sound waves. It is mainly used in headphones, speakers, automobiles, and other fields to provide a better listening experience and reduce noise interference. The core of an active noise cancellation system is an algorithm based on the principle of sound wave interference. When two sound waves with opposite phases meet, they cancel each other out, thus reducing noise. An active noise cancellation system first captures ambient noise through a microphone, then the controller analyzes these noise signals in real time and generates a cancellation signal that is opposite to the noise signal. Finally, the speaker or headphones play the original audio signal and the cancellation signal, effectively reducing noise.

[0049] (2) Reverse acoustic wave signal: The generation of reverse acoustic wave signals is usually based on the principle of acoustic wave interference. When two acoustic waves meet, if they have the same frequency and opposite phase (i.e., opposite vibration directions), they will cancel each other out, resulting in a reduction or elimination of acoustic wave energy. Based on this principle, reverse acoustic wave signals are designed to match the noise acoustic waves in frequency and phase, so as to produce a cancellation effect when they meet.

[0050] (3) Active noise reduction adaptive filtering: Based on the principle of acoustic interference, when two acoustic waves meet, if their frequencies are the same and their phases are opposite, they will cancel each other out, resulting in a reduction or elimination of acoustic energy. By detecting ambient noise in real time and generating an acoustic wave that is out of phase with the noise, i.e., an inverse acoustic wave signal, noise cancellation is achieved.

[0051] (4) Normalized Least Mean Square Filter: When the system transfer function is unknown, the filter weights are continuously adjusted according to the input signal to approximate the ideal response of the system. That is, the weights are updated by regressing the residual between the output of the normalized least mean square filter and the desired response, thereby realizing the function of digital filtering.

[0052] (5) Harmonic Enhancement Processing: Harmonics refer to sound components in an audio signal whose frequencies are in a certain proportion to the high-frequency components of the original sound. Audio signals, such as human voices and instrument sounds, are composed of several harmonics. Harmonic enhancement processing enriches the spectrum of the audio signal by increasing these harmonic components, thereby improving sound quality.

[0053] (6) Equalizer equalization processing: used to adjust the gain value of different frequency components in the audio signal to improve sound quality, enhance the effect of specific frequency bands or satisfy personal listening preferences. It can precisely control each frequency band of the audio to make the sound more harmonious and balanced.

[0054] Active noise cancellation is a key research area in audio processing. Its core objective is to suppress ambient reference signals by generating a signal whose sound wave is opposite to that of the ambient reference signal. This technology has profound implications for improving users' communication experience in noisy environments, enhancing speech clarity, and improving overall auditory comfort.

[0055] In related technologies, active noise cancellation technology suppresses ambient audio signals indiscriminately. However, when there are valid audio signals that need to be retained in the ambient audio signals, such as human voices or vehicle sounds, the playback effect of these valid audio signals is often damaged by active noise cancellation, making it impossible for users to clearly capture important information in the ambient audio signals.

[0056] This application provides an audio processing method, apparatus, device, computer-readable storage medium, and computer program product, which can reduce the damage to the effective audio signal and the input audio signal during the active noise reduction process while actively reducing noise on the environmental reference signal, thereby comprehensively improving the effect of active noise reduction. The following describes exemplary applications of the electronic devices provided in this application. These electronic devices can be implemented as various types of terminals such as laptops, tablets, desktop computers, set-top boxes, smartphones, smart speakers, smartwatches, smart TVs, in-vehicle terminals, and headphones, or as servers. The following will describe exemplary applications when the device is implemented as a terminal.

[0057] See Figure 1 , Figure 1 This is a schematic diagram of the audio processing system architecture provided in the embodiments of this application. The audio processing system 100 supports an audio processing application. The terminal 400 connects to the server 200 through the network 300. The server 200 uploads data to the database 500. The network 300 can be a wide area network or a local area network, or a combination of the two.

[0058] Terminal 400 is a noise reduction device. It collects ambient audio signals and device feedback signals, and uploads these signals, along with its input audio signal, to server 200. Server 200 filters the ambient audio signals to obtain a valid audio signal and an environmental reference signal. It then mixes the valid audio signal and the input audio signal to obtain a mixed audio signal. This mixed signal is then enhanced to obtain an enhanced audio signal. Active noise reduction processing is performed on the device feedback signal and the environmental reference signal to obtain a canceled signal. Finally, the enhanced signal, the environmental reference signal, and the canceled signal are combined to obtain the playback signal for terminal 400, which is then sent to terminal 400 for playback.

[0059] In some embodiments, server 200 may be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms. Terminals and servers can be connected directly or indirectly via wired or wireless communication, which is not limited in this embodiment.

[0060] See Figure 2 , Figure 2 This is a schematic diagram of the structure of the audio processing device provided in the embodiments of this application. Figure 2The terminal 400 shown includes at least one processor 410, a memory 450, at least one network interface 420, and a user interface 430. The various components in the terminal 400 are coupled together via a bus system 440. It is understood that the bus system 440 is used to implement communication between these components. In addition to a data bus, the bus system 440 also includes a power bus, a control bus, and a status signal bus. However, for clarity, ... Figure 2 The general labeled all buses as Bus System 440.

[0061] The processor 410 can be an integrated circuit chip with signal processing capabilities, such as a general-purpose processor, a digital signal processor (DSP), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor can be a microprocessor or any conventional processor, etc.

[0062] User interface 430 includes one or more output devices 431 that enable the presentation of media content, including one or more speakers and / or one or more visual displays. User interface 430 also includes one or more input devices 432, including user interface components that facilitate user input, such as a keyboard, mouse, microphone, touch screen display, camera, other input buttons and controls.

[0063] The memory 450 may be removable, non-removable, or a combination thereof. Exemplary hardware devices include solid-state storage, hard disk drives, optical disk drives, etc. The memory 450 may optionally include one or more storage devices physically located away from the processor 410.

[0064] The memory 450 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be read-only memory (ROM), and the volatile memory may be random access memory (RAM). The memory 450 described in this application embodiment is intended to include any suitable type of memory.

[0065] In some embodiments, memory 450 is capable of storing data to support various operations, examples of which include programs, modules, and data structures or subsets or supersets thereof, as illustrated below.

[0066] Operating system 451 includes system programs for handling various basic system services and performing hardware-related tasks, such as the framework layer, core library layer, driver layer, etc., for implementing various basic business functions and handling hardware-based tasks;

[0067] The network communication module 452 is used to reach other electronic devices via one or more (wired or wireless) network interfaces 420, exemplary network interfaces 420 including: Bluetooth, WiFi, and Universal Serial Bus (USB), etc.

[0068] Presentation module 453 is configured to enable the presentation of information (e.g., a user interface for operating peripheral devices and displaying content and information) via one or more output devices 431 (e.g., a display screen, a speaker, etc.) associated with user interface 430;

[0069] The input processing module 454 is used to detect and translate one or more user inputs or interactions from one or more input devices 432.

[0070] In some embodiments, the apparatus provided in this application can be implemented in software. Figure 2 An audio processing device 455 stored in memory 450 is shown. This device can be software in the form of programs and plugins, and includes the following software modules: a filtering module 4551, a mixing module 4552, an enhancement module 4553, an active noise reduction module 4554, a signal combination module 4555, an audio filtering model training module 4556, an audio enhancement model training module 4557, a filtering configuration module 4558, and an environmental information determination module 4559. These modules are logically connected and can therefore be arbitrarily combined or further separated according to their implemented functions. The functions of each module will be described below.

[0071] In other embodiments, the audio processing apparatus provided in this application can be implemented in hardware. As an example, the audio processing apparatus provided in this application can be a processor in the form of a hardware decoding processor, which is programmed to execute the audio processing method provided in this application. For example, the processor in the form of a hardware decoding processor can be one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), programmable logic devices (PLDs), complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or other electronic components.

[0072] In some embodiments, the terminal or server can implement the audio processing method provided in this application by running various computer-executable instructions or computer programs. For example, computer-executable instructions can be microprogram-level commands, machine instructions, or software instructions. Computer programs can be native programs or software modules in an operating system. In summary, the aforementioned computer-executable instructions can be any form of instruction, and the aforementioned computer programs can be any form of application program, module, or plugin.

[0073] The audio processing method provided in this application will be described below with reference to exemplary applications and implementations of the terminal provided in the embodiments of this application.

[0074] The audio processing method provided in the embodiments of this application will be described below. As mentioned above, the electronic device implementing the audio processing method of the embodiments of this application can be a terminal, a server, or a combination of both. Therefore, the executing entity of each step will not be described again below.

[0075] See Figure 3 , Figure 3 This is a flowchart illustrating the audio processing method provided in the embodiments of this application, which will be combined with... Figure 3 The steps shown are explained below. Figure 3 The main component of the process is the terminal.

[0076] In step 101, the ambient audio signal collected by the noise reduction device is filtered to obtain an effective audio signal and an ambient reference signal.

[0077] In some embodiments, see Figure 4 , Figure 3 Before step 101 shown, steps 106 to 110 can be performed to train the audio filtering model, which will be discussed below. Figure 4 This needs to be explained. It should be noted that the filtering process is implemented using an audio filtering model.

[0078] In step 106, valid audio signal samples are obtained.

[0079] As an example, the content of the valid audio signal sample obtained in step 106 can be artificially synthesized, that is, the content of the valid audio signal sample is a clean, noise-free sound signal, such as a clear human voice signal or a pure car horn sound signal.

[0080] In step 107, the effective audio signal sample and the environmental reference signal sample are fused to obtain the environmental audio signal sample.

[0081] As an example, the content of the environmental reference signal sample is a noise signal, such as the noise signal of machine operation or wind noise in the natural environment. The effective audio signal sample and the environmental reference signal sample are fused. That is, noise is added to the effective audio signal sample. The resulting environmental audio signal sample is an effective audio signal sample with superimposed noise.

[0082] In step 108, the environmental audio signal samples are filtered using an audio filtering model to obtain a predicted valid audio signal.

[0083] As an example, the environmental audio signal samples are filtered using an audio filtering model. In other words, the predicted valid audio signals in the environmental audio signal samples are filtered and sorted out by the audio filtering model.

[0084] In step 109, the filtering loss is determined based on the error between the predicted valid audio signal and the valid audio signal sample.

[0085] As an example, due to the accuracy limitations of the audio filtering model, there is an error between the predicted effective audio signal and the effective audio signal sample. In order to adjust the audio filtering model so that the predicted effective audio signal is closer to the effective audio signal sample, it is necessary to calculate the error between the predicted effective audio signal and the effective audio signal sample. The error can be calculated in various ways, such as mean square error, root mean square error, signal-to-noise ratio, etc. These indicators can quantitatively describe the difference between the predicted effective audio signal and the effective audio signal sample. The filtering loss of the audio filtering model can be determined by the quantified error.

[0086] In step 110, the parameters of the audio filtering model are updated based on the filtering loss to obtain the updated audio filtering model.

[0087] As an example, after obtaining the filtering loss, the audio filtering model can be continuously adjusted and optimized based on the filtering loss. The main approach is iterative training to adjust the audio filtering model. Specifically, the audio filtering model is first used to filter environmental audio signal samples to obtain predicted valid audio signals. Then, the error between the predicted valid audio signal and the valid audio signal samples is calculated, and the filtering loss is determined. The parameters or structure of the audio filtering model are adjusted based on the filtering loss to obtain an updated audio filtering model. Based on the updated audio filtering model, the environmental audio signal samples are filtered again, and the filtering loss is repeatedly calculated and the audio filtering model is updated until the error between the predicted valid audio signal and the valid audio signal samples obtained through the updated audio filtering model is within the required range, for example, when the error between the predicted valid audio signal and the valid audio signal samples is less than the error threshold of 0.01, the updating of the audio filtering model is stopped.

[0088] Through the embodiments of this application, effective audio signal samples and environmental reference signal samples can be obtained and fused to generate environmental audio signal samples that are more relevant to the environment and rich in environmental features, thereby improving the accuracy of subsequent filtering processes. Using an audio filtering model to filter the environmental audio signal samples can effectively remove noise and interference, resulting in a predicted effective audio signal. The error between the predicted effective audio signal and the effective audio signal samples is identified, and the filtering loss is determined. Based on this filtering loss, the audio filtering model is updated, continuously optimizing its performance and making the predicted effective audio signal closer to the effective audio signal samples.

[0089] In some embodiments, Figure 3 Before step 101 shown, the following technical solution can also be performed to configure the filtering of the ambient audio signal: in response to the filtering configuration operation of the object, a filtering mode is obtained, wherein the filtering mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; the filtering processing of the ambient audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: filtering the ambient audio signal collected by the noise reduction device to obtain the environmental reference signal and an effective audio signal that satisfies the filtering mode.

[0090] As an example, users can customize filtering configuration operations to filter out specific valid audio signals required by the user during the filtering process of ambient audio signals. Filtering configuration operations include configuring the loudness range, pitch range, and timbre information of the filtered valid audio signals. Loudness range refers to the relative intensity of the valid audio signal, measured in decibels (dB); pitch range refers to the frequency of the valid audio signal, measured in hertz (Hz); timbre information includes the characteristics of the sound-producing body, such as the shape, size, and material of the sound-producing body made of different materials, and the shape, size, and material of the resonating cavity. During the filtering configuration operation, the loudness range, pitch range, and timbre information of the valid audio signal can be set. For example, the loudness range of the filtered valid audio signal can be 40 to 60 dB, the pitch range 100 to 150 Hz, and the timbre information can be the sound produced by a musical instrument. Different filtering configuration operations correspond to different filtering modes. For example, when the filtering configuration operation configures the loudness range of the valid audio signal, the corresponding filtering mode is the preset loudness range filtering mode; when the filtering configuration operation configures the pitch range of the valid audio signal, the corresponding filtering mode is the preset pitch range filtering mode; when the filtering configuration operation configures the timbre information of the valid audio signal, the corresponding filtering mode is the timbre information configuration filtering mode. After determining the filtering mode based on the filtering configuration operation, filtering the ambient audio signal will yield the valid audio signal and the ambient reference signal that satisfy the filtering mode.

[0091] As an example, different filtering modes require different training data for the audio filtering models. When the filtering mode filters valid audio signals within a preset loudness range, the valid audio signal samples used during training should conform to the preset loudness range requirements. Similarly, when the filtering mode filters valid audio signals within a preset pitch range, the valid audio signal samples used during training should conform to the preset pitch range requirements. And when the filtering mode filters valid audio signals with configured timbre information, the valid audio signal samples used during training should conform to the configured timbre information requirements. The training process of these audio filtering models is similar to... Figure 4 Steps 106 to 110 shown are the same and will not be repeated here. Different audio filtering models corresponding to different filtering modes can be used simultaneously, so that the effective audio signal obtained by filtering can meet the requirements of different filtering configuration operations.

[0092] This application provides an embodiment that allows users to customize effective audio signals according to their actual needs. By flexibly configuring filtering modes, including preset loudness ranges, preset pitch ranges, and configuring timbre information, users can precisely control the noise reduction device's filtering of ambient audio signals. This customization capability not only improves the flexibility of audio processing but also ensures that the obtained effective audio signal and environmental reference signal better meet the user's expectations. Furthermore, it can adapt to different application scenarios and needs, bringing users a more personalized and high-quality experience.

[0093] In some embodiments, Figure 3 Before step 101 shown, the following technical solution can also be performed to automatically determine the content of the effective audio signal: obtain the environmental information of the object using the noise reduction device, and determine the filtering mode based on the environmental information, wherein the filtering mode includes at least one of the following: preset loudness range, preset pitch range, and configured timbre information; the step of filtering the environmental audio signal collected by the noise reduction device to obtain the effective audio signal and the environmental reference signal includes: filtering the environmental audio signal collected by the noise reduction device to obtain the environmental reference signal and the effective audio signal that satisfies the filtering mode.

[0094] As an example, users do not need to configure filtering for ambient audio signals. The noise reduction device can collect environmental information about the user's environment and intelligently determine and select a suitable filtering mode based on this information. For instance, when the noise reduction device determines that the user's environment is a traffic scene, to ensure the user's safety, the effective audio signal filtered by the appropriate filtering mode will include vehicle horn sounds.

[0095] As an example, different filtering modes require different training data for the audio filtering models. When the filtering mode filters valid audio signals within a preset loudness range, the valid audio signal samples used during training should conform to the preset loudness range requirements. Similarly, when the filtering mode filters valid audio signals within a preset pitch range, the valid audio signal samples used during training should conform to the preset pitch range requirements. And when the filtering mode filters valid audio signals with configured timbre information, the valid audio signal samples used during training should conform to the configured timbre information requirements. The training process of these audio filtering models is similar to... Figure 4 Steps 106 to 110 shown are the same and will not be repeated here. Different audio filtering models corresponding to different filtering modes can be used simultaneously, so that the effective audio signal obtained by filtering can meet the requirements of different filtering configuration operations.

[0096] Through the embodiments of this application, the environmental information of the user of the noise reduction device can be automatically obtained, and the appropriate filtering mode can be intelligently determined based on this information. This enables efficient and accurate filtering of environmental audio signals, which not only simplifies the user's operation process, but also ensures that high-quality effective audio signals and environmental reference signals can be obtained in different scenarios, thereby improving the overall efficiency of audio processing.

[0097] In some embodiments, see Figure 5 , Figure 3 Step 101 shown can be implemented through steps 1011 to 1012 below, in conjunction with the following. Figure 5 Please provide an explanation.

[0098] In step 1011, the audio filtering model is invoked to filter the environmental audio signal to obtain the effective audio signal.

[0099] As an example, the audio filtering model used in step 1011 is through Figure 4 The training results are shown in steps 106 to 110.

[0100] In step 1012, the difference between the ambient audio signal and the valid audio signal is obtained as the ambient reference signal.

[0101] As an example, the ambient audio signal contains a valid audio signal and an ambient reference signal. Therefore, the difference between the ambient audio signal and the valid audio signal can be calculated by subtracting the valid audio signal from the ambient audio signal, resulting in the ambient reference signal. In active noise cancellation, the ambient reference signal, i.e., the noise, is the part that needs to be canceled, while the valid audio signal is the part that needs to be preserved. For example, in a traffic scenario, when pedestrians wear noise-canceling devices to cancel the ambient reference signal (i.e., noise), they need to hear the sound of car horns to ensure safety. Here, the sound of car horns is the signal that needs to be preserved during active noise cancellation, i.e., the valid audio signal.

[0102] Through the embodiments of this application, in the process of active noise cancellation, the environmental reference signal, i.e., the noise, is the part that needs to be canceled. By using an audio filtering model, the effective audio signal in the environmental audio signal can be effectively obtained, and the environmental reference signal can be determined based on the environmental audio signal and the effective audio signal. The effective audio signal and the environmental reference signal in the environmental audio signal can be accurately identified. This method of filtering the environmental audio signal makes the active noise cancellation process more targeted, and can better suppress the environmental reference signal while ensuring the playback effect of the effective audio signal.

[0103] After performing step 101, the effective audio signal and the input audio signal can be enhanced to obtain an enhanced audio signal, which can be achieved through steps 102 to 103.

[0104] See also Figure 3 In step 102, the effective audio signal and the input audio signal are mixed to obtain a mixed audio signal.

[0105] As an example, first, the valid audio signal and the input audio signal are aligned in the time domain, meaning their start times and sampling rates are consistent. Then, the valid audio signal and the input audio signal are fed into the same mix track. The volume balance of the valid audio signal and the input audio signal is adjusted proportionally on the mix track. Finally, a mixed audio signal containing the characteristics of both the valid audio signal and the input audio signal is output.

[0106] In step 103, the mixed audio signal is enhanced to obtain an enhanced audio signal.

[0107] In some embodiments, see Figure 6 , Figure 3Before step 103 shown, steps 111 to 115 can be performed to train the audio enhancement model, which will be discussed below. Figure 6 This needs to be explained. It should be noted that the enhancement processing is implemented through an audio enhancement model.

[0108] In step 111, the original audio signal sample is obtained.

[0109] As an example, the original audio signal sample obtained in step 111 can be an artificially synthesized audio signal or an audio signal acquired by an audio acquisition device; there are no restrictions here.

[0110] In step 112, the original audio signal sample is subjected to quality attenuation processing to obtain attenuated audio signal samples corresponding to the original audio signal samples.

[0111] As an example, quality attenuation can involve reducing volume, increasing noise, or adjusting audio frequency response. Quality attenuation reduces the volume, clarity, or frequency response of the original audio signal sample, resulting in a weakened audio signal sample with lower sound quality than the original audio signal sample.

[0112] In step 113, the weakened audio sample is enhanced using the audio enhancement model to obtain a predicted enhanced audio signal.

[0113] In step 114, the audio enhancement loss is determined based on the difference between the predicted enhanced audio signal and the original audio signal sample.

[0114] As an example, due to the accuracy limitations of audio enhancement models, there is a difference between the predicted enhanced audio signal and the original audio signal sample. In order to adjust the enhancement model so that the predicted enhanced audio signal is closer to the original audio signal sample, it is necessary to calculate the difference between the predicted enhanced audio signal and the original audio signal sample. The difference can be calculated in various ways, such as mean square error, root mean square error, signal-to-noise ratio, etc. These indicators can quantitatively describe the difference between the predicted enhanced audio signal and the original audio signal sample. The audio enhancement loss of the audio enhancement model can be determined by the quantified difference.

[0115] In step 115, the parameters of the audio enhancement model are updated based on the audio enhancement loss to obtain the updated audio enhancement model.

[0116] As an example, after obtaining the audio enhancement loss, the audio enhancement model can be continuously adjusted and optimized based on the audio enhancement loss. The main approach is iterative training to adjust the audio enhancement model. Specifically, the audio enhancement model is first used to weaken audio samples for enhancement, resulting in a predicted enhanced audio signal. Then, the difference between the predicted enhanced audio signal and the original audio signal sample is calculated, and the audio enhancement loss is determined. The parameters or structure of the audio enhancement model are adjusted based on the audio enhancement loss to obtain an updated audio enhancement model. Based on the updated audio enhancement model, the weakened audio samples are enhanced again, and the calculation of weakened audio samples and updates to the audio enhancement model are repeated until the difference between the predicted enhanced audio signal and the original audio signal sample obtained through the updated audio enhancement model is within a required range, for example, the difference between the predicted enhanced audio signal and the original audio signal sample is less than a difference threshold of 0.01, at which point the updating of the audio enhancement model stops.

[0117] This application provides an embodiment that allows for the acquisition and quality attenuation of original audio signal samples, simulating the attenuation conditions that audio signals may encounter in real-world environments. By predicting the difference between the enhanced audio signal and the original audio signal sample, the audio enhancement loss is determined, and the audio enhancement model is updated based on this loss. This process not only improves the performance of the audio enhancement model but also makes it more adaptable to processing various attenuated audio signals, thereby achieving high-quality audio signal restoration and making the predicted enhanced audio signal closer to the original audio signal sample.

[0118] In some embodiments, see Figure 7 , Figure 3 Step 103 shown can be implemented through steps 1031 to 1035 below, in conjunction with the following... Figure 7 Please provide an explanation.

[0119] In step 1031, the mixed audio signal is subjected to time-frequency conversion processing to obtain a frequency domain mixed audio signal.

[0120] As an example, steps 1031 to 1034 are to enhance the low-frequency part of the mixed audio signal. Therefore, in step 1031, the time-domain form of the mixed audio signal needs to be converted to a time-frequency signal. The time-frequency conversion uses Fourier transform to obtain the frequency-domain form of the mixed audio signal, i.e., the frequency-domain mixed audio signal.

[0121] In step 1032, a first audio signal below the first frequency in the frequency domain mixed audio signal is obtained;

[0122] As an example, since steps 1031 to 1034 are to enhance the low-frequency part of the frequency-domain mixed audio signal, it is necessary to obtain the low-frequency part of the frequency-domain mixed audio signal. The low-frequency part of the frequency-domain mixed audio signal can be the first audio signal below the first frequency in the frequency-domain mixed audio signal. The value of the first frequency should be related to the listener's auditory perception. Taking human hearing as an example, the range of audio signal frequencies that humans can hear is roughly between 20 Hz and 20 kHz. Audio signals below 500 Hz can be considered low-frequency audio signals. At this time, the value of the first frequency is 500 Hz.

[0123] In step 1033, the first audio signal is enhanced to obtain an enhanced first audio signal.

[0124] As an example, during the enhancement process in step 1033, the method used is... Figure 6 The audio enhancement model obtained by training in steps 111 to 115 is shown.

[0125] In step 1034, the enhanced first audio signal and the frequency domain mixed audio signal are fused to obtain the enhanced frequency domain mixed audio signal.

[0126] As an example, since the audio enhancement model enhances the low-frequency part of the frequency-domain mixed audio signal, i.e., the first audio signal, after obtaining the enhanced first audio signal, it needs to be fused with the frequency-domain mixed audio signal. The low-frequency part of the frequency-domain mixed audio signal, i.e., the first audio signal, is replaced with the enhanced first audio signal. In this way, the enhanced frequency-domain mixed audio signal can be obtained. The enhanced frequency-domain mixed audio signal contains the high-frequency part of the frequency-domain mixed audio signal and the low-frequency part of the frequency-domain mixed audio signal, i.e., the first audio signal.

[0127] In step 1035, the enhanced frequency-domain mixed audio signal is subjected to time-frequency inverse conversion processing to obtain the enhanced audio signal.

[0128] As an example, steps 1031 to 1034 are to enhance the low-frequency part of the mixed audio signal. The resulting enhanced frequency-domain mixed audio signal is a signal in frequency domain representation. Therefore, it is necessary to perform time-frequency inverse transformation processing, i.e., inverse Fourier transform, on the enhanced frequency-domain mixed audio signal to obtain the enhanced audio signal in time domain representation.

[0129] Through the embodiments of this application, time-frequency conversion is performed to obtain the low-frequency component of the frequency-domain mixed audio signal, namely the first audio signal, and targeted enhancement of the first audio signal effectively improves the low-frequency response of the audio. This processing not only makes the mixed audio signal sound fuller and more powerful, but also enhances the spatial sense and immersion of the audio. At the same time, enhancing the low-frequency part of the mixed audio signal, namely the first audio signal, can improve the playback effect and quality of the mixed audio signal, making the played mixed audio signal clearer.

[0130] See also Figure 3 In step 104, a cancellation signal is generated based on the device feedback signal collected by the noise reduction device and the environmental reference signal.

[0131] In some embodiments, see Figure 8 , Figure 3 Step 104 shown can be implemented through steps 1041 to 1044 below, in conjunction with the following... Figure 8 Please provide an explanation.

[0132] In step 1041, the error signal between the environmental reference signal and the device feedback signal is obtained.

[0133] As an example, the active noise cancellation process uses an active noise cancellation adaptive filtering algorithm. Its working principle is to continuously adjust and update the adaptive filter weights W so that the output speaker's cancellation signal and the environmental reference signal (i.e., noise) can cancel each other out, thereby reducing the level of the environmental reference signal. In the active noise cancellation adaptive filtering algorithm, the adjustment of the adaptive filter weights W is performed by comparing the environmental reference signal Ref and the device feedback signal X to calculate the error signal. The specific calculation formula is given in formula (1).

[0134] E(n)=Ref(n)-X(n) (1)

[0135] Where n is the current time value, E(n) is the error signal at the current time, Ref(n) is the environmental reference signal at the current time, and X(n) is the real-time environmental audio signal collected at the position closest to the ear at the current time, which is also the device feedback signal.

[0136] In step 1042, the active noise reduction weights are adjusted based on the error signal, the device feedback signal, and the step size parameter to obtain the adjusted active noise reduction weights.

[0137] In some embodiments, see Figure 9 , Figure 8 Step 1042 shown can be implemented through steps 1042A to 1042B as described below. Figure 9 Please provide an explanation.

[0138] In step 1042A, the weight change value is determined based on the error signal, the device feedback signal, and the step size parameter.

[0139] In some embodiments, step 1042A can be implemented by the following technical solution: obtaining the square of the amplitude of the device feedback signal; obtaining the ratio of the device feedback signal to the square of the amplitude; multiplying the ratio, the error signal, and the step size parameter to obtain the weight change value.

[0140] Through the embodiments of this application, the square of the amplitude of the device feedback signal and its ratio are calculated and multiplied with the error signal and the step size parameter to obtain the weight change value, which can adjust the weight more accurately and improve the accuracy and efficiency of signal processing.

[0141] In step 1042B, the active noise reduction weights are adjusted based on the weight change value to obtain the adjusted active noise reduction weights.

[0142] As an example, the active noise reduction weights are adjusted by the weight change values ​​determined by the error signal, the device feedback signal and the step size parameter. Taking the normalized minimum mean square filter as an example, the calculation formula for the adjusted active noise reduction weights is shown in the following formula (2).

[0143]

[0144] Where n is the current value, W(n) is the current filter coefficient weight (i.e., the active noise reduction weight), W(n+1) is the next filter coefficient weight (i.e., the adjusted active noise reduction weight), and μ is the step size parameter of the adaptive filter. The values ​​represent the weight changes. X(n) is the real-time ambient audio signal captured at the position closest to the ear at the current moment, which is also the device feedback signal. e(n) is the signal value of the error signal E(n) at the current moment. The values ​​of X(n) and e(n) can be in time-domain representation or in frequency-domain representation after Fourier transform. The time-domain representation is converted to the frequency-domain representation because it is more commonly used in the frequency domain, and processing in the frequency domain is more effective than processing in the time domain.

[0145] By obtaining the squared amplitude and ratio of the device feedback signal through the embodiments of this application, and combining it with the error signal and step size parameter to calculate the weight change value, the accuracy and efficiency of weight adjustment can be improved. Adjusting the active noise reduction weights based on the weight change value can further optimize the active noise reduction effect. This adaptive adjustment mechanism ensures that optimal active noise reduction performance is provided under different noise levels, i.e., environmental reference signals, maintaining the stability and consistency of the active noise reduction effect.

[0146] See also Figure 8In step 1043, the environmental reference signal is convolved based on the adjusted active noise reduction weights to obtain the convolved environmental reference signal.

[0147] As an example, before performing convolution, the active denoising weights are typically flipped. This flip usually refers to a 180-degree rotation along the spatial dimensions of the active denoising weights; for example, for a two-dimensional convolution kernel, this is a flip along its rows and columns. The purpose of this flip is to mathematically ensure the consistency of the convolution operation, accurately reflecting the interaction between the ambient reference signal and the active denoising weights. The flipped active denoising weights are then slid along the ambient reference signal. At each sliding position, the product of the active denoising weight and the corresponding portion of the ambient reference signal is calculated, and these products are accumulated. After the sliding convolution is complete, the result is the convolution-processed ambient reference signal.

[0148] In step 1044, a phase flipping operation is performed on the convolutional environmental reference signal to obtain the cancellation signal.

[0149] In some embodiments, see Figure 10 , Figure 8 Step 1044 shown can be implemented through steps 1044A to 1044D as described below. Figure 10 Please provide an explanation.

[0150] In step 1044A, the convolutional environmental reference signal is subjected to time-frequency conversion processing to obtain a frequency domain environmental reference signal.

[0151] As an example, steps 1044A to 1044D involve performing a phase-based flipping operation on the convolutional environmental reference signal. Therefore, to obtain the phase information of the convolutional environmental reference signal, it is necessary to convert the time-domain representation of the convolutional environmental reference signal into a frequency-domain representation to obtain the frequency-domain environmental reference signal. Here, the time-frequency conversion uses Fourier transform.

[0152] In step 1044B, the phase value of the frequency domain environment reference signal is determined.

[0153] As an example, after performing a time-frequency conversion, i.e., a Fourier transform, the resulting frequency domain environmental reference signal is a complex number, which contains the amplitude and phase information of the frequency domain environmental reference signal. The real part of the complex number represents the cosine component of the frequency domain environmental reference signal, and the imaginary part represents the sine component of the frequency domain environmental reference signal. In the frequency domain, the phase value represents the phase angle of the signal at different frequency components. Therefore, the phase value of the frequency domain environmental reference signal can be obtained by calculating the phase angle of the complex number. The arctangent function can be used to determine the phase angle based on the real and imaginary parts of the complex number. For the specific calculation formula, please refer to formula (3) below.

[0154] phase=arctan2(imag,real) (3)

[0155] Where phase represents the phase angle, or phase value, arctan2 is a two-parameter arctangent function, and real and img represent the real and imaginary parts of the frequency domain environment reference signal, respectively.

[0156] In step 1044C, the phase value is adjusted based on a preset coefficient to obtain the adjusted phase value.

[0157] As an example, the preset coefficient is -1, meaning that adjusting the phase value is an operation of multiplying the phase value by -1 in the complex plane. Since in the frequency domain, the phase value represents the phase angle of the signal at different frequency components, adjusting the phase value based on the preset coefficient is equivalent to rotating the phase angle by 180 degrees.

[0158] In step 1044D, the adjusted phase value is subjected to time-frequency inverse conversion to obtain the cancellation signal.

[0159] As an example, since the signal played in the noise reduction device should be in time-domain representation, after obtaining the adjusted phase value in frequency-domain representation, an inverse time-frequency conversion process is needed to obtain the time-domain representation of the adjusted phase value, i.e., the canceled signal. Here, the inverse time-frequency conversion process is the inverse Fourier transform.

[0160] Through the embodiments of this application, time-frequency conversion is performed on the convolutionally processed environmental reference signal to obtain the phase information of the frequency domain environmental reference signal. The phase value is adjusted by preset coefficients to obtain an adjusted phase value that is opposite in phase to the frequency domain environmental reference signal. The adjusted phase value is then subjected to inverse time-frequency conversion. The resulting cancellation signal, because it is opposite in phase to the environmental reference signal, can more accurately cancel it out, significantly improving the active noise reduction effect.

[0161] This application's embodiments enable real-time acquisition of the error between the environmental reference signal and the device feedback signal, intelligently adjusting the active noise reduction weights, and significantly improving the accuracy and efficiency of active noise reduction. The adjusted weights are used to convolve the environmental reference signal, a process that accurately simulates and predicts the characteristics of noise, i.e., the environmental reference signal. The resulting convolved environmental reference signal is then phase-flipped to generate a cancellation signal with the opposite phase. This cancellation signal can cancel out the environmental reference signal when played simultaneously with it, thereby achieving precise active noise reduction. This not only effectively reduces the environmental reference signal (i.e., noise) but also enhances the clarity of the played signal.

[0162] See also Figure 3 In step 105, the playback signal of the noise reduction device is generated based on the enhanced signal, the environmental reference signal, and the cancellation signal.

[0163] As an example, the playback signal of the noise reduction device can be generated by directly combining the enhancement signal, the environmental reference signal, and the cancellation signal. The result of the combination is the playback signal of the noise reduction device. The calculation formula for the playback signal obtained by direct combination is shown in formula (4) below.

[0164] P(t)=X(t)+Y(t)+Z(t) (4)

[0165] Where P(t) is the playback signal, X(t) is the enhancement signal, Y(t) is the cancellation signal, and Z(t) is the environmental reference signal.

[0166] In some embodiments, Figure 3 The step 105 shown can also be implemented by the following technical solution: obtaining a first weighted signal corresponding to the enhanced signal, a second weighted signal corresponding to the canceled signal, and a third weighted signal corresponding to the environmental reference signal; superimposing the first weighted signal, the second weighted signal, and the third weighted signal to obtain the playback signal.

[0167] As an example, the playback signal generated by the noise reduction device can be achieved by combining (superimposing) the enhancement signal, the environmental reference signal, and the cancellation signal according to the weights of each signal, and adjusting the proportion of each signal in the playback signal. The weighting formulas are shown in formulas (5) to (7) below.

[0168] X′(t)=a·X(t) (5)

[0169] Y′(t)=b·Y(t) (6)

[0170] Z′(t)=c·Z(t) (7)

[0171] Where X(t) is the enhancement signal, X'(t) is the first weighted signal, a is the weight of the enhancement signal, Y(t) is the cancellation signal, Y'(t) is the second weighted signal, b is the weight of the cancellation signal, Z(t) is the environmental reference signal, Z'(t) is the third weighted signal, and c is the weight of the environmental reference signal.

[0172] As an example, the second weighted signal can also be phase-adjusted to obtain a phase-adjusted second weighted signal; then, when combining, the phase-adjusted second weighted signal is used for combination. For the specific adjustment method, please refer to the following formula (8).

[0173] Y″(t)=Y′(t-Δt) (8)

[0174] Where Y'(t) is the second weighted signal, Y'(t) is the second weighted signal after phase adjustment, and Δt is the time-domain delay corresponding to the phase adjustment, which is equivalent to a phase rotation of the second weighted signal in the frequency domain. Phase adjustment is performed because it can compensate for phase distortion in the transmission of the cancellation signal, ensure that the signals at the receiving end and the transmitting end are in phase, and improve the accuracy of the cancellation signal. Phase adjustment of the cancellation signal can effectively reduce the phase cancellation problem between signals.

[0175] As an example, the combination processing can be a linear combination. A linear combination is when the first weighted signal, the phase-adjusted second weighted signal, and the third weighted signal are combined, the output playback signal is the algebraic sum of the outputs of each signal acting alone. See formula (9) below for the specific calculation formula.

[0176] P(t)=X′(t)+Y″(t)+Z′(t) (9)

[0177] Wherein, P(t) is the playback signal, X'(t) is the first weighted signal, Y'(t) is the second weighted signal after phase adjustment, and Z'(t) is the third weighted signal.

[0178] As an example, the combination processing can be non-linear. When combining the first weighted signal, the second weighted signal after phase adjustment, and the third weighted signal, more complex operations are involved, such as multiplication and convolution. The signals are combined through complex operations to obtain the playback signal.

[0179] Through the embodiments of this application, the enhanced signal, the canceled signal, and the environmental reference signal can be acquired and weighted respectively. Since the integration of the environmental reference signal can enhance the realism, the final weighting method can make the playback signal clearer, more coherent, and more spatial and three-dimensional.

[0180] In this embodiment, the ambient audio signal collected by the noise reduction device is filtered to obtain an effective audio signal and an ambient reference signal. The effective audio signal is separated before active noise reduction, thus preventing it from being denoised. The effective audio signal and the input audio signal are then enhanced to obtain an enhanced audio signal. By filtering the ambient audio signal to obtain the effective audio signal and the ambient reference signal, and enhancing both the effective and input audio signals, the degree of cancellation of the effective and input audio signals during noise reduction is reduced. A cancellation signal is generated based on the device feedback signal and the ambient reference signal collected by the noise reduction device. A playback signal for the noise reduction device is generated based on the enhanced signal, the ambient reference signal, and the cancellation signal. By actively denoising the device feedback signal and the ambient reference signal, the resulting cancellation signal can cancel the ambient reference signal during playback, improving the effectiveness of active noise reduction. Furthermore, while actively denoising the ambient reference signal, the damage to the effective and input audio signals during active noise reduction is reduced, comprehensively improving the effectiveness of active noise reduction.

[0181] The following will describe an exemplary application of the embodiments of this application in a real-world application scenario.

[0182] This application proposes a custom active noise cancellation method involving effective signal filtering and pre-enhancement processing. In this method, users can interactively select different active noise cancellation modes. Some of these modes predefine the effective signal category; for example, human voice signals or specific sound signals within the external sound signal are defined as effective signals. A deep network extracts the defined effective and ineffective signals. The effective signals are then superimposed on the original input signal and pre-enhanced to obtain a pre-enhanced signal, thus preventing damage to the sound signal played after active noise cancellation. Here, the original input signal is the sound played through headphones, such as music. Ineffective signals are the primary target of active noise cancellation. These ineffective signals are input to an active noise cancellation adaptive filter and processed together with the external sound signal collected by the feedback microphone. The output signal is the opposite of the ineffective signal's sound wave. The ineffective signal, the signal with the opposite sound wave, and the pre-enhanced signal are then sent to a speaker for playback, thereby canceling out the ineffective signals in the weakened external sound signal. This application identifies external sound signals, allowing users to communicate normally with people while wearing active noise cancellation devices such as headphones, without affecting the communication quality. The application scenarios of this application embodiment can be used in application scenarios where the surrounding environment is relatively noisy and on-site communication is required, such as public places such as game venues and factory operations, or audio and video input devices for applications such as instant messaging, conferencing, and gaming, which can achieve a good noise reduction effect.

[0183] Active noise cancellation is a technology that suppresses ineffective signals, or noise, in external sound signals by generating inverse sound waves. It is primarily used in audio devices such as headphones and earphones to provide a better listening experience. Active noise cancellation is based on the principle of sound wave interference; that is, when two sound waves of the same frequency and amplitude are 180 degrees out of phase, they will cancel each other out, thereby suppressing ineffective signals in the external sound signal. See also Figure 11 Active noise cancellation systems typically include the following components:

[0184] External microphone 201: Used to capture external sound signals.

[0185] First internal microphone 202 and second internal microphone 203: used to capture sound signals inside the headphones for closed-loop control and to improve noise reduction.

[0186] Signal processor 204: processes the captured external sound signals to generate reverse sound wave signals.

[0187] First driver and speaker 205 and second driver and speaker 206: Drive the headphone speaker to play sound waves that cancel out ineffective signals, i.e. noise, in the external sound signal, according to the reverse sound wave signal generated by the signal processor 204.

[0188] Control circuit: Responsible for controlling and regulating the entire system.

[0189] Working process: The external microphone captures external sound signals and converts them into electrical signals. Simultaneously, the internal microphone captures external sound signals collected by the headphones and converts them into electrical signals. The electrical signals captured by both the external and internal microphones are transmitted to the signal processor, where they are compared and analyzed. Based on the signals from the internal and external microphones, the signal processor adjusts the frequency, amplitude, and phase of the inverse sound wave. The driver then generates a sound wave that cancels out ineffective signals (noise) in the external sound signal, based on the inverse sound wave signal generated by the signal processor.

[0190] Noise cancellation performance: Active noise cancellation technology mainly targets low-frequency noise suppression, and typically performs well in the frequency range of 20 Hz to 1000 Hz. For mid-to-high frequency noise above 1000 Hz, the effect of active noise cancellation gradually weakens, at which point passive noise cancellation, such as the physical sound isolation of earplugs or earmuffs, becomes more important.

[0191] Active noise cancellation methods in related technologies treat the entire collected external sound signal as the "noise" signal to be canceled, thereby generating corresponding inverse sound waves for spatial sound wave cancellation processing within the headphones. However, the sound wave cancellation process in these technologies can damage effective signals in the external sound signal, such as human voice signals. In some application scenarios, such as noisy outdoor environments, users need to wear active noise-canceling headphones while simultaneously listening to and communicating with other people in the area. Active noise-canceling headphones treat the external sound signal as a whole for cancellation. Although human voice signals above 1000 Hz are still present and not completely suppressed, frequencies below 1000 Hz are typically where the main energy of human voice signals is concentrated. After active noise cancellation, users wearing headphones may find it difficult to hear other voices in the environment. In addition, because active noise cancellation strongly suppresses low-frequency sound signals, it can put users in danger in some scenarios. For example, the sound of car horns on the road is suppressed very cleanly by active noise cancellation headphones, so users wearing active noise cancellation headphones cannot hear the sound of nearby car horns, which can easily lead to accidents.

[0192] While existing active noise cancellation solutions and products offer multi-level noise reduction parameters for interactive user selection, they merely provide functions such as noise reduction on / off, weak noise reduction, and strong noise reduction to control the noise reduction intensity. They lack the ability to intelligently identify and filter effective signals, and they do not employ pre-enhancement processing to prevent significant damage to effective signals during active noise cancellation. Therefore, to address the problem that existing active noise cancellation solutions indiscriminately suppress external sound signals, leading to the damage or complete elimination of important sound information, this application proposes a new active noise cancellation processing scheme. This scheme incorporates effective signal filtering and pre-enhancement modules into the closed-loop control process of existing active noise cancellation solutions to prevent damage to effective signals from the active noise cancellation algorithm.

[0193] The main application scenarios of this application include outdoor calling scenarios or calling scenarios in noisy environments. See the application scenarios below. Figure 12 For example, in noisy public gaming venues, multiple users need to wear active noise-canceling headphones to listen to the audio signals played through the headphones, such as game sound effects, or to protect their ears from strong noise interference. They also need to hear the effective signals of external sound, such as the voices of their teammates. Similarly, in cars, active noise cancellation is used to reduce auditory fatigue caused by vehicle noise, but passengers still need to be able to communicate normally. In noisy production workshops, operators need to wear active noise-canceling headphones to shield their ears while still listening to instructions from colleagues. In these applications, active noise cancellation technology suppresses ineffective signals from the external sound spectrum, providing a quiet listening experience while allowing for normal communication and preserving the effective signals.

[0194] In terms of interaction, since users are in different scenarios and have different application needs, they can configure different active noise cancellation modes or levels through user interface settings, such as selecting through the terminal screen interface, setting through buttons on the headphones, or setting through voice recognition. These correspond to different active noise cancellation auditory experience functions. The modes or levels can be defined as follows: for example, the near-field human voice mode: filters effective signals that meet certain conditions in terms of volume level collected by the external microphone to avoid being suppressed by active noise cancellation, and pre-enhances the effective signals, such as low-frequency harmonic enhancement, thereby offsetting the problem of active noise cancellation accidentally damaging the effective signals; another example is the near-field important sound mode: in addition to filtering near-field human voice signals, this mode also filters some prominent sound signals, such as obvious horn sounds and other sudden high-energy sounds, to prevent these sound signals defined as effective signals from being damaged or even eliminated by active noise cancellation.

[0195] The embodiments of this application provide users with more choices in auditory experience. They are not limited to the active noise cancellation functions in related technologies, which only offer mechanical noise cancellation level selection, typically functions like on / off, weak noise cancellation, and strong noise cancellation to control the intensity of noise cancellation, without effectively identifying, filtering, or pre-enhancing external sound signals.

[0196] This application's embodiment is designed to meet the need for normal communication and reception of external sounds in noisy environments, while wearing active noise-canceling headphones and configuring the noise cancellation mode according to the user's settings. See also Figure 13 External sound signals include valid and invalid signals. The valid signals, as defined here, are defined by noise reduction modes pre-configured through user interface interaction. These can be human voice signals or specific sound signals set by the manufacturer's product, etc. Figure 13 The reference sensor 301 collects external sound signals. The reference sensor can be a microphone sensor embedded in headphones, helmets, or other devices. It converts the external sound signals from electrical signals to digital signals 302. The digital signal 302 of the external sound signals will pass through the effective signal filtering module 303. This module can be implemented based on a deep learning network or a statistical sound model. Its input is the digital signal 302. After the filtering module separates the signals, it outputs two signals: an effective signal 304 and an ineffective signal 305. The ineffective signal is the sound signal in the external sound signal other than the effective signal. The effective signal 304 is mixed with the original input signal 306 to form a mixed signal 307, which is then pre-enhanced to obtain an enhanced signal 308. The original input signal is the sound played by the headphones, such as music. The pre-enhancing process can be low-frequency enhancement or deep network audio enhancement. The ineffective signal 305 is sent to the active noise reduction adaptive filtering algorithm 309. The active noise reduction adaptive filtering algorithm is also fed with the X signal 311, which is converted from an electrical signal to a digital signal by the feedback sensor 310. The X signal is the real-time external sound signal collected at the position closest to the ear during each active noise reduction. The active noise reduction adaptive filtering algorithm works by continuously adjusting and updating the adaptive filter weight W so that the reverse sound wave signal of the output speaker cancels out the ineffective signal, i.e., noise, thereby reducing the noise level. The active noise reduction adaptive filtering compares the ineffective signal and the X signal to calculate the error signal. The specific calculation formula is shown in formula (10).

[0197] E(n) = Ref(n) - X(n) (10)

[0198] Where n is the current time value, E(n) is the error signal at the current time, Ref(n) is the invalid signal at the current time, and X(n) is the real-time external sound signal collected at the position closest to the ear at the current time.

[0199] The filter coefficient weights W of the active noise reduction adaptive filter are adjusted and updated by the X signal and the error signal. Taking the normalized minimum mean square filter as an example, the calculation formula of its filter coefficient weights W is shown in formula (11).

[0200]

[0201] Where n is the current time value, W(n) is the current time value of the filter coefficients, W(n+1) is the next time value of the filter coefficients, μ is the adaptive filter step size parameter, X(n) is the real-time external sound signal collected at the position closest to the ear at the current time, and e(n) is the signal value of the error signal E(n) at the current time. The values ​​of X(n) and e(n) can be time domain values ​​or frequency domain values ​​after Fourier transform. The conversion to the frequency domain is because the frequency domain has more applications, and processing in the frequency domain is more effective than processing in the time domain.

[0202] The enhanced signal 308, the cancellation signal output by the active noise cancellation adaptive filtering algorithm 309 (which is out of phase with the ineffective signal sound wave), and the output signal 312 (the result of superimposing the ineffective signals) are driven by the speaker for playback. The playback signal includes the enhanced signal of the effective signal and the original input sound signal, the ineffective signal, and the cancellation signal (which is out of phase with the ineffective signal sound wave). In other words, what the human ear ultimately hears is mainly the effective signal and the original input signal. Active noise cancellation is continuous during user operation; each noise cancellation cycle outputs a signal and drives the speaker for playback.

[0203] This application proposes that separating custom valid and invalid signals from external sound signals can be achieved using deep learning networks. See [link to relevant documentation]. Figure 14 , Figure 14 The encoder-decoder convolutional neural network shown is trained with a large number of valid signals that meet the definition, such as human voice signals and specific sound signals, to achieve custom valid signal filtering output. For example, the input external sound signal is Sig, and the valid signal output by the deep learning network is Val. The non-valid signal Ref can be obtained by the following formula (12).

[0204] Ref = Sig-Val (12)

[0205] Here, Ref represents the inactive signal, Sig represents the external sound signal, and Val represents the active signal. The Mix signal, resulting from the superposition of the Val signal and the original input signal Org, requires pre-enhancement processing. This is to prevent the damage to the Mix signal caused by the out-of-phase cancellation signal of the active noise reduction adaptive filtering algorithm, which is opposite to the inactive Ref signal. Since the active noise reduction adaptive filtering algorithm is mainly effective for low-frequency signals below 1000 Hz, it can be achieved simply by performing harmonic enhancement or equalization processing (EQ equalization) on the low-frequency signal, or by using deep network enhancement processing, such as conventional recurrent neural networks (RNNs) or convolutional neural networks (CNNs).

[0206] This application provides an active noise cancellation scheme based on effective signal filtering and pre-enhancement, effectively solving the problem that active noise cancellation schemes in related technologies easily damage human voice or other specific effective signals. Users can customize the active noise cancellation mode or level to suit their current application scenario through user interface interaction, ensuring that active noise cancellation does not inadvertently damage effective signals. This application embodiment meets the needs of practical application scenarios and can bring users a differentiated auditory experience.

[0207] The following description continues to illustrate the exemplary structure of the audio processing device 455 provided in the embodiments of this application as a software module. In some embodiments, see [link to relevant documentation]. Figure 2 The software modules stored in the audio processing device 455 of the memory 450 may include:

[0208] The filtering module 4551 is used to filter the ambient audio signals collected by the noise reduction device to obtain effective audio signals and ambient reference signals.

[0209] The mixing module 4552 is used to mix the effective audio signal and the input audio signal to obtain a mixed audio signal;

[0210] Enhancement module 4553 is used to enhance the mixed audio signal to obtain an enhanced audio signal;

[0211] The active noise reduction module 4554 is used to generate a cancellation signal based on the device feedback signal collected by the noise reduction device and the environmental reference signal;

[0212] The signal combination module 4555 is used to generate the playback signal of the noise reduction device based on the enhanced signal, the environmental reference signal and the cancellation signal.

[0213] In some embodiments, the audio processing apparatus further includes an audio filtering model training module 4556, configured to acquire valid audio signal samples, fuse the valid audio signal samples with environmental reference signal samples to obtain environmental audio signal samples; filter the environmental audio signal samples using an audio filtering model to obtain predicted valid audio signals; determine a filtering loss based on the error between the predicted valid audio signals and the valid audio signal samples; and update the parameters of the audio filtering model based on the filtering loss to obtain an updated audio filtering model.

[0214] In some embodiments, the filtering module 4551 is further configured to call the audio filtering model to filter the ambient audio signal to obtain the effective audio signal; and to obtain the difference between the ambient audio signal and the effective audio signal as the ambient reference signal.

[0215] In some embodiments, the audio processing apparatus further includes an audio enhancement model training module 4557, configured to acquire original audio signal samples, perform quality attenuation processing on the original audio signal samples to obtain attenuated audio signal samples corresponding to the original audio signal samples; perform enhancement processing on the attenuated audio samples using the audio enhancement model to obtain a predicted enhanced audio signal; determine an audio enhancement loss based on the difference between the predicted enhanced audio signal and the original audio signal samples; and update the parameters of the audio enhancement model based on the audio enhancement loss to obtain an updated audio enhancement model.

[0216] In some embodiments, the enhancement module 4553 is further configured to perform time-frequency conversion processing on the mixed audio signal to obtain a frequency-domain mixed audio signal; acquire a first audio signal below a first frequency in the frequency-domain mixed audio signal; perform enhancement processing on the first audio signal to obtain an enhanced first audio signal; fuse the enhanced first audio signal and the frequency-domain mixed audio signal to obtain an enhanced frequency-domain mixed audio signal; and perform inverse time-frequency conversion processing on the enhanced frequency-domain mixed audio signal to obtain the enhanced audio signal.

[0217] In some embodiments, the active noise reduction module 4554 is further configured to: acquire an error signal between the environmental reference signal and the device feedback signal; adjust the active noise reduction weights based on the error signal, the device feedback signal, and the step size parameter to obtain adjusted active noise reduction weights; perform convolution processing on the environmental reference signal based on the adjusted active noise reduction weights to obtain a convolutionally processed environmental reference signal; and perform a phase flip operation on the convolutionally processed environmental reference signal to obtain the cancellation signal.

[0218] In some embodiments, the active noise reduction module 4554 is further configured to determine a weight change value based on the error signal, the device feedback signal, and the step size parameter; and to adjust the active noise reduction weight based on the weight change value to obtain the adjusted active noise reduction weight.

[0219] In some embodiments, the active noise reduction module 4554 is further configured to obtain the square of the amplitude of the device feedback signal; obtain the ratio of the device feedback signal to the square of the amplitude; and multiply the ratio, the error signal, and the step size parameter to obtain the weight change value.

[0220] In some embodiments, the active noise reduction module 4554 is further configured to perform time-frequency conversion processing on the convolutional environmental reference signal to obtain a frequency domain environmental reference signal; determine the phase value of the frequency domain environmental reference signal; adjust the phase value based on a preset coefficient to obtain an adjusted phase value; and perform time-frequency inverse conversion processing on the adjusted phase value to obtain the cancellation signal.

[0221] In some embodiments, the audio processing apparatus further includes a filter configuration module 4558, configured to obtain a filter mode in response to a filter configuration operation of an object, wherein the filter mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; the step of filtering the ambient audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: filtering the ambient audio signal collected by the noise reduction device to obtain the environmental reference signal and an effective audio signal that satisfies the filter mode.

[0222] In some embodiments, the audio processing apparatus further includes an environmental information determination module 4559, configured to acquire environmental information of the object using the noise reduction device, and determine a filtering mode based on the environmental information, wherein the filtering mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; the step of filtering the environmental audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: filtering the environmental audio signal collected by the noise reduction device to obtain the environmental reference signal and an effective audio signal that satisfies the filtering mode.

[0223] In some embodiments, the signal combining module 4555 is further configured to acquire a first weighted signal corresponding to the enhanced signal, a second weighted signal corresponding to the canceled signal, and a third weighted signal corresponding to the environmental reference signal; perform phase adjustment processing on the second weighted signal to obtain a phase-adjusted second weighted signal; and superimpose the first weighted signal, the phase-adjusted second weighted signal, and the third weighted signal to obtain the playback signal.

[0224] This application provides a computer program product including a computer program or computer-executable instructions stored in a computer-readable storage medium. A processor of an electronic device reads the computer-executable instructions from the computer-readable storage medium and executes the computer-executable instructions, causing the electronic device to perform the audio processing method described above in this application.

[0225] This application provides a computer-readable storage medium storing computer-executable instructions or computer programs. When the computer-executable instructions or computer programs are executed by a processor, the processor will execute the audio processing method provided in this application.

[0226] In some embodiments, the computer-readable storage medium may be a memory such as RAM, ROM, flash memory, magnetic surface memory, optical disk, or CD-ROM; or it may be a variety of devices including one or any combination of the above-mentioned memories.

[0227] In some embodiments, computer-executable instructions may take the form of programs, software, software modules, scripts, or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as stand-alone programs or as modules, components, subroutines, or other units suitable for use in a computing environment.

[0228] As an example, computer-executable instructions may, but do not necessarily, correspond to files in a file system. They may be stored as part of a file that holds other programs or data, for example, in one or more scripts in a Hyper Text Markup Language (HTML) document, in a single file dedicated to the program in question, or in multiple co-located files (e.g., files that store one or more modules, subroutines, or code sections).

[0229] As an example, computer-executable instructions can be deployed to execute on a single electronic device, or on multiple electronic devices located at one location, or on multiple electronic devices distributed across multiple locations and interconnected via a communication network.

[0230] In summary, through the embodiments of this application, the environmental audio signal collected by the noise reduction device can be filtered to obtain an effective audio signal and an environmental reference signal. Using an audio filtering model, the effective audio signal in the environmental audio signal can be effectively obtained, and the environmental reference signal can be determined based on the environmental audio signal and the effective audio signal. This method of filtering environmental audio signals makes the active noise reduction process more targeted, effectively suppressing the environmental reference signal while ensuring the playback effect of the effective audio signal. The effective audio signal and the input audio signal are mixed to obtain a mixed audio signal; the mixed audio signal is then enhanced to obtain an enhanced audio signal; through time-frequency conversion, the low-frequency component of the frequency domain mixed audio signal, i.e., the first audio signal, can be obtained, and the first audio signal is specifically enhanced, effectively improving the low-frequency response of the audio. This processing not only makes the mixed audio signal sound fuller and more powerful but also enhances the spatial sense and immersion of the audio. Simultaneously, enhancing the low-frequency part of the mixed audio signal, i.e., the first audio signal, improves the playback effect and quality of the mixed audio signal, making the played mixed audio signal clearer. By filtering the ambient audio signal, an effective audio signal and an ambient reference signal are obtained. The effective audio signal in the ambient audio signal is mixed and enhanced with the input audio signal, which reduces the degree of cancellation of the effective audio signal and the input audio signal during the cancellation process of the ambient reference signal. Based on the device feedback signal and the ambient reference signal collected by the noise reduction device, a cancellation signal is generated. The error between the ambient reference signal and the device feedback signal can be obtained in real time, and the active noise reduction weights can be intelligently adjusted, significantly improving the accuracy and efficiency of active noise reduction. The adjusted weights are used to perform convolution processing on the ambient reference signal. This process can accurately simulate and predict the characteristics of noise, i.e., the ambient reference signal. The phase of the obtained convolutional ambient reference signal is flipped to generate a cancellation signal with the opposite phase to the convolutional ambient reference signal. The cancellation signal can cancel the ambient reference signal when played simultaneously with it, thereby achieving accurate active noise reduction. This not only effectively weakens the ambient reference signal, i.e., noise, but also enhances the clarity of the played signal. The playback signal of the noise reduction device is generated based on the enhanced signal, the ambient reference signal, and the cancellation signal. By actively denoising the device feedback signal and the environmental reference signal, the resulting cancellation signal can cancel out the environmental reference signal during playback, thus improving the effectiveness of active noise reduction. The method provided in this application can reduce the damage to the effective audio signal and the input audio signal during active noise reduction while simultaneously reducing the environmental reference signal, thereby comprehensively improving the effectiveness of active noise reduction. Furthermore, it allows users to customize the effective audio signal according to their actual needs.By flexibly configuring filtering modes, including preset loudness ranges, preset pitch ranges, and timbre information, users can precisely control the noise-canceling device's filtering of ambient audio signals. This customization capability not only improves the flexibility of audio processing but also ensures that the obtained effective audio signals and environmental reference signals better meet the user's expectations. It can also adapt to different application scenarios and needs, providing users with a more personalized and high-quality experience. Furthermore, it can automatically acquire environmental information about the user's environment and intelligently determine the appropriate filtering mode based on this information. This enables efficient and accurate filtering of ambient audio signals, simplifying the user's operation process and ensuring high-quality effective audio signals and environmental reference signals are obtained in different scenarios, thereby improving the overall efficiency of audio processing.

[0231] The above description is merely an embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, and improvements made within the spirit and scope of this application are included within the scope of protection of this application.

Claims

1. An audio processing method, characterized in that, The method includes: The ambient audio signal collected by the noise reduction device is filtered to obtain an effective audio signal and an environmental reference signal; The effective audio signal and the input audio signal are enhanced to obtain an enhanced audio signal; Based on the device feedback signal and the environmental reference signal collected by the noise reduction device, a cancellation signal is generated; The playback signal of the noise reduction device is generated based on the enhanced signal, the environmental reference signal, and the cancellation signal.

2. The method according to claim 1, characterized in that, The filtering process is implemented through an audio filtering model. Before filtering the ambient audio signal collected by the noise reduction device to obtain the effective audio signal and the ambient reference signal, the method further includes: Valid audio signal samples are acquired, and the valid audio signal samples are fused with environmental reference signal samples to obtain environmental audio signal samples. The environmental audio signal samples are filtered using an audio filtering model to obtain the predicted valid audio signal; The filtering loss is determined based on the error between the predicted effective audio signal and the effective audio signal sample; The parameters of the audio filtering model are updated based on the filtering loss to obtain the updated audio filtering model.

3. The method according to claim 1, characterized in that, The enhancement processing is implemented through an audio enhancement model, and the method further includes: Obtain the original audio signal sample and perform quality attenuation processing on the original audio signal sample to obtain the attenuated audio signal sample corresponding to the original audio signal sample; The weakened audio samples are enhanced using the audio enhancement model to obtain a predicted enhanced audio signal; The audio enhancement loss is determined based on the difference between the predicted enhanced audio signal and the original audio signal sample; The parameters of the audio enhancement model are updated based on the audio enhancement loss to obtain the updated audio enhancement model.

4. The method according to claim 1, characterized in that, The enhancement processing of the effective audio signal and the input audio signal to obtain the enhanced audio signal includes: The effective audio signal and the input audio signal are mixed to obtain a mixed audio signal; The mixed audio signal is enhanced to obtain the enhanced audio signal.

5. The method according to claim 4, characterized in that, The enhancement process for the mixed audio signal to obtain the enhanced audio signal includes: The mixed audio signal is subjected to time-frequency conversion processing to obtain a frequency domain mixed audio signal; Obtain the first audio signal located below the first frequency in the frequency domain mixed audio signal; The first audio signal is enhanced to obtain an enhanced first audio signal; The enhanced first audio signal and the frequency domain mixed audio signal are fused to obtain the enhanced frequency domain mixed audio signal; The enhanced frequency-domain hybrid audio signal is subjected to inverse time-frequency conversion processing to obtain the enhanced audio signal.

6. The method according to claim 1, characterized in that, The generation of a cancellation signal based on the device feedback signal and the environmental reference signal collected by the noise reduction device includes: Obtain the error signal between the environmental reference signal and the device feedback signal; The active noise reduction weights are adjusted based on the error signal, the device feedback signal, and the step size parameter to obtain the adjusted active noise reduction weights. The environmental reference signal is convolved based on the adjusted active noise reduction weights to obtain a convolved environmental reference signal. The phase-flipping operation is performed on the convolutional environmental reference signal to obtain the cancellation signal.

7. The method according to claim 6, characterized in that, The step of adjusting the active noise reduction weights based on the error signal, the device feedback signal, and the step size parameter to obtain the adjusted active noise reduction weights includes: The weight change value is determined based on the error signal, the device feedback signal, and the step size parameter; The active noise reduction weights are adjusted based on the weight change values ​​to obtain the adjusted active noise reduction weights.

8. The method according to claim 7, characterized in that, The step of determining the weight change value based on the error signal, the device feedback signal, and the step size parameter includes: Obtain the square of the amplitude of the device feedback signal; Obtain the ratio of the device feedback signal to the square of the amplitude; The weight change value is obtained by multiplying the ratio, the error signal, and the step size parameter.

9. The method according to claim 6, characterized in that, The step of performing a phase-flipping operation on the convolutional environmental reference signal to obtain the cancellation signal includes: The convolutional environmental reference signal is subjected to time-frequency conversion to obtain a frequency domain environmental reference signal; Determine the phase value of the frequency domain environmental reference signal; The phase value is adjusted based on a preset coefficient to obtain the adjusted phase value; The adjusted phase value is subjected to time-frequency inverse conversion to obtain the cancellation signal.

10. The method according to claim 1, characterized in that, Before filtering the ambient audio signal collected by the noise reduction device to obtain the effective audio signal and the ambient reference signal, the method further includes: In response to a filtering configuration operation of an object, a filtering mode is obtained, wherein the filtering mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information; The filtering process of the environmental audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: The ambient audio signal collected by the noise reduction device is filtered to obtain the ambient reference signal and the effective audio signal that satisfies the filtering mode.

11. The method according to claim 1, characterized in that, Before filtering the ambient audio signal collected by the noise reduction device to obtain the effective audio signal and the ambient reference signal, the method further includes: The system acquires environmental information about the object using the noise reduction device and determines a filtering mode based on the environmental information. The filtering mode includes at least one of the following: a preset loudness range, a preset pitch range, and configured timbre information. The filtering process of the environmental audio signal collected by the noise reduction device to obtain an effective audio signal and an environmental reference signal includes: The ambient audio signal collected by the noise reduction device is filtered to obtain the ambient reference signal and the effective audio signal that satisfies the filtering mode.

12. The method according to claim 1, characterized in that, The step of generating the playback signal for the noise reduction device based on the enhanced signal, the environmental reference signal, and the cancellation signal includes: Obtain a first weighted signal corresponding to the enhanced signal, a second weighted signal corresponding to the canceled signal, and a third weighted signal corresponding to the environmental reference signal; The first weighted signal, the second weighted signal, and the third weighted signal are superimposed to obtain the playback signal.

13. An audio processing apparatus, characterized in that, The device includes: The filtering module is used to filter the ambient audio signals collected by the noise reduction device to obtain effective audio signals and ambient reference signals; An enhancement module is used to enhance the effective audio signal and the input audio signal to obtain an enhanced audio signal; An active noise reduction module is used to generate a cancellation signal based on the device feedback signal collected by the noise reduction device and the environmental reference signal; The signal combination module is used to generate the playback signal of the noise reduction device based on the enhanced signal, the environmental reference signal and the cancellation signal.

14. An electronic device, characterized in that, The electronic device includes: Memory is used to store executable instructions for a computer; A processor, when executing computer-executable instructions stored in the memory, implements the method according to any one of claims 1 to 12.

15. A computer-readable storage medium storing computer-executable instructions, characterized in that, When the computer-executable instructions are executed by a processor, they implement the method according to any one of claims 1 to 12.

Images