Audio signal processing method and apparatus, storage medium, and vehicle

Abstract

This application relates to an audio signal processing method and apparatus, a storage medium, and a vehicle. The method includes: obtaining a first audio signal collected by a sound sensor; processing one or more of human voice information, harmonic information, and burst sound information included in the first audio signal to determine a first noise signal in the first audio signal; adjusting a second audio signal based on the first noise signal and the second audio signal to obtain a third audio signal, where the second audio signal is the original audio source of a playback device, and the adjustment includes amplitude adjustment; and playing the third audio signal by using the playback device. According to the embodiments of this application, the current noise level can be accurately estimated, and as a result, the adjusted audio signal has a better noise masking effect, and the user's auditory experience is improved. Further, in the above process, the dependence on non-acoustic state information is avoided, and this adjustment method can be used in multiple scenarios, is more flexible, and supports rapid deployment.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular, to an audio signal processing method and apparatus, a storage medium, and a vehicle.

Background Art

[0002] In audio playback scenarios such as music playback, voice calls, navigation prompts, or human-machine interaction, the noise level affects people's audio experience. To obtain a better audio experience, audio signals can be processed through volume adjustment and the like to reduce the energy of the noise perceived by people and reduce the noise interference received by people. However, for example, in a vehicle driving scenario, when the audio volume is manually adjusted, people's attention becomes scattered. This causes a safety risk and affects the driving experience.

[0003] In current solutions, audio signals are usually processed by using non-acoustic measurement values (such as vehicle speed) to reduce noise interference. However, in this case, the relationship between the non-acoustic measurement value and the noise needs to be calibrated by relying on a large number of experiments, and it is difficult to accurately determine the noise and adjust the reproduced audio signal when the external environment changes. This causes a degraded user's auditory experience.

Summary of the Invention

[0004] In view of this, an audio signal processing method and apparatus, a storage medium, and a vehicle are proposed.

[0005] According to a first aspect, an embodiment of the present application provides an audio signal processing method. The method includes the steps of: acquiring a first audio signal collected by a sound sensor; processing the first audio signal to determine a first noise signal in the first audio signal; adjusting a second audio signal based on the first noise signal and a second audio signal to acquire a third audio signal, wherein the second audio signal is the original audio source of a playback device, and the adjustment includes amplitude adjustment; and playing the third audio signal using a playback device.

[0006] According to embodiments of this application, the noise signal is determined by acquiring and processing an audio signal collected by a sound sensor. The audio signal collected by the sound sensor is fully utilized, and dependence on non-acoustic state information is avoided. Furthermore, the current noise level can be accurately estimated, and as a result, the estimated noise is more similar to the actual noise. The original audio source of the playback device is adjusted based on the noise signal and the original audio source to obtain an adjusted audio signal for playback by the playback device. The original audio source may also be adjusted using the noise signal, thereby achieving an effect of adapting to the noisy environment, and as a result, the adjusted audio signal has a better noise masking effect, resulting in a better auditory experience for the user.

[0007] According to the first embodiment, in a first possible implementation of an audio signal processing method, the step of processing a first audio signal to determine a first noise signal in the first audio signal includes processing one or more of the human voice information, harmonic information and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal.

[0008] According to this embodiment of the application, one or more of the human speech information, harmonic information, and burst sound information contained in the acquired audio signal collected by the sound sensor are processed to determine the noise signal. Therefore, the current noise level can be accurately estimated, and as a result the estimated noise is more similar to the actual noise, the adjusted audio signal has a better noise masking effect, and the user's auditory experience is improved. This adjustment method can be used in multiple scenarios, is more flexible, and supports rapid deployment.

[0009] According to the first embodiment, in a first possible implementation of an audio signal processing method, the steps of obtaining a third audio signal by adjusting a second audio signal based on a first noise signal and a second audio signal include: determining a second noise signal based on a first noise signal and transmission information, wherein the second noise signal is an estimated noise signal perceived by the user; and adjusting a second audio signal based on a second noise signal and a second audio signal to obtain a third audio signal.

[0010] According to embodiments of this application, a first noise signal is processed to obtain a second noise signal that is more similar to the noise actually perceived by the user, and the obtained third audio signal can better mask the noise, thereby improving the user's auditory experience.

[0011] According to a first possible implementation of the first embodiment, in a second possible implementation of the audio signal processing method, the transmitted information includes transmitted information from a sound sensor to the user's human ear and / or transmitted information within the human ear.

[0012] According to embodiments of this application, the noise transmission path may be simulated more realistically, and as a result, the determined second noise signal will more closely resemble the noise actually perceived by the user.

[0013] According to a first or second possible implementation of the first embodiment, in a third possible implementation of an audio signal processing method, the step of adjusting the second audio signal based on a second noise signal and a second audio signal to obtain a third audio signal includes the steps of determining a gain curve based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain curve to obtain a third audio signal.

[0014] According to the embodiment of this application, the second audio signal can be adjusted using a gain curve to obtain a third audio signal, thereby achieving a noise masking effect by using the third audio signal and ensuring the user's hearing.

[0015] According to the first, second, or third possible implementations of the first embodiment, in a fourth possible implementation of the audio signal processing method, the step of adjusting the second audio signal based on a second noise signal and a second audio signal to obtain a third audio signal includes the steps of determining a gain value based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain value to obtain a third audio signal.

[0016] According to this embodiment of the application, the second audio signal is adjusted by replacing the gain curve with a gain value, and as a result, the acquired third audio signal does not have a sense of modulation, resulting in a better auditory experience for the user.

[0017] According to the first, second, third, or fourth possible implementations of the first embodiment, a fifth possible implementation of an audio signal processing method includes the steps of: obtaining a third audio signal by adjusting the second audio signal based on a second noise signal and the second audio signal; determining a noise masking threshold for the second audio signal based on the second audio signal and psychoacoustic information, wherein the masking threshold indicates a volume threshold for noise that is masked by the second audio signal at each frequency, and noise whose volume is lower than the volume threshold at each frequency is masked by the second audio signal; and obtaining a third audio signal by adjusting the second audio signal based on the second noise signal and the masking threshold.

[0018] According to this embodiment of the application, the noise masking threshold of the second audio signal is determined by using psychoacoustic information, and as a result, the third audio signal can be adjusted in a more targeted manner. In this way, a better noise masking effect can be obtained, and the user's hearing can be ensured.

[0019] According to the first embodiment, or the first, second, third, fourth, or fifth possible implementations of the first embodiment, in a sixth possible implementation of the audio signal processing method, the step of determining a first noise signal in the first audio signal by processing one or more of the human voice information, harmonic information, and burst sound information contained in the first audio signal includes the step of determining a first noise signal in the first audio signal by processing echo information and one or more of the human voice information, harmonic information, and burst sound information contained in the first audio signal.

[0020] According to the embodiments of this application, echo information is processed so that the first audio signal can be processed in a more targeted manner, and various scenarios can be considered. Therefore, the noise signal can be separated more accurately from the first audio signal, and as a result, the noise estimation becomes more stable. In this way, after the second audio signal is adjusted, the noise signal can be masked more effectively, resulting in a better user experience.

[0021] According to the first embodiment, or the first, second, third, fourth, fifth, or sixth possible implementations of the first embodiment, in a seventh possible implementation of the audio signal processing method, the step of processing one or more of human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal includes the step of determining that the first noise signal is the first noise signal of the previous frame when it is determined that the first audio signal contains human voice information and / or harmonic information.

[0022] According to embodiments of this application, the first noise signal may be obtained directly by using the first noise signal of the previous frame as the determined noise signal, and no processing to remove other information is required. This reduces the workload of the adjustment process and lowers costs.

[0023] According to the first embodiment, or the first, second, third, fourth, fifth, sixth, or seventh possible implementations of the first embodiment, in an eighth possible implementation of the audio signal processing method, the first audio signal includes a first audio signal collected from the current N frames, the second audio signal includes a second audio signal to be adjusted from the current N frames, and the third audio signal includes a third audio signal from the current N frames, where N is a positive integer.

[0024] According to embodiments of this application, the number of audio signal frames used is not limited, and as a result, the computational complexity in the adjustment process can be flexibly adjusted based on the actual situation, thereby facilitating deployment in different scenarios.

[0025] According to a second aspect, an embodiment of the present application provides an audio signal processing device. The device includes an acquisition module configured to acquire a first audio signal collected by a sound sensor; a first determination module configured to process one or more of human speech information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal; a second determination module configured to acquire a third audio signal by adjusting a second audio signal based on the first noise signal and a second audio signal, wherein the second audio signal is the original audio source of a playback device, and the adjustment includes amplitude adjustment; and a playback module configured to play the third audio signal using a playback device.

[0026] According to a second embodiment, in a first possible implementation of the audio signal processing device, the second determination module is configured to determine a second noise signal based on a first noise signal and transmitted information, the second noise signal being an estimated noise signal perceived by the user, and is configured to adjust the second audio signal based on the second noise signal and the second audio signal to obtain a third audio signal.

[0027] According to the first possible implementation of the second embodiment, in the second possible implementation of the audio signal processing device, the transmitted information includes transmitted information from a sound sensor to the user's human ear and / or transmitted information within the human ear.

[0028] According to the first or second possible implementation manner of the second aspect, in the third possible implementation manner of the audio signal processing apparatus, obtaining the third audio signal by adjusting the second audio signal based on the second noise signal and the second audio signal includes determining a gain curve based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain curve to obtain the third audio signal.

[0029] According to the first, second or third possible implementation manner of the second aspect, in the fourth possible implementation manner of the audio signal processing apparatus, obtaining the third audio signal by adjusting the second audio signal based on the second noise signal and the second audio signal includes determining a gain value based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain value to obtain the third audio signal.

[0030] According to the first, second, third or fourth possible implementation manner of the second aspect, in the fifth possible implementation manner of the audio signal device method, obtaining the third audio signal by adjusting the second audio signal based on the second noise signal and the second audio signal includes determining a noise masking threshold of the second audio signal based on the second audio signal and psychoacoustic information, where the masking threshold indicates a volume threshold of noise masked by the second audio signal at each frequency, noise with a volume lower than the volume threshold at each frequency is masked by the second audio signal, and adjusting the second audio signal based on the second noise signal and the masking threshold to obtain the third audio signal.

[0031] According to the second aspect, or the first, second, third, fourth or fifth possible implementation manner of the second aspect, in the sixth possible implementation manner of the audio signal processing apparatus, one determination module is configured to process echo information and one or more of human voice information, harmonic information and burst sound information included in the first audio signal to determine the first noise signal in the first audio signal.

[0032] According to the second embodiment, or the first, second, third, fourth, fifth, or sixth possible implementations of the second embodiment, in a seventh possible implementation of the audio signal processing device, the first determination module is configured to determine that the first noise signal is the first noise signal of the previous frame when it is determined that the first audio signal contains human speech information and / or harmonic information.

[0033] According to the second embodiment, or the first, second, third, fourth, fifth, sixth, or seventh possible implementations of the second embodiment, in an eighth possible implementation of the audio signal processing device, the first audio signal includes a first audio signal collected from the current N frames, the second audio signal includes a second audio signal to be adjusted from the current N frames, and the third audio signal includes a third audio signal from the current N frames, where N is a positive integer.

[0034] According to a third aspect, an embodiment of the present application provides an audio signal processing apparatus including a processor and a memory. The memory is configured to store a program, and the processor is configured to execute the program stored in the memory, so that the apparatus implements an audio signal processing method according to the first aspect or one or more possible implementations of the first aspect.

[0035] According to a fourth aspect, an embodiment of the present application provides a terminal device. The terminal device may perform an audio signal processing method according to the first aspect or one or more possible implementations of the first aspect.

[0036] According to a fifth aspect, an embodiment of the present application provides a computer-readable storage medium. The computer-readable storage medium stores program instructions, and when the program instructions are executed by a computer, the computer can implement an audio signal processing method according to the first aspect or one or more possible implementations of the first aspect.

[0037] According to a sixth aspect, an embodiment of this application provides a computer program product including program instructions. When the program instructions are executed by a computer, the computer can implement an audio signal processing method according to the first aspect or one or more possible implementations of the first aspect.

[0038] According to a seventh aspect, an embodiment of the present application provides a vehicle, the vehicle including a processor, the processor configured to perform an audio signal processing method according to the first aspect or one or more possible implementations of the first aspect.

[0039] These and other aspects of this application will be more concise and comprehensive in the following description of the (multiple) embodiments. [Brief explanation of the drawing]

[0040] The accompanying drawings included in and constituting part of this specification, and this specification, together illustrate exemplary embodiments, features, and aspects of this application and are intended to illustrate the principles of this application. [Figure 1] (a) is a schematic diagram of an application scenario according to an embodiment of this application. (b) is a schematic diagram of an application scenario according to an embodiment of this application. [Figure 2] This is a schematic diagram showing how to adjust an audio signal according to an embodiment of this application. [Figure 3] This is a flowchart of the audio signal processing method according to an embodiment of this application. [Figure 4] This is a flowchart for processing a first audio signal according to an embodiment of this application. [Figure 5] This is a diagram showing the structure of an audio signal processing device according to an embodiment of this application. [Figure 6] This is a diagram showing the structure of an electronic device according to an embodiment of this application. [Figure 7]This is a diagram showing the structure of an electronic device according to an embodiment of this application. [Figure 8] This is a diagram showing the structure of an electronic device according to an embodiment of this application. [Figure 9] This is a diagram showing the structure of an electronic device according to an embodiment of this application. [Modes for carrying out the invention]

[0041] Various exemplary embodiments, features, and aspects of this application are described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate elements having the same or similar function. Various aspects of the embodiments are shown in the accompanying drawings, which are not necessarily drawn proportionally unless otherwise specified.

[0042] The specific term "example" used herein means "used as an example, embodiment, or illustration." Any embodiment described as "exemplary" is not necessarily described as being superior or better than any other embodiment.

[0043] Furthermore, in order to better illustrate this application, numerous specific details are given in the following particular embodiments. Those skilled in the art should understand that this application can also be realized without some of the specific details. In some examples, methods, means, elements and circuits that are well known to those skilled in the art are not described in detail, and as a result, the subject matter of this application is emphasized.

[0044] For example, in a vehicle driving scenario, there are numerous audio usage scenarios such as music playback, voice calls, navigation prompts, and human-machine interaction. The magnitude of environmental noise affects people's auditory experience in audio usage scenarios. To obtain a better auditory experience, methods such as adjusting the volume may be used to adapt to the noisy environment, reduce the energy of the noise perceived by people, and reduce noise interference to people. However, frequent manual adjustment of audio volume distracts people's attention, creates safety risks, and affects the driving experience. In current solutions, audio signals are usually processed by using non-acoustic measurements (e.g., vehicle speed) to reduce noise interference. However, in this case, the relationship between non-acoustic measurements and noise needs to be calibrated by relying on a large amount of experimentation, and it is difficult to accurately determine the noise and adjust the audio signal to be played when the external environment changes. As an alternative, the collected acoustic signal is used to fuzzy estimate the noise. However, in this case, the noise estimation is not accurate. Therefore, the user has a degraded auditory experience.

[0045] To solve the above technical problems, this application provides an audio signal processing method. In the audio signal processing method according to embodiments of this application, an audio signal collected by a sound sensor is acquired, and the audio signal is processed to determine the noise signal within the audio signal. Thus, the current noise level can be accurately estimated by using acoustic measurements, avoiding dependence on non-acoustic state information, and the estimated noise is more similar to the actual noise. The original audio source of the playback device is adjusted based on the noise signal and the original audio source to obtain an adjusted audio signal for playback by the playback device. In this way, the original audio source can be adjusted using the noise signal, thereby achieving the effect of adapting to a noisy environment, and as a result, the adjusted audio signal has a better noise masking effect, resulting in a better auditory experience for the user. Furthermore, in the above process, based on the audio signal collected by the sound sensor, this adjustment method can be used in multiple scenarios, is more flexible, and supports rapid deployment.

[0046] Figures 1(a) and 1(b) are schematic diagrams of application scenarios according to embodiments of this application. As shown in Figures 1(a) and 1(b), in possible application scenarios, the audio signal processing method according to embodiments of this application may be applied to scenarios in which noise masking is performed on a vehicle. The reproduced audio signal is adjusted to reduce the driver's perception of noise in the vehicle. The audio signal processing system according to embodiments of this application may be installed in a vehicle and includes a sound sensor, a processor, and a playback device.

[0047] A sound sensor (which may be the one shown in Figure 1(a), or a microphone, for example) may be placed anywhere inside the vehicle, for example, near the driver inside the vehicle, and is configured to collect audio signals inside the vehicle (which may be called the first audio signal in Figure 1(a) and Figure 1(b)) and to determine the ambient noise perceived by the user inside the vehicle.

[0048] A processor, such as a system-on-a-chip (SoC) or digital signal processing (DSP) chip, may be integrated into the vehicle's head unit (or audio system) as an in-vehicle computing unit. The processor may determine a noise signal corresponding to ambient noise perceived by the user in the vehicle, based on audio signals collected by sound sensors. Based on the determined noise signal and the original audio source, the processor may further adjust the original audio source of the playback device (which may be referred to as the second audio signal with reference to Figure 1(a) and Figure 1(b)) to determine the adjusted audio signal (which may be referred to as the third audio signal with reference to Figure 1(a) and Figure 1(b)). Alternatively, the processor may be located externally in a cloud server. The server and the vehicle may communicate wirelessly, for example, by using mobile communication technologies such as 2G / 3G / 4G / 5G, using wireless communication methods such as Wi-Fi, Bluetooth, frequency modulation (FM), data transmission radio, or satellite communication. Through communication between the vehicle and the server, the server may collect audio signals gathered by sound sensors for calculation and return the calculation results to the corresponding vehicle.

[0049] The playback device (see (b) in Figure 1) may be located in the vehicle, may include speakers, etc., and may be configured to play an audio signal adjusted by a processor. Figure 2 is a schematic diagram of adjusting an audio signal according to an embodiment of the present invention. As shown in Figure 2, for example, in a scenario where music is played in a vehicle, if the noise outside the vehicle increases (for example, when the vehicle is passing through a congested section of road), and the playback device is still playing unadjusted music, then as the noise increases, the noise perceived by the driver will also increase, and there is no doubt that the driver's auditory experience will be affected. However, according to an embodiment of the present invention, the audio signal collected by a sound sensor is used to adjust the music to be played in this case, and the playback device plays the adjusted audio signal. For example, due to a change in volume (as shown in the drawing, the volume increases), the music the driver hears can mask the noise perceived by the driver. In other words, because the music being played is adjusted, the adjusted music can affect the noise auditory effect on the driver's ears, and the driver's perception of noise in the vehicle can be reduced. In this way, the driver cannot hear the noise, thus improving the driver's auditory experience when music is played in the vehicle.

[0050] Although Figures 1(a) and 1(b) show only one sound sensor, one processor, and one playback device, it should be understood that the audio signal processing system may, alternatively, include other numbers of sound sensors, processors, and playback devices.

[0051] It should be noted that the audio signal processing method in the embodiments of this application may also be applied to other scenarios where noise masking needs to be performed, other than the in-vehicle scenarios shown in Figure 1(a), Figure 1(b), and Figure 2, and may be applied to usage scenarios corresponding to electronic devices having audio interaction capabilities and microphones, such as mobile phones or smart homes. This is not limited to this application.

[0052] In the following, an automotive scenario will be used as an example to describe in detail the audio signal processing method in the embodiments of this application based on the audio signal processing system described above.

[0053] Figure 3 is a flowchart of an audio signal processing method according to an embodiment of this application. This method may be applied to the audio signal processing system described above. As shown in Figure 3, the method may include the following steps.

[0054] Step S301: Acquire the first audio signal collected by the sound sensor.

[0055] For the sound sensor, see (a) in Figure 1. The first audio signal may be the signal of one or more frames. In the case of multiple frames, the first audio signal may be the signals of N consecutive frames (the value of N may be predetermined), or it may be the signals of N spaced-out frames (for example, the N frames may include signals determined by the interval between frames). It includes ambient acoustic information collected by the sound sensor. For example, it may include human speech information, harmonic information, burst sound information, echo information, and noise information.

[0056] Human voice information may include the driver's voice inside the vehicle collected by a sound sensor. Harmonic information may include long vowels in the driver's voice, speaker sounds, etc., inside the vehicle, collected by a sound sensor. Burst sound information may include short burst sounds collected by a sound sensor, such as those generated when a door is opened or closed. Echo information may include sounds collected by a sound sensor and played back by a playback device. The playback device may play audio such as music, navigation broadcasts, or other voice broadcasts. Noise information may include ambient noise inside and outside the vehicle.

[0057] The ambient noise intensity perceived by the user inside the vehicle may be estimated by removing non-noise information from the first audio signal, thereby adjusting the audio signal in a more targeted manner to better mask the noise. See below for a detailed process.

[0058] Step S302: Process the first audio signal to determine the first noise signal within the first audio signal.

[0059] The processing includes removing corresponding information from the first audio signal. To determine the first noise signal, for example, human voice information, harmonic information, burst sound information, echo information, etc., contained in the first audio signal may be removed. Figure 4 is a flowchart of processing the first audio signal according to an embodiment of this application. As shown in Figure 4, the process of processing the first audio signal may include the following steps.

[0060] Step S401: Process human voice information in the first audio signal.

[0061] Human voice information may correspond to audio information generated by the voice of the driver or a person outside the vehicle. Methods such as voice activity detection (VAD) may be used for processing. For example, whether or not the first audio signal contains human voice information may be determined by using the VAD method. If it is determined that the first audio signal contains human voice information, the human voice information contained in the current first audio signal may be removed based on the first audio signal of previous frames (e.g., the first 3 to 5 frames) using a method such as smooth interpolation, or the human voice information may be processed in other ways. This is not limited to this application.

[0062] Step S402: Process the harmonic information in the first audio signal.

[0063] The harmonic information may correspond to audio information generated by long vowels in speech, speaker sounds, etc. Methods such as long vowel detection (LVD) may be used for processing. For example, the LVD method may include collecting statistics on the energy peaks of a first audio signal in the frequency domain, thereby determining whether the first audio signal contains harmonic information.

[0064] If the first audio signal is determined to contain human speech information and / or harmonic information, the first noise signal may be determined to be the first noise signal of the previous frame.

[0065] For example, in a frame-by-frame processing scenario, the first noise signal from the previous frame may be used directly as the first noise signal for the current frame. Therefore, the first noise signal can be acquired directly, and no processing to remove other information is required. This reduces the workload of the adjustment process and lowers costs.

[0066] Step S403: Process burst sound information in the first audio signal.

[0067] The burst sound information may correspond to audio information generated by short-duration sounds, such as when a vehicle door is opened or closed. Methods such as minimum statistics (MS) may be used for processing. For example, the burst sound information contained in the first audio signal may be estimated using the MS method, and the estimated burst sound information is removed. Alternatively, the burst sound information contained in the first audio signal may be estimated using a method other than MS, thereby removing the burst sound information contained in the first audio signal. This is not limited to this application.

[0068] The sequence for processing the above information is not limited in this application. For example, burst sound information may be processed after human voice information and harmonic information have been processed. Therefore, when burst sound information is processed, any remaining human voice or harmonic information may be further removed. A first noise signal may be determined after one or more of the above types of information contained in the first audio signal have been removed.

[0069] Through the process described above, one or more of the human speech information, harmonic information, and burst sound information contained in the acquired audio signal collected by the sound sensor are processed. As a result, the current noise level can be estimated more accurately, and the estimated noise more closely resembles the actual noise. Thus, the noise masking effect of the subsequently adjusted audio signal may also be better, resulting in a better auditory experience for the user. Furthermore, this adjustment method can be used in multiple scenarios, is more flexible, and supports rapid deployment.

[0070] Optionally, the process for processing the first audio signal may further include the following steps:

[0071] Step S404: Process the echo information in the first audio signal.

[0072] The echo information may correspond to audio information generated by the audio played back by the playback device. Therefore, the echo information in the first audio signal may be removed first by using a method such as a frequency domain adaptive filter (FDAF). This method may be a linear suppression method, or alternatively, any line echo cancellation (LEC) method other than FDAF may be used.

[0073] When echo information is removed using a linear suppression method, residual echo information may still exist, and the estimated noise value may not be sufficiently accurate. This can lead to subsequent misadjustment and a chain reaction in the second audio signal. Therefore, based on linear suppression, residual echo information may be removed by using the residual echo suppression (RES) method. In this process, the first audio signal from several frames (e.g., frames 3-5) prior to the first audio signal of the current frame may be used.

[0074] In the process of removing echo information using the FDAF and RES methods, spectral holes may be generated, meaning that over-cancellation may exist at some frequencies of the first audio signal in the frequency domain. When the spectrum of the noise signal is considered to be generally smooth, frequency smoothing (FS) may be used to further compensate for spectral holes.

[0075] Thus, the first audio signal may be processed in a more targeted manner, and various scenarios can be considered. Consequently, the noise signal can be separated more accurately from the first audio signal, resulting in more stable noise estimation. In this way, after the second audio signal has been adjusted, the noise signal can be masked more effectively, resulting in a better user experience.

[0076] It should be noted that the order in which echo information is processed and human voice information, harmonic information, and burst sound information is processed is not limited in this application; that is, the order in which steps S401 to S404 are performed is not limited. For example, echo information may be processed first, followed by one or more of the human voice information, harmonic information, and burst sound information.

[0077] After removing echo information and one or more of the following: human voice information, harmonic information, and burst sound information, the acquired signal may be considered a first noise signal, i.e., estimated ambient noise perceived by the user in the vehicle, and may be adjusted based on the first noise signal and the original audio source of the playback device to determine an adjusted audio signal for playback. In this way, the adjusted audio signal can mask the ambient noise perceived by the user in the vehicle, thereby achieving a noise masking effect. For a detailed process, please refer again to Figure 3 below.

[0078] Step S303: The second audio signal is adjusted based on the first noise signal and the second audio signal to obtain the third audio signal.

[0079] The second audio signal is the original audio source of the playback device, and the adjustment may include amplitude adjustment. For the playback device, see (b) in Figure 1. The original audio source may be music, navigation sounds, voice call sounds, etc. This is not limited to this application. The third audio signal obtained by performing amplitude adjustment on the second audio signal can mask ambient noise, that is, it can reduce the user's perception of ambient noise, thereby achieving a noise masking effect.

[0080] Since there is a difference between the first noise signal and the noise actually perceived by the user's human ear, the first noise signal may be processed to more closely resemble the magnitude of the noise actually perceived by the user, in order to enable the third audio signal obtained through adjustment to mask the noise more effectively. See below.

[0081] Step S303 is This may include determining a second noise signal based on a first noise signal and transmitted information.

[0082] The second noise signal is the estimated noise signal perceived by the user.

[0083] For example, the second noise signal may be determined by weighting the first noise signal based on the transmission information. Alternatively, the second noise signal may be determined based on other methods using the transmission information. This is not limited to this application.

[0084] The transmitted information may include information transmitted from the sound sensor to the user's human ear, and / or information transmitted within the human ear.

[0085] For example, the information transmitted from the sound sensor to the user's human ear may indicate the noise transmission path from the sound sensor to the user's human ear (e.g., the area of ​​the ear's position), or it may be determined by determining the relative position between the sound sensor and the user's human ear (or an area near the human ear). The information transmitted within the human ear may indicate the noise transmission path within the semicircular canals of the user's path (e.g., from the outer ear to the middle ear). The information transmitted may be determined, for example, by using an attenuation function from the outer ear to the middle ear or an A-weighted method, which can reduce computational complexity and improve performance. The second noise signal may be determined by other methods, as an alternative, and is not limited to this application.

[0086] According to embodiments of this application, the noise transmission path may be simulated more realistically, and as a result, the determined second noise signal is more similar to the noise actually perceived by the user.

[0087] After the second noise signal is determined, the second audio signal may be adjusted based on the second noise signal and the second audio signal to obtain a third audio signal.

[0088] According to embodiments of this application, a first noise signal is processed to obtain a second noise signal that is more similar to the noise actually perceived by the user, and the obtained third audio signal can better mask the noise, thereby improving the user's auditory experience.

[0089] For example, the second audio signal may be multiplied by a gain based on the second noise signal and the second audio signal, thereby obtaining a third audio signal, where the gain may be a gain curve or a gain value. See the explanation below for details.

[0090] The second audio signal is adjusted based on the second noise signal and the second audio signal to obtain a third audio signal. This may include determining a gain curve based on a second noise signal and the second audio signal.

[0091] Determining the gain curve based on a second noise signal and a second audio signal may also be done by determining the gain curve based on noise masking thresholds for the second noise signal and the second audio signal, where the masking threshold may represent the volume threshold of the noise masked by the second audio signal at each frequency. See the following description for a method of obtaining the masking threshold. For example, the amplitude of the second noise signal corresponding to each frequency in the frequency domain may be subtracted from the volume threshold at the corresponding frequency in the masking threshold, thereby determining the gain curve. The gain curve may represent the amplitude gain corresponding to each frequency in the frequency domain.

[0092] After the gain curve has been determined, the second audio signal may be adjusted based on the gain curve to obtain the third audio signal.

[0093] For example, the amplitude of a third audio signal corresponding to each frequency in the frequency domain may be determined by multiplying the value corresponding to each frequency on the gain curve by the amplitude of a second audio signal corresponding to the frequency in the frequency domain, thereby determining the third audio signal.

[0094] According to the embodiment of this application, the second audio signal can be adjusted using a gain curve to obtain a third audio signal, thereby achieving a noise masking effect by using the third audio signal and ensuring the user's hearing.

[0095] When a second audio signal is adjusted using a gain curve, the adjusted audio signal may have a noticeable modulation feel and may even exhibit distortion. Therefore, the overall gain value may be used to replace the gain curve, thereby reducing the modulation feel of the audio signal and avoiding the excessive influence of certain singular values ​​in the gain curve on the adjustment of the second audio signal. See the explanation below for further details.

[0096] The second audio signal is adjusted based on the second noise signal and the second audio signal to obtain a third audio signal. This may include determining a gain value based on a second noise signal and a second audio signal.

[0097] Determining the gain value based on a second noise signal and a second audio signal may also be done by determining a gain curve based on noise masking thresholds for the second noise signal and the second audio signal, and then determining the gain value based on the gain curve. The gain value may be, for example, the root mean square of all or some of the values ​​on the gain curve, or a weighted mean. This is not limited to this application.

[0098] The gain value may be a single value, and this single gain value may be determined based on values ​​corresponding to all frequencies on the gain curve, or it may be determined based on values ​​corresponding to several frequencies (e.g., 20 frequencies) on the gain curve. Alternatively, the gain value may be multiple values ​​(e.g., 2 to 5 values), and these multiple gain values ​​may be determined individually, for example, based on the high-frequency, mid-frequency, and low-frequency portions of the gain curve.

[0099] After the gain value is determined, the second audio signal may be adjusted based on the gain value to obtain a third audio signal.

[0100] For example, the amplitude of a third audio signal corresponding to each frequency in the frequency domain may be determined by multiplying the gain value by the amplitude of the second audio signal at each frequency in the frequency domain, thereby determining the third audio signal. If the corresponding gain values ​​are determined based on the high-frequency, intermediate-frequency, and low-frequency portions of the gain curve, the third audio signal may be determined by multiplying the gain value corresponding to the high frequencies by the amplitude of the second audio signal corresponding to the high-frequency portion in the frequency domain, the gain value corresponding to the intermediate frequencies by the amplitude of the second audio signal corresponding to the intermediate-frequency portion in the frequency domain, and the gain value corresponding to the low frequencies by the amplitude of the second audio signal corresponding to the low-frequency portion in the frequency domain.

[0101] According to this embodiment of the application, the second audio signal is adjusted by replacing the gain curve with a gain value, and as a result, the acquired third audio signal does not have a sense of modulation, resulting in a better auditory experience for the user.

[0102] To simulate noise masking of an audio signal, a noise masking threshold for a second audio signal may be further obtained based on the determination of a second noise signal, which forms the basis for calculating the gain value or gain curve in the above description. See the following description for details.

[0103] The second audio signal is adjusted based on the second noise signal and the second audio signal to obtain a third audio signal. This may include determining a noise masking threshold for the second audio signal based on the second audio signal and psychoacoustic information.

[0104] The masking threshold may represent the volume threshold of noise that is masked by the second audio signal at each frequency, and noise whose volume is lower than the volume threshold at each frequency may be masked by the second audio signal. For example, if the masking threshold of the second audio signal at 400 Hz is 30 dBspl, noise signals below 30 dBspl may not be perceived by the user, thereby achieving the effect of masking the noise.

[0105] Psychoacoustic information may include, for example, information such as the user's hearing threshold, loudness, pitch, and sound masking. Psychoacoustic information may be obtained based on, for example, a psychoacoustic model, such as a perceptual evaluation of audio quality (PEAQ) model, a Johnston model, or a Terhardt model, but is not limited to this application. The volume thresholds of noise that can be masked by a second audio signal at different frequencies may be determined based on the psychoacoustic information.

[0106] After the masking threshold is determined, the second audio signal may be adjusted based on the second noise signal and the masking threshold to obtain a third audio signal.

[0107] See, for example, the above explanation. The gain value or gain curve may be obtained by using a second noise signal and a masking threshold, and as a result the second audio signal may be adjusted to obtain a third audio signal.

[0108] According to this embodiment of the application, the noise masking threshold of the second audio signal is determined by using psychoacoustic information, and as a result, the third audio signal can be adjusted in a more targeted manner. In this way, a better noise masking effect can be obtained, and the user's hearing can be ensured.

[0109] Since the human ear has different sensitivities to different loudness levels, the third audio signal determined in step S303 may be further corrected in the frequency domain to ensure the user's hearing (for example, equal-loudness compensation is performed). For example, the third audio signal may be corrected based on loudness information, auditory threshold information, etc., in psychoacoustic information, and the loudness information may include an equal-loudness curve. That is, the compensation correction may be performed on the third audio signal at different frequencies based on the relationship between different pure sound pressure levels and frequencies, which is obtained when the loudness perceived by the user's human ear is the same within the auditory frequency range, thereby adapting to the sensitivity of the human ear to different loudness levels.

[0110] Step S304: Play the third audio signal using the playback device.

[0111] According to embodiments of this application, the noise signal is determined by acquiring and processing an audio signal collected by a sound sensor. The audio signal collected by the sound sensor is fully utilized, and dependence on non-acoustic state information is avoided. Furthermore, the current noise level can be accurately estimated, and as a result, the estimated noise is more similar to the actual noise. The original audio source of the playback device is adjusted based on the noise signal and the original audio source to obtain an adjusted audio signal for playback by the playback device. The original audio source may also be adjusted using the noise signal, thereby achieving an effect of adapting to the noisy environment, and as a result, the adjusted audio signal has a better noise masking effect, resulting in a better auditory experience for the user.

[0112] In the process of processing audio signals, the first audio signal may include the first audio signal collected from the current N frames, the second audio signal may include the second audio signal to be adjusted from the current N frames, and the third audio signal may include the third audio signal from the current N frames, where N is a positive integer.

[0113] The value of N may be predetermined, and the signals of the N frames may be N spaced-out frames (for example, N frames may contain signals determined at the interval of one frame), or N consecutive frames. This is not limited to this application.

[0114] For example, N may be 1, and as a result, frame-by-frame processing may be performed to dynamically determine the third audio signal.

[0115] Therefore, the computational complexity in the adjustment process may be flexibly adjusted based on the actual situation, thereby facilitating deployment in different scenarios.

[0116] Figure 5 is a diagram showing the structure of an audio signal processing device according to an embodiment of this application. As shown in Figure 5, the device is An acquisition module 501 configured to acquire a first audio signal collected by a sound sensor, A first determination module 502 is configured to process one or more of the human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal, A second determination module 503 is configured to obtain a third audio signal by adjusting a second audio signal based on a first noise signal and a second audio signal, wherein the second audio signal is the original audio source of the playback device, and the adjustment includes amplitude adjustment. A playback module 504 configured to play a third audio signal using a playback device, and Includes.

[0117] According to embodiments of this application, an audio signal collected by a sound sensor is acquired, and one or more of the human speech information, harmonic information, and burst sound information contained in the audio signal are processed to determine the noise signal. Thus, the current noise level can be accurately estimated, and as a result, the estimated noise is more similar to the actual noise. The original audio source of the playback device is adjusted based on the noise signal and the original audio source to obtain an adjusted audio signal for playback by the playback device. The original audio source may also be adjusted using the noise signal, thereby achieving the effect of adapting to the noisy environment, and as a result, the adjusted audio signal has a better noise masking effect, resulting in a better auditory experience for the user. Furthermore, in the above process, non-acoustic measurements are not used, and the audio signal collected by the sound sensor is fully utilized, avoiding dependence on non-acoustic state information. This adjustment method can be used in multiple scenarios, is more flexible, and supports rapid deployment.

[0118] Optionally, the first determination module 502 may be configured to determine that the first noise signal is the first noise signal from the previous frame when it is determined that the first audio signal contains human speech information and / or harmonic information.

[0119] According to embodiments of this application, the first noise signal may be obtained directly by using the first noise signal of the previous frame as the determined noise signal, and no processing to remove other information is required. This reduces the workload of the adjustment process and lowers costs.

[0120] Optionally, the first determination module 502 may be configured to process echo information and one or more of the human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal.

[0121] According to the embodiments of this application, echo information is processed so that the first audio signal can be processed in a more targeted manner, and various scenarios can be considered. Therefore, the noise signal can be separated more accurately from the first audio signal, and as a result, the noise estimation becomes more stable. In this way, after the second audio signal is adjusted, the noise signal can be masked more effectively, resulting in a better user experience.

[0122] For example, the second decision module 503 may be configured to determine a second noise signal based on a first noise signal and transmitted information, the second noise signal being an estimated noise signal perceived by the user, and may be configured to adjust the second audio signal based on the second noise signal and the second audio signal to obtain a third audio signal.

[0123] According to embodiments of this application, a first noise signal is processed to obtain a second noise signal that is more similar to the noise actually perceived by the user, and the obtained third audio signal can better mask the noise, thereby improving the user's auditory experience.

[0124] The transmitted information may include information transmitted from the sound sensor to the user's human ear, and / or information transmitted within the human ear.

[0125] According to embodiments of this application, the noise transmission path may be simulated more realistically, and as a result, the determined second noise signal will more closely resemble the noise actually perceived by the user.

[0126] Optionally, obtaining a third audio signal by adjusting a second audio signal based on a second noise signal and the second audio signal may include determining a gain curve based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain curve to obtain a third audio signal.

[0127] According to the embodiment of this application, the second audio signal can be adjusted using a gain curve to obtain a third audio signal, thereby achieving a noise masking effect by using the third audio signal and ensuring the user's hearing.

[0128] Optionally, obtaining a third audio signal by adjusting a second audio signal based on a second noise signal and the second audio signal may include determining a gain value based on the second noise signal and the second audio signal, and adjusting the second audio signal based on the gain value to obtain a third audio signal.

[0129] According to this embodiment of the application, the second audio signal is adjusted by replacing the gain curve with a gain value, and as a result, the acquired third audio signal does not have a sense of modulation, resulting in a better auditory experience for the user.

[0130] Optionally, obtaining a third audio signal by adjusting a second audio signal based on a second noise signal and the second audio signal may include determining a noise masking threshold for the second audio signal based on the second audio signal and psychoacoustic information, where the masking threshold indicates the volume threshold of noise that is masked by the second audio signal at each frequency, and noise whose volume is lower than the volume threshold at each frequency is masked by the second audio signal, and adjusting the second audio signal based on the second noise signal and the masking threshold to obtain a third audio signal.

[0131] According to this embodiment of the application, the noise masking threshold of the second audio signal is determined by using psychoacoustic information, and as a result, the third audio signal can be adjusted in a more targeted manner. In this way, a better noise masking effect can be obtained, and the user's hearing can be ensured.

[0132] The first audio signal may include the first audio signal collected from the current N frames, the second audio signal may include the second audio signal to be adjusted from the current N frames, and the third audio signal may include the third audio signal from the current N frames, where N is a positive integer.

[0133] According to embodiments of this application, the number of audio signal frames used is not limited, and as a result, the computational complexity in the adjustment process can be flexibly adjusted based on the actual situation, thereby facilitating deployment in different scenarios.

[0134] Figure 6 is a diagram illustrating the structure of an electronic device according to an embodiment of this application. The electronic device may be a terminal, for example, a vehicle or a head unit, or a chip embedded in a terminal, and may implement the steps of the audio signal processing method shown in Figures 3 and 4, or may implement the functions of the audio signal processing module shown in Figure 5. As shown in Figure 6, the electronic device 600 includes a processor 601 and an interface circuit 602 coupled to the processor. Although only one processor and one interface circuit are shown in Figure 6, it should be understood that the electronic device 600 may include a number of other processors and interface circuits.

[0135] The interface circuit 602 is configured to connect to other terminal components, such as memory or another processor. The processor 601 is configured to perform signal interaction with the other components by using the interface circuit 602. The interface circuit 602 may also be an input / output interface for the processor 601.

[0136] The processor 601 may be a processor within an in-vehicle device such as a head unit, or it may be a separately sold processing unit.

[0137] For example, the processor 601 reads a computer program or instruction in memory connected to the processor 601 by using the interface circuit 602, decodes the computer program or instruction, and executes it. When the corresponding program or instruction is decoded and executed by the processor 601, the electronic device 600 may be able to implement the solution in the audio signal processing method provided in the embodiments of this application.

[0138] Optionally, these programs or instructions are stored in memory outside of the electronic device 600. When the above programs or instructions are decoded and executed by the processor 601, the memory temporarily stores some or all of the contents of the above programs or instructions.

[0139] Optionally, these programs or instructions are stored in the memory of the electronic device 600. When the memory of the electronic device 600 stores the programs or instructions, the electronic device 600 may be located in the terminal in the embodiment of this application.

[0140] Optionally, some of the contents of these programs or instructions are stored in memory outside the electronic device 600, and other contents of these programs or instructions are stored in memory within the electronic device 600.

[0141] Figure 7 is a diagram illustrating the structure of an electronic device according to an embodiment of this application. The electronic device may be a terminal, for example, a vehicle or a head unit, or a chip embedded in a terminal, and may implement steps of the audio signal processing method shown in Figures 3 and 4, or implement functions of an audio signal processing module shown in Figure 5. As shown in Figure 7, the electronic device 700 includes a processor 701 and a memory 702 coupled to the processor. Although only one processor and one memory are shown in Figure 7, it should be understood that the electronic device 700 may include other numbers of processors and memories.

[0142] The memory 702 is configured to store computer programs or computer instructions. When these computer programs or instructions are executed by the processor 701, the electronic device 700 may be able to implement the steps of the audio signal processing method in the embodiment of this application.

[0143] Figure 8 is a diagram showing the structure of an electronic device according to an embodiment of this application. As shown in Figure 8, the electronic device 800 may be a terminal, for example, a vehicle or a head unit, or a chip embedded in a terminal, and may implement the steps of the audio signal processing method shown in Figures 3 and 4, or may implement the functions of the audio signal processing module shown in Figure 5. The electronic device 800 includes at least one processor 1801, at least one memory 1802, and at least one communication interface 1803. Furthermore, the electronic device may further include common components such as an antenna. Details are not described here.

[0144] The components within the electronic device 800 will be described in detail below with reference to Figure 8.

[0145] The processor 1801 may be a general-purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to control the execution of the above solution program. The processor 1801 may include one or more processing units. For example, the processor 1801 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, a neural network processing unit (NPU), etc. Different processing units may be independent components or may be integrated into one or more processors.

[0146] The communication interface 1803 is configured to communicate with other electronic devices or communication networks, such as Ethernet, a wireless access network (RAN), a core network, or a wireless local area network (WLAN).

[0147] Memory 1802 may be read-only memory (ROM) or other types of static storage devices capable of storing static information and instructions, or random access memory (RAM) or other types of dynamic storage devices capable of storing information and instructions, or it may be, but is not limited to, electrically erasable programmable read-only memory (EEPROM), compact disc read-only memory (CD-ROM) or other compact disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital multipurpose discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium that can be used to carry or store program code expected in the form of instructions or data structures and is accessible to the computer. Memory may exist independently or be connected to the processor via a bus. Alternatively, memory may be integrated with the processor.

[0148] Memory 1802 is configured to store application code for executing the above solution, and processor 1801 controls the execution. Processor 1801 is configured to execute the application code stored in memory 1802.

[0149] For example, referring to the audio signal processing device shown in Figure 5, the acquisition module 501 in Figure 5 may be implemented by the communication interface 1803 in Figure 8. The first decision module 502 and the second decision module 503 in Figure 5 may be implemented by the processor 1801 in Figure 8.

[0150] Figure 9 is a diagram of the structure of an electronic device according to an embodiment of this application. The electronic device may be the above-mentioned terminal, for example, a vehicle or a head unit, or a chip embedded in the terminal, and may perform the audio signal processing method shown in either Figure 3 or Figure 4, or may implement the functions of the audio signal processing module shown in Figure 5. The electronic device 900 includes a sound sensor 901, a processing unit 902 coupled to the sound sensor 901, and a speaker 903 coupled to the processing unit 902. Although only one sound sensor, one speaker, and one processing unit are shown in Figure 9, it should be understood that the electronic device 900 may include other numbers of sound sensors, speakers, and processing units.

[0151] The sound sensor 901 may include a capacitive microphone, a moving-coil microphone, a laser microphone, etc. The sound sensor 901 is configured to collect the first audio signal described above. The processing unit 902 may be configured to process the first audio signal to determine the noise signal within the first audio signal, and may further adjust the original audio source based on the noise signal and the original audio source to obtain an adjusted audio signal. The speaker 903 may be configured to play the adjusted audio signal, resulting in a better noise masking effect in the played audio, thereby improving the user's auditory experience.

[0152] It should be understood that the electronic device in the embodiments of this application may be implemented by software, for example, a computer program or instructions, and the corresponding computer program or instructions may be stored in the memory of the terminal. The processor reads the corresponding computer program or instructions from the memory and performs the above functions. Alternatively, the electronic device in the embodiments of this application may be implemented by hardware. Processing unit 902 is a processor.

[0153] In the embodiments described above, each description has its own focus. For aspects not described in detail in the embodiments, refer to the relevant descriptions in other embodiments.

[0154] A computer-readable storage medium may be a tangible device capable of holding and storing instructions for use by an instruction execution device. A computer-readable storage medium may be, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exclusive list) of computer-readable storage media include portable computer disks, hard disks, random-access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (Erasable PROM, EPROM), static random-access memory (SRAM), compact disc read-only memory (CD-ROM), digital video discs (DVD), memory sticks, floppy disks, mechanical coding devices for storing instructions, such as punched cards or grooved structures, and any suitable combination thereof.

[0155] The computer-readable program instructions or code described herein may be downloaded from a computer-readable storage medium to each computing / processing device, or they may be downloaded to an external computer or external storage device via a network such as the Internet, a local area network, a wide area network, and / or a wireless network. The network may include copper transmission cables, optical fiber transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface within each computing / processing device receives computer-readable program instructions from the network and transfers the computer-readable program instructions for storage in a computer-readable storage medium within each computing / processing device.

[0156] The computer program instructions used to perform the operations in this application may be assembly instructions, Instruction Set Architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages. Programming languages ​​include object-oriented programming languages ​​such as Smalltalk and C++, and traditional procedural programming languages ​​such as C or similar languages. Computer-readable program instructions may run entirely on a user computer, partially on a user computer, as a standalone software package, partially on a user computer and partially on a remote computer, or entirely on a remote computer or server. If a remote computer is involved, the remote computer may be connected to the user computer on a network of any kind, including a Local Area Network (LAN) or a Wide Area Network (WAN), or it may be connected to an external computer (for example, connected over the Internet using an Internet Service Provider). In some embodiments, electronic circuits, such as programmable logic circuits, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), are customized by using state information from computer-readable program instructions. The electronic circuits may execute computer-readable program instructions to implement various embodiments of this application.

[0157] Various aspects of this application are described herein with reference to flowcharts and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of this application. It should be understood that each block in the flowcharts and / or block diagrams, and combinations of blocks in the flowcharts and / or block diagrams, may be implemented by computer-readable program instructions.

[0158] These computer-readable program instructions may be provided to the processor of a general-purpose computer, a dedicated computer, or other programmable data processing device to generate a machine, which, when executed by the processor of the computer or other programmable data processing device, creates a device for realizing the functions / operations specified in one or more blocks in a flowchart and / or block diagram. Alternatively, these computer-readable program instructions may be stored in a computer-readable storage medium. These instructions enable the computer, programmable data processing device, and / or other device to operate in a particular manner. Therefore, a computer-readable medium storing instructions contains artifacts that include instructions for realizing various aspects of the functions / operations specified in one or more blocks in a flowchart and / or block diagram.

[0159] Computer-readable program instructions may, alternatively, be loaded into a computer, another programmable data processing device, or other device, resulting in a series of operational steps being executed on the computer, other programmable data processing device, or other device to generate a computer-implemented process. Thus, instructions executed on the computer, other programmable data processing device, or other device implement the functions / operations specified in one or more blocks in the flowchart and / or block diagram.

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the system architecture, functions, and operation of possible implementations of the apparatus, system, method, and computer program product according to several embodiments of this application. In this regard, each block in the flowchart or block diagram may represent a module, program segment, or part of an instruction, and a module, program segment, or part of an instruction contains one or more executable instructions for implementing a specified logical function. In some alternative embodiments, the functions marked in a block may also be performed in an order different from the order marked in the accompanying drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and may, depending on the functions involved, be executed in reverse order in some cases.

[0161] It should also be noted that each block in a block diagram and / or flowchart, as well as combinations of blocks in a block diagram and / or flowchart, may be implemented by hardware that performs the corresponding function or operation (e.g., a circuit or an ASIC (Application-Specific Integrated Circuit)), or by a combination of hardware and software, such as firmware.

[0162] While this application is described with reference to embodiments, in the process of realizing this application for which protection is claimed, a person skilled in the art may understand and realize other variations of the disclosed embodiments by looking at the accompanying drawings, the disclosed content and the accompanying claims. In the claims, “comprising” does not exclude other components or other steps, and “one” does not exclude multiple cases. A single processor or other unit may realize some of the functions enumerated in the claims. Although some means are described in different dependent claims, this does not mean that these means cannot be combined to produce a better effect.

[0163] Embodiments of this application are described above. The above description is illustrative and not exhaustive, and is not limited to the embodiments disclosed. Many modifications and variations will be apparent to those skilled in the art without departing from the scope of the embodiments described. The choice of terms used herein is intended to best describe the principles of the embodiments, their practical applications or improvements in the market, or to enable other those skilled in the art to understand the embodiments disclosed herein.

Claims

1. An audio signal processing method, A step of acquiring a first audio signal collected by a sound sensor, The steps include: processing human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal; The steps include: obtaining a third audio signal by adjusting the second audio signal based on the first noise signal and the second audio signal, wherein the second audio signal is the original audio source of the playback device, and the adjustment includes amplitude adjustment; The steps include: playing back the third audio signal using the playback device; A method that includes this.

2. The step of adjusting the second audio signal based on the first noise signal and the second audio signal to obtain a third audio signal is: The steps include determining a second noise signal based on the first noise signal and transmitted information, wherein the second noise signal is an estimated noise signal perceived by the user, and The steps include: adjusting the second audio signal based on the second noise signal and the second audio signal to obtain the third audio signal; The method according to claim 1, including the method described in claim 1.

3. The method according to claim 2, wherein the transmitted information includes information transmitted from the sound sensor to the user's human ear and / or information transmitted within the human ear.

4. The step of adjusting the second audio signal based on the second noise signal and the second audio signal to obtain the third audio signal is: A step of determining a gain curve based on the second noise signal and the second audio signal, The steps include adjusting the second audio signal based on the gain curve to obtain the third audio signal, and The method according to claim 2, including the method described in claim 2.

5. The step of adjusting the second audio signal based on the second noise signal and the second audio signal to obtain the third audio signal is: A step of determining a gain value based on the second noise signal and the second audio signal, The steps include adjusting the second audio signal based on the gain value to obtain the third audio signal, and The method according to claim 2, including the method described in claim 2.

6. The step of adjusting the second audio signal based on the second noise signal and the second audio signal to obtain the third audio signal is: The steps include determining a noise masking threshold for the second audio signal based on the second audio signal and psychoacoustic information, wherein the noise masking threshold represents the volume threshold of noise that is masked by the second audio signal at each frequency, and noise whose volume is lower than the volume threshold at each frequency is masked by the second audio signal. A step of obtaining the third audio signal by adjusting the second audio signal based on the second noise signal and the noise masking threshold. The method according to claim 2, including the method described in claim 2.

7. The step of processing human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal is: The method according to claim 1, comprising the step of processing echo information and the human voice information, harmonic information and burst sound information contained in the first audio signal to determine the first noise signal in the first audio signal.

8. An audio signal processing device, An acquisition module configured to acquire a first audio signal collected by a sound sensor, A first determination module configured to process human voice information, harmonic information, and burst sound information contained in the first audio signal to determine a first noise signal in the first audio signal, A second determination module configured to obtain a third audio signal by adjusting the second audio signal based on the first noise signal and the second audio signal, wherein the second audio signal is the original audio source of the playback device, and the adjustment includes amplitude adjustment. A playback module configured to reproduce the third audio signal using the aforementioned playback device, Audio signal processing device including

9. The aforementioned second decision module is, The system is configured to determine a second noise signal based on the first noise signal and transmitted information, wherein the second noise signal is an estimated noise signal perceived by the user. The audio signal processing device according to claim 8, configured to adjust the second audio signal based on the second noise signal and the second audio signal to obtain the third audio signal.

10. The audio signal processing apparatus according to claim 9, wherein the transmitted information includes information transmitted from the sound sensor to the user's human ear and / or information transmitted within the human ear.

11. The aforementioned second decision module is, A gain curve is determined based on the second noise signal and the second audio signal. The audio signal processing apparatus according to claim 9, configured to adjust the second audio signal based on the gain curve to obtain the third audio signal.

12. The aforementioned second decision module is, The gain value is determined based on the second noise signal and the second audio signal. The audio signal processing apparatus according to claim 9, configured to adjust the second audio signal based on the gain value to obtain the third audio signal.

13. The aforementioned second decision module is, The system is configured to determine a noise masking threshold for the second audio signal based on the second audio signal and psychoacoustic information, wherein the noise masking threshold indicates the volume threshold of noise that is masked by the second audio signal at each frequency, and noise whose volume is lower than the volume threshold at each frequency is masked by the second audio signal. The audio signal processing device according to claim 9, configured to adjust the second audio signal based on the second noise signal and the noise masking threshold to obtain the third audio signal.

14. The first decision module described above is: The audio signal processing device according to claim 8, configured to process echo information and the human voice information, harmonic information, and burst sound information contained in the first audio signal to determine the first noise signal in the first audio signal.

15. A computer-readable storage medium, The computer-readable storage medium stores program instructions, and when the program instructions are executed by a computer, the computer is able to implement the method according to claim 1.

Images