Signal processing method and device and electronic equipment

Abstract

The invention provides a signal processing method and apparatus, and an electronic device. The signal processing method comprises the steps of obtaining a first audio signal of an audio output apparatus and a second audio signal collected by an audio collection apparatus; obtaining a target operation state parameter of the audio output device and / or the audio acquisition device; performing target audio processing on the first audio signal based on the target operation state parameter to obtain a target reference echo signal; performing echo cancellation processing on the second audio signal based on the target reference echo signal; wherein if the target operation state parameter is the first operation state parameter, the target audio processing is the first audio processing; if the target operation state parameter is a second operation state parameter, the target audio processing is second audio processing; processing parameters of the first audio processing and the second audio processing are different.

Description

Technical Field

[0001] This disclosure relates to the field of signal processing technology, specifically to a signal processing method, apparatus, and electronic device. Background Technology

[0002] Traditional echo cancellation typically uses an electrical signal input to an audio playback device as the reference signal. However, after mechanical conversion and air propagation, the electrical signal will differ from the final output sound signal, resulting in inaccurate echo cancellation. Summary of the Invention

[0003] In view of the above problems, this disclosure provides a signal processing method, apparatus and electronic device.

[0004] According to a first aspect of this disclosure, a signal processing method is provided, comprising: acquiring a first audio signal from an audio output device and a second audio signal acquired by an audio acquisition device; acquiring target operating state parameters of the audio output device and / or the audio acquisition device; performing target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal; and performing echo cancellation processing on the second audio signal based on the target reference echo signal; wherein, if the target operating state parameter is a first operating state parameter, the target audio processing is a first audio processing; if the target operating state parameter is a second operating state parameter, the target audio processing is a second audio processing; and the processing parameters of the first audio processing and the second audio processing are different.

[0005] According to embodiments of this disclosure, the processing parameters for the target audio processing include a first processing parameter and a second processing parameter; wherein the first processing parameter is related to the first audio signal, and the second processing parameter is related to the target operating state parameter.

[0006] According to embodiments of this disclosure, the target audio processing includes a first sub-target audio processing and a second sub-target audio processing, with a first processing parameter corresponding to the first sub-target audio processing and a second processing parameter corresponding to the second sub-target audio processing. The target audio processing of the first audio signal based on target operating state parameters to obtain a target reference echo signal includes: performing first sub-target audio processing on the first audio signal based on the first processing parameter to obtain a first sub-target audio signal; performing second sub-target audio processing on the first sub-target audio signal based on the second processing parameter to obtain a target reference echo signal; or performing second sub-target audio processing on the first audio signal based on the second processing parameter to obtain a second sub-target audio signal; and performing first sub-target audio processing on the second sub-target audio signal based on the first processing parameter to obtain a target reference echo signal.

[0007] According to embodiments of this disclosure, the target audio processing includes at least: adding an interference signal to a first audio signal based on second processing parameters; wherein the interference signal includes an audio signal added during the output of the first audio signal by the audio output device, and / or an audio signal added during the acquisition of the second audio signal by the audio acquisition device; the target reference echo signal contains the interference signal, and the interference signal is different depending on the target operating state parameters.

[0008] According to embodiments of this disclosure, the first sub-target audio processing is selected from one of the following combinations: adjusting the frequency response of the first audio signal based on first sub-processing parameters; the first processing parameters include the first sub-processing parameters; adjusting the amplitude of the first audio signal based on second sub-processing parameters; the first processing parameters include the second sub-processing parameters; adjusting the phase of the first audio signal based on third sub-processing parameters; the first processing parameters include the third sub-processing parameters.

[0009] According to an embodiment of this disclosure, when the first processing parameter includes at least two of a first sub-processing parameter, a second sub-processing parameter, and a third sub-processing parameter, the phase of the first audio signal is adjusted based on the third sub-processing parameter during a preset processing stage.

[0010] According to embodiments of this disclosure, the method further includes: adjusting the acquisition parameters of the audio acquisition device in response to a change in the target operating state parameter of the audio acquisition device; wherein, if the target operating state parameter of the audio acquisition device is a third operating parameter, the acquisition parameter is a first acquisition parameter; if the target operating state parameter of the audio acquisition device is a fourth operating parameter, the acquisition parameter is a second acquisition parameter, and the first acquisition parameter is different from the second acquisition parameter.

[0011] According to an embodiment of this disclosure, the method further includes: adjusting the output parameters of the audio output device in response to a change in the target operating state parameter of the audio output device; wherein, if the target operating state parameter of the audio output device is a fifth operating parameter, the output parameter is a first output parameter; if the target operating state parameter of the audio output device is a sixth operating parameter, the output parameter is a second output parameter, and the first output parameter is different from the second output parameter.

[0012] A second aspect of this disclosure provides a signal processing apparatus, comprising: a first acquisition module for acquiring a first audio signal from an audio output device and a second audio signal acquired by an audio acquisition device; a second acquisition module for acquiring target operating state parameters of the audio output device and / or the audio acquisition device; an audio processing module for performing target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal; and an echo cancellation module for performing echo cancellation processing on the second audio signal based on the target reference echo signal; wherein, if the target operating state parameter is a first operating state parameter, the target audio processing is the first audio processing; if the target operating state parameter is a second operating state parameter, the target audio processing is the second audio processing; and the processing parameters for the first audio processing and the second audio processing are different.

[0013] A third aspect of this disclosure provides an electronic device, comprising: an audio acquisition device for acquiring audio signals; an audio output device for outputting audio signals; and one or more processors for performing: acquiring a first audio signal from the audio output device and a second audio signal acquired by the audio acquisition device; acquiring target operating state parameters of the audio output device and / or the audio acquisition device; performing target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal; and performing echo cancellation processing on the second audio signal based on the target reference echo signal; wherein, if the target operating state parameter is a first operating state parameter, the target audio processing is a first audio processing; if the target operating state parameter is a second operating state parameter, the target audio processing is a second audio processing; and the processing parameters of the first audio processing and the second audio processing are different.

[0014] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0015] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0016] Figure 1 A schematic diagram of conventional echo cancellation in related technologies is shown;

[0017] Figure 2 The diagram illustrates an application scenario of the signal processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0018] Figure 3 A flowchart illustrating a signal processing method according to an embodiment of the present disclosure is shown schematically.

[0019] Figure 4A schematic diagram illustrating echo cancellation according to an embodiment of the present disclosure is shown.

[0020] Figure 5 A circuit schematic based on a lumped model according to an embodiment of the present disclosure is illustrated.

[0021] Figure 6 A flowchart illustrating echo cancellation according to an embodiment of the present disclosure is shown schematically;

[0022] Figure 7 A flowchart illustrating echo reference signal modeling according to an embodiment of the present disclosure is shown schematically;

[0023] Figure 8 A schematic block diagram of a signal processing apparatus according to an embodiment of the present disclosure is shown; and

[0024] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a signal processing method according to an embodiment of the present disclosure. Detailed Implementation

[0025] The embodiments of the present disclosure will now be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the disclosure. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the present disclosure for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concepts of the present disclosure.

[0026] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit this disclosure. The terms “comprising,” “including,” etc., as used herein indicate the presence of features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0027] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0028] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0029] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples. Moreover, without contradiction, those skilled in the art can combine and integrate the different embodiments or examples described in this specification, as well as the features of those different embodiments or examples.

[0030] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0031] Echoes have nonlinear components and acoustic characteristics of audio output devices (such as loudspeakers). Due to the influence of nonlinear components and the acoustic structure of loudspeakers, traditional echo cancellation methods cannot accurately reduce echoes.

[0032] Traditional echo cancellation methods suffer from drawbacks such as echo remnants and poor two-way communication performance, which are particularly noticeable in hands-free mode. Typically, the performance of hands-free two-way communication (such as call clarity and echo cancellation effectiveness) can differ by 10-30 dB between hands-free and hands-free modes. This is mainly because the echo loudness is higher in hands-free mode, and the non-linear components significantly impact echo cancellation performance. If the echo reference signal is closer to the actual echo signal, better echo cancellation and improved two-way communication performance can be achieved. Therefore, making the characteristics of the echo and the echo reference signal as close as possible helps enhance echo cancellation and two-way communication effects.

[0033] Figure 1 A schematic diagram of conventional echo cancellation in related technologies is shown.

[0034] like Figure 1As shown, a common traditional echo reference signal is an electrical signal from the speaker path. The speaker's (SPK) power amplifier (PA) amplifies this weak electrical signal to a level sufficient to drive the speaker diaphragm. However, the electrical signal ignores a portion of the acoustic signal output from the speaker cavity. The difference between the acoustic and electrical signals can stem from important factors such as speaker characteristics, the size of the sound outlet, cavity size, acoustic duct, and the echo processed by the audio acquisition device (e.g., a microphone). Therefore, this echo reference signal differs significantly from the actual echo signal, resulting in inaccurate and unclean echo cancellation, reducing call quality, especially in hands-free mode. Because the loudness in hands-free mode is much greater than with a handset or headset, the echo volume is also greater and more difficult to cancel effectively, leading to residual echoes or excessive nonlinear echo cancellation to suppress residual echoes, ultimately affecting the normal transmission of voice.

[0035] In view of the above, this disclosure provides a signal processing method, apparatus, and electronic device, which will be described below with reference to the accompanying drawings.

[0036] Figure 2 Figure 200 schematically illustrates an application scenario of a signal processing method, apparatus, and electronic device according to embodiments of the present disclosure.

[0037] It is important to note that Figure 2 The examples shown are merely examples of scenarios in which the embodiments of this disclosure can be applied, to help those skilled in the art understand the technical content of this disclosure, but do not mean that the embodiments of this disclosure cannot be used in other devices, systems, environments or scenarios.

[0038] like Figure 2 As shown, application scenario 200 according to this embodiment may include a terminal device, including but not limited to smartphones, smart speakers, laptops, tablets, and in-vehicle audio systems. The terminal device may integrate an audio output device 201 and an audio acquisition device 202. The audio output device 201 and the audio acquisition device 202 can interact through the audio signal processing link within the terminal device to achieve specific audio functions, such as echo cancellation.

[0039] The audio output device 201 is a device with audio playback functionality, which may include speakers, headphones, audio systems, and head-mounted displays (with built-in audio). It can receive audio data sources (such as music files, call voice streams, and video audio tracks) from internal or external terminal devices, and after a series of processes such as decoding and digital-to-analog conversion, convert the digital audio signal into an analog audio signal and drive the speaker to produce sound, thereby presenting clear, high-quality audio content.

[0040] The audio acquisition device 202 is an audio data acquisition terminal, which may include audio acquisition devices such as built-in microphones, external microphones, and microphone arrays. It can acquire various sound signals from the surrounding environment (such as ambient sounds, user voice, and echoes) and convert them into electrical signals for subsequent processing and analysis by the terminal device. The audio acquisition device 202 can possess characteristics such as high sensitivity and low noise to ensure the quality of the acquired audio signals.

[0041] It should be understood that Figure 2 The number of terminal devices, audio output devices, and audio acquisition devices shown is merely illustrative. Depending on the implementation requirements, any number of terminal devices, audio output devices, and audio acquisition devices can be included.

[0042] It should be noted that the signal processing method provided in this disclosure can generally be applied to terminal devices, and correspondingly, the apparatus and electronic equipment provided in this disclosure can be installed in terminal devices. The signal processing method provided in this disclosure can also be executed by a server or server cluster that is different from the terminal device but capable of communicating with it. Correspondingly, the signal processing apparatus provided in this disclosure can also be installed in a server or server cluster that is different from the terminal device but capable of communicating with it. It should be understood that... Figure 2 The number of terminal devices shown is merely illustrative. Depending on implementation needs, any number of terminal devices can be used.

[0043] The following will be based on Figure 2 The described scene, through Figures 3-7 The signal processing method of the present disclosure will be described in detail.

[0044] like Figure 3 As shown, the signal processing method of this embodiment may include S310~S340.

[0045] In S310, the first audio signal from the audio output device and the second audio signal acquired by the audio acquisition device are obtained.

[0046] The first audio signal can be the original electrical signal input to the audio output device. This original electrical signal can be the initial digital representation of the audio content, that is, before it has undergone physical conversion and acoustic processing by the audio output device, and can completely carry all the information of the audio.

[0047] The second audio signal can be acquired when the audio output device outputs. The second audio signal can contain not only the external sound that is expected to be acquired, but also the echo component that is captured by the microphone after the sound output by the audio output device is reflected by the environment.

[0048] In S320, the target operating status parameters of the audio output device and / or audio acquisition device are acquired.

[0049] Target operating status parameters can be used to characterize the actual operating status of audio output devices and / or audio acquisition devices. For audio output devices, target operating status parameters may include operating temperature, current, voice coil displacement, volume, frequency response characteristics, and output power, etc.; for audio acquisition devices, target operating status parameters may include sensitivity, gain, sampling rate, and signal-to-noise ratio, etc.

[0050] In S330, the first audio signal is processed based on the target operating state parameters to obtain the target reference echo signal.

[0051] Depending on the different target operating state parameters, different audio processing methods can be used to process the first audio signal. These methods may include gain adjustment, frequency response adjustment, and distortion compensation, to approximate the real echo signal. If the target operating state parameter is the first operating state parameter, the target audio processing can be the first audio processing; if the target operating state parameter is the second operating state parameter, the target audio processing can be the second audio processing. Furthermore, the processing parameters for the first and second audio processing are different.

[0052] For the same first audio signal, different processing parameters can result in different target reference echo signals. That is, for the same first audio signal, the target reference echo signal obtained after the first processing can be the first reference echo signal, and the target reference echo signal obtained after the second processing can be the second reference echo signal.

[0053] For example, when the target operating state parameter is the first operating state parameter (e.g., the operating temperature of the audio output device is high), the voice coil resistance increases. According to Ohm's law, under the same voltage, the current will decrease, and the voice coil displacement will also decrease accordingly, resulting in a decrease in the loudness of the sound. At this time, the processing parameters of the first audio signal, such as gain and frequency response, can be adjusted according to the above changes to perform the first processing on the first audio signal. At the same time, the signal after target audio processing can be further adjusted by combining the target operating state parameters of the audio acquisition device. For example, factors such as the sensitivity and gain of the audio acquisition device can be considered to simulate the process of sound being output from the audio output device and acquired by the audio acquisition device, so as to generate a target reference echo signal that is closer to the real echo.

[0054] In some exemplary embodiments, multiple processing modes can be preset according to different combinations of target operating state parameters to accelerate processing efficiency. For example, when the operating temperature is within the normal range, the current is moderate, and the voice coil displacement is normal, a standard processing mode can be used; when the operating temperature is too high, the current increases, and the voice coil displacement is large, a high-temperature, high-load processing mode can be used, and each mode can correspond to different processing parameters.

[0055] In S340, echo cancellation processing is performed on the second audio signal based on the target reference echo signal.

[0056] Figure 4 A schematic diagram of echo cancellation according to an embodiment of the present disclosure is shown.

[0057] like Figure 4 As shown, the speaker's power amplifier can drive the speaker diaphragm to output a first audio signal. After the first audio signal is adjusted by the target audio processing, it is matched with the real echo collected by the audio acquisition device. After obtaining a more accurate target reference echo signal, the second audio signal collected by the audio acquisition device can be processed for echo cancellation based on the target reference echo signal, such as by using an adaptive filtering algorithm or a minimum mean square error algorithm.

[0058] For example, an adaptive filtering algorithm can be used, which automatically adjusts the filter coefficients based on the characteristics of the target reference echo signal, making the filter output as close as possible to the echo components in the second audio signal. Then, the filter output can be subtracted from the second audio signal to obtain the clean audio signal after echo removal.

[0059] Target audio processing can be integrated into an Advanced Digital Signal Processing (ADSP) platform. This platform can be equipped with high-performance processors, such as digital signal processor chips or field-programmable gate arrays (FPGAs) with powerful digital signal processing capabilities. This allows for the execution of complex mathematical operations at extremely high speeds, performing tasks such as spectral analysis and filtering involved in the target audio processing. ADSP also enables the rapid development and deployment of various algorithms to meet different target audio processing needs. Through software programming, audio processing parameters, such as gain and filter coefficients, can be easily adjusted to adapt to different audio sources and application scenarios.

[0060] For example, the raw electrical signal input to the speaker can be obtained from an audio source (such as the audio stream of a conference speaker or the audio signal of a multimedia playback device) as the first audio signal. Simultaneously, a microphone can collect sound from the conference room in real time to obtain a second audio signal, which includes the speaker's voice and the echoes formed by the sound from the speaker output after reflections from the conference room walls, tables, and chairs. Then, a temperature sensor and current detection circuit can be installed on each speaker to obtain the speaker's operating temperature and current in real time. Furthermore, the voice coil displacement can be estimated using a displacement sensor or by analyzing the electrical signal characteristics of the speaker drive circuit. Based on the obtained speaker operating temperature, current, and voice coil displacement parameters (i.e., target operating state parameters), the processing parameters for the target audio processing can be determined to process the first audio signal, such as adjusting the gain and frequency response of the first audio signal, to obtain the target reference echo signal. Finally, based on the target reference echo signal, the second audio signal can be echo-cancelled using a minimum mean square error algorithm, i.e., continuously optimizing the filter parameters to minimize the echo component in the output signal.

[0061] Understandably, by acquiring the target operating status parameters of the audio output device and the audio acquisition device in real time, and performing target audio processing based on these parameters, a target reference echo signal that more closely approximates the real echo can be generated. This allows for more accurate identification and elimination of echo components in the second audio signal, improving the accuracy of echo cancellation and effectively reducing echo residue. Furthermore, dynamically adjusting the audio processing method based on the target operating status parameters can adapt to performance changes in the audio output device and the audio acquisition device under different operating conditions, thereby maintaining good audio processing results and improving the stability and reliability of audio processing.

[0062] Based on the above embodiments, in this embodiment, the processing parameters for target audio processing include a first processing parameter and a second processing parameter; wherein, the first processing parameter is related to the first audio signal, and the second processing parameter is related to the target operating state parameter.

[0063] It is understandable that the first processing parameter is related to the first audio signal, and the second processing parameter is related to the target operating state parameter. That is, if the first audio signal is different but the target operating state parameter is the same, the first processing parameter is different and the second processing parameter is the same; if the first audio signal is the same but the target operating state parameter is different, the first processing parameter is the same and the second processing parameter is different; if the first audio signal is different but the target operating state parameter is different, both the first and second processing parameters are different. In other words, the first processing parameter corresponds to the linear part, or the part that cannot be customized. This part of the variation is universal and objective, and it is difficult to precisely quantify and adjust it according to specific operating state parameters. Therefore, a relatively standardized processing method can be adopted to minimize its impact on echo cancellation. The second processing parameter, on the other hand, corresponds to the non-linear part, or the customizable part, and can be quantized according to the operating state parameter.

[0064] When different first audio signals have varying frequency response, amplitude, and other characteristics, the first processing parameters can differ even if the target operating parameters remain consistent. Taking frequency band processing of audio signals as an example, the first processing parameters can be specifically set based on factors such as the energy distribution and frequency characteristics of the audio signal across different frequency bands. For instance, if the first audio signal has strong energy in the high-frequency band, the gain control and filter coefficients for the high-frequency band can be adjusted accordingly to prevent excessively strong high-frequency signals from causing incomplete echo cancellation or generating new distortions. Similarly, for the mid-frequency and low-frequency bands, processing parameters can be precisely set based on their energy proportion and frequency characteristics to ensure that audio signals in each frequency band are appropriately processed, thus providing a good signal foundation for echo cancellation.

[0065] Furthermore, the environment can also affect echo cancellation; that is, the environment can influence the inherent characteristics of the first audio signal (frequency response, amplitude, etc.). For example, low-frequency signals can be amplified due to the resonance effect of a room, while high-frequency signals can be attenuated by absorption by objects such as walls. In this case, the first processing parameters can be adjusted according to the characteristics of the first audio signal after the change in environment.

[0066] The second processing parameter can be closely related to the target operating state parameters of the audio output device and / or the audio acquisition device. When the first audio signal is the same, but the target operating state parameters are different, the second processing parameter can change accordingly. Different operating state parameters can cause different degrees of nonlinear distortion in the audio signal. By determining the second processing parameter according to the target operating state parameters, the first audio signal can be quantitatively adjusted, thereby enabling refined and personalized audio processing.

[0067] When the first audio signal and the target operating state parameters are different, the first processing parameter and the second processing parameter will also be different accordingly. In this case, the changes in the first audio signal and the target operating state parameters can be considered comprehensively, and the first processing parameter and the second processing parameter can be adjusted in a coordinated manner.

[0068] For example, in a smartphone voice call scenario, due to differences in voice characteristics among different people, such as pitch and volume, the frequency response and amplitude of the first audio signal vary. For the high-frequency band, if the speaker's voice is sharp, there may be a lot of high-frequency noise; in this case, the first processing parameter can focus on enhancing high-frequency noise suppression. For the low-frequency band, if the voice is deep and resonant, the low-frequency gain compensation parameter can be adjusted to avoid low-frequency echo interference. During a call, the speaker may experience increased operating temperature due to prolonged use, causing changes in its physical characteristics, such as increased impedance and a shift in the frequency response curve. Therefore, this state parameter can be used to determine the second processing parameter for the target audio processing, and the filter coefficients can be adjusted based on the second processing parameter to compensate for the effects of speaker temperature changes, ultimately generating the target reference echo signal.

[0069] Understandably, dividing the processing parameters into first and second processing parameters, each corresponding to different processing logics and objectives, improves the targeting and flexibility of audio processing. Simultaneously, establishing a quantitative relationship between the second processing parameter and the target operating state parameter allows audio processing to be adjusted according to the device's operating state, achieving customizable audio processing.

[0070] Based on the above embodiments, in this embodiment, the target audio processing includes a first sub-target audio processing and a second sub-target audio processing, with a first processing parameter corresponding to the first sub-target audio processing and a second processing parameter corresponding to the second sub-target audio processing; performing target audio processing on the first audio signal based on target running state parameters to obtain a target reference echo signal includes: performing first sub-target audio processing on the first audio signal based on the first processing parameter to obtain a first sub-target audio signal; performing second sub-target audio processing on the first sub-target audio signal based on the second processing parameter to obtain a target reference echo signal; or performing second sub-target audio processing on the first audio signal based on the second processing parameter to obtain a second sub-target audio signal; and performing first sub-target audio processing on the second sub-target audio signal based on the first processing parameter to obtain a target reference echo signal.

[0071] The first sub-target audio processing corresponding to the first processing parameter can be used to process the non-customizable parts of the first audio signal or the second sub-target audio signal, such as processing the frequency response, amplitude, and phase.

[0072] For example, based on the frequency response curves of the audio output device and the audio acquisition device, the frequency response of the first audio signal or the second sub-target audio signal can be adjusted to make its energy distribution at different frequencies more consistent with the actual situation. Simultaneously, regarding amplitude and phase, the amplitude and phase angle of the first audio signal or the second sub-target audio signal can be adjusted according to the spatial relationship between the microphone and the speaker and the sound propagation path to make it more consistent with the sound propagation characteristics in the actual environment, thereby obtaining the first sub-target audio signal or the target reference echo signal after processing the first sub-target audio.

[0073] The second sub-target audio processing corresponding to the second processing parameter can be used to process customizable parts of the first audio signal or the first sub-target audio signal, such as nonlinear processing like distortion compensation.

[0074] For example, when the audio output device is in a high-volume operating state, harmonic distortion may occur. At this time, the corresponding second processing parameter can be determined according to the high-volume operating state parameter to compensate for the harmonic distortion of the first audio signal or the first sub-target audio signal, so as to make it closer to the real echo signal, thereby obtaining an accurate target reference echo signal or the second sub-target audio signal.

[0075] It should be noted that the order of processing the first and second sub-target audio signals can be chosen based on specific needs, computational resources, and desired audio processing effects. First, linear processing is performed on the first audio signal (i.e., first sub-target audio processing), which removes the influence of non-customizable factors. Then, non-linear processing is performed to compensate for distortion. This sequential approach ensures that non-linear processing is based on a relatively standardized signal, which helps improve the accuracy of distortion compensation. Alternatively, performing non-linear processing first to initially compensate for distortion components in the audio signal, followed by linear processing to further optimize signal quality, can remove significant distortion problems more quickly in certain situations, improving processing efficiency.

[0076] Understandably, by performing first and second sub-target audio processing on the audio signal—that is, processing it according to the different characteristics of the audio signal—distortion and interference generated during signal transmission and acquisition can be effectively reduced. Furthermore, the processing order of the first and second sub-target audio processing can be flexibly selected, thereby improving adaptability to different application scenarios.

[0077] In embodiments of this disclosure, target audio processing may include at least: adding an interference signal to a first audio signal based on a second processing parameter; wherein the interference signal includes an audio signal added during the output of the first audio signal by the audio output device, and / or an audio signal added during the acquisition of the second audio signal by the audio acquisition device; the target reference echo signal contains the interference signal, and the interference signal is different depending on the target operating state parameters.

[0078] In traditional echo cancellation, distortion is a non-linear component. Because it's difficult to effectively assess the distortion caused by factors such as the speaker and its structure, echo cancellation is often inaccurate. For example, harmonic distortion, a type of non-linear distortion in speakers, can cause high-frequency component attenuation, leading to distortion in the audio signal during transmission and processing, thus affecting the accuracy of echo cancellation.

[0079] To address the aforementioned issues, the target audio processing provided in this disclosure embodiment can compare the distortion of the echo and perform distortion compensation on the first audio signal based on the second processing parameters, thereby maximizing the approximation of actual distortion.

[0080] Figure 5 A schematic diagram of a circuit based on a lumped model according to an embodiment of the present disclosure is shown.

[0081] like Figure 5 As shown, it can include multiple lumped elements, such as resistive elements like R. e (T v R2(x), R MS R AP (q p ) etc.; inductive components such as L e L(x) and L2(x) are used to store and release magnetic field energy; capacitor elements such as C MS (x), C AP (P box C R (P rear Mutual inductance elements, such as M, can be used for charge storage and filtering; MS M AP Used to achieve magnetic coupling between different circuit sections; controlled source F m (x,i,i2) is used to generate the corresponding output based on the input variables; switch S D It is used to control the on / off state of the circuit.

[0082] Figure 5This demonstrates the conversion process of a loudspeaker from electrical energy to mechanical energy and then to acoustic energy. However, traditional echo reference signals can only pick up the electrical signal, ignoring the influence of electro-to-mechanical energy conversion and acoustic factors. These neglected factors are a significant reason for the inaccurate echo cancellation. Total Harmonic Distortion (THD), as an important parameter in nonlinearity, represents the total harmonic distortion generated during signal amplification or transmission. This total harmonic distortion, i.e., the interference signal, can include the audio signal added during the output of the first audio signal by the audio output device, and / or the audio signal added during the acquisition of the second audio signal by the audio acquisition device. In the Lumped-model, common parameter factors include force factor (Bl) and mechanical compliance (C). ms ) and induction (L e Factors such as , , and can all have their impact on distortion calculated using finite element analysis.

[0083] The aforementioned parameter factors are constant under fixed conditions. However, in actual working environments, when variables change, such as changes in current, voice coil displacement, and horn temperature (i.e., target operating state parameters), these parameter factors will no longer be constant. Based on this situation, a simplified nonlinear differential equation can be derived:

[0084]

[0085] Among them, L e (x) represents the sensitivity of the voice coil, measured in mH or uH; Bl(x) represents the driving capability of the voice coil, measured in N / A; M ms R represents the mass of the diaphragm, voice coil, and air resistance, measured in g or kg. ms The machine resistance of the suspension edge is expressed in units of (kg / s or Ns / m); C ms (x,t) represents the compliance of the dangling edge, in units of (um / N), and Cms = 1 / Kms.

[0086] Figure 6 A flowchart illustrating echo cancellation according to an embodiment of the present disclosure is shown schematically.

[0087] Based on the above nonlinear differential equations, we can gradually eliminate B1 and L. e C ms By determining the corresponding current and voice coil displacement, the relationships between Bl and L under different operating states can be obtained. e And C ms At the same time, such as Figure 6As shown, two-tone excitation can be used as the input signal for measurement, that is, the loudspeaker is simultaneously excited by signals of two frequencies, and the total harmonic distortion generated under different voltages is measured. During the measurement process, the sensor can convert the acquired loudspeaker audio signal into current, and multiply it with relevant signals to generate an error signal. This error signal can reflect the difference between the loudspeaker output and the expected output. Simultaneously, the current across the loudspeaker and the actual displacement of the voice coil can be recorded, and the loudspeaker parameter factors (Bl, L) can be adjusted based on the error. e and C ms Adjustments are made to determine appropriate Bl and L. e and C ms Finally, the least mean square algorithm can be used to approximate the true THD, and the result can be fed back to the second sub-target audio processing module to form a closed-loop feedback. Through continuous iterative adjustments, the speaker output can be made closer to the expected target, thus obtaining the THD corresponding to different target operating state parameters. During echo cancellation, the corresponding THD can be determined based on different target operating state parameters, and the predicted THD (i.e., the interference signal) can be added to the first audio signal to achieve more accurate echo cancellation.

[0088] Understandably, by adding distortion-related interference signals to the first audio signal, the target reference echo signal can better reflect the actual distortion situation, thereby improving the accuracy of echo cancellation.

[0089] Based on the above embodiments, in this embodiment, the first sub-target audio processing can be selected from one of the following combinations: adjusting the frequency response of the first audio signal based on the first sub-processing parameters; the first processing parameters include the first sub-processing parameters; adjusting the amplitude of the first audio signal based on the second sub-processing parameters; the first processing parameters include the second sub-processing parameters; adjusting the phase of the first audio signal based on the third sub-processing parameters; the first processing parameters include the third sub-processing parameters.

[0090] The frequency response of the first audio signal is adjusted based on the first sub-processing parameter, that is, the gain of the first audio signal in different frequency bands is modified, thereby changing its frequency response characteristics. After frequency response adjustment, the obtained target reference echo signal and the real echo can have the same frequency response. The amplitude of the first audio signal is adjusted based on the second sub-processing parameter, that is, the amplitude of the first audio signal is linearly scaled to control its overall energy level, so that the obtained target reference echo signal and the real echo have equal amplitude. The phase of the first audio signal is adjusted based on the third sub-processing parameter, that is, the phase relationship of each frequency component of the signal is changed, so that the obtained target reference echo signal and the echo are in phase. For example, under the condition that only speaker output exists and there is no other noise interference, in order to obtain the real echo signal collected by the microphone, a log node can be inserted on the processing path after the echo signal enters the microphone to record the characteristics of the echo signal, such as frequency response characteristics, amplitude characteristics, and phase characteristics. Then, different echo signal scenarios can be simulated, and the characteristic parameters such as frequency response, amplitude and phase of the echo signal can be artificially changed. At the same time, the sub-processing parameters that achieve the best echo cancellation effect under each combination of features can be recorded, and finally a mapping table between signal features and sub-processing parameters can be formed.

[0091] Based on the frequency response characteristics in the log nodes, the first sub-processing parameters can be determined. For example, by analyzing the logs regarding frequency response characteristics, it can be found that the actual echo signal has weak energy in the low-frequency range and strong energy in the high-frequency range. In this case, the first sub-processing parameter can be set to attenuate the gain in the low-frequency range and appropriately increase the gain in the high-frequency range. Based on this first sub-processing parameter, the first sub-target audio processing can be performed, that is, adjusting the frequency response of the first audio signal. For example, an equalizer can be used to adjust the gain or attenuation of signals in different frequency ranges so that the target reference echo signal and the echo have the same frequency response.

[0092] Based on the amplitude characteristics in the log nodes, the second sub-processing parameters can be determined. For example, if the echo signal amplitude is weak in certain frequency bands, the second sub-processing parameter can be set to attenuate the gain in these bands. Based on this second sub-processing parameter, the first sub-target audio processing can be performed, i.e., adjusting the amplitude of the first audio signal. For example, the Multi-Band Dynamic Gain Compensation (MBDRC) method can be used, which increases the gain of the echo signal in the corresponding frequency band according to the second sub-processing parameter, so that the target reference echo signal and the echo have the same amplitude. At the same time, MBDRC can also prevent unwanted digital clipping. Digital clipping refers to the truncation of the signal when the signal amplitude exceeds the maximum range that the system can process, resulting in sound distortion. By dynamically adjusting the gain, it can be ensured that the signal amplitude is always within a reasonable range, guaranteeing audio quality. In addition, the amplitude difference between the echo signal and the reference signal can be eliminated by setting the gain parameter, achieving amplitude matching.

[0093] Based on the phase characteristics in the log nodes, the third sub-processing parameters can be determined. For example, by analyzing the phase characteristic logs, the phase difference can be determined, thereby identifying the third sub-processing parameters, such as the amplitude of phase adjustment, frequency range, and adjustment time window. Based on these third sub-processing parameters, the first sub-target audio processing can be performed, that is, adjusting the phase of the first audio signal so that the adjusted target reference echo signal is in phase with the real echo.

[0094] Understandably, by adjusting the frequency response, amplitude, and phase of the first audio signal, the characteristics of the target reference echo signal can be made closer to the real echo signal from multiple dimensions, thereby better eliminating the echo.

[0095] Based on the above embodiments, in this embodiment, when the first processing parameter includes at least two of the first sub-processing parameter, the second sub-processing parameter, and the third sub-processing parameter, the phase of the first audio signal is adjusted based on the third sub-processing parameter during the preset processing stage.

[0096] During the adjustment of the frequency response of the first audio signal based on the first sub-processing parameters, the frequency components change, indirectly affecting the propagation characteristics of the sound waveform and thus disturbing the phase relationship. For example, when a high-frequency signal is significantly boosted, the propagation speed and direction of the high-frequency sound may change subtly, causing a change in its phase difference with other frequencies. Similarly, when the first audio signal is amplitude-processed based on the second sub-processing parameters, amplitude changes also affect the phase characteristics of the sound to some extent. For instance, during signal amplification, if the amplifier circuit exhibits nonlinear distortion, it may cause a phase shift in the signal.

[0097] Since both frequency response processing and amplitude processing will affect the phase to some extent, phase processing can be arranged as the last stage of each processing stage, that is, the preset processing stage can be the last processing stage.

[0098] After completing frequency response and amplitude processing, the frequency response and amplitude distribution of the first audio signal can be basically determined. At this point, the first audio signal after frequency response and amplitude adjustment can be analyzed, including feature extraction and evaluation of frequency components, amplitude changes, and phase relationships, thereby determining the third sub-processing parameters to achieve precise adjustment of the phase of the first audio signal.

[0099] Understandably, by scheduling phase adjustment in the last pre-defined processing stage, it is possible to avoid the phase changing again due to other processing operations after phase adjustment, thereby ensuring the accuracy and effectiveness of phase adjustment.

[0100] In embodiments of this disclosure, the method may further include: adjusting the acquisition parameters of the audio acquisition device in response to a change in the target operating state parameter of the audio acquisition device; wherein, if the target operating state parameter of the audio acquisition device is a third operating parameter, the acquisition parameter is a first acquisition parameter; if the target operating state parameter of the audio acquisition device is a fourth operating parameter, the acquisition parameter is a second acquisition parameter, and the first acquisition parameter is different from the second acquisition parameter.

[0101] During operation, the target operating state parameters of an audio acquisition device can change due to various factors, such as fluctuations in ambient noise levels or changes in the acquisition scene. In such cases, the acquisition parameters of the audio acquisition device can be adjusted based on these changes. Specifically, if the target operating state parameter is the third operating parameter, the acquisition parameter can be the first acquisition parameter; and if the target operating state parameter is the fourth operating parameter, the acquisition parameter can be the second acquisition parameter.

[0102] It is understandable that, under the same environmental conditions (including the audio output device outputting the same audio with the same operating parameters), the second audio signal acquired by the audio acquisition device will be different under different acquisition parameters. This difference can be reflected in the acquisition effect of signals at different frequencies. Therefore, in one embodiment, after determining the interference frequency of the interfering audio signal added by the audio signal output by the audio output device, the acquisition sensitivity of the audio acquisition device for the interfering frequency is adjusted. The sensitivity adjustment can be downward, reducing the interference signal acquired by the audio acquisition device; or it can be upward, increasing the interference signal acquired by the audio acquisition device so that the interference signal is more accurately identified and eliminated during subsequent echo cancellation processing.

[0103] For example, when the ambient noise level increases (corresponding to the third operating parameter as the target operating state parameter), the first acquisition parameter can increase the acquisition sensitivity and enhance the noise suppression function (corresponding to the aforementioned echo cancellation processing, adjusting the corresponding processing parameters to enhance the noise suppression function) to ensure that the second audio signal can still be clearly acquired in noisy environments. When the ambient noise level decreases (corresponding to the fourth operating parameter as the target operating state parameter), the second acquisition parameter can decrease the acquisition sensitivity, deeply suppress potential interference frequencies, and maintain a flat audio response in the target frequency band to avoid signal degradation caused by oversampling in low-noise environments. At the same time, it provides a cleaner input signal for subsequent echo cancellation.

[0104] Understandably, by adjusting the acquisition parameters in real time according to the target operating status parameters of the audio acquisition device, the audio acquisition device can always be in the best working state, thereby improving the quality and accuracy of audio acquisition.

[0105] In embodiments of this disclosure, the method may further include: adjusting the output parameters of the audio output device in response to a change in the target operating state parameter of the audio output device; wherein, if the target operating state parameter of the audio output device is a fifth operating parameter, the output parameter is a first output parameter; if the target operating state parameter of the audio output device is a sixth operating parameter, the output parameter is a second output parameter, and the first output parameter is different from the second output parameter.

[0106] It is understandable that, under the same input audio signal (including the same first audio signal input to the audio output device), the actual audio signal output by the audio output device will differ under different output parameters. This difference can be reflected in the output effect of signals at different frequencies. Therefore, in one embodiment, when the target operating state parameters of the audio output device are determined, the output parameters of the audio output device for the audio signal at the interference frequency can be adjusted. This parameter can be adjusted downwards to reduce the interference signal in the audio signal output by the audio output device; or it can be adjusted upwards to amplify the interference signal output by the audio output device, so that the interference signal can be more accurately identified and eliminated during subsequent echo cancellation processing.

[0107] During operation, the target operating parameters of an audio output device can be affected by various factors, such as changes in the load on the output device, changes in the type of audio content, or personalized settings for the output effect by the user. In this case, the output parameters of the audio output device can be adjusted according to changes in the target operating parameters. Specifically, when the target operating parameter is the fifth operating parameter, the output parameter can be the first output parameter; and when the target operating parameter is the sixth operating parameter, the output parameter can be the second output parameter.

[0108] For example, when the target operating status parameters indicate that the audio output device is at risk of overload, such as when the voice coil displacement is close to its limit or the temperature is too high (corresponding to the fifth operating parameter), the first output parameter can be adjusted downwards to reduce the gain of the interference frequency through frequency band filtering, thereby reducing distortion and hardware loss and reducing the distortion component in the echo signal caused by overload. Conversely, when the target operating status parameters indicate that the audio output device is in a region of significant nonlinear distortion, such as when the frequency response curve shows abnormal fluctuations in the critical frequency band (corresponding to the sixth operating parameter), the first output parameter can be adjusted upwards to amplify the characteristic frequency band of the interference signal. At the same time, the phase shift can be calibrated to enhance the significance of the interference signal, ensuring that subsequent echo cancellation processing can more accurately identify and eliminate the interference.

[0109] Understandably, by adjusting the output parameters in real time according to the target operating status parameters of the audio output device, the audio output effect can be effectively optimized, ensuring stable output of high-quality audio under different operating conditions and meeting diverse usage needs.

[0110] Figure 7 A flowchart illustrating echo reference signal modeling according to an embodiment of the present disclosure is shown schematically.

[0111] like Figure 7 The diagram illustrates the echo reference signal modeling process disclosed in this disclosure. The characteristics of the speaker downlink and microphone uplink (i.e., the sound propagation path from the speaker to the microphone) will have a certain impact on echo cancellation. During the echo reference signal modeling process, it is necessary to first adjust the downlink to meet basic downlink metrics, such as the Receiving Loudness Rating (RLR) and Receiving Frequency Response (RFR), and the uplink to meet basic uplink metrics, such as the Sending Loudness Rating (SLR) and Sending Frequency Response (SFR). These metrics affect the loudness and frequency response of the echo signal. The uplink metrics are consistent with actual application scenarios. For example, adjusting the sending loudness can optimize the loudness of the speech signal according to the actual application scenario and user needs, ensuring it is neither too weak to be clearly heard nor too loud to cause auditory discomfort. Adjusting the sending frequency response can correct the response characteristics of the speech signal at different frequencies to improve the timbre and sound quality of the speech, making it more natural and clear. Similarly, the downward indicators are consistent with actual application scenarios.

[0112] After adjustment, the characteristics of the echo signal processed after entering the microphone can be captured from the log node to determine the signal characteristics of the current echo. Then, based on the frequency response and amplitude of the real echo signal, parameters for adjusting the frequency response and amplitude of the first audio signal can be determined to obtain an echo reference signal with a frequency response and amplitude close to the real echo signal. Next, based on the frequency response and amplitude of the real echo signal, parameters for distortion compensation of the first audio signal can be determined to obtain an echo reference signal close to the real echo signal. Then, based on the phase characteristics of the real echo signal, parameters for adjusting the phase of the first audio signal can be determined to obtain an echo reference signal with a phase close to the real echo signal. Finally, it can be determined whether echo cancellation criteria (such as enhanced echo return loss) and dual-talk criteria (such as dual-talk detection accuracy) are met. If they are met, the process ends; otherwise, the relevant parameters of the first audio signal can be readjusted.

[0113] Figure 8 A block diagram of a signal processing apparatus according to an embodiment of the present disclosure is shown schematically.

[0114] like Figure 8 As shown, the signal processing device 800 includes a first acquisition module 810, a second acquisition module 820, an audio processing module 830, and an echo cancellation module 840.

[0115] According to some embodiments of this disclosure, the signal processing device 800 can be used to implement a reference. Figures 3-7 The signal processing method described according to embodiments of the present disclosure.

[0116] The first acquisition module 810 can perform, for example, operation S310, to acquire a first audio signal from the audio output device and a second audio signal acquired by the audio acquisition device.

[0117] The second acquisition module 820 can perform, for example, operation S320, to acquire target operating status parameters of the audio output device and / or audio acquisition device.

[0118] The audio processing module 830 can perform, for example, operation S330, to perform target audio processing on the first audio signal based on the target operating state parameters to obtain the target reference echo signal.

[0119] The echo cancellation module 840 can perform, for example, operation S340, to perform echo cancellation processing on the second audio signal based on the target reference echo signal; wherein, if the target operating state parameter is the first operating state parameter, the target audio processing is the first audio processing; if the target operating state parameter is the second operating state parameter, the target audio processing is the second audio processing; the processing parameters of the first audio processing and the second audio processing are different.

[0120] For example, any plurality of the first acquisition module 810, the second acquisition module 820, the audio processing module 830, and the echo cancellation module 840 can be combined into one module, or any one of these modules can be split into multiple modules. Alternatively, at least part of the functionality of one or more of these modules can be combined with at least part of the functionality of other modules and implemented in one module. According to embodiments of this disclosure, at least one of the first acquisition module 810, the second acquisition module 820, the audio processing module 830, and the echo cancellation module 840 can be at least partially implemented as hardware circuitry, such as a field-programmable gate array (FPGA), a programmable logic array (PLA), a system-on-a-chip, a system-on-a-substrate, a system-on-package, an application-specific integrated circuit (ASIC), or any other reasonable means of integrating or packaging the circuitry, or implemented in software, hardware, or firmware, or in any suitable combination of any of these three implementation methods. Alternatively, at least one of the first acquisition module 810, the second acquisition module 820, the audio processing module 830, and the echo cancellation module 840 may be at least partially implemented as a computer program module, which can perform corresponding functions when the computer program module is run.

[0121] It should be understood that the signal processing device in the embodiments of this disclosure corresponds to the signal processing method in the embodiments of this disclosure, and their specific implementation details are the same, which will not be repeated here.

[0122] It should be noted that the collection, storage, use, processing, transmission, provision, disclosure, and application of user personal information in this disclosed technical solution comply with relevant laws and regulations, necessary confidentiality measures have been taken, and it does not violate public order and good morals. In this disclosed technical solution, user authorization or consent has been obtained before acquiring or collecting user personal information.

[0123] Embodiments of this disclosure also provide an electronic device, which may include an audio acquisition device, an audio output device, and one or more processors. The audio acquisition device is used to acquire audio signals; the audio output device is used to output audio signals; and the one or more processors are used to perform the following actions: acquiring a first audio signal from the audio output device and a second audio signal acquired by the audio acquisition device; acquiring target operating state parameters of the audio output device and / or the audio acquisition device; performing target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal; and performing echo cancellation processing on the second audio signal based on the target reference echo signal. Wherein, if the target operating state parameter is a first operating state parameter, the target audio processing is first audio processing; if the target operating state parameter is a second operating state parameter, the target audio processing is second audio processing; and the processing parameters for the first audio processing and the second audio processing are different.

[0124] Figure 9 A block diagram schematically illustrates an electronic device suitable for implementing a signal processing method according to an embodiment of the present disclosure.

[0125] like Figure 9 As shown, an electronic device 900 according to an embodiment of the present disclosure includes a processor 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage portion 908 into a random access memory (RAM) 903. The processor 901 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 901 may also include onboard memory for caching purposes. The processor 901 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0126] RAM 903 stores various programs and data required for the operation of electronic device 900. Processor 901, ROM 902, and RAM 903 are interconnected via bus 904. Processor 901 performs various operations of the method flow according to embodiments of the present disclosure by executing programs in ROM 902 and / or RAM 903. It should be noted that programs may also be stored in one or more memories other than ROM 902 and RAM 903. Processor 901 may also perform various operations of the method flow according to embodiments of the present disclosure by executing programs stored in one or more memories.

[0127] According to embodiments of this disclosure, the electronic device 900 may further include an input / output (I / O) interface 905, which is also connected to a bus 904. The electronic device 900 may also include one or more of the following components connected to the I / O interface 905: an input section 906 including target hardware, etc.; an output section 907 including a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 908 including a hard disk, etc.; and a communication section 909 including a network interface card such as a LAN card, a modem, etc. The communication section 909 performs communication processing via a network such as the Internet. A drive 910 is also connected to the I / O interface 905 as needed. A removable medium 911, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 910 as needed so that computer programs read from it can be installed into the storage section 908 as needed.

[0128] This disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs that, when executed, implement the method according to the embodiments of this disclosure.

[0129] According to embodiments of this disclosure, the computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this disclosure, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of this disclosure, the computer-readable storage medium may include ROM 902 and / or RAM 903 and / or one or more memories other than ROM 902 and RAM 903 described above.

[0130] Embodiments of this disclosure also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the signal processing methods provided in the embodiments of this disclosure.

[0131] When the computer program is executed by the processor 901, it performs the functions defined in the system / apparatus of this disclosure embodiments. According to embodiments of this disclosure, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0132] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and downloaded and installed via the communication section 909, and / or installed from a removable medium 911. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0133] In such an embodiment, the computer program can be downloaded and installed from a network via the communication section 909, and / or installed from the removable medium 911. When the computer program is executed by the processor 901, it performs the functions defined in the system of this disclosure embodiment. According to embodiments of this disclosure, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0134] According to embodiments of this disclosure, program code for executing the computer programs provided in embodiments of this disclosure can be written in any combination of one or more programming languages. Specifically, these computational programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. Programming languages ​​include, but are not limited to, languages ​​such as Java, C++, Python, "C", or similar programming languages. The program code can execute entirely on a user's computing device, partially on a user's device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0135] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0136] Those skilled in the art will understand that the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways, even if such combinations or combinations are not explicitly described in this disclosure. In particular, the features described in the various embodiments and / or claims of this disclosure can be combined and / or combined in various ways without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0137] The embodiments of this disclosure have been described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of this disclosure. Although various embodiments have been described above, this does not mean that the measures in the various embodiments cannot be used advantageously in combination. The scope of this disclosure is defined by the appended claims and their equivalents. Various substitutions and modifications can be made by those skilled in the art without departing from the scope of this disclosure, and all such substitutions and modifications should fall within the scope of this disclosure.

Claims

1. A signal processing method, comprising: Acquire the first audio signal from the audio output device and the second audio signal acquired by the audio acquisition device; Obtain the target operating status parameters of the audio output device and / or the audio acquisition device; Based on the target operating state parameters, the first audio signal is subjected to target audio processing to obtain a target reference echo signal; The second audio signal is subjected to echo cancellation processing based on the target reference echo signal; Wherein, if the target running state parameter is the first running state parameter, the target audio processing is the first audio processing; If the target operating state parameter is the second operating state parameter, the target audio processing is the second audio processing; the processing parameters of the first audio processing and the second audio processing are different.

2. The method according to claim 1, wherein, The processing parameters for the target audio processing include a first processing parameter and a second processing parameter; The first processing parameter is related to the first audio signal, and the second processing parameter is related to the target operating state parameter.

3. The method according to claim 2, wherein, The target audio processing includes a first sub-target audio processing and a second sub-target audio processing, wherein the first processing parameter corresponds to the first sub-target audio processing and the second processing parameter corresponds to the second sub-target audio processing; The step of performing target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal includes: Based on the first processing parameters, the first audio signal is subjected to first sub-target audio processing to obtain the first sub-target audio signal; Based on the second processing parameters, the first sub-target audio signal is subjected to second sub-target audio processing to obtain the target reference echo signal; or Based on the second processing parameters, the first audio signal is subjected to second sub-target audio processing to obtain a second sub-target audio signal; Based on the first processing parameters, the second sub-target audio signal is subjected to first sub-target audio processing to obtain the target reference echo signal.

4. The method according to claim 2, wherein the target audio processing includes at least: Based on the second processing parameters, the interference signal is added to the first audio signal; The interference signal includes an audio signal added during the output of the first audio signal by the audio output device, and / or an audio signal added during the acquisition of the second audio signal by the audio acquisition device; the target reference echo signal contains the interference signal, and the interference signal varies depending on the target's operating state parameters.

5. The method of claim 3, wherein the first sub-target audio processing is selected from one of the following combinations: The frequency response of the first audio signal is adjusted based on the first sub-processing parameters; the first processing parameters include the first sub-processing parameters. The amplitude of the first audio signal is adjusted based on the second sub-processing parameter; the first processing parameter includes the second sub-processing parameter. The phase of the first audio signal is adjusted based on the third sub-processing parameter; the first processing parameter includes the third sub-processing parameter.

6. The method according to claim 5, wherein, When the first processing parameter includes at least two of the first sub-processing parameter, the second sub-processing parameter, and the third sub-processing parameter, the phase of the first audio signal is adjusted based on the third sub-processing parameter during the preset processing stage.

7. The method according to claim 1, further comprising: In response to a change in the target operating state parameters of the audio acquisition device, the acquisition parameters of the audio acquisition device are adjusted. Wherein, if the target operating state parameter of the audio acquisition device is a third operating parameter, the acquisition parameter is a first acquisition parameter; if the target operating state parameter of the audio acquisition device is a fourth operating parameter, the acquisition parameter is a second acquisition parameter, and the first acquisition parameter is different from the second acquisition parameter.

8. The method according to claim 1, further comprising: In response to a change in the target operating state parameters of the audio output device, the output parameters of the audio output device are adjusted. Wherein, if the target operating state parameter of the audio output device is the fifth operating parameter, the output parameter is the first output parameter; if the target operating state parameter of the audio output device is the sixth operating parameter, the output parameter is the second output parameter, and the first output parameter is different from the second output parameter.

9. A signal processing apparatus, comprising: The first acquisition module is used to acquire the first audio signal from the audio output device and the second audio signal acquired by the audio acquisition device. The second acquisition module is used to acquire the target operating status parameters of the audio output device and / or the audio acquisition device; The audio processing module is used to perform target audio processing on the first audio signal based on the target operating state parameters to obtain a target reference echo signal; An echo cancellation module is used to perform echo cancellation processing on the second audio signal based on the target reference echo signal; Wherein, if the target running state parameter is the first running state parameter, the target audio processing is the first audio processing; If the target operating state parameter is the second operating state parameter, the target audio processing is the second audio processing; the processing parameters of the first audio processing and the second audio processing are different.

10. An electronic device, comprising: An audio acquisition device used to acquire audio signals; Audio output device, used to output audio signals; One or more processors are used to execute: Acquire a first audio signal from an audio output device and a second audio signal from an audio acquisition device; acquire target operating status parameters of the audio output device and / or the audio acquisition device; perform target audio processing on the first audio signal based on the target operating status parameters to obtain a target reference echo signal; The second audio signal is subjected to echo cancellation processing based on the target reference echo signal; wherein, if the target operating state parameter is a first operating state parameter, the target audio processing is a first audio processing. If the target operating state parameter is the second operating state parameter, the target audio processing is the second audio processing; the processing parameters of the first audio processing and the second audio processing are different.

Images