Signal processing method and acoustic system

Abstract

The invention provides a signal processing method and an acoustic system.The method comprises the steps that a first signal and a second signal are obtained, the first signal is obtained by collecting environment sound when a first sound sensor in a sound sensor module works, and the second signal is obtained by collecting environment sound when a second sound sensor in the sound sensor module works; the environment sound at least comprises target sound output when the loudspeaker module works, a first target operation is executed on the first signal and the second signal to generate a comprehensive signal, then a second target operation is executed on the comprehensive signal, the comprehensive signal is a synthesized signal of a signal of a first frequency band and a signal of a second frequency band, and the signal of the first frequency band and the signal of the second frequency band are different. The signal of the first frequency band is from a pickup result signal under the condition that the sound sensor module points to the loudspeaker module in the zero pickup direction. According to the comprehensive signal generated by the scheme, signal components from the loudspeaker module are reduced, so that howling can be suppressed or echoes can be eliminated.

Description

Signal processing method and acoustic system Technical Field

[0001] The present application relates to the field of acoustic technology, and in particular to a signal processing method and an acoustic system. Background Art

[0002] Some acoustic systems include both speakers and sound sensors. In these acoustic systems, the ambient sound collected by the sound sensor may include the sound emitted from the speaker, which is detrimental to the operation of the acoustic system. For example, when a hearing aid system is working, it collects ambient sound through a sound sensor, amplifies the ambient sound, and then plays it through the speaker to compensate for the wearer's hearing loss. When the sound emitted by the speaker is re-collected by the sound sensor, the acoustic system forms a closed loop. As a result, the sound emitted by the speaker is continuously amplified in the closed loop, causing howling in the acoustic system and making the wearer feel uncomfortable. For example, in a telephone system or conference system, the voice signal emitted by the remote user is played through the local speaker and is collected by the local sound sensor together with the voice emitted by the local user. It is then transmitted back to the remote end, so that the remote user is disturbed by the echo.

[0003] Summary of the Invention

[0004] The present application provides a signal processing method and an acoustic system, which can reduce the signal component from the speaker module in the integrated signal, thereby suppressing howling or eliminating echo.

[0005] In a first aspect, the present application provides a signal processing method, comprising: obtaining a first signal, wherein the first signal is obtained by collecting ambient sound when a first sound sensor in a sound sensor module is working, and the ambient sound includes at least a target sound, and the target sound is the sound output by a speaker module when working; obtaining a second signal, wherein the second signal is obtained by collecting the ambient sound when a second sound sensor in the sound sensor module is working; performing a first target operation on the first signal and the second signal to generate a comprehensive signal, wherein the comprehensive signal is a composite signal of a signal of a first frequency band and a signal of a second frequency band, and the signal of the first frequency band comes from a sound pickup result signal of the sound sensor module in a target sound pickup state, and the target sound pickup state corresponds to a zero-point sound pickup direction of the sound sensor module pointing to the speaker module; and performing a second target operation on the comprehensive signal.

[0006] In some embodiments, the signal in the second frequency band comes from the first signal.

[0007] In some embodiments, the first target operation includes: a zero-point differential operation, performing zero-point differential on the first signal and the second signal to adjust the zero-point pickup direction of the sound sensor module to point toward the speaker module, thereby obtaining the pickup result signal; and a signal synthesis operation, synthesizing the component of the first frequency band in the pickup result signal with the component of the second frequency band in the first signal to obtain the composite signal.

[0008] In some embodiments, the zero-point differential operation includes: a first delay operation, delaying the second signal to obtain a second delayed signal; a first differential operation, differentiating the first signal and the second delayed signal to obtain a first differential signal; and a gain compensation operation, performing gain compensation on signals of at least part of the frequency band in the first differential signal to obtain the pickup result signal.

[0009] In some embodiments, the zero-point differential operation includes: a first delay operation, delaying the second signal to obtain a second delayed signal; a second delay operation, delaying the first signal to obtain a first delayed signal; a first differential operation, performing a differential operation on the first signal and the second delayed signal to obtain a first differential signal; a second differential operation, performing a differential operation on the second signal and the first delayed signal to obtain a second differential signal; a third differential operation, performing a differential operation on the first differential signal and the second differential signal to obtain a third differential signal; and a gain compensation operation, performing gain compensation on signals of at least part of the frequency band in the third differential signal to obtain the pickup result signal.

[0010] In some embodiments, the zero-point differential operation also includes: a target parameter generation operation, which generates a target parameter with the goal of minimizing the signal component corresponding to the target sound in the third differential signal; and a multiplication operation, which is performed between the second differential operation and the third differential operation, multiplying the target parameter with the second differential signal to obtain a multiplication result, so that the third differential operation performs a differential operation on the first differential signal and the multiplication result to obtain the third differential signal.

[0011] In some embodiments, the signal synthesis operation includes: performing a first filtering on the first signal to obtain the component of the second frequency band in the first signal; performing a second filtering on the sound pickup result signal to obtain the component of the first frequency band in the sound pickup result signal; and synthesizing the component of the first frequency band with the component of the second frequency band to obtain the integrated signal.

[0012] In some embodiments, the first filtering and the second filtering are complementary filtering.

[0013] In some embodiments, the first target operation includes: performing a first filtering on the first signal to obtain a first sub-signal corresponding to the first frequency band in the first signal, and performing a second filtering on the first signal to obtain a second sub-signal corresponding to the second frequency band in the first signal; performing the first filtering on the second signal to obtain a third sub-signal corresponding to the first frequency band in the second signal; performing a zero-point difference on the first sub-signal and the third sub-signal to obtain a target sub-signal; and synthesizing the second sub-signal with the target sub-signal to obtain the integrated signal.

[0014] In some embodiments, a phase of a signal component corresponding to the target sound in the second signal is earlier than or equal to a phase of a signal component corresponding to the target sound in the first signal.

[0015] In some embodiments, frequencies in the first frequency band are higher than frequencies in the second frequency band.

[0016] In some embodiments, the second frequency band includes a frequency band corresponding to the background noise of the current environment, and the first frequency band includes a frequency band other than the second frequency band.

[0017] In some embodiments, the method further includes: determining a background noise characteristic corresponding to the current environment based on the first signal and the second signal; and determining a frequency range corresponding to the first frequency band and a frequency range corresponding to the second frequency band based on the background noise characteristic.

[0018] In some embodiments, the speaker module includes a first speaker and a second speaker, the first frequency band includes a first sub-frequency band and a second sub-frequency band, the sound frequency band of the first speaker includes the first sub-frequency band, and the sound frequency band of the second speaker includes the second sub-frequency band; the signal of the first sub-frequency band in the integrated signal comes from a first sound pickup result signal obtained by the sound sensor module in a first sound pickup state, and the first sound pickup state corresponds to the zero-point pickup direction of the sound sensor module pointing to the first speaker; and the signal of the second sub-frequency band in the integrated signal comes from a second sound pickup result signal obtained by the sound sensor module in a second sound pickup state, and the second sound pickup state corresponds to the zero-point pickup direction of the sound sensor module pointing to the second speaker, wherein the pickup direction patterns corresponding to the first sound pickup state and the second sound pickup state are different.

[0019] In some embodiments, the frequencies in the first sub-band are lower than the frequencies in the second sub-band; and the sound pickup direction pattern corresponding to the first sound pickup state is a cardioid, and the sound pickup direction pattern corresponding to the second sound pickup state is a figure-8.

[0020] In some embodiments, the second target operation includes: performing gain amplification on the integrated signal and sending the gain-amplified signal to the speaker module, so that the speaker module emits sound.

[0021] In some embodiments, the speaker module and the sound sensor module are deployed on a first acoustic device, the first acoustic device is communicatively connected to a second acoustic device; and the second target operation includes: sending the integrated signal to the second acoustic device to reduce the echo of the second acoustic device.

[0022] In a second aspect, the present application also provides an acoustic system, comprising: a speaker module, which receives an input signal and outputs a target sound when in operation; a sound sensor module, which at least comprises: a first sound sensor and a second sound sensor, wherein the first sound sensor collects ambient sound and generates a first signal when in operation, and the second sound sensor collects the ambient sound and generates a second signal when in operation, wherein the ambient sound includes at least the target sound; and a signal processing circuit, which is connected to the sound sensor module and which executes any one of the methods described in the first aspect when in operation.

[0023] In some embodiments, the signal processing circuit includes: at least one storage medium storing at least one instruction set for signal processing; and at least one processor communicatively connected to the sound sensor module and the at least one storage medium, wherein, when the acoustic system is running, the at least one processor reads the at least one instruction set and executes any one of the methods described in the first aspect according to the instructions of the at least one instruction set.

[0024] In some embodiments, the acoustic system is any one of a hearing aid system, a sound amplification system, a headphone system, a telephone system, and a conference system.

[0025] In some embodiments, the acoustic system is a hearing aid system, which further includes a shell, and the speaker module, the sound sensor module and the signal processing circuit are arranged in the shell, wherein, when the acoustic system is worn on the user's head, the sound output end of the speaker module is facing the user's head, and the pickup end of at least one sound sensor in the sound sensor module is located on the side of the shell away from the user's head.

[0026] It can be seen from the above technical solution that the present application provides a signal processing method and an acoustic system, which method includes: obtaining a first signal and a second signal, the first signal being obtained by collecting ambient sound when the first sound sensor in the sound sensor module is working, and the second signal being obtained by collecting ambient sound when the second sound sensor in the sound sensor module is working, the ambient sound at least including the target sound output when the speaker module is working, performing a first target operation on the first signal and the second signal to generate a comprehensive signal, and performing a second target operation on the comprehensive signal, wherein the comprehensive signal is a composite signal of the signal of the first frequency band and the signal of the second frequency band, the signal of the first frequency band comes from the sound pickup result signal of the sound sensor module in the target sound pickup state, and the target sound pickup state corresponds to the zero-point pickup direction of the sound sensor module pointing to the speaker module. In the above scheme, since the sound pickup result signal is obtained when the sound sensor module picks up sound when the zero-point pickup direction is directed towards the speaker module, the sound pickup result signal does not contain or contains relatively few signal components from the speaker module. Furthermore, since the first frequency band in the integrated signal comes from the sound pickup result signal, the signal component from the speaker module in the integrated signal is reduced, thereby reducing the sound sensor module's pickup of the target sound emitted by the speaker module, so that the above-mentioned acoustic system can achieve the effect of suppressing howling or eliminating echoes.

[0027] Other features of the acoustic system and the signal processing method for the acoustic system provided by the present application will be partially listed in the following description. The creative aspects of the acoustic system and the signal processing method for the acoustic system provided by the present application can be fully explained by practicing or using the methods, devices and combinations described in the following detailed examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0029] FIG1 is a schematic diagram showing a howling scenario according to an embodiment of the present application;

[0030] FIG2 shows a schematic diagram of an echo scenario provided according to an embodiment of the present application;

[0031] FIG3A shows a schematic structural diagram of an acoustic system provided according to an embodiment of the present application;

[0032] FIG3B shows a logic diagram of an acoustic system provided according to an embodiment of the present application;

[0033] FIG4 shows a schematic diagram of a design of the acoustic system shown in FIG3A-3B ;

[0034] FIG5 shows a flow chart of a signal processing method provided according to an embodiment of the present application;

[0035] FIG6 shows a schematic diagram of a signal processing process provided according to an embodiment of the present application;

[0036] FIG7 shows a schematic diagram of a zero-point differential operation provided according to an embodiment of the present application;

[0037] FIG8 is a schematic diagram showing a sound pickup direction pattern corresponding to the zero-point differential operation shown in FIG7 ;

[0038] FIG9 is a schematic diagram showing a frequency response curve of a first differential signal obtained by the zero-point differential operation shown in FIG7 ;

[0039] FIG10 shows a schematic diagram of another zero-point differential operation provided according to an embodiment of the present application;

[0040] FIG11 is a schematic diagram showing the principle of adjusting the sound pickup direction corresponding to the zero-point differential operation shown in FIG10 ;

[0041] FIG12 is a schematic diagram showing the sound pickup direction patterns of the zero-point differential operation shown in FIG10 in three different scenarios;

[0042] FIG13 shows a schematic diagram of another zero-point differential operation provided according to an embodiment of the present application;

[0043] FIG14 shows a schematic diagram of adjusting the zero-point pickup direction using the target parameter β according to an embodiment of the present application;

[0044] FIG15 is a schematic diagram showing another signal processing process provided according to an embodiment of the present application;

[0045] FIG16 is a schematic diagram showing a frequency response curve corresponding to a set of complementary filtering operations provided according to an embodiment of the present application;

[0046] FIG17 is a schematic diagram showing the background noise components corresponding to the sound pickup result signal and the integrated signal in FIG15 ;

[0047] FIG18 is a schematic diagram showing zero-point attenuation effects corresponding to different zero-point differential operations provided according to an embodiment of the present application; and

[0048] FIG19 shows a schematic diagram of another signal processing process provided according to an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following description provides specific application scenarios and requirements of the present application, with the purpose of enabling those skilled in the art to make and use the content of this application. Various local modifications to the disclosed embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments and applications without departing from the spirit and scope of this application. Therefore, this application is not limited to the embodiments shown, but is intended to be of the widest scope consistent with the claims.

[0050] The terms used herein are for the purpose of describing specific example embodiments only and are not intended to be limiting. For example, unless the context clearly indicates otherwise, as used herein, the singular forms "a," "an," and "the" may also include the plural forms. When used in this application, the terms "comprise," "include," and / or "contain" are meant to refer to the presence of associated integers, steps, operations, elements, and / or components, but do not exclude the presence of one or more other features, integers, steps, operations, elements, components, and / or groups or the addition of other features, integers, steps, operations, elements, components, and / or groups in the system / method.

[0051] With the following description in mind, these and other features of the present application, as well as the operation and function of the associated elements of the structure, and the economical assembly and manufacture of the components, can be significantly improved. Reference is made to the accompanying drawings, all of which form a part of this application. However, it should be clearly understood that the drawings are for illustration and description purposes only and are not intended to limit the scope of this application. It should also be understood that the drawings are not drawn to scale.

[0052] The flowcharts used in this application illustrate the operations of the system implementation according to some embodiments of the present application. It should be clearly understood that the operations of the flowcharts may not be implemented in sequence. Instead, the operations may be implemented in reverse order or simultaneously. In addition, one or more additional operations may be added to the flowcharts. One or more operations may be removed from the flowcharts.

[0053] For the convenience of description, the terms appearing in this application are first explained.

[0054] Howling: Howling is a common phenomenon in acoustic systems. The following describes the process of howling with reference to Figure 1. Figure 1 shows a schematic diagram of a howling scenario 001 provided according to an embodiment of the present application, wherein howling scenario 001 can correspond to scenarios such as a sound amplification system, a hearing aid / assisted listening system, etc. As shown in Figure 1, howling scenario 001 includes a loudspeaker 110-A, a sound sensor 120-A, and a gain amplifier 130. When the sound sensor 120-A is working, it collects ambient sound. During this process, if the loudspeaker 110-A is also playing sound synchronously, the sound played by the loudspeaker 110-A will also be collected by the sound sensor 120-A. In this way, the ambient sound collected by the sound sensor 120-A includes both the sound from the target sound source 160 and the sound from the loudspeaker 110-A. Furthermore, the above-mentioned ambient sound is input into the gain amplifier 130 for gain amplification, and then played through the loudspeaker 110-A. In this way, a closed loop of "loudspeaker-sound sensor-loudspeaker" is formed in the acoustic system. In this case, howling occurs when sound signals of certain frequencies self-oscillate. This howling can cause discomfort to the user and, when severe, can damage the acoustic equipment. Furthermore, the presence of this howling also limits the gain amplification factor of the gain amplifier 130, thereby restricting the maximum sound gain that the acoustic system 003 can achieve.

[0055] Echo: Echo is also a phenomenon that often occurs in acoustic systems. The process of generating echo is explained below in conjunction with Figure 2. Figure 2 shows a schematic diagram of an echo scene 002 provided according to an embodiment of the present application. The echo scene 002 can correspond to scenarios such as a telephone system, a conference system, and a voice call system. As shown in Figure 2, the echo scene 002 includes a local end and a remote end, wherein the local end includes a local user 140-A, a speaker 110-A, and a sound sensor 120-A, and the remote end includes a remote user 140-B, a speaker 110-B, and a sound sensor 120-B. The local end and the remote end can be connected through a network. The network is a medium for providing a communication connection between the local end and the remote end, which can facilitate the exchange of information or data between the two. In some embodiments, the network can be any type of wired or wireless network, or a combination thereof. For example, the network may include a cable network, a wired network, a fiber optic network, a telecommunications network, an intranet, the Internet, a local area network (LAN), a wide area network (WAN), a wireless local area network (WLAN), a metropolitan area network (MAN), a wide area network (WAN), a public switched telephone network (PSTN), a Bluetooth network, a ZigBee network, a near field communication (NFC) network, or the like. In some embodiments, the network may include one or more network access points. For example, the network may include a wired or wireless network access point, such as a base station or an Internet exchange point, through which a local end and a remote end can connect to the network to exchange data or information.

[0056] Continuing with Figure 2, during a call between local user 140-A and remote user 140-B, the remote voice emitted by remote user 140-B is collected by sound sensor 120-B and transmitted to the local end, where it is then played through local speaker 110-A. The remote voice played by speaker 110-A, along with the local voice emitted by local user 140-A, is collected by local sound sensor 120-A, then transmitted back to the remote end and played through remote speaker 110-B. As a result, remote user 140-B hears its own echo, which interferes with the call. It should be noted that Figure 2 illustrates the process in which remote user 140-B is disturbed by the echo. It should be understood that local user 140-A may also be disturbed by the echo. The echo generation process on the local end is similar to that described above and will not be elaborated upon here. Such an echo can interfere with the user's normal conversation.

[0057] Background noise: This refers to the ambient noise outside the sound source being measured. In this application, any sound that is unwelcome, unwanted, or disturbing to the user's hearing can be referred to as noise.

[0058] Pickup pattern: This refers to a pattern used to characterize the sensitivity of a sound sensor / sound sensor module to sounds coming from different directions. Simply put, the pickup pattern represents the ability of a sound sensor / sound sensor module to pick up sounds from different directions. Common pickup patterns include omnidirectional, cardioid, figure-8, and supercardioid.

[0059] Zero-point pickup direction: In theory, if the sensitivity of the sound sensor / sound sensor module to sounds in a certain direction is 0 or close to 0, then this direction is called the zero-point pickup direction. It should be understood that when the sound source is located in the zero-point pickup direction, the sound sensor / sound sensor module will not theoretically collect the sound emitted by the sound source. In actual situations, due to the manufacturing errors of the sound sensor / sound sensor module and the fact that the sound source in reality is not necessarily an ideal point, the sound sensor / sound sensor module can still collect less sound in the zero-point pickup direction. It should be noted that in the present application, the zero-point pickup direction can refer to a specific direction, or it can refer to a range of directions including multiple directions.

[0060] Far-field sound source: may refer to a sound source that is farther away from the sound sensor / sound sensor module. Generally speaking, when the distance between the sound source to be measured and the sound sensor / sound sensor module is greater than N times the physical size of the sound sensor / sound sensor module, the sound source can be approximately regarded as a far-field sound source. It should be noted that the value of N may be different in different application scenarios. For example, in the scenario of headphones, the physical size of the sound sensor / sound sensor module may be less than or equal to 0.01m, and the value of N may be greater than or equal to 10. That is to say, a sound source that is greater than or equal to 0.1m from the sound sensor / sound sensor module can be regarded as a far-field sound source. Compared with near-field sound sources, the sound waves of the far-field sound sources are approximately plane, and the amplitude of the sound waves decreases less with propagation.

[0061] Near-field sound source: It can refer to a sound source that is closer to the sound sensor / sound sensor module. Generally speaking, when the distance between the sound source to be measured and the sound sensor / sound sensor module is less than 2 to 3 times the physical size of the sound sensor / sound sensor module, the sound source can be approximately regarded as a far-field sound source. For example, in the scenario of headphones, the sound source with a distance of less than 0.1m can be considered a near-field sound source. Compared with the aforementioned far-field sound source, the sound wave of the near-field sound source is closer to a sphere, and the amplitude of the sound wave decreases more as it propagates.

[0062] Before describing the specific embodiments of the present application, the application scenarios of the present application are first introduced as follows: the signal processing method and acoustic system provided by the present application can be applied in scenarios where howling suppression is required (such as the scenario shown in FIG1 ), and can also be applied in scenarios where echo cancellation is required (such as the scenario shown in FIG2 ). In the above scenarios, the acoustic system collects ambient sound through a sound sensor module, and uses the signal processing method described in the present application to process the collected signal to generate a composite signal, so as to reduce the signal component from the speaker in the composite signal, thereby achieving the purpose of suppressing howling or canceling echoes.

[0063] It should be noted that the above-mentioned howling suppression and echo cancellation scenarios are only some of the multiple usage scenarios provided by this application. The signal processing method and acoustic system provided by this application can also be applied to other similar scenarios. Those skilled in the art should understand that the application of the signal processing method and acoustic system provided by this application to other usage scenarios is also within the scope of protection of this application.

[0064] Figure 3A shows a schematic diagram of the structure of an acoustic system 003 provided according to an embodiment of the present application; Figure 3B shows a logical diagram of an acoustic system 003 provided according to an embodiment of the present application. Acoustic system 003 may be a sound reinforcement system, a hearing aid / assisted hearing system, a telephone system, a conference system, or a voice call system. Acoustic system 003 may include: a speaker module 110, a sound sensor module 120, and a signal processing circuit 150. The sound sensor module 120 includes at least a first sound sensor 120-1 and a second sound sensor 120-2.

[0065] It should be noted that in the acoustic system 003 shown in Figures 3A-3B, the physical position relationship between the speaker module 110, the first sound sensor 120-1, and the second sound sensor 120-2 can be arbitrary, and Figures 3A-3B do not show the physical position relationship between the three. For example, in some embodiments, the speaker module 110, the first sound sensor 120-1, and the second sound sensor 120-2 can be arranged in a straight line. For another example, in some embodiments, the speaker module 110 can be located (or substantially located) on the perpendicular bisector of the first sound sensor 120-1 and the second sound sensor 120-2. In this case, the distance between the speaker module 110 and the first sound sensor 120-1 is equal to (or substantially equal to) the distance between the speaker module 110 and the second sound sensor 120-2. Of course, the speaker module 110, the first sound sensor 120-1, and the second sound sensor 120-2 can also have other physical position relationships, which are not listed one by one in this application.

[0066] In some embodiments, the acoustic system 003 is a hearing aid system, and the acoustic system 003 may further include a housing 115, and the speaker module 110, the sound sensor module 120, and the signal processing circuit 150 may be arranged in the housing 115. The housing 115 protects the components inside it and is convenient for the user to pick up and wear. The acoustic system 003 can be worn on the user's head. For example, the acoustic system 003 can be worn on the user's ear in an in-ear or hanging-ear manner. When the acoustic system 003 is worn on the user's head, the sound output end of the speaker module 110 is directed toward the user's head, for example, toward the user's ear canal or toward the vicinity of the ear canal. The sound pickup end of at least one sound sensor in the sound sensor module 120 is located on the side of the housing 115 away from the user's head. In this way, on the one hand, it is convenient to pick up ambient sound, and on the other hand, the pickup of the sound emitted by the speaker module 110 can be minimized.

[0067] The first sound sensor 120-1 and the second sound sensor 120-2 can be the same sound sensor or different sound sensors. For ease of description, in this application, the sound sensor in the sound sensor module 120 that is closer to the speaker module 110 is referred to as the second sound sensor 120-2, and the sound sensor that is farther away from the speaker module 110 is referred to as the first sound sensor. That is, the second sound sensor 120-2 is closer to the speaker module 110 than the first sound sensor 120-1. In this way, the sound emitted by the speaker module 110 will be collected by the second sound sensor 120-2 first, and then by the first sound sensor 120-1. In this case, the phase of the signal component corresponding to the target sound in the second signal is earlier than the phase of the signal component corresponding to the target sound in the first signal. When the distance between the two sound sensors and the speaker module 110 is equal, any one of them can be referred to as the first sound sensor 120-1, and the other can be referred to as the second sound sensor 120-2. In this case, the phase of the signal component corresponding to the target sound in the second signal is equal to the phase of the signal component corresponding to the target sound in the first signal. In addition, the speaker module 110 , the first sound sensor 120 - 1 , and the second sound sensor 120 - 2 may be integrated into the same electronic device, or may be independent of each other, which is not limited in this application.

[0068] The speaker module 110 may include one speaker or multiple speakers. In the subsequent description, unless otherwise specified, the speaker module 110 is illustrated as an example of a speaker. When the speaker module 110 includes multiple speakers, the multiple speakers may be arranged in an array, such as a linear array, a planar array, a spherical array, or other arrays. A speaker, which may also be referred to as an electroacoustic converter, is a device for converting an electrical signal into an acoustic signal. For example, the speaker may be a horn.

[0069] The speaker module 110 receives input signals when working and converts them into audio for playback. The above-mentioned input signal refers to an electrical signal that carries sound information, and the above-mentioned audio refers to the sound played through the speaker module 110. In some embodiments, the input signal received by the speaker module 110 may come from the sound sensor module 120. This situation may correspond to the acoustic scene shown in Figure 1. For example, after the sound sensor module 120 collects ambient sound and generates an electrical signal, the electrical signal is provided to the speaker module 110 for playback, or the electrical signal is provided to the speaker module 110 for playback after being subjected to preset processing, wherein the above-mentioned preset processing may include at least one of amplification processing, noise reduction processing, and enhancement processing. In some embodiments, the input signal received by the speaker module 110 may also come from other electronic devices. This situation may correspond to the acoustic scene shown in Figure 2. For example, the acoustic system 003 can receive an input signal from a remote device and convert the input signal into audio through the speaker module 110 for playback, or perform preset processing on the input signal and convert it into audio for playback through the speaker module 110, wherein the preset processing can include at least one of amplification processing, noise reduction processing, and enhancement processing.

[0070] The sound sensor (first sound sensor 120-1 and / or second sound sensor 120-2) can also be called an acoustic-to-electrical converter or a sound pickup device. It is a device for collecting sound and converting it into an electrical signal. For example, the sound sensor can be a microphone (MIC). When working, the sound sensor collects ambient sound and converts it into an electrical signal that carries sound information. Among them, the sound sensor can be an omnidirectional sound sensor, in which case the sound sensor can collect ambient sound from all directions. The sound sensor can also be a directional sound sensor, in which case the sound sensor can collect ambient sound from some directions.

[0071] Continuing to refer to Figures 3A-3B, in some embodiments, the acoustic system 003 can sense and process sounds from the target sound source 160. For example, the target sound source 160 can be an electronic device with a sound playback function (such as a TV, a speaker, a mobile phone, etc.). For another example, the target sound source 160 can also be a person's throat. In this case, the ambient sound includes both the sound from the target sound source 160 and the sound from the speaker module 110. When the first sound sensor 120-1 is working, it collects the ambient sound and generates a first signal, which includes the sound signal from the target sound source 160 and the sound signal from the speaker module 110. When the second sound sensor 120-2 is working, it collects the ambient sound and generates a second signal, which includes the sound signal from the target sound source 160 and the sound signal from the speaker module 110. For ease of description, the sound emitted by the speaker module 110 is referred to as the target sound in this application. In this way, the ambient sound includes at least the target sound, the first signal includes at least the signal corresponding to the target sound, and the second signal includes at least the signal corresponding to the target sound.

[0072] The signal processing circuit 150 is connected to the sound sensor module 120. As shown in Figures 3A-3B, the signal processing circuit 150 is connected to the first sound sensor 120-1 and the second sound sensor 120-2, respectively. In this way, the signal processing circuit 150 can obtain a first signal from the first sound sensor 120-1 and a second signal from the second sound sensor 120-2. Furthermore, the signal processing circuit 150 can perform a first target operation 40 on the first signal and the second signal to generate a composite signal. The composite signal is a composite signal of the signal in the first frequency band and the signal in the second frequency band. The signal in the first frequency band is a sound pickup result signal of the sound sensor module 120 in a target sound pickup state. The target sound pickup state corresponds to the zero-point sound pickup direction of the sound sensor module 120 pointing toward the speaker module 110. It can be understood that because the sound pickup signal is obtained when the sound sensor module 120 is pointing toward the speaker module 110 with the zero-point pickup direction, the sound pickup signal does not contain or contains relatively few signal components from the speaker. Consequently, the first frequency band in the integrated signal is derived from the sound pickup signal, which can reduce the signal component from the speaker module 110 in the integrated signal. In some embodiments, "reducing the signal component from the speaker module 110 in the integrated signal" can mean reducing the signal component from the speaker module 110 in the integrated signal relative to the first signal and / or the second signal. In other words, the signal component from the speaker module 110 in the integrated signal is less than the signal component from the speaker module 110 in the first signal and / or less than the signal component from the speaker module 110 in the second signal. In some embodiments, "reducing the signal component from the speaker module 110 in the integrated signal" can mean reducing the signal strength from the speaker module 110 in the integrated signal relative to the first signal and / or the second signal. That is, the signal strength from the speaker module 110 in the integrated signal is smaller than the signal strength from the speaker module 110 in the first signal and / or smaller than the signal strength from the speaker module 110 in the second signal.

[0073] Continuing to refer to Figures 3A-3B, after generating the integrated signal, the signal processing circuit 150 can perform a second target operation 50 on the integrated signal. In some embodiments, the second target operation 50 may include: performing gain amplification on the integrated signal and sending the gain-amplified signal to the speaker module 110, so that the speaker module 110 converts it into sound. The above scheme can be applied to the howling suppression scenario as shown in Figure 1. It should be understood that since the signal processing circuit 150 reduces the pickup of the target sound by the sound sensor module 120, the signal component from the speaker module 110 is reduced in the integrated signal (or the signal strength from the speaker module 110 is reduced), which destroys the condition for the sound emitted by the speaker module 110 to generate howling in the closed loop shown in Figure 1, thereby achieving the effect of suppressing howling.

[0074] In some embodiments, when the speaker module 110, the sound sensor module 120 and the signal processing circuit 150 are deployed in the first acoustic device, the first acoustic device can be communicatively connected with the second acoustic device. In this case, the second target operation 50 may include: sending the integrated signal to the second acoustic device to reduce the echo of the second acoustic device. The above scheme can be applied to the echo cancellation scenario shown in Figure 2. For example, the first acoustic device can be a local device, and the second acoustic device can be a remote device. Since the signal component from the speaker module is reduced in the integrated signal sent by the first acoustic device to the second acoustic device (or the signal strength from the speaker module is reduced), it is equivalent to eliminating / reducing the sound from the second acoustic device. Therefore, when the second acoustic device receives the integrated signal and plays it, the user on the second acoustic device side (i.e., the remote user) will not hear the echo, thereby achieving the effect of echo cancellation.

[0075] The signal processing circuit 150 can be configured to perform the signal processing method described in this application. In some embodiments, the signal processing circuit 150 may include multiple hardware circuits with a connection relationship, each hardware circuit includes one or more electrical components, and when working, implements one or more steps in the signal processing method described in this application. The above multiple hardware circuits cooperate with each other to implement the signal processing method described in this application when working. In some embodiments, the signal processing circuit 150 may include a hardware device with data information processing functions and the necessary programs required to drive the hardware device to work. The hardware device implements the signal processing method described in this application by executing the programs. The signal processing method will be described in detail later.

[0076] FIG4 shows a schematic diagram of a design of the acoustic system 003. As shown in FIG4 , the signal processing circuit 150 may include: at least one storage medium 210 and at least one processor 220. The at least one processor 220 is communicatively connected to the speaker module 110, the first sound sensor 120-1, and the second sound sensor 120-2. It should be noted that, for the purpose of demonstration only, the signal processing circuit 150 in this application includes at least one storage medium 210 and at least one processor 220. It will be understood by those skilled in the art that the signal processing circuit 150 may also include other hardware circuit structures, which are not limited in this application, as long as they can meet the functions mentioned in this application without deviating from the spirit of this application.

[0077] Continuing with FIG4 , in some embodiments, the acoustic system 003 may further include a communication port 230. The communication port 230 is used for data communication between the acoustic system and the outside world. For example, the communication port 230 may be used for data communication between the acoustic system and other devices / systems. In some embodiments, the acoustic system 003 may further include an internal communication bus 240. The internal communication bus 240 may connect different system components. For example, the speaker module 110, the first sound sensor 120-1, the second sound sensor 120-2, the processor 220, the storage medium 210, and the communication port 230 may all be connected via the internal communication bus 240.

[0078] The storage medium 210 may include a data storage device. The data storage device may be a non-transitory storage medium or a temporary storage medium. For example, the data storage device may include one or more of a disk 2101, a read-only storage medium (ROM) 2102, or a random access storage medium (RAM) 2103. The storage medium 210 also includes at least one instruction set stored in the data storage device. The instruction set includes instructions, which are computer program codes. The computer program codes may include programs, routines, objects, components, data structures, processes, modules, etc. that execute the signal processing method provided in this application.

[0079] At least one processor 220 is used to execute the at least one instruction set mentioned above. When the acoustic system 003 is running, the at least one processor 220 reads the at least one instruction set and, according to the instructions of the at least one instruction set, executes the signal processing method provided in the present application. The processor 220 can perform all or part of the steps included in the above-mentioned signal processing method. The processor 220 can be in the form of one or more processors. In some embodiments, the processor 220 can include one or more hardware processors, such as a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), any circuit or processor capable of performing one or more functions, etc., or any combination thereof. For illustration purposes only, the acoustic system 003 shown in Figure 4 illustrates a case where only one processor 220 is included. However, it should be noted that the acoustic system 003 provided in the present application may also include multiple processors. Therefore, the operations and / or method steps disclosed in the present application may be performed by a single processor or jointly by multiple processors. For example, if the processor 220 of the acoustic system in the present application performs steps A and B, it should be understood that steps A and B may also be performed jointly or separately by two different processors 220 (e.g., the first processor performs step A, the second processor performs step B, or the first and second processors perform steps A and B together).

[0080] FIG5 shows a flow chart of a signal processing method according to an embodiment of the present application. The signal processing method P100 described in the present application can be applied to the acoustic system 003 as described above. Specifically, the signal processing circuit 150 can execute the signal processing method P100. As shown in FIG5 , the signal processing method P100 may include:

[0081] S310: Obtain a first signal, where the first signal is obtained by collecting ambient sound when a first sound sensor in the sound sensor module is working. The ambient sound at least includes a target sound, and the target sound is the sound output by the speaker module when the speaker module is working.

[0082] S320: Obtain a second signal, where the second signal is obtained when the second sound sensor in the sound sensor module collects the ambient sound when in operation.

[0083] The signal processing circuit 150 can obtain a first signal from the first sound sensor 120-1 and a second signal from the second sound sensor 120-2. As previously described, since the ambient sound includes both the sound emitted by the target sound source 160 and the target sound emitted by the speaker module 110, the first signal and the second signal obtained by the signal processing circuit 150 both include signal components from the target sound source 160 and signal components from the speaker module 110.

[0084] It should be noted that the present application does not limit the execution order of S310 and S320. The execution order of the two can be interchanged, or the two can be executed simultaneously.

[0085] S330: Perform a first target operation on the first signal and the second signal to generate a composite signal, wherein the composite signal is a composite signal of a signal in a first frequency band and a signal in a second frequency band, and the signal in the first frequency band comes from a sound pickup result signal of the sound sensor module in a target sound pickup state, and the target sound pickup state corresponds to a zero-point sound pickup direction of the sound sensor module pointing to the speaker module.

[0086] The signal processing circuit 150 can generate a composite signal by performing a first target operation 40 on the first signal and the second signal. The purpose of the first target operation 40 is to reduce the pickup of the target sound by the sound sensor module 120, thereby reducing the signal component from the speaker module 110 in the composite signal (i.e., reducing the signal component of the target sound in the composite signal). "Reducing the pickup of the target sound by the sound sensor module 120" means that, compared to the pickup of the target sound by the sound sensor module 120 when the first target operation is not performed, the pickup of the target sound by the sound sensor module 120 is reduced when the first target operation is performed.

[0087] The composite signal is a signal synthesized in frequency bands. Specifically, the composite signal may be a composite signal of a signal in a first frequency band and a signal in a second frequency band. The signal in the first frequency band comes from the sound pickup result signal of the sound sensor module 120 in a target sound pickup state, and the target sound pickup state corresponds to the zero-point sound pickup direction of the sound sensor module 120 pointing to the speaker module 110. The signal in the second frequency band may not be taken from the sound pickup result signal, but may be obtained by other means. In other words, the signals in some frequency bands in the composite signal come from the sound pickup result signal, while the signals in other frequency bands do not come from the sound pickup result signal.

[0088] It should be noted that the above-mentioned “zero-point pickup direction points to the speaker module 110” should be understood as the zero-point pickup direction generally pointing to the speaker module 110. For example, the zero-point pickup direction may point to the center point of the speaker module 110. For another example, the zero-point pickup direction may point to any point on the sound-emitting surface of the speaker module 110. For another example, the zero-point pickup direction may point to a preset area on the sound-emitting surface of the speaker module 110. For another example, assuming that the direction angle corresponding to the center point of the speaker module 110 is θ, the direction angle corresponding to the zero-point pickup direction may be located at within the range.

[0089] The sound pickup result signal refers to a single-channel signal obtained by merging / superimposing the sound pickup signal of the first sound sensor 120-1 and the sound pickup signal of the second sound sensor 120-2 when the sound sensor module 120 is pointing toward the speaker module 110 in the zero-point sound pickup direction. It should be understood that when the zero-point sound pickup direction of the sound sensor module 120 is pointing toward the speaker module 110 (i.e., in the target sound pickup state), the sound emitted by the speaker module 110 is not collected by the sound sensor module 120, or is less collected by the sound sensor module 120. Therefore, the sound pickup result signal does not contain a signal component from the speaker module 110, or contains a lesser signal component from the speaker module 110.

[0090] It can be understood that since the sound pickup result signal does not contain or contains less signal components from the speaker module 110, when the first frequency band in the integrated signal is taken from the sound pickup result signal, the integrated signal also does not contain or contains less signal components from the speaker module 110. Therefore, the integrated signal can reduce the signal components from the speaker module 110 compared with the first signal and the second signal.

[0091] In some embodiments, the signal of the second frequency band may come from the first signal or the second signal. Since the first signal and the second signal are both original signals collected by the sound sensor, compared to the sound pickup result signal, the first signal and the second signal can more accurately reflect certain characteristics of the real environment sound (such as background noise characteristics). Therefore, when the signal of the second frequency band comes from the first signal or the second signal, the components of the original signal are retained in the integrated signal, so that the integrated signal can more accurately reflect the characteristics of the real environment sound. It can be understood that in the case where the signal of the first frequency band in the integrated signal comes from the sound pickup result signal and the signal of the second frequency band comes from the first signal or the second signal, since the integrated signal retains the components of the original signal collected by the sound sensor and reduces the signal components from the speaker module 110, the integrated signal can reduce the signal components from the speaker module 110 while reflecting the real environment sound as accurately as possible, thereby improving the accuracy of the integrated signal.

[0092] For ease of description, the following description uses the example of the signal of the second frequency band coming from the first signal. It should be understood that when the signal of the second frequency band comes from the second signal, its implementation method is similar, and this application will not elaborate on this.

[0093] FIG6 shows a schematic diagram of a signal processing process provided according to an embodiment of the present application. As shown in FIG6 , in some embodiments, the first target operation 40 may include: a zero-point differential operation 41 and a signal synthesis operation 42. The zero-point differential operation 41 is configured to perform a zero-point differential on the first signal and the second signal to adjust the zero-point pickup direction of the sound sensor module 120 to point toward the speaker module 110, thereby obtaining the sound pickup result signal. In this application, "zero-point differential" refers to a differential operation capable of adjusting the zero-point pickup direction of the sound sensor module 120. The signal synthesis operation 42 is configured to synthesize the components of the first frequency band in the sound pickup result signal with the components of the second frequency band in the first signal to obtain the composite signal. By performing the above-mentioned zero-point differential operation 41 and signal synthesis operation 42 on the first signal and the second signal, the signal processing circuit 150 generates a composite signal in which the components of the first frequency band are derived from the sound pickup result signal, and the components of the second frequency band are derived from the first signal.

[0094] In some embodiments, the aforementioned zero-point differential operation 41 and / or signal synthesis operation 42 can be implemented by the processor 220 in the signal processing circuit 150, i.e., the processor 220 executes an instruction set and performs the zero-point differential operation 41 and / or signal synthesis operation 42 according to the instructions of the instruction set. In some embodiments, the signal processing circuit 150 may include a zero-point differential circuit, and the aforementioned zero-point differential operation 41 can be implemented by the zero-point differential circuit. In some embodiments, the signal processing circuit 150 may include a signal synthesis circuit, and the aforementioned signal synthesis operation 42 can be implemented by the signal synthesis circuit.

[0095] Several implementations of the zero-point differential operation 41 are described below with reference to FIG. 7 to FIG. 14 .

[0096] FIG7 shows a schematic diagram of a zero-point differential operation 41a according to an embodiment of the present application. As shown in FIG7 , the zero-point differential operation 41a may include a first delay operation 411 and a first differential operation 413. The first delay operation 411 is configured to delay the second signal to obtain a second delayed signal. The first differential operation 413 is configured to differentiate the first signal from the second delayed signal (e.g., subtract the first signal from the second delayed signal) to obtain a first differential signal.

[0097] The zero-point differential operation 41a shown in FIG7 can be applied to a scenario where the speaker module 110 is a far-field sound source. In this scenario, the distance between the speaker module 110 and the sound sensor module 120 is relatively far, and it can be assumed that the amplitude and direction of the sound signals collected by the two sound sensors are the same, and there is only a time difference (i.e., a phase difference) between the two. In this case, when the second sound sensor 120-2 is closer to the speaker module 110 than the first sound sensor 120-1, the sound emitted by the speaker module 110 is first collected by the second sound sensor 120-2 and then by the first sound sensor 120-1. In other words, the phase of the signal component from the speaker module 110 in the second signal precedes the phase of the signal component from the speaker module 110 in the first signal. Therefore, the signal processing circuit 150 can align the phase of the signal component from the speaker module 110 in the second delayed signal with the phase of the signal component from the speaker module 110 in the first signal by performing the above-mentioned first delay operation 411 (i.e., delaying the second signal to obtain a second delayed signal).

[0098] In some embodiments, when the signal processing circuit 150 performs the first delay operation 411 , it may determine the delay duration T corresponding to the second signal based on the following formula (1): T=d / c Formula (1)

[0099] Wherein, d is the distance between the first sound sensor 120 - 1 and the second sound sensor 120 - 2 , and c is the speed of sound.

[0100] After executing the above-mentioned first delay operation 411, since the phase of the signal component from the speaker module 110 in the second delayed signal is aligned with the phase of the signal component from the speaker module 110 in the first signal, the signal processing circuit 150 can cancel each other out by executing the first differential operation 413 (i.e., subtracting the first signal from the second delayed signal), so that the signal component from the speaker module 110 in the first signal and the signal component from the speaker module 110 in the second delayed signal can be made to exhibit a pickup zero-point characteristic in the direction of the speaker module 110.

[0101] FIG8 is a schematic diagram illustrating the sound pickup direction pattern corresponding to the zero-point differential operation 41a shown in FIG7 . As shown in FIG8 , the sound sensor module exhibits a sound pickup null characteristic in the 180-degree direction. When the speaker module 110 is located at or near the 180-degree direction, the sound sensor module 120 will not collect (or will rarely collect) the sound emitted by the speaker module 110. Thus, the zero-point differential operation 41a shown in FIG7 can adjust the sound pickup direction pattern of the sound sensor module 120 to a cardioid shape, with the null-point pickup direction pointing toward the speaker module 110. Therefore, the first differential signal obtained after the signal processing circuit 150 performs the zero-point differential operation 41a shown in FIG7 on the first signal and the second signal does not contain (or contains relatively few) signal components from the speaker module 110.

[0102] Furthermore, the inventors analyzed and experimented with the zero-point differential operation 41a in FIG7 and found that the zero-point differential operation 41a has an attenuation effect on components in some frequency bands. This is illustrated below with reference to FIG9 . FIG9 shows a schematic diagram of the frequency response curve of the first differential signal obtained by using the zero-point differential operation 41a shown in FIG7 . As shown in FIG9 , curve 1 represents the frequency response curve corresponding to the first differential signal in the 0-degree direction, and curve 2 represents the frequency response curve corresponding to the first differential signal in the 90-degree direction. It can be seen from curves 1 and 2 that the components of the first differential signal in some frequency bands (for example, below 1000 Hz) are attenuated, that is, the sensitivity of the sound sensor module 120 to the sound signals in these frequency bands is low.

[0103] Therefore, in some embodiments, referring again to FIG7 , the zero-point differential operation 41a may further include a gain compensation operation 416 . The gain compensation operation 416 is configured to perform gain compensation on at least a portion of the frequency band of the first differential signal to obtain a sound pickup result signal. For example, gain compensation can be performed on components in the first differential signal with attenuated frequency bands (e.g., below 1000 Hz), so that the sound sensor module 120 also has a high sensitivity to sound signals in these frequency bands. Referring again to FIG9 , Curve 3 shows the frequency response curve of the compensated first differential signal (i.e., the sound pickup result signal) at 0 degrees, and Curve 4 shows the frequency response curve of the compensated first differential signal (i.e., the sound pickup result signal) at 90 degrees. Thus, it can be seen that by performing the gain compensation operation 416 (i.e., performing gain compensation on at least a portion of the frequency band of the first differential signal), the signal processing circuit 150 enables the sound sensor module 120 to have a high and relatively flat sensitivity to sound signals across the entire frequency band.

[0104] As mentioned above, the zero-point differential operation 41a shown in FIG7 is applicable to the scenario where the speaker module 110 is a far-field sound source. During the research process of the inventor, it was found that when the distance between the speaker module 110 and the sound sensor module 120 is close (for example, less than or equal to 0.1m), the above-mentioned far-field assumption (that is, the amplitude and direction of the sound signals collected by the two sound sensors are the same, and there is only a time difference between the two) is no longer valid. Therefore, for the near-field sound source scenario, if the zero-point differential operation 41a shown in FIG7 is used to adjust the pickup direction, the adjusted pickup direction pattern no longer has a pickup zero point. To this end, the present application also provides another zero-point differential operation, which can be applied to both scenarios where the speaker module 110 is a far-field sound source and where the speaker module 110 is a near-field sound source. This will be explained below in conjunction with FIG10.

[0105] Figure 10 shows a schematic diagram of another zero-point differential operation 41b provided according to an embodiment of the present application. As shown in Figure 10, the zero-point differential operation 41b may include: a first delay operation 411, a second delay operation 412, a first differential operation 413, a second differential operation 414, and a third differential operation 415.

[0106] Among them, the first delay operation 411 is configured to delay the second signal to obtain a second delayed signal. In some embodiments, the delay length of the second signal can be determined based on the aforementioned formula (1), which is not described in detail here. The second delay operation 412 is configured to delay the first signal to obtain a first delayed signal. In some embodiments, the delay length of the first signal can be determined based on the aforementioned formula (1), which is not described in detail here. The first differential operation 413 is configured to perform a differential operation on the first signal and the second delayed signal (i.e., subtract the first signal from the second delayed signal) to obtain a first differential signal. The second differential operation 414 is configured to perform a differential operation on the second signal and the first delayed signal (i.e., subtract the second signal from the first delayed signal) to obtain a second differential signal. The third differential operation 415 is configured to perform a differential operation on the first differential signal and the second differential signal (i.e., subtract the first differential signal from the second differential signal) to obtain a third differential signal.

[0107] It should be understood that the principle of adjusting the sound pickup direction of the zero-point differential operation 41b shown in FIG10 is similar to that of FIG7 . The difference between the two is that the zero-point differential operation 41b in FIG10 requires two delay operations. Therefore, the scheme shown in FIG7 can be referred to as a single-delay zero-point differential scheme, while the scheme shown in FIG10 can be referred to as a double-delay zero-point differential scheme. FIG11 shows a schematic diagram of the sound pickup direction adjustment principle corresponding to the zero-point differential operation 41b shown in FIG10 . As shown in FIG11 , the signal processing circuit 150 can generate a cardioid pattern with a zero-point pickup direction pointing to 180 degrees (see pattern I in FIG11 ) by performing a first delay operation 411 and a first differential operation 413. This construction principle is the same as that of FIG7 and will not be further described here. Furthermore, the signal processing circuit 150 can generate a cardioid pattern with a zero-point pickup direction pointing to 0 degrees (see pattern II in FIG11 ) by performing a second delay operation 412 and a second differential operation 414. This construction principle is also similar to that of FIG7 and will not be further described here. It should be understood that the cardioid pattern with the null-point pickup direction pointing to 180 degrees (i.e., pattern I) in FIG11 corresponds to the first differential signal, and the cardioid pattern with the null-point pickup direction pointing to 0 degrees (i.e., pattern II) corresponds to the second differential signal. The signal processing circuit 150 can obtain an 8-shaped pattern with the null-point pickup direction pointing to 90 degrees and 270 degrees (see pattern III in FIG11 ) by performing the third differential operation 415 (equivalent to taking the difference between pattern I and pattern II).

[0108] As can be seen from the figure-eight sound pickup direction pattern (i.e., pattern III), the sound sensor module 120 exhibits a sound pickup null-point characteristic in the 90-degree and 270-degree directions. When the speaker module 110 is located in the 90-degree / 270-degree direction or near the 90-degree / 270-degree direction, the sound sensor module 120 does not pick up (or only picks up very little) sound emitted by the speaker module 110. Therefore, the double-delay-based zero-point differential operation 41b shown in FIG10 can adjust the sound pickup direction pattern of the sound sensor module 120 into a figure-eight shape, with the null-point sound pickup direction pointing toward the speaker module 110. Therefore, the third differential signal obtained by the signal processing circuit 150 through the zero-point differential operation 41b does not contain (or contains very little) signal components from the speaker module 110.

[0109] FIG12 illustrates schematic diagrams of the sound pickup direction patterns of the zero-point differential operation 41b shown in FIG10 in three different scenarios. As shown in FIG12 , Pattern I illustrates the sound pickup direction pattern obtained when the distance between the speaker module 110 and the sound sensor module 120 is 1 meter, Pattern II illustrates the sound pickup direction pattern obtained when the distance between the speaker module 110 and the sound sensor module 120 is 0.1 meter, and Pattern III illustrates the sound pickup direction pattern obtained when the distance between the speaker module 110 and the sound sensor module 120 is 0.06 meter. As can be seen from FIG12 , regardless of the distance between the speaker module 110 and the sound sensor module 120, the zero-point differential operation 41b shown in FIG10 can produce an 8-shaped sound pickup direction pattern. Therefore, the zero-point differential operation 41b shown in FIG10 can be applied to scenarios where the speaker module 110 is a near-field sound source, as well as scenarios where the speaker module 110 is a far-field sound source. Therefore, the zero-point differential operation 41 b shown in FIG. 10 can effectively reduce the signal component from the speaker module 110 in the third differential signal in all scenarios.

[0110] Furthermore, referring to FIG10 , the zero-point differential operation 41b may also include a gain compensation operation 416. Gain compensation operation 416 is configured to perform gain compensation on at least a portion of the frequency band of the third differential signal to produce a sound pickup result signal. It should be understood that the principles and functions of gain compensation operation 416 in FIG10 are the same as those of gain compensation operation 416 in FIG7 , and are not further described here.

[0111] When using the zero-point differential operation 41b shown in FIG10 , the sound pickup null direction of the sound sensor module 120 points at 90 degrees and 270 degrees. That is, each position on the perpendicular midline between the first sound sensor 120-1 and the second sound sensor 120-2 is located in the sound pickup null direction of the sound sensor module 120. Therefore, during product design, the speaker module 110 can be positioned on the perpendicular midline between the first sound sensor 120-1 and the second sound sensor 120-2, thereby eliminating the signal component from the speaker module 110 in the sound pickup signal of the sound sensor module 120. However, in actual applications, due to various factors such as product form, production tolerances, and wearing posture, the speaker module 110 is often not positioned strictly on the perpendicular midline between the first sound sensor 120-1 and the second sound sensor 120-2. As a result, the signal component from the speaker module 110 in the sound pickup signal of the sound sensor module 120 is reduced compared to the sound pickup signal without the zero-point differential operation, but the signal component from the speaker module 110 is still included. To this end, the present application also provides another zero-point differential operation that can adaptively adjust the zero-point sound pickup direction so that the signal component from the speaker module 110 in the sound pickup result signal is minimized. This will be explained below with reference to FIG13 .

[0112] Figure 13 shows a schematic diagram of another zero-point differential operation 41c provided according to an embodiment of the present application. As shown in Figure 13, based on the zero-point differential operation 41b shown in Figure 10, the zero-point differential operation 41c can also include a multiplication operation 417 and a target parameter generation operation 418. Among them, the target parameter generation operation 418 can generate a target parameter β with the goal of minimizing the signal component from the speaker module 110 in the third differential signal. The multiplication operation 417 is performed between the second differential operation 414 and the third differential operation 415, or between the first differential operation 413 and the third differential operation 415. For the convenience of description, in the following text of this application and in the accompanying drawings, "the multiplication operation 417 is performed between the second differential operation 414 and the third differential operation 415" is used as an example for illustration. Specifically, the multiplication operation 417 is configured to multiply the target parameter β with the second differential signal to obtain a multiplication result, so that the third differential operation 415 performs a differential operation on the first differential signal and the multiplication result to obtain the third differential signal.

[0113] It should be understood that the second differential signal corresponds to pattern II in Figure 11 (i.e., a cardioid pattern with the null-point pickup direction pointing to 0 degrees), and the second differential signal is adjusted by multiplying the target parameter β with the second differential signal. Furthermore, by performing a third differential operation 415 based on the first differential signal and the adjusted second differential signal, the null-point pickup direction can be adjusted from 90 degrees / 270 degrees to other angles. FIG14 shows a schematic diagram of adjusting the null-point pickup direction using the target parameter β provided in accordance with an embodiment of the present application. As shown in FIG14 , when the target parameter β is 0.99, the null points in the pickup direction pattern point to 90 degrees and 270 degrees. When the target parameter β is updated to 0.16, the null points in the pickup direction pattern point to 135 degrees and 225 degrees. Thus, the signal processing circuit 150 can adaptively adjust the null-point pickup direction by performing the multiplication operation 417 and the target parameter generation operation 418, so that the signal component from the speaker module 110 in the pickup result signal is minimized.

[0114] It should be noted that the various zero-point differential operations 41 involved in this application (such as the zero-point differential operation 41a shown in Figure 7, the zero-point differential operation 41b shown in Figure 10, and the zero-point differential operation 41c shown in Figure 13) can be performed in the time domain or in the frequency domain, and this application does not limit this.

[0115] In some embodiments, any one or more of the first delay operation 411, the second delay operation 412, the first differential operation 413, the second differential operation 414, the third differential operation 415, the gain compensation operation 416, the multiplication operation 417, and the target parameter generation operation 418 can be implemented by the processor 220 in the signal processing circuit 150, i.e., the processor 220 executes an instruction set and performs one or more of the above operations according to the instructions of the instruction set. In some embodiments, the signal processing circuit 150 may include a first delay circuit, and the first delay operation 411 may be implemented by the first delay circuit. In some embodiments, the signal processing circuit 150 may include a second delay circuit, and the second delay operation 412 may be implemented by the second delay circuit. In some embodiments, the signal processing circuit 150 may include a first differential circuit, and the first differential operation 413 may be implemented by the first differential circuit. In some embodiments, the signal processing circuit 150 may include a second differential circuit, and the second differential operation 414 may be implemented by the second differential circuit. In some embodiments, the signal processing circuit 150 may include a third differential circuit, and the third differential operation 415 described above may be implemented by the third differential circuit. In some embodiments, the signal processing circuit 150 may include a gain compensation circuit, and the gain compensation operation 416 described above may be implemented by the gain compensation circuit. In some embodiments, the signal processing circuit 150 may include a multiplication circuit, and the multiplication operation 417 described above may be implemented by the multiplication circuit. In some embodiments, the signal processing circuit 150 may include a target parameter generation circuit, and the target parameter generation operation 418 described above may be implemented by the target parameter generation circuit.

[0116] As can be seen from the foregoing description, the zero-point differential operation 41 (e.g., the zero-point differential operation 41a shown in FIG7 , the zero-point differential operation 41b shown in FIG10 , and the zero-point differential operation 41c shown in FIG13 ) all include a gain compensation operation 416. This gain compensation operation 416 inevitably causes the noise floor components in certain frequency bands of the sound pickup signal to be boosted. When the noise floor is boosted to a certain sound intensity, it can affect the user's auditory experience. Therefore, in some embodiments, the second frequency band can include the frequency band corresponding to the current environment's noise floor, and the first frequency band includes frequency bands other than the second frequency band. In this way, when the component of the second frequency band (i.e., the frequency band corresponding to the noise floor) in the integrated signal comes from the first signal, and the component of the first frequency band (the frequency bands other than the frequency band corresponding to the noise floor) comes from the sound pickup signal, since the first signal does not undergo the zero-point differential operation 41, it can truly reflect the noise floor characteristics of the current environment, thereby avoiding the problem of boosted noise floor components.

[0117] In some embodiments, the signal processing method P100 may further include: determining the background noise characteristics of the current environment based on the first signal and the second signal, and then determining the frequency range corresponding to the first frequency band and the frequency range corresponding to the second frequency band based on the background noise characteristics. For example, the background noise characteristics may include the frequency band corresponding to the background noise, or include other characteristics that can indicate the frequency band corresponding to the background noise. In this way, the signal processing circuit 150 can identify the frequency band corresponding to the background noise based on the background noise characteristics, and then determine the frequency band corresponding to the background noise as the second frequency band, and determine the remaining frequency bands as the first frequency band. In the above scheme, the signal processing circuit 150 can adaptively adjust the frequency range of the first frequency band and the second frequency band based on the background noise characteristics of the current environment, so that in any scenario, the signal component from the speaker module 110 in the integrated signal can be reduced without raising the background noise.

[0118] In practical applications, considering that the background noise in the environment is usually low-frequency noise, in some embodiments, the frequency in the first frequency band may be higher than the frequency in the second frequency band. For example, the first frequency band may be a high frequency band, and the second frequency band may be a low frequency band. For another example, the first frequency band may be a medium-high frequency band, and the second frequency band may be a low frequency band. For another example, the first frequency band may be a high frequency band, and the second frequency band may be a medium-low frequency band. For another example, the first frequency band may be a medium frequency band, and the second frequency band may be a low frequency band, and so on. In other words, the frequency band corresponding to the lower frequency range in the audio pickup band of the sound sensor module 120 is used as the second frequency band, and the component of the second frequency band in the integrated signal is taken from the first signal, thereby avoiding the problem of the background noise component being raised.

[0119] The low frequency band refers to a frequency band generally below 1 kHz, the mid-frequency band refers to a frequency band generally between 1 kHz and 4 kHz, the high frequency band refers to a frequency band above 4 kHz, the mid-low frequency band refers to a frequency band generally below 4 kHz, and the mid-high frequency band refers to a frequency band generally above 1 kHz. Those skilled in the art will appreciate that the distinction between the above frequency bands is merely an example of a rough range. The definition of the above frequency bands may vary depending on different industries, different application scenarios, and different classification standards. For example, in some application scenarios, the low frequency band may refer to a frequency band generally between 20 Hz and 150 Hz, the mid-frequency band may refer to a frequency band generally between 150 Hz and 5 kHz, the high frequency band may refer to a frequency band generally between 5 kHz and 20 kHz, the mid-low frequency band may refer to a frequency band generally between 150 Hz and 500 Hz, and the mid-high frequency band may refer to a frequency band generally between 500 Hz and 5 kHz. For example, in some other application scenarios, the low frequency band refers to the frequency band of roughly 20Hz to 80Hz, the mid-low frequency band may refer to the frequency band of roughly 80Hz-160Hz, the mid-frequency band may refer to the frequency band of roughly 160Hz to 1280Hz, the mid-high frequency band may refer to the frequency band of roughly 1280Hz-2560Hz, and the high frequency band may refer to the frequency band of roughly 2560Hz to 20KHz.

[0120] FIG15 is a schematic diagram of another signal processing process provided according to an embodiment of the present application, which is used to provide a detailed description of the signal synthesis operation 42 in FIG6. It should be understood that the zero-point differential operation 41 in FIG15 can adopt the zero-point differential operation 41a shown in FIG7, the zero-point differential operation 41b shown in FIG10, or the zero-point differential operation 41c shown in FIG13. For details, please refer to the relevant descriptions above and will not be repeated here.

[0121] As shown in Figure 15, the signal synthesis operation 42 may include: a first filtering operation 421, a second filtering operation 422, and a synthesis operation 424. The first filtering operation 421 is configured to perform a first filtering on the first signal to obtain a component in the second frequency band of the first signal. For example, when the second frequency band is a low frequency band, the first filtering operation 421 can be implemented using a low-pass filter. The second filtering operation 422 is configured to perform a second filtering on the sound pickup result signal to obtain a component in the first frequency band of the sound pickup result signal. For example, when the first frequency band is a high frequency band, the second filtering operation 422 can be implemented using a high-pass filter. The synthesis operation 424 is configured to synthesize the component in the second frequency band obtained by the first filtering operation 421 with the component in the first frequency band obtained by the second filtering operation 422 to obtain the composite signal. In some embodiments, the synthesis operation 424 can be implemented using an adder.

[0122] Thus, it can be seen that the signal processing circuit 150 extracts the components of the second frequency band from the first signal by performing the first filtering operation 421, and extracts the components of the first frequency band from the sound pickup result signal by performing the second filtering operation 422, and then synthesizes the components of the first frequency band and the components of the second frequency band to obtain a composite signal by performing the synthesis operation 424, so that the first frequency band in the composite signal comes from the sound pickup result signal, and the second frequency band comes from the first signal.

[0123] In some embodiments, the first filtering and the second filtering may be complementary filtering, or in other words, the sum of the transfer function of the first filtering and the transfer function of the second filtering is equal to 1. For example, assuming that the transfer function corresponding to the first filtering is expressed as the following formula (2), the transfer function corresponding to the second filtering can be expressed as the following formula (3).

[0124] It can be seen from formula (2) and formula (3) that the denominator expressions corresponding to the transfer functions of the two filtering operations are the same, both A(z), and the numerator expressions corresponding to the transfer functions of the two filtering operations are B(z) and A(z)-B(z), respectively. This design makes the sum of the transfer functions of the two filtering operations equal to 1, so that the two filtering operations exhibit an all-pass characteristic when working in combination. For ease of understanding, Figure 16 shows a schematic diagram of the frequency response curves corresponding to a set of complementary filtering operations provided according to an embodiment of the present application. As shown in Figure 16, curve 1 can correspond to the frequency response curve of the first filtering operation 421 in Figure 15, showing a low-pass characteristic, and curve 2 can correspond to the frequency response curve of the second filtering operation 422 in Figure 15, showing a high-pass characteristic. The above two filtering operations will exhibit an all-pass characteristic when used in combination.

[0125] It should be noted that the complementary filtering operations exemplified in the above formulas (2) and (3) are only used as a possible example. Those skilled in the art will appreciate that any filter group capable of frequency division and synthesis is feasible. For example, in some embodiments, the above-mentioned first filtering and second filtering can also be implemented by a first filter and a second filter having the same cutoff frequency. The cutoff frequency of the first filter is w1, and the cutoff frequency of the second filter is w2. The cutoff frequencies of the two filters are the same: w1 = w2, and the amplitude responses of the two filters at the cutoff frequency satisfy

[0126] FIG17 is a schematic diagram showing the background noise components corresponding to the sound pickup result signal and the integrated signal in FIG15 . As shown in FIG17 , Curve 1 shows the background noise component in the first signal collected by the first sound sensor 120-1, and Curve 2 shows the background noise component in the second signal collected by the second sound sensor 120-1. It can be understood that since the first signal and the second signal are the original signals collected by the sound sensors, Curve 1 and Curve 2 can more accurately reflect the background noise conditions of the current environment. Curve 3 shows the background noise component in the sound pickup result signal obtained by using the zero-point differential operation 41a shown in FIG7 (corresponding to the cardioid pickup direction pattern). Curve 4 shows the background noise component in the sound pickup result signal obtained by using the zero-point differential scheme 41b shown in FIG10 (corresponding to the figure-8 pickup direction pattern). It can be seen from Curve 3 and Curve 4 that both the zero-point differential operation 41a (single-delay zero-point differential solution) shown in FIG7 and the zero-point differential operation 41b (double-delay zero-point differential solution) shown in FIG10 will significantly increase the noise floor in the low-frequency band (i.e., the second frequency band), thereby causing greater noise interference to the user.

[0127] Continuing with FIG. 17 , Curve 5 shows the noise floor component of the composite signal obtained by performing frequency-band synthesis on the sound pickup signal obtained by the zero-point differential operation 41a (corresponding to the cardioid pickup direction pattern) shown in FIG. 7 and the first signal. Curve 6 shows the noise floor component of the composite signal obtained by performing frequency-band synthesis on the sound pickup signal obtained by the zero-point differential operation 41b (corresponding to the figure-eight pickup direction pattern) shown in FIG. 10 and the first signal. As can be seen from Curves 5 and 6, the signal processing circuit 150, by performing frequency-band synthesis on the sound pickup signal and the first signal to obtain the composite signal, can make the noise floor component in the composite signal closer to the noise floor component in the originally collected first signal, thereby avoiding the noise floor increase problem caused by the zero-point differential operation 41 and thus preventing noise interference to the user.

[0128] In some embodiments, in a scenario where the speaker module 110 includes multiple speakers, when different speakers correspond to different sound frequency bands, different zero-point differential schemes can be used for different frequency bands, so that the zero-point pickup direction of the sound sensor module 120 in different frequency bands is directed to the speaker of the corresponding frequency band, thereby minimizing the signal component from each speaker in the integrated signal. Take the example of the speaker module 110 including the first speaker 110-1 and the second speaker 110-2 as an example. Assume that the sound frequency band of the first speaker 110-1 includes the first sub-band, and the sound frequency band of the second speaker 110-2 includes the second sub-band. In this case, the signal of the first sub-band in the integrated signal comes from the first sound pickup result signal obtained by the sound sensor module 120 in the first sound pickup state, and the first sound pickup state corresponds to the zero-point pickup direction of the sound sensor module 120 pointing to the first speaker 110-1. The signal of the second sub-frequency band in the integrated signal comes from the second sound pickup result signal obtained by the sound sensor module 120 in the second sound pickup state. The second sound pickup state corresponds to the sound sensor module 120 having a null-point pickup direction pointing toward the second speaker 120-2. The first and second sound pickup states correspond to different sound pickup direction patterns. That is, when adjusting the null-point pickup direction, the signal processing circuit 150 can employ different null-point differential operations 41 for the first and second sub-frequency bands, respectively, so that the sound pickup direction pattern corresponding to the first sub-frequency band differs from the sound pickup direction pattern corresponding to the second sub-frequency band. In practical applications, considering that each null-point differential scheme may have different null-point attenuation effects (i.e., sound intensity attenuation in the null-point pickup direction) in different frequency bands, it is possible to refer to the null-point attenuation performance of each null-point differential operation 41 in different frequency bands and employ the null-point differential operation 41 with the better null-point attenuation effect for the first and second sub-frequency bands, thereby improving the null-point attenuation effect across the entire frequency band.

[0129] FIG18 is a schematic diagram illustrating the zero-point attenuation effects corresponding to different zero-point differential operations provided in accordance with an embodiment of the present application. As shown in FIG18 , Curve 1 illustrates a frequency response diagram of the first signal collected by the first sound sensor 120-1. Taking the first sound sensor 120-1 as an omnidirectional sound sensor as an example, Curve 1 represents the frequency response of the first signal in any pickup direction. Curve 2 illustrates a frequency response diagram of the pickup result signal obtained using the zero-point differential operation 41a shown in FIG7 (corresponding to a cardioid pickup direction pattern) in the zero-point pickup direction. Curve 3 illustrates a frequency response diagram of the pickup result signal obtained using the zero-point differential operation 41b shown in FIG10 (corresponding to an 8-shaped pickup direction pattern) in the zero-point pickup direction. As shown in Figure 18 , within the 1kHz-4kHz frequency band, the attenuation of Curve 2 relative to Curve 1 is greater than the attenuation of Curve 3 relative to Curve 1. In other words, the sound intensity attenuation in the null-point pickup direction of the null-point differential operation 41a (corresponding to a cardioid pickup pattern) shown in Figure 7 is greater than the sound intensity attenuation in the null-point pickup direction of the null-point differential operation 41b (corresponding to a figure-eight pickup pattern) shown in Figure 10 . Therefore, within the 1kHz-4kHz frequency band, the null-point attenuation effect of the null-point differential operation 41a (corresponding to a cardioid pickup pattern) shown in Figure 7 is superior to that of the null-point differential operation 41b (corresponding to a figure-eight pickup pattern) shown in Figure 10 .

[0130] Continuing with FIG18 , within the 4kHz-8kHz frequency band, the attenuation of Curve 3 relative to Curve 1 is more pronounced than the attenuation of Curve 2 relative to Curve 1. In other words, the sound intensity attenuation in the null-point pickup direction of the null-point differential operation 41b (corresponding to the figure-eight pickup direction pattern) shown in FIG10 is greater than the sound intensity attenuation in the null-point pickup direction of the null-point differential operation 41a (corresponding to the cardioid pickup direction pattern) shown in FIG7 . Therefore, within the 4kHz-8kHz frequency band, the null-point attenuation effect of the null-point differential operation 41b (corresponding to the figure-eight pickup direction pattern) shown in FIG10 is superior to the null-point differential operation 41a (corresponding to the cardioid pickup direction pattern) shown in FIG7 .

[0131] Based on the comparison of the zero-point attenuation effects shown in FIG18 , in some embodiments, the first frequency band can be divided into a first sub-band and a second sub-band, where the frequencies in the first sub-band are lower than the frequencies in the second sub-band. For example, the first sub-band can be a mid-frequency band (e.g., 1kHz-4kHz as shown in FIG18 ), and the second sub-band can be a high-frequency band (e.g., 4kHz-8kHz as shown in FIG18 ). In this case, the zero-point differential operation 41a shown in FIG7 can be used for the first sub-band, in which case the sound sensor module 120 correspondingly has a cardioid pickup direction pattern. The zero-point differential operation 41b shown in FIG10 or the zero-point differential operation 41c shown in FIG13 can be used for the second sub-band, in which case the sound sensor module 120 correspondingly has an 8-shaped pickup direction pattern. As a result, both the first and second sub-bands have greater attenuation in the null-point pickup direction, thereby improving the overall zero-point attenuation effect.

[0132] FIG19 shows a schematic diagram of another signal processing process provided according to an embodiment of the present application, which is applicable to the scenario where different zero-point differential operations are respectively used for the first sub-band and the second sub-band. As shown in FIG19 , it is assumed that the speaker module 110 includes a first speaker 110-1 and a second speaker 110-2, the sound frequency band of the first speaker 110-1 includes the first sub-band, and the sound frequency band of the second speaker 110-2 includes the second sub-band. The zero-point differential operation 41 may include a first zero-point differential operation 41-1 and a second zero-point differential operation 41-2. Among them, the first zero-point differential operation 41-1 can adopt the zero-point differential operation 41a shown in FIG7 (i.e., a single-delay zero-point differential scheme, corresponding to a cardioid pickup direction pattern), and the second zero-point differential operation 41-2 can adopt the zero-point differential operation 41b shown in FIG10 or the zero-point differential operation 41c shown in FIG13 (i.e., a double-delay zero-point differential scheme, corresponding to an 8-shaped pickup direction pattern). After obtaining the first and second signals, the signal processing circuit 150 may perform a first zero-point differential operation 41-1 on the first and second signals to direct the null-point sound pickup direction of the sound sensor module 120 toward the first speaker 110-1, thereby obtaining a first sound pickup result signal. Furthermore, the signal processing circuit 150 may perform a second zero-point differential operation 41-2 on the first and second signals to direct the null-point sound pickup direction of the sound sensor module 120 toward the second speaker 110-2, thereby obtaining a second sound pickup result signal. The first sound pickup result signal is a sound pickup result signal obtained based on a cardioid sound pickup direction pattern, and the second sound pickup result signal is a sound pickup result signal obtained based on a figure-8 sound pickup direction pattern.

[0133] Continuing with FIG. 19 , the signal synthesis operation 42 may include a first filtering operation 421, a second filtering operation 422, a third filtering operation 423, and a synthesis operation 424. The first filtering operation 421 is configured to perform a first filtering operation on the first signal to obtain a component in a second frequency band within the first signal. For example, the second frequency band may be a low frequency band (e.g., less than 1000 Hz), in which case the first filtering operation 421 may be implemented using a low-pass filter. The second filtering operation 422 is configured to perform a second filtering operation on the first sound pickup signal to obtain a component in a first sub-frequency band within the first sound pickup signal. For example, the first sub-frequency band may be a high frequency band (e.g., greater than 4 kHz), in which case the second filtering operation 422 may be implemented using a high-pass filter. The third filtering operation 423 is configured to perform a third filtering operation on the second sound pickup signal to obtain a component in a second sub-frequency band within the second sound pickup signal. For example, the second sub-frequency band may be a mid-frequency band (e.g., 1 kHz-4 kHz), in which case the second filtering operation 422 may be implemented using a band-pass filter. The synthesis operation 424 is configured to synthesize the components of the second frequency band obtained by the first filtering operation 421, the components of the first sub-frequency band obtained by the second filtering operation 422, and the components of the second sub-frequency band obtained by the third filtering operation 423 to obtain a composite signal. In some embodiments, the synthesis operation 424 can be implemented using an adder. Continuing with FIG19 , after obtaining the composite signal, the signal processing circuit 150 can perform a gain amplification operation 51 on the composite signal to obtain a gain-amplified signal. Furthermore, based on the sound frequency bands of the first speaker 110-1 and the second speaker 110-2, the gain-amplified signal can be subjected to a frequency division operation 52, and the frequency-divided signals can be sent to the first speaker 110-1 and the second speaker 110-2, respectively. For example, assuming that the sound frequency band of the first speaker 110-1 includes the second frequency band and the first sub-band, and the sound frequency band of the second speaker 110-2 includes the second sub-band, in this case, the frequency division operation 52 can extract the first target signal corresponding to the second frequency band and the first sub-band from the gain-amplified signal, and extract the second target signal corresponding to the second sub-band. Furthermore, the signal processing circuit 150 can send the first target signal to the first speaker 110-1 and the second target signal to the second speaker 110-2. The frequency division operation 52 can be implemented using a filter or other feasible methods, which are not limited in this application.

[0134] It should be noted that, in a scenario where the speaker module 110 includes a first speaker 110-1 and a second speaker 110-2, the signal processing process shown in FIG19 is only one possible example. In practical applications, after obtaining the first sound pickup result signal and the second sound pickup result signal, the signal processing circuit 150 may also employ other signal processing methods. For example, in some embodiments, the signal processing circuit 150 may perform a first filtering operation 421 on the first signal to obtain a component in the second frequency band of the first signal, perform a second filtering operation 422 on the first sound pickup result signal to obtain a component in the first sub-frequency band of the first sound pickup result signal, and perform a third filtering operation 423 on the second sound pickup result signal to obtain a component in the second sub-frequency band of the second sound pickup result signal. Furthermore, the signal processing circuit synthesizes the components in the first sub-frequency band and the components in the second frequency band to obtain a first composite signal, performs a first gain amplification operation on the first composite signal, and then transmits it to the first speaker 110-1. The signal processing circuit performs a second gain amplification operation on the components in the second sub-frequency band, and then transmits it to the second speaker 110-2.

[0135] Similar to FIG15 , the first filtering operation 421, the second filtering operation 422, and the third filtering operation 423 in FIG19 can be referred to as a set of complementary filters. In other words, the sum of the transfer function corresponding to the first filtering operation 421, the transfer function corresponding to the second filtering operation 422, and the transfer function corresponding to the third filtering operation 423 is equal to 1. For the explanation and effects of complementary filters, please refer to the relevant description above and will not be repeated here.

[0136] In some embodiments, any one or more of the first filtering operation 421, the second filtering operation 422, the third filtering operation 423, and the synthesis operation 424 described above can be implemented by the processor 220 in the signal processing circuit 150, i.e., the processor 220 executes an instruction set and performs one or more of the above operations according to the instructions of the instruction set. In some embodiments, the signal processing circuit 150 may include a first filtering circuit, and the first filtering operation 421 described above may be implemented by the first filtering circuit. In some embodiments, the signal processing circuit 150 may include a second filtering circuit, and the second filtering operation 422 described above may be implemented by the second filtering circuit. In some embodiments, the signal processing circuit 150 may include a third filtering circuit, and the third filtering operation 423 described above may be implemented by the third filtering circuit. In some embodiments, the signal processing circuit 150 may include a synthesis circuit, and the synthesis operation 424 described above may be implemented by the synthesis circuit.

[0137] In the aforementioned embodiment, when generating the composite signal, the signal processing circuit 150 first performs a zero-point differential operation 41 on the first signal and the second signal to obtain a sound pickup result signal, and then filters the sound pickup result signal and the first signal to extract the components of the first frequency band from the sound pickup result signal and the components of the second frequency band from the first signal, and then synthesizes the components of the two frequency bands to obtain the composite signal. In some embodiments, the signal processing circuit 150 may also interchange the order of the zero-point differential operation 41 and the filtering. Specifically, taking the two-frequency band synthesis scheme shown in FIG15 as an example, the signal processing circuit 150 may generate the composite signal in the following manner: performing a first filtering on the first signal to obtain a first sub-signal corresponding to the first frequency band in the first signal; performing a second filtering on the first signal to obtain a second sub-signal corresponding to the second frequency band in the first signal; and performing the first filtering on the second signal to obtain a third sub-signal corresponding to the first frequency band in the second signal; then performing a zero-point differential operation on the first sub-signal and the third sub-signal to obtain a target sub-signal; and then synthesizing the second sub-signal with the target sub-signal to obtain the composite signal. It should be understood that the specific implementation methods of the above-mentioned zero-point difference operation, filtering operation and synthesis operation are the same as or similar to those described above, and will not be repeated here.

[0138] S340: Perform a second target operation on the integrated signal.

[0139] After obtaining the integrated signal, the signal processing circuit 150 can perform a second target operation 50 on the integrated signal based on the requirements of the application scenario. In some embodiments, referring to Figures 15 and 19, the signal processing circuit 150 can also be connected to the speaker module 110. In this case, the second target operation 50 may include a gain amplification operation 51. After obtaining the integrated signal, the signal processing circuit 150 can perform gain amplification on the integrated signal by performing the gain amplification operation 51, and then send the gain-amplified signal to the speaker module 110 so that the speaker module 110 emits sound. The above scheme can be applied to the howling suppression scenario shown in Figure 1. It should be understood that since the signal component from the speaker module 110 is reduced in the integrated signal (or the signal strength from the speaker module 110 is reduced), the conditions for the sound emitted by the speaker module 110 to generate howling in the closed loop shown in Figure 1 are destroyed, thereby achieving the effect of suppressing howling. In some embodiments, the above-mentioned gain amplification operation 51 can be implemented by the processor 220 in the signal processing circuit 150, that is, the processor 220 executes the instruction set and performs the above-mentioned gain amplification operation 51 according to the instructions of the instruction set. In some embodiments, the signal processing circuit 150 may include a gain amplification circuit, and the above-mentioned gain amplification operation 51 may be implemented by the gain amplification circuit.

[0140] In some embodiments, the speaker module 110, the sound sensor module 120 and the signal processing circuit 150 are integrated in the first acoustic device, and the first acoustic device is communicatively connected to the second acoustic device. In this case, the second target operation 50 may include: sending the integrated signal to the second acoustic device to reduce the echo of the second acoustic device. The above scheme can be applied to the echo cancellation scenario shown in Figure 2. For example, the first acoustic device can be a local device, and the second acoustic device can be a remote device. Since the signal component from the speaker is reduced in the integrated signal (or the signal strength from the speaker is reduced), it is equivalent to reducing the sound from the second acoustic device. Therefore, when the second acoustic device receives the integrated signal and plays it, the user on the second acoustic device side (i.e., the remote user) will not hear or hear less of the echo, thereby achieving the effect of echo cancellation.

[0141] In summary, the present application provides a signal processing method and acoustic system. The method includes: obtaining a first signal and a second signal, wherein the first signal is obtained by a first sound sensor in a sound sensor module when operating to collect ambient sound, and the second signal is obtained by a second sound sensor in the sound sensor module when operating, wherein the ambient sound includes at least a target sound output by a speaker when operating; performing a first target operation on the first and second signals to generate a composite signal; and performing a second target operation on the composite signal. The composite signal is a composite signal of a signal in a first frequency band and a signal in a second frequency band, wherein the signal in the first frequency band is derived from a sound pickup result signal of the sound sensor module in a target sound pickup state, wherein the target sound pickup state corresponds to a sound pickup direction of the sound sensor module pointing toward the speaker. In the above scheme, since the sound pickup result signal is obtained when the sound sensor module is collecting sound with the zero-point pickup direction pointing toward the speaker, the sound pickup result signal does not contain or contains relatively little signal components from the speaker. Furthermore, since the first frequency band in the composite signal is derived from the sound pickup result signal, the signal components from the speaker in the composite signal are reduced, thereby achieving the effect of suppressing howling or eliminating echoes.

[0142] On the other hand, the present application provides a non-transitory storage medium that stores at least one set of executable instructions for performing signal processing. When the executable instructions are executed by a processor, the executable instructions instruct the processor to implement the steps of the signal processing method P100 described in this application. In some possible implementations, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product is run on the acoustic system 003, the program code is used to cause the acoustic system 003 to perform the steps of the signal processing method P100 described in this application. The program product for implementing the above method can use a portable compact disc read-only memory (CD-ROM) to include program code and can be run on the acoustic system 003. However, the program product of the present application is not limited to this. In the present application, a readable storage medium can be any tangible medium that contains or stores a program that can be used by or in combination with an instruction execution system. The program product can use any combination of one or more readable media. The readable medium can be a readable signal medium or a readable storage medium. The readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media include: an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing. The computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such a propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The computer-readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing. The program code for performing the operations of the present application may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, and conventional procedural programming languages ​​such as C or similar programming languages. The program code may be executed entirely on the acoustic system 003, partially on the acoustic system 003, as a stand-alone software package, partially on the acoustic system 003 and partially on a remote computing device, or entirely on a remote computing device.

[0143] The foregoing description describes specific embodiments of the present application. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims may be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0144] In summary, after reading this detailed disclosure, those skilled in the art will appreciate that the foregoing detailed disclosure may be presented by way of example only and may not be limiting. Although not expressly stated herein, those skilled in the art will understand that the present application requires various reasonable changes, improvements, and modifications to the embodiments. Such changes, improvements, and modifications are intended to be proposed by the present application and are within the spirit and scope of the exemplary embodiments of the present application.

[0145] In addition, certain terms in this application have been used to describe embodiments of the application. For example, "one embodiment," "an embodiment," and / or "some embodiments" mean that a particular feature, structure, or characteristic described in conjunction with that embodiment may be included in at least one embodiment of the application. Therefore, it is emphasized and should be understood that two or more references to "an embodiment," "one embodiment," or "an alternative embodiment" in various sections of this application do not necessarily refer to the same embodiment. Furthermore, particular features, structures, or characteristics may be appropriately combined in one or more embodiments of the application.

[0146] It should be understood that in the foregoing description of the embodiments of this application, in order to facilitate understanding of a feature and to simplify this application, this application combines various features into a single embodiment, figure, or description thereof. However, this does not mean that the combination of these features is required. When reading this application, it is entirely possible for a person skilled in the art to mark out some of the devices and understand them as separate embodiments. In other words, the embodiments of this application can also be understood as the integration of multiple secondary embodiments. This also applies when the content of each secondary embodiment is less than all the features of a single aforementioned disclosed embodiment.

[0147] Each patent, patent application, patent application publication, and other materials, such as articles, books, specifications, publications, documents, articles, and the like, cited herein, except any historical prosecution documents to which it relates, any equivalent that may be inconsistent or conflicting with this document, or any equivalent historical prosecution documents that may have a limiting effect on the broadest scope of the claims, is hereby incorporated by reference for all purposes now or hereafter connected with this document. In addition, in the event of any inconsistency or conflict between the descriptions, definitions, and / or use of terms associated with any incorporated material and the terminology, descriptions, definitions, and / or use associated with this document, the terminology in this document shall control.

[0148] Finally, it should be understood that the embodiments of the application disclosed herein are illustrations of the principles of the embodiments of the present application. Other modified embodiments are also within the scope of the present application. Therefore, the embodiments disclosed in the present application are merely examples and not limitations. Those skilled in the art can adopt alternative configurations based on the embodiments in the present application to implement the applications in the present application. Therefore, the embodiments of the present application are not limited to the embodiments precisely described in the application.

Claims

1. A signal processing method, characterized in that: include: Obtaining a first signal, wherein the first signal is obtained by collecting environmental sound when the first sound sensor in the sound sensor module is working, wherein the environmental sound at least includes a target sound, and the target sound is the sound output when the speaker module is working; Obtaining a second signal, where the second signal is obtained by collecting the environmental sound when the second sound sensor in the sound sensor module is working; Performing a first target operation on the first signal and the second signal to generate a composite signal, wherein the composite signal is a composite signal of a signal in a first frequency band and a signal in a second frequency band, the signal in the first frequency band comes from a sound pickup result signal of the sound sensor module in a target sound pickup state, and the target sound pickup state corresponds to a zero-point sound pickup direction of the sound sensor module pointing to the speaker module; as well as A second target operation is performed on the composite signal.

2. The method according to claim 1, characterized in that The signal in the second frequency band comes from the first signal.

3. The method according to claim 2, characterized in that The first target operation includes: A zero-point differential operation is performed to perform a zero-point differential on the first signal and the second signal, so as to adjust the zero-point sound pickup direction of the sound sensor module to point to the speaker module, thereby obtaining the sound pickup result signal; and The signal synthesis operation synthesizes the component of the first frequency band in the sound pickup result signal and the component of the second frequency band in the first signal to obtain the integrated signal.

4. The method according to claim 3, characterized in that The zero point differential operation includes: A first delay operation is performed to delay the second signal to obtain a second delayed signal; A first differential operation is performed to differentiate the first signal from the second delayed signal to obtain a first differential signal; and A gain compensation operation is performed to perform gain compensation on signals of at least a partial frequency band in the first differential signal to obtain the sound pickup result signal.

5. The method according to claim 3, characterized in that: The zero point differential operation includes: A first delay operation is performed to delay the second signal to obtain a second delayed signal; A second delay operation is performed to delay the first signal to obtain a first delayed signal; A first differential operation is performed to perform a differential operation on the first signal and the second delayed signal to obtain a first differential signal; A second differential operation is performed to perform a differential operation on the second signal and the first delayed signal to obtain a second differential signal; a third differential operation, performing a differential operation on the first differential signal and the second differential signal to obtain a third differential signal; and The gain compensation operation performs gain compensation on signals of at least a partial frequency band in the third differential signal to obtain the sound pickup result signal.

6. The method according to claim 5, characterized in that The zero point differential operation further includes: a target parameter generating operation of generating a target parameter with the goal of minimizing a signal component corresponding to the target sound in the third differential signal; and A multiplication operation is performed between the second differential operation and the third differential operation, multiplying the target parameter with the second differential signal to obtain a multiplication result, so that the third differential operation performs a differential operation on the first differential signal and the multiplication result to obtain the third differential signal.

7. The method according to claim 3, characterized in that The signal synthesis operation includes: Performing a first filtering on the first signal to obtain a component of the second frequency band in the first signal; Performing a second filtering on the sound pickup result signal to obtain a component of the first frequency band in the sound pickup result signal; and The component of the first frequency band is synthesized with the component of the second frequency band to obtain the integrated signal.

8. The method according to claim 7, characterized in that The first filtering and the second filtering are complementary filtering.

9. The method according to claim 2, characterized in that: The first target operation includes: Performing a first filtering on the first signal to obtain a first sub-signal corresponding to the first frequency band in the first signal, and performing a second filtering on the first signal to obtain a second sub-signal corresponding to the second frequency band in the first signal; Performing the first filtering on the second signal to obtain a third sub-signal corresponding to the first frequency band in the second signal; Performing zero-point difference on the first sub-signal and the third sub-signal to obtain a target sub-signal; and The second sub-signal is synthesized with the target sub-signal to obtain the integrated signal.

10. The method according to claim 2, characterized in that A phase of a signal component in the second signal corresponding to the target sound is earlier than or equal to a phase of a signal component in the first signal corresponding to the target sound.

11. The method according to claim 2, characterized in that Frequencies in the first frequency band are higher than frequencies in the second frequency band.

12. The method according to claim 2, characterized in that: The second frequency band includes a frequency band corresponding to the background noise of the current environment, and the first frequency band includes a frequency band other than the second frequency band.

13. The method according to claim 12, characterized in that The method further comprises: Determining a background noise feature corresponding to a current environment based on the first signal and the second signal; and Based on the background noise characteristics, a frequency range corresponding to the first frequency band and a frequency range corresponding to the second frequency band are determined.

14. The method according to claim 1, characterized in that The speaker module includes a first speaker and a second speaker, the first frequency band includes a first sub-frequency band and a second sub-frequency band, the sound frequency band of the first speaker includes the first sub-frequency band, and the sound frequency band of the second speaker includes the second sub-frequency band; The signal of the first sub-frequency band in the integrated signal comes from a first sound pickup result signal obtained by the sound sensor module in a first sound pickup state, and the first sound pickup state corresponds to the zero-point sound pickup direction of the sound sensor module pointing to the first speaker; as well as The signal of the second sub-frequency band in the comprehensive signal comes from a second sound pickup result signal obtained by the sound sensor module in a second sound pickup state, and the second sound pickup state corresponds to the zero-point sound pickup direction of the sound sensor module pointing to the second speaker, wherein the sound pickup direction patterns corresponding to the first sound pickup state and the second sound pickup state are different.

15. The method according to claim 14, characterized in that The frequencies in the first frequency sub-band are lower than the frequencies in the second frequency sub-band; and The sound pickup direction pattern corresponding to the first sound pickup state is a heart shape, and the sound pickup direction pattern corresponding to the second sound pickup state is an 8 shape.

16. The method according to claim 1, characterized in that The second target operation includes: performing gain amplification on the integrated signal and sending the gain-amplified signal to the speaker module, so that the speaker module emits sound.

17. The method according to claim 1, characterized in that The speaker module and the sound sensor module are deployed on a first acoustic device, and the first acoustic device is communicatively connected with a second acoustic device; as well as The second target operation includes sending the integrated signal to the second acoustic device to reduce an echo of the second acoustic device.

18. An acoustic system, characterized in that: include: The speaker module receives input signals and outputs target sounds when in operation; The sound sensor module comprises at least: a first sound sensor and a second sound sensor, wherein the first sound sensor collects environmental sounds and generates a first signal when in operation, and the second sound sensor collects the environmental sounds and generates a second signal when in operation, wherein the environmental sounds at least include the target sound; as well as The signal processing circuit is connected to the sound sensor module and executes the method described in any one of claims 1 to 17 when in operation.

19. The acoustic system according to claim 18, characterized in that The signal processing circuit comprises: at least one storage medium storing at least one instruction set for performing signal processing; and At least one processor is communicatively connected to the sound sensor module and the at least one storage medium, wherein, when the acoustic system is running, the at least one processor reads the at least one instruction set and executes the method of any one of claims 1-17 according to the instructions of the at least one instruction set.

20. The acoustic system according to claim 18, characterized in that The acoustic system is any one of a hearing aid system, a sound amplification system, a headphone system, a telephone system, and a conference system.

21. The acoustic system according to claim 18, characterized in that The acoustic system is a hearing aid system, and the acoustic system further comprises a housing, wherein the speaker module, the sound sensor module and the signal processing circuit are arranged in the housing, wherein When the acoustic system is worn on the user's head, the sound output end of the speaker module faces the user's head, and the sound pickup end of at least one sound sensor in the sound sensor module is located on a side of the housing away from the user's head.

Images