Sound signal processing method and device and earphone

Abstract

The invention discloses a sound signal processing method and device and an earphone, the sound signal processing method is applied to the earphone, the earphone is provided with at least one microphone, and the method comprises the following steps: acquiring a first sound signal obtained by acquiring an external sound signal by the at least one microphone of the earphone, acquiring a first sound source orientation of a sound source of the external sound signal; performing convolution processing on the first sound signal according to a first head transfer function impact response corresponding to the first sound source orientation to obtain a second sound signal; determining amplitude information and phase information in the second sound signal according to the second sound signal; performing amplitude adjustment processing on the amplitude information of the second sound signal according to a first head transfer function corresponding to the first sound source orientation to obtain adjusted amplitude information; and obtaining a target sound signal according to the adjusted amplitude information and the phase information of the second sound signal.

Description

Technical Field

[0001] The present invention relates to the field of earphone technology, and more specifically, to a sound signal processing method, device and earphone. Background Art

[0002] Headphone-based sound signal processing technology is widely used in video live broadcast, VR, AR, movies, television, games, music and other fields. The application of headphone-based sound signal processing technology involves multiple links such as sound collection, processing, encoding, transmission and reproduction, with the aim of providing users with an immersive and high-fidelity auditory experience. However, most of the sounds reproduced in the current headphone-based recording technology are presented in a mono or dual-channel undirectional form, which lacks an indication of the location of the sound source and cannot provide a sense of direction and space for the sound, thereby reducing the user's immersive and viewing experience. Summary of the invention

[0003] One purpose of an embodiment of the present application is to provide a new sound signal processing method, which can add sound direction information to the sound signal, enhance the sense of direction and space of the sound, and improve the user's immersive experience and viewing experience.

[0004] According to a first aspect of the present invention, a method for processing a sound signal is provided. The method for processing a sound signal is applied to any earphone, wherein the earphone is provided with at least one microphone. The method comprises:

[0005] Acquire a first sound signal obtained by collecting an external sound signal by at least one microphone of the headset, and acquire a first sound source position of a sound source of the external sound signal;

[0006] Performing convolution processing on the first sound signal according to a first head transfer function impulse response corresponding to the first sound source orientation to obtain a second sound signal;

[0007] Determining amplitude information and phase information of the second sound signal according to the second sound signal;

[0008] performing amplitude adjustment processing on the amplitude information of the second sound signal according to a first head transfer function corresponding to the first sound source orientation to obtain adjusted amplitude information;

[0009] A target sound signal is obtained according to the adjusted amplitude information and the phase information.

[0010] Optionally, the first head transfer function impulse response includes a first impulse response corresponding to a plurality of reference sampling points at a set sampling frequency, the first sound signal includes first sound data corresponding to a plurality of actual sampling points at the set sampling frequency, and the convolution processing is performed on the first sound signal according to the first head transfer function impulse response corresponding to the first sound source orientation to obtain the second sound signal, comprising:

[0011] For any actual sampling point among the multiple actual sampling points, selecting a first impulse response corresponding to a reference sampling point matching the actual sampling point from the first head transfer function impulse response as the first impulse response corresponding to the actual sampling point;

[0012] According to the first impulse response corresponding to the actual sampling point, convolution processing is performed on the first sound data corresponding to the actual sampling point to obtain convolution sound data corresponding to the actual sampling point;

[0013] The second sound signal is determined according to the convolved sound data corresponding to each of the actual sampling points.

[0014] Optionally, the first impulse response includes a first phase response and a first amplitude response, the first sound data includes a first sound amplitude and the first sound phase, and the convolution processing is performed on the first sound data corresponding to the actual sampling point according to the first impulse response corresponding to the actual sampling point to obtain the convolution sound data corresponding to the actual sampling point, including:

[0015] Determine the convolution sound phase corresponding to the actual sampling point according to the first phase response and the first sound phase corresponding to the actual sampling point, and determine the convolution sound amplitude corresponding to the actual sampling point according to the first amplitude response and the first sound amplitude corresponding to the actual sampling point;

[0016] The convolution sound amplitude and the convolution sound phase corresponding to the actual sampling point are used as the convolution sound data corresponding to the actual sampling point.

[0017] Optionally, determining amplitude information and phase information in the second sound signal according to the second sound signal includes:

[0018] The second sound signal is subjected to time domain-frequency domain conversion processing to determine amplitude information and phase information in the second sound signal.

[0019] Optionally, the adjusted amplitude information is adjusted frequency domain amplitude information, the phase information is frequency domain phase information, and obtaining the target sound signal according to the adjusted amplitude information and the phase information of the second sound signal includes:

[0020] Determining an adjusted frequency domain sound signal according to the adjusted frequency domain amplitude information and the frequency domain phase information;

[0021] The adjusted frequency domain sound signal is subjected to frequency domain-time domain conversion processing to obtain a target sound signal.

[0022] Optionally, the amplitude information includes measured amplitude values ​​of the second sound signal at multiple second frequencies, the first head transfer function includes reference amplitude values ​​at multiple first frequencies, and the amplitude adjustment processing is performed on the amplitude information of the second sound signal according to the first head transfer function corresponding to the first sound source orientation to obtain the adjusted amplitude information, including:

[0023] For any second frequency, determining a measured amplitude value and a reference amplitude value corresponding to the second frequency according to the amplitude information and the first head transfer function; wherein the reference amplitude value corresponding to the second frequency is an amplitude value of the first head transfer function when the first frequency is equal to the second frequency;

[0024] Determine the target amplitude value corresponding to the second frequency according to the measured amplitude value and the reference amplitude value corresponding to the second frequency, and obtain the target amplitude value corresponding to each second frequency; wherein the target amplitude value corresponding to the second frequency is the amplitude difference between the measured amplitude value and the reference amplitude value corresponding to the second frequency;

[0025] The adjusted amplitude information is determined according to the target amplitude value corresponding to each second frequency.

[0026] Optionally, the earphone includes a left earphone and a right earphone, the left earphone and the right earphone are respectively provided with at least one microphone, and the first sound source orientation of the sound source of the external sound signal is obtained, including:

[0027] Acquire a third sound signal obtained by collecting external sound signals by two symmetrical microphones in each ear; wherein the third sound signal includes a left sound signal collected by the microphone arranged at the position of the left earphone and a right sound signal collected by the microphone arranged at the position of the right earphone;

[0028] Performing cross-correlation analysis on the left sound signal and the right sound signal to determine a first time difference between the left sound signal and the right sound signal;

[0029] Determine, according to the first time difference, candidate coordinate values ​​of the sound source of the external sound signal relative to a headphone coordinate system; wherein the headphone coordinate system is a coordinate system constructed based on the left headphone and the right headphone;

[0030] According to the root mean square value of the amplitude of the left sound signal and / or the root mean square value of the amplitude of the right sound signal, a target coordinate value is selected from the candidate coordinate values ​​as the first sound source orientation of the sound source of the external sound signal.

[0031] Optionally, the earphone includes a first microphone array provided in the left earphone and a second microphone array provided in the right earphone, and the first sound source orientation of the sound source for obtaining the external sound signal includes:

[0032] Acquire a fourth sound signal obtained by collecting external sound signals with two symmetrical microphones at each ear;

[0033] Determine a target microphone array according to the fourth sound signal; wherein the target microphone array is one of the first microphone array and the second microphone array;

[0034] Acquire a fifth sound signal obtained by collecting the external sound signal after the target microphone array performs beamforming in the second sound source orientation;

[0035] According to the fifth sound signal, the target sound source orientation of the sound source of the external sound signal relative to the target microphone array is determined as the first sound source orientation of the sound source of the external sound signal.

[0036] According to a second aspect of the present invention, there is further provided a sound signal processing device, comprising a memory and a processor, wherein the memory is used to store executable instructions; the processor is used to operate according to the control of the instructions to execute the method as described in the first aspect.

[0037] According to a third aspect of the present invention, there is further provided an earphone, the earphone comprising at least one microphone and the sound signal processing device as described in the second aspect.

[0038] One beneficial effect of the present invention is that the first sound signal is convoluted according to the impulse response of the first head transfer function corresponding to the first sound source orientation of the sound source of the external sound signal to obtain a second sound signal, the amplitude information and phase information of the second sound signal are determined according to the second sound signal, the amplitude information of the second sound signal is adjusted according to the first head transfer function corresponding to the first sound source orientation to obtain the adjusted amplitude information, and the target sound signal is obtained according to the adjusted amplitude information and the phase information. It can be achieved that when the target sound signal containing the sound source position is obtained, the sense of direction and space of the sound can be improved. Moreover, by performing amplitude adjustment processing on the amplitude information, the amplitude influence introduced by the first head transfer function in the second sound signal can be removed, the sound distortion and auditory discomfort caused by the amplitude influence of the first head transfer function introduced into the second sound signal can be avoided, and the distortion of the sound signal can be avoided while improving the sense of space and direction of the sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] The accompanying drawings, which are incorporated in and constitute a part of the specification, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the invention.

[0040] Figure 1 is a schematic diagram of the hardware structure of the headset according to an embodiment of the present application;

[0041] Figure 2 is a flow chart of a sound signal processing method according to an embodiment of the present invention;

[0042] Figure 3 is a schematic diagram of the arrangement of each preset sound source orientation according to an example of the present invention;

[0043] Figure 4 is a schematic diagram of a head transfer function corresponding to each preset sound source orientation according to an example of the present invention;

[0044] Figure 5 is a schematic diagram of an impulse response of a head transfer function corresponding to each preset sound source orientation according to an example of the present invention;

[0045] Figure 6 is a schematic flow chart of a sound source localization method according to an embodiment of the present invention;

[0046] Figure 7 is a sound propagation path for transmitting external sound signals to the position of a human ear when wearing headphones according to an embodiment of the present invention;

[0047] Figure 8 is a schematic flow chart of a sound source localization method according to another embodiment of the present invention;

[0048] Fig. 9 is a schematic diagram of the angular relationship between the target sound source orientation and the target microphone array according to an example of the present invention;

[0049] Fig.10 is a schematic structural diagram of a sound signal processing device according to an embodiment of the present invention;

[0050] Fig.11 2 is a schematic diagram of the structure of an earphone according to an embodiment of the present invention. DETAILED DESCRIPTION

[0051] Various exemplary embodiments of the present invention will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present invention unless otherwise specifically stated.

[0052] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the invention, its application, or uses.

[0053] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the technologies, methods, and equipment should be considered part of the specification.

[0054] In all examples shown and discussed herein, any specific values ​​should be interpreted as merely exemplary and not as limiting. Therefore, other examples of the exemplary embodiments may have different values.

[0055] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0056] <Hardware Configuration>

[0057] Figure 1 is a block diagram of the hardware configuration of the headset 100 according to an embodiment of the present application.

[0058] like Figure 1 As shown, the earphone 100 includes at least one microphone 1000 and a sound signal processing device 2000 .

[0059] The earphone 100 can be a left earphone or a right earphone, which is not limited here.

[0060] The headset 100 may be provided with a single microphone or a microphone array, and the number of microphones provided in the headset 100 is not limited herein.

[0061] At least one microphone 1000 of the headset 100 is communicatively connected to the sound signal processing device 2000. The at least one microphone 1000 is used to collect external sound signals to obtain a first sound signal and send it to the sound signal processing device 2000. The sound signal processing device 2000 executes the sound signal processing method of the embodiment of the present application based on the first sound signal to obtain a target sound signal.

[0062] In this embodiment, refer to Figure 1 As shown, the sound signal processing device 2000 may include a processor 2100, a memory 2200, a communication device 2300, a filter 2400, and the like.

[0063] Processor 2100 may be a mobile version processor. Memory 2200 may include, for example, ROM (read-only memory), RAM (random access memory), non-volatile memory such as a hard disk, etc. Communication device 2300 may, for example, be capable of wired or wireless communication. Communication device 2300 may include a short-range communication device, such as any device for short-range wireless communication based on short-range wireless communication protocols such as Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. Communication device 2300 may also include a remote communication device, such as any device for WLAN, GPRS, 2G / 3G / 4G / 5G remote communication. Filter 2400 is used to filter the sound signal obtained by collecting external sound signals.

[0064] In this embodiment, the memory 2200 of the sound signal processing device 2000 is used to store instructions, and the instructions are used to control the processor 2100 to operate to at least perform the method performed by the sound signal processing device according to any embodiment of the present invention. A technician can design instructions according to the scheme disclosed in the present invention. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0065] Despite Figure 1 , multiple devices of the sound signal processing device 2000 are shown, but the present invention may only involve some of the devices, for example, the sound signal processing device 2000 only involves the memory 2200 and the processor 2100.

[0066] In some other embodiments, the earphone includes a left earphone, a right earphone and a sound signal processing device.

[0067] The left earphone and the right earphone are each provided with at least one microphone to collect external sound signals. That is to say, the left earphone and the right earphone can each be provided with one microphone, or can each be provided with a microphone array, and the number of microphones provided with the left earphone and the right earphone is not limited here.

[0068] In these embodiments, the left earphone may include a processor, a memory, at least one microphone array, a communication device, and the like.

[0069] The processor may be a mobile processor. The memory may include, for example, ROM (read-only memory), RAM (random access memory), non-volatile memory such as a hard disk, etc. At least one microphone may collect external sound signals. The communication device may be capable of, for example, wired or wireless communication. The communication device may include a short-range communication device, such as any device for short-range wireless communication based on short-range wireless communication protocols such as Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. The communication device may also include a remote communication device, such as any device for WLAN, GPRS, 2G / 3G / 4G / 5G remote communication.

[0070] In this embodiment, the memory of the left earphone is used to store instructions, and the instructions are used to control the processor to operate to at least perform the method performed by the left earphone according to any embodiment of the present invention. A technician can design instructions according to the scheme disclosed in the present invention. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0071] In these embodiments, the right earphone may include a processor, a memory, at least one microphone, a communication device, and the like.

[0072] The processor may be a mobile version processor. The memory may include, for example, ROM (read-only memory), RAM (random access memory), non-volatile memory such as a hard disk, etc. At least one microphone may collect external sound signals. The communication device may be capable of, for example, wired or wireless communication. The communication device may include a short-range communication device, such as any device for short-range wireless communication based on short-range wireless communication protocols such as Hilink protocol, WiFi (IEEE 802.11 protocol), Mesh, Bluetooth, ZigBee, Thread, Z-Wave, NFC, UWB, LiFi, etc. The communication device may also include a remote communication device, such as any device for WLAN, GPRS, 2G / 3G / 4G / 5G remote communication.

[0073] In this embodiment, the memory of the right earphone is used to store instructions, and the instructions are used to control the processor to operate to at least perform the method performed by the right earphone according to any embodiment of the present invention. A technician can design instructions according to the scheme disclosed in the present invention. How instructions control the processor to operate is well known in the art, so it will not be described in detail here.

[0074] The left earphone and the right earphone are respectively communicatively connected to the sound signal processing device to send the sound signals collected by them to the sound signal processing device for sound signal processing, thereby adding sound source orientation information to the sound signal collected by at least one microphone of the left earphone (i.e., obtaining the target sound signal corresponding to the left earphone) and adding sound source orientation information to the sound signal collected by at least one microphone of the right earphone (i.e., obtaining the target sound signal corresponding to the right earphone).

[0075] <Method Example>

[0076] Figure 2 is a flow chart of a sound signal processing method according to an embodiment of the present application, the method is applied to a sound signal processing device of a headset 100, that is, Figure 1 The sound signal processing device 2000 in the embodiment of the present invention. The earphone 100 may be a left earphone or a right earphone, which is not limited here. The earphone 100 is provided with at least one microphone.

[0077] according to Figure 2 As shown, the sound signal processing method of this embodiment may include the following steps S2100 to S2500:

[0078] Step S2100: obtaining a first sound signal obtained by collecting an external sound signal by at least one microphone of the earphone, and obtaining a first sound source direction of a sound source of the external sound signal.

[0079] In this embodiment, in an example where the headset is provided with a microphone array, the first sound signal obtained in step S2100 is a sound signal obtained by any microphone in the microphone array collecting an external sound signal. In an example where the headset is provided with a single microphone, the first sound signal obtained in step S2100 is a sound signal obtained by the microphone collecting an external sound signal.

[0080] The first sound source direction of the sound source of the external sound signal can be obtained based on the positioning function of the earphone itself, or it can be a sound signal processing device sent to the earphone after positioning by an external electronic device, or it can be the sound source direction manually set by the user, etc. Those skilled in the art should understand that the specific positioning method for the sound source of the external sound signal is not limited here.

[0081] Exemplarily, when locating a sound source based on the positioning function of the earphone itself, the earphone includes a left earphone and a right earphone, each of which is provided with at least one microphone, and the first sound source orientation of the sound source of the external sound signal can be a target coordinate value of the sound source of the external sound signal relative to the earphone coordinate system. The earphone coordinate system can be as follows: Figure 3 The coordinate system shown is constructed with the midpoint of the line between the left earphone and the right earphone as the center of the circle, the center of the circle as the starting point, and the directions perpendicular to the line between the left earphone and the right earphone are defined as 0° and 180° respectively. The first sound source orientation of the sound source of the external sound signal can be Figure 3 45° direction.

[0082] Step S2200: performing convolution processing on the first sound signal according to a first head transfer function impulse response corresponding to the first sound source orientation to obtain a second sound signal.

[0083] In this embodiment, the impulse response of the head transfer function (Impulse Response) is the representation of the head transfer function (HRTF) in the time domain, which reflects how the sound changes from the sound source to the ear over time when a short and strong pulse (impact) sound source emits sound. The impulse response of the head transfer function is a time series data, which can record the evolution of the sound signal in time, including the beginning, duration and end of the sound. The head transfer function (HRTF) reflects the frequency response characteristics caused by the influence of anatomical structures such as the head, auricle, and ear canal when the sound propagates from the sound source to the human ear. The head transfer function HRTF is a representation in the frequency domain. Both the head transfer function HRTF and the head transfer function impulse response can describe the response when the sound is transmitted from the sound source to the ear. The difference is that the head transfer function is the representation of the sound transfer process in the frequency domain, and the impulse response of the head transfer function is the representation of the sound transfer process in the time domain. In addition, by performing a time domain-frequency domain conversion process on the head transfer function impulse response, the head transfer function HRTF in the frequency domain can be obtained.

[0084] From the above analysis, it can be seen that the first head transfer function corresponding to the first sound source orientation can be obtained by performing time domain-frequency domain conversion processing on the first head transfer function impulse response corresponding to the first sound source orientation. The first head transfer function impulse response can be obtained by performing frequency domain-time domain conversion processing on the first head transfer function.

[0085] In some embodiments, the impulse response of the first head transfer function corresponding to the first sound source orientation is determined by the following steps SA1 and SA2:

[0086] Step SA1, obtaining the head transfer function impulse response corresponding to each preset sound source orientation.

[0087] In this embodiment, the process of obtaining the impulse response of the head transfer function corresponding to each preset sound source position may be: a sound response test is performed on the simulated human head in advance when a short and strong pulse (impact) sound source emits sound, and the impulse response of the head transfer function corresponding to each preset sound source position is obtained. The arrangement of each preset sound source position may be as follows: Figure 3 The setting shown is for the center position of the left earphone and the right earphone, and can also be set for the left ear side or the right ear side, which is not limited here.

[0088] Step SA2: according to the head transfer function impulse response corresponding to each preset sound source orientation, determine the head transfer function impulse response corresponding to the preset sound source orientation matching the first sound source orientation as the first head transfer function impulse response.

[0089] Exemplarily, the plurality of preset sound source positions may be as follows: Figure 3 The sound source orientation shown in FIG. 1 , correspondingly, the head transfer function corresponding to each preset sound source orientation obtained through simulation test can be as follows: Figure 4 As shown, the impulse response of the head transfer function corresponding to each preset sound source orientation obtained by simulation test can be as follows: Figure 5 If the first sound source orientation is determined to be 45°, the impulse response of the first head transfer function is Figure 5 The head transfer function impulse response corresponding to 45° in the middle, the first head transfer function is as follows Figure 4 The head transfer function corresponding to 45° is shown.

[0090] After determining the impulse response of the first head transfer function corresponding to the first sound source orientation, the first sound signal and the impulse response of the first head transfer function are time aligned, and then the first sound signal and the impulse response of the first head transfer function corresponding to the same time point are convolved to obtain a second sound signal. The second sound signal is time series data, that is, the second sound signal reflects the change of the sound over time.

[0091] In order to clearly present the changes of the head transfer function impulse response over time and to facilitate convolution processing with the sound data corresponding to multiple actual sampling points collected, the head transfer function impulse response can be represented by the impulse responses corresponding to multiple discrete reference sampling points.

[0092] Based on this, in some embodiments, the impulse response of the first head transfer function includes the first impulse response corresponding to multiple reference sampling points at a set sampling frequency, and the first sound signal includes the first sound data corresponding to multiple actual sampling points at the set sampling frequency. That is, the time interval between any two adjacent reference sampling points is the same as the time interval between any two adjacent actual sampling points. At this time, if the number of multiple reference sampling points of the first head transfer function impulse response is the same as the number of multiple actual sampling points of the first sound signal, the first head transfer function impulse response and the first sound signal are completely aligned in time, and the first sound data corresponding to each actual sampling point can be directly convolved with the first impulse response of the reference sampling point that is time-aligned with the actual sampling point to obtain the second sound signal. If the number of multiple reference sampling points of the first head transfer function impulse response is not the same as the number of multiple actual sampling points of the first sound signal, it is necessary to select the first impulse response of the reference sampling point that matches the actual sampling point in the first head transfer function impulse response as the first impulse response corresponding to the actual sampling point.

[0093] Based on this, in step S2200, the first sound signal is convolved according to the first head transfer function impulse response corresponding to the first sound source orientation to obtain a second sound signal, including: steps S2200.1 to S2200.3.

[0094] Step S2200.1: for any actual sampling point among the multiple actual sampling points, select a first impulse response corresponding to a reference sampling point matching the actual sampling point from the first head transfer function impulse response as the first impulse response corresponding to the actual sampling point.

[0095] Exemplarily, the first head transfer function impulse response includes 300 reference sampling points, the first sound signal includes 200 actual sampling points, and the time interval between two adjacent reference sampling points is equal to the time interval between two adjacent actual sampling points. At this time, for any actual sampling point among the 200 actual sampling points, the first impulse response corresponding to the reference sampling point matching the actual sampling point is selected from the first head transfer function impulse response, that is, for the nth actual sampling point, the first impulse response corresponding to the nth reference sampling point is selected as the first impulse response corresponding to the actual sampling point.

[0096] Step S2200.2: performing convolution processing on the first sound data corresponding to the actual sampling point according to the first impulse response corresponding to the actual sampling point to obtain convolution sound data corresponding to the actual sampling point.

[0097] In this embodiment, the first impulse response corresponding to the actual sampling point may be multiplied by the first sound data to obtain the convolution sound data corresponding to the actual sampling point.

[0098] In one example, y(n)=x(i)*h in formula (1) can be used. θ (n) Calculate the convolution sound data corresponding to the actual sampling point.

[0099]

[0100] Wherein, vector x(i), i=1, 2...L, represents the first sound data corresponding to L actual sampling points, h θ (n) is the impulse response of the first head transfer function corresponding to the first sound source orientation θ, and n is the reference sampling point of the impulse response of the first head transfer function. L can represent the length of the first sound signal or the length of the impulse response of the first head transfer function involved in the convolution.

[0101] In some embodiments, in order to facilitate time-domain-frequency domain conversion (ie, Fast Fourier Transform (FFT)) of the second sound signal, the length of L is an integer power of 2.

[0102] Step S2200.3, determining the second sound signal according to the convolution sound data corresponding to each of the actual sampling points.

[0103] In this embodiment, the convolution sound data corresponding to each actual sampling point is arranged in time sequence to form a new signal sequence, and the new signal sequence is the second sound signal.

[0104] Continuing with the above example, we can use formula (1) A second sound signal is obtained by calculation.

[0105] The impulse response of the head transfer function includes a phase response and an amplitude response. The phase response indicates the temporal change of the phase when the sound is transmitted from the sound source to the ear, and the amplitude response indicates the temporal change of the amplitude when the sound is transmitted from the sound source to the ear. Correspondingly, the first impulse response corresponding to any reference sampling point in the first impulse response of the head transfer function also includes the first phase response and the first amplitude response corresponding to the reference sampling point. The first sound signal is composed of multiple actual sampling points, and the first sound data corresponding to each actual sampling point includes the first sound amplitude and the first sound phase of the sound signal at that time point.

[0106] Based on this, in step S2200.2, according to the first impulse response corresponding to the actual sampling point, the first sound data corresponding to the actual sampling point is convolved to obtain the convolved sound data corresponding to the actual sampling point, including: step S2200.21 and step S2200.22.

[0107] Step S2200.21, determining the convolution sound phase corresponding to the actual sampling point according to the first phase response and the first sound phase corresponding to the actual sampling point, and determining the convolution sound amplitude corresponding to the actual sampling point according to the first amplitude response and the first sound amplitude corresponding to the actual sampling point.

[0108] In this embodiment, the first phase response corresponding to the actual sampling point and the first sound phase are multiplied to obtain the convolution sound phase corresponding to the actual sampling point, and the first amplitude response corresponding to the actual sampling point and the first sound amplitude are multiplied to obtain the convolution sound amplitude corresponding to the actual sampling point.

[0109] Step S2200.22, taking the convolution sound amplitude and the convolution sound phase corresponding to the actual sampling point as the convolution sound data corresponding to the actual sampling point.

[0110] In this embodiment, the convolution sound amplitude and the convolution sound phase are used together as the convolution sound data.

[0111] Step S2300: Determine amplitude information and phase information of the second sound signal according to the second sound signal.

[0112] In this embodiment, the second sound signal is time series data. The second sound signal can be converted from the time domain to the frequency domain through fast Fourier transform (FFT) and short-time Fourier transform (STFT) to obtain the amplitude information and phase information of the second sound signal. The second sound signal can also be analyzed through Hilbert transform to obtain the amplitude information and phase information of the second sound signal. The time-frequency characteristics of the second sound signal can also be obtained through wavelet transform to obtain the amplitude information and phase information of the second sound signal.

[0113] Those skilled in the art should understand that the specific manner of determining the amplitude information and the phase information of the second sound signal is not limited here.

[0114] Since the amplitude information and phase information of the second sound signal are determined in different ways, the obtained amplitude information and phase information may be time domain phase information and time domain amplitude information, or may be frequency domain phase information and frequency domain amplitude information.

[0115] The time domain amplitude information can be used to reflect the intensity or size of the second sound signal changing over time, and the time domain phase information can be used to reflect the phase offset of the second sound signal waveform relative to the reference point at each moment. The frequency domain amplitude information can be used to reflect the intensity distribution of the second sound signal at multiple second frequencies, and the frequency domain phase information can be used to reflect the phase offset of the second sound signal at each frequency component.

[0116] In some embodiments, determining amplitude information and phase information in the second sound signal according to the second sound signal in step S2300 includes step S2311.

[0117] Step S2311: perform time domain-frequency domain conversion processing on the second sound signal to determine amplitude information and phase information in the second sound signal.

[0118] In this embodiment, the time domain-frequency domain conversion processing can be a fast Fourier transform (Fast Fourier Transform, FFT), a short-time Fourier transform (Short-Time Fourier Transform, STFT), etc., which is not limited here.

[0119] The process of obtaining the amplitude information and the phase information of the second sound signal through fast Fourier transform may be to obtain the second sound signal in the frequency domain through fast Fourier transform of the second sound signal in the time domain.

[0120] Each frequency component of the second sound signal in the frequency domain can be represented by a complex number, that is, X(k)=A(k)+iB(k), where A(k) is the real part, B(k) is the imaginary part, and k is the frequency.

[0121] For any frequency component, that is, any value of k, the modulus of the complex number can be calculated to obtain the amplitude corresponding to the frequency component, and the phase angle of the complex number can be calculated to obtain the phase corresponding to the frequency component. Finally, the phase corresponding to each frequency component is used as phase information, and the amplitude corresponding to each frequency component is used as amplitude information.

[0122] Step S2400: performing amplitude adjustment processing on the amplitude information of the second sound signal according to the first head transfer function corresponding to the first sound source orientation to obtain adjusted amplitude information.

[0123] In this embodiment, after executing step S2200, the first sound signal is convolved according to the impulse response of the first head transfer function corresponding to the first sound source orientation to obtain the second sound signal, the obtained second sound signal includes the amplitude influence and phase influence introduced by the impulse response of the first head transfer function (or, the first head transfer function). Among them, the amplitude influence introduced by the first head transfer function is due to the resonance effect of the auricle and the ear canal included in the first head transfer function, which raises the high-frequency energy of the external sound signal.

[0124] The design purpose of this scheme is mainly to enhance the sense of direction and space of the sound, which is related to the phase effect introduced by the first head transfer function, but has nothing to do with the amplitude effect introduced by the first head transfer function. Moreover, after the second sound signal is broadcast by different playback devices, the listener's own auricle and ear canal can also adjust the amplitude of the sound (that is, the listener himself also introduces the amplitude effect of the head transfer function). If the amplitude effect introduced by the first head transfer function in the second sound signal is not removed, the sound heard by the listener will also be distorted due to the amplitude effect of the head transfer function introduced twice. Therefore, based on these two reasons, it is necessary to remove the amplitude effect introduced by the first head transfer function in the amplitude information of the second sound signal, and retain the phase effect introduced by the first head transfer function.

[0125] In other words, the amplitude information of the second sound signal is subjected to amplitude adjustment processing through the first head transfer function, so as to remove the amplitude influence introduced by the first head transfer function in the second sound signal.

[0126] In one example, the first head transfer function may be obtained by performing a time domain-frequency domain conversion process on the impulse response of the first head transfer function.

[0127] In another example, the first head transfer function corresponding to the first sound source orientation is determined by following steps SE1 and SE2:

[0128] Step SE1, obtaining the head transfer function corresponding to each preset sound source orientation.

[0129] In this embodiment, a test is performed on a simulated human head in advance to obtain a head transfer function corresponding to each preset sound source orientation. The preset sound source orientation may be as follows: Figure 3 The sound source orientation shown may also be set to other sound source orientations according to accuracy requirements, which is not limited here.

[0130] Step SE2: Determine a first head transfer function corresponding to the first sound source orientation according to the head transfer function corresponding to each preset sound source orientation.

[0131] Exemplarily, the amplitude information of the second sound signal may be frequency domain amplitude information. If the orientation of the first sound source is 45°, the first head transfer function is: Figure 4 The head transfer function corresponding to 45° in the middle can be directly processed by the first head transfer function for amplitude adjustment. The amplitude information can also be time domain amplitude information, which can be converted into frequency domain amplitude information, and then processed by the first head transfer function for amplitude adjustment.

[0132] In some embodiments, the amplitude information includes measured amplitude values ​​of the second sound signal at a plurality of second frequencies, and the first head transfer function includes reference amplitude values ​​at the plurality of first frequencies.

[0133] In this embodiment, the amplitude information is frequency domain amplitude information. The multiple first frequencies and the multiple second frequencies may be completely the same or partially the same, which is not limited here.

[0134] In these embodiments, in step S2400, the amplitude information of the second sound signal is adjusted according to the first head transfer function corresponding to the first sound source orientation to obtain the adjusted amplitude information, including: steps S2400.1 to S2400.3.

[0135] Step S2400.1: for any second frequency, determine a measured amplitude value and a reference amplitude value corresponding to the second frequency according to the amplitude information and the first head transfer function.

[0136] In this embodiment, the reference amplitude value corresponding to the second frequency is the amplitude value of the first head transfer function when the first frequency is equal to the second frequency.

[0137] This step is to use, for any second frequency included in the amplitude information, an amplitude value corresponding to a first frequency equal to the second frequency in the first head transfer function as a reference amplitude value of the second frequency.

[0138] For example, the plurality of second frequencies include 1000 Hz, and the first head transfer function is Figure 4 The head transfer function corresponding to 45° in the figure can be used to determine the measured amplitude value corresponding to 1000HZ based on the amplitude information. Figure 4 Determine the reference amplitude value corresponding to 1000HZ.

[0139] Step S2400.2, determining the target amplitude value corresponding to the second frequency according to the measured amplitude value and the reference amplitude value corresponding to the second frequency, and obtaining each target amplitude value corresponding to the second frequency.

[0140] In this embodiment, after obtaining the reference amplitude value and the measured amplitude value corresponding to any second frequency, the measured amplitude value and the reference amplitude value corresponding to any second frequency are subtracted to obtain the target amplitude value corresponding to the second frequency. The target amplitude value is the amplitude difference between the measured amplitude value and the reference amplitude value;

[0141] Step S2400.3, determining the adjusted amplitude information according to the target amplitude value corresponding to each second frequency.

[0142] In this embodiment, the target amplitude values ​​corresponding to each second frequency are combined to obtain adjusted amplitude information.

[0143] By performing amplitude adjustment processing on the amplitude information, the amplitude influence introduced by the first head transfer function in the second sound signal can be removed, the problem of sound distortion caused by the amplitude influence of the first head transfer function introduced into the second sound signal can be avoided, and the distortion of the sound signal can be avoided while improving the spatial sense and directional sense of the sound.

[0144] Step S2500: obtaining a target sound signal according to the adjusted amplitude information and the phase information.

[0145] In this embodiment, the adjusted amplitude information may be in the time domain or in the frequency domain. The phase information may be in the frequency domain or in the time domain. The target sound signal may be in the time domain (i.e., the time domain target sound signal) or in the frequency domain (i.e., the frequency domain target sound signal), which is not limited here.

[0146] In some embodiments, after obtaining the target sound signal in step S2500, the method further includes: outputting the target sound signal.

[0147] In this embodiment, outputting the target sound signal may be, for example, playing the target sound signal through a speaker, wherein the target sound signal for output is a time domain target sound signal.

[0148] In some embodiments, the adjusted amplitude information is adjusted frequency domain amplitude information, and the phase information is frequency domain phase information.

[0149] In this embodiment, the adjusted amplitude information and phase information are both frequency domain data. In order to facilitate the output of the target sound signal, the frequency domain phase information and the adjusted frequency domain amplitude information can be first subjected to frequency domain-time domain conversion processing and then combined to obtain the target sound signal. Alternatively, the frequency domain phase information and the adjusted frequency domain amplitude information can be combined in the frequency domain to obtain the frequency domain target sound signal, and then the frequency domain target sound signal can be subjected to frequency domain-time domain conversion processing to obtain the target sound signal that can be played.

[0150] In some embodiments, obtaining a target sound signal according to the adjusted amplitude information and the phase information of the second sound signal in step S2500 includes: step S2500.1 and step S2500.2.

[0151] Step S2500.1, determining an adjusted sound signal according to the adjusted amplitude information and the aroma information of the second sound signal.

[0152] In this embodiment, the adjusted sound signal is the frequency domain target sound signal.

[0153] Step S2500.2, performing frequency domain-time domain conversion processing on the adjusted sound signal to obtain a target sound signal.

[0154] In this embodiment, the target sound signal is a time-domain target sound signal.

[0155] According to an embodiment of the present application, the first sound signal is convolved with the impulse response of the first head transfer function corresponding to the first sound source orientation of the sound source of the external sound signal to obtain a second sound signal, the amplitude information and phase information of the second sound signal are determined according to the second sound signal, the amplitude information of the second sound signal is adjusted according to the first head transfer function corresponding to the first sound source orientation to obtain the adjusted amplitude information, and the target sound signal is obtained according to the adjusted amplitude information and the phase information. It can be achieved that when the target sound signal containing the sound source position is obtained, the sense of direction and space of the sound can be improved. Moreover, by performing amplitude adjustment processing on the amplitude information, the amplitude influence introduced by the first head transfer function in the second sound signal can be removed, the sound distortion and auditory discomfort caused by the amplitude influence of the first head transfer function introduced into the second sound signal can be avoided, and the distortion of the sound signal can be avoided while improving the sense of space and direction of the sound.

[0156] In some embodiments, the earphone includes a left earphone and a right earphone, and the left earphone and the right earphone are respectively provided with at least one microphone. Figure 6 As shown, obtaining the first sound source orientation of the sound source of the external sound signal in step S2100 includes: steps S1100 to S1400.

[0157] Step S1100: Acquire a third sound signal obtained by collecting external sound signals using two symmetrical microphones at both ears.

[0158] In this embodiment, the two symmetrical microphones refer to two microphones in the left earphone and the right earphone that are symmetrically placed relative to the center of the wearer's head in terms of spatial layout. Both microphones can be called feedforward microphones.

[0159] like Figure 7 As shown, it can represent the sound propagation path of the external sound signal transmitted to the position of the human ear when wearing headphones. The external sound signal is x(t). The third sound signal obtained by collecting the external sound signal with two symmetrical microphones on each ear can be expressed as x(t)*R(s), where R(s) represents the total transfer function of the external sound signal from the environment to the position of two symmetrical microphones on each ear.

[0160] The third sound signal includes a left sound signal collected by the microphone arranged at the position of the left earphone and a right sound signal collected by the microphone arranged at the position of the right earphone.

[0161] R(s) may be the sum of a first transfer function R1(s) for transmitting an external sound signal from the environment to the microphone position set at the left earphone position and a second transfer function R2(s) for transmitting an external sound signal from the environment to the microphone position set at the right earphone position. Then, the left sound signal may be expressed as x(t)*R1(s), and the right sound signal may be expressed as x(t)*R2(s).

[0162] Step S1200: performing cross-correlation analysis on the left sound signal and the right sound signal to determine a first time difference between the left sound signal and the right sound signal.

[0163] In this embodiment, cross-correlation may be used to measure the similarity between the left sound signal and the right sound signal to determine the time delay between the two, that is, the first time difference.

[0164] In some embodiments, step S1200 performs cross-correlation analysis on the left sound signal and the right sound signal to determine a first time difference between the left sound signal and the right sound signal, including:

[0165] A cross-correlation analysis is performed on the left sound signal and the right sound signal, and a time difference between the left sound signal and the right sound signal corresponding to a maximum value in a result of the cross-correlation analysis is used as the first time difference.

[0166] In this embodiment, x(n) is the left sound signal collected by the microphone set in the left earphone, n is the sampling point index of the left sound signal, n=1, 2...N, N is the total number of sampling points of x(n). y(m) is the right sound signal collected by the microphone set in the right earphone, m is the sampling point index of the right sound signal, m=1, 2...M, M is the total number of sampling points of y(m). The cross-correlation function can be shown as formula (2):

[0167]

[0168] Wherein, c(iN) represents the value of the cross-correlation function, that is, the result of the cross-correlation analysis, iN is the time offset, and i is the index variable when summing, which is used to traverse the sampling points of the left sound signal and the right sound signal when calculating the cross-correlation function.

[0169] When the left sound signal and the right sound signal are completely aligned in time, the value c(iN) of the cross-correlation function will reach a maximum value. The maximum value of the cross-correlation function indicates that the correlation between the two signals is the strongest, and the time difference between the left sound signal and the right sound signal when the cross-correlation function is the maximum value of the two signals is taken as the first time difference. The first time difference is caused by the distance difference of the different paths of the sound source to the microphone set in the left earphone and the microphone set in the right earphone.

[0170] According to an embodiment of the present application, by using the time difference between the left sound signal and the right sound signal corresponding to the maximum value in the result of the cross-correlation analysis as the first time difference, the accuracy of determining the time difference can be improved, thereby improving the accuracy of positioning.

[0171] In order to avoid affecting the accuracy of determining the first time difference due to the fact that the wavelength of the sound waves in the left sound signal and the right sound signal is smaller than the distance between each microphone that is symmetrical between the two ears, in some embodiments, step S1200 performs a cross-correlation analysis on the left sound signal and the right sound signal to determine the first time difference between the left sound signal and the right sound signal, including: step S1200.11 and step S1200.12.

[0172] Step S1200.11, performing a first bandpass filter on the left sound signal and the right sound signal within a first set frequency range to obtain a first left filtered sound signal and a first right filtered sound signal.

[0173] In this embodiment, the inventors have found that since the wavelength of sound waves with a frequency above 1.5 kHz is smaller than the distance between the two symmetrical microphones of the two ears, the determination of the first time difference is affected. Therefore, the first set frequency range can be, for example, 100 Hz to 1500 Hz, or other frequency ranges less than 1500 Hz, which is not limited here.

[0174] By performing a first bandpass filtering within a first set frequency range on the left sound signal and the right sound signal, only the low-frequency components in the left sound signal and the right sound signal can be retained, and the high-frequency components can be filtered out to obtain a first left filtered sound signal and a first right filtered sound signal.

[0175] Step S1200.12: performing cross-correlation analysis on the first left-side filtered sound signal and the first right-side filtered sound signal to determine a first time difference between the first left-side filtered sound signal and the first right-side filtered sound signal.

[0176] In this embodiment, the first left-side filtered sound signal and the first right-side filtered sound signal are substituted into the above formula (2) to calculate the cross-correlation function, that is, x(n)n=1,2…N is the first left-side filtered sound signal, and y(m)m=1,2…M is the first right-side filtered sound signal. The time difference between the first left-side filtered sound signal and the first right-side filtered sound signal when the two signals reach the maximum cross-correlation value is then taken as the first time difference.

[0177] In this way, by performing the first bandpass filtering within the first set frequency range on the left sound signal and the right sound signal, only the low-frequency components in the left sound signal and the right sound signal can be retained for cross-correlation analysis, thereby improving the accuracy of determining the first time difference between the two signals.

[0178] Step S1300: determining candidate coordinate values ​​of the sound source of the external sound signal relative to the headphone coordinate system according to the first time difference.

[0179] In this embodiment, the first time difference may be converted into a corresponding phase difference, and then the candidate coordinate value may be determined according to the phase difference, or the candidate coordinate value may be directly determined according to the first time difference, which is not limited here.

[0180] The headset coordinate system is a coordinate system constructed based on the left headset and the right headset. The headset coordinate system can be a coordinate system constructed with the midpoint of the line between the left headset and the right headset as the center of the circle and the line between the left headset and the right headset as the reference line (that is, the directions of the line between the left headset and the right headset are 0° and 180° respectively), or it can be a coordinate system constructed with the midpoint of the line between the left headset and the right headset as the center of the circle and the line perpendicular to the left headset and the right headset as the reference line (that is, the directions perpendicular to the line between the left headset and the right headset are 0° and 180°), which is not limited here.

[0181] For example, Figure 3 As shown, the midpoint of the line between the left earphone and the right earphone can be used as the center of the earphone coordinate system, and the center of the circle can be used as the starting point. The directions perpendicular to the line between the left earphone and the right earphone are defined as 0° and 180° respectively, thereby constructing the earphone coordinate system.

[0182] The candidate coordinate value may be the angle value of the sound source of the external sound signal in the headphone coordinate system. In other words, the candidate coordinate value may be the angle value between the line connecting the sound source of the external sound signal and the center of the circle and the reference line of the headphone coordinate system.

[0183] In one example, the process of determining the candidate coordinate value according to the first time difference may be: the preset sound source position may include: Figure 3The five preset sound source orientations of 0°, 45°, 90°, 135°, and 180° are tested on the simulated human head in advance to obtain the head transfer function corresponding to each preset sound source orientation. The head transfer function includes the amplitude and phase response corresponding to each preset sound source orientation. Figure 4 As shown. Figure 4 The data shown show that the time difference is the largest in the 90° direction (estimated to be about 0.7ms based on the diameter of the human head), followed by 45° / 135° (about 0.35), and the time difference of 0° / 180° is close to 0. If the first time difference is 0.35ms, then the candidate coordinate values ​​are 45° and 135°. If the first time difference is 0, then the candidate coordinate values ​​are 0° and 180°.

[0184] In some embodiments, step S1300 determines the candidate coordinate values ​​of the sound source of the external sound signal relative to the headphone coordinate system according to the first time difference, including: steps S1300.1 to S1300.2.

[0185] Step S1300.1: determine a second time difference according to the first time difference and the sampling rate of the third sound signal.

[0186] In this embodiment, the "data point difference" obtained by the cross-correlation analysis actually refers to the time difference between two signals at the sampling points in the cross-correlation function, rather than the actual time difference.

[0187] By dividing the first time difference by the sampling rate of the third sound signal to obtain the second time difference, the time difference between the two signals at the sampling points can be converted into an actual time difference, i.e., the second time difference. The sampling rate refers to the number of times the microphone samples per unit time, which determines the actual time interval between the sampling points.

[0188] Step S1300.2: determine the candidate coordinate value according to the second time difference.

[0189] Since the second time difference is more accurate than the first time difference, determining the candidate coordinate value according to the second time difference can improve the positioning accuracy.

[0190] Step S1400: Select a target coordinate value from the candidate coordinate values ​​as the first sound source orientation of the sound source of the external sound signal according to the root mean square value of the left sound signal amplitude and / or the root mean square value of the right sound signal amplitude.

[0191] In this embodiment, since the auricle can enhance the reflection of sound waves, the sound signal energy transmitted to the ear by the sound source located in front of the wearer is greater than the sound signal energy transmitted to the ear by the sound source located behind the wearer, and the sound signal energy can be represented by the root mean square value of the amplitude of the sound signal. Based on this, the target coordinate value can be selected from the candidate coordinate values ​​according to the root mean square value of the amplitude of the left sound signal or the root mean square value of the amplitude of the right sound signal.

[0192] In some embodiments, in step S1400, the target coordinate value is selected from the candidate coordinate values ​​according to the root mean square value of the left sound signal amplitude and / or the root mean square value of the right sound signal amplitude, including: steps S1400.1 to S1400.4.

[0193] Step S1400.1, obtaining the head transfer function corresponding to each preset sound source orientation.

[0194] This step is the same as the above step SE1 and will not be described again here.

[0195] The head transfer function corresponding to each preset sound source orientation includes the amplitude and phase response corresponding to each preset sound source orientation. Moreover, for any preset sound source orientation, the head transfer function of the preset sound source orientation includes a first head transfer function corresponding to the left earphone and a second head transfer function corresponding to the right earphone.

[0196] Step S1400.2, determining a candidate head transfer function corresponding to the candidate coordinate value according to the head transfer function corresponding to each preset sound source orientation.

[0197] In this embodiment, from among multiple head transfer functions corresponding to multiple preset sound source orientations, a head transfer function whose preset sound source orientation is consistent with the candidate coordinate value is determined as a candidate head transfer function corresponding to the candidate coordinate value. The candidate head transfer function includes a first head transfer function corresponding to the left earphone and a second head transfer function corresponding to the right earphone.

[0198] Step S1400.3: Determine the reference signal energy corresponding to the candidate coordinate value according to the candidate head transfer function.

[0199] In this embodiment, the reference signal energy is the root mean square value of the amplitude response of the candidate head transfer function.

[0200] If the candidate coordinate values ​​are 45° and 135°, the reference signal energy corresponding to the candidate coordinate values ​​45° and 135° can be determined according to the RMS value of the amplitude response of the head transfer function when the preset sound source orientation is 45° and 135°. Among them, since the auricle will enhance the reflection of sound waves, the signal energy of the sound source located in front of the wearer is greater than the signal energy located behind the wearer, so the calculated reference signal energy corresponding to the candidate coordinate value 45° is greater than the reference signal energy corresponding to the candidate coordinate value 135°.

[0201] The reference signal energy includes a first reference signal energy corresponding to the left earphone and a second reference signal energy corresponding to the right earphone. For example, the reference signal energy corresponding to the candidate coordinate value 45° includes a first reference signal energy corresponding to 45° and a second reference signal energy corresponding to 45°.

[0202] Step S1400.4, selecting a target coordinate value from the candidate coordinate values ​​according to the RMS value of the left sound signal amplitude and / or the RMS value of the right sound signal amplitude and the reference signal energy of the candidate coordinate values.

[0203] In one example, if the candidate coordinate values ​​are 45° and 135°, it is impossible to determine whether the sound source is in front of or behind the wearer. Through the above steps S1400.1 to S1400.3, the reference signal energy corresponding to 45° and the reference signal energy corresponding to 135° are determined. Then the root mean square value of the left sound signal amplitude is compared with the first reference signal energy corresponding to 45° and the first reference signal energy corresponding to 135°. If the root mean square value of the left sound signal amplitude is greater than the first reference signal energy corresponding to 135° and less than the first reference signal energy corresponding to 45°, the target coordinate value is determined to be 45°. If the root mean square value of the left sound signal amplitude is greater than the first reference signal energy corresponding to 45°, the target coordinate value is determined to be 45°. If the root mean square value of the left sound signal amplitude is less than the first reference signal energy corresponding to 135°, the target coordinate value is determined to be 135°.

[0204] In another example, after determining the reference signal energy corresponding to 45° and the reference signal energy corresponding to 135°, the RMS value of the left sound signal amplitude and the RMS value of the right sound signal amplitude can be compared with the reference signal energy. If the RMS value of the left sound signal amplitude is greater than the first reference signal energy corresponding to 45°, and the RMS value of the right sound signal amplitude is greater than the second reference signal energy corresponding to 45°, then the target coordinate value is determined to be 45°. If the RMS value of the left sound signal amplitude is less than the first reference signal energy corresponding to 135°, and the RMS value of the right sound signal amplitude is less than the second reference signal energy corresponding to 135°, then the target coordinate value is determined to be 135°.

[0205] By comparing at least one of the actually collected root mean square value of the left sound signal amplitude and the root mean square value of the right sound signal amplitude with the reference signal energy corresponding to the candidate coordinate value, the specific sound source orientation can be selected from the candidate coordinate values, thereby improving the positioning accuracy of the sound source orientation.

[0206] The inventors have found that the reflection of sound waves by the auricle mainly acts on high-frequency sounds. In order to improve the accuracy of determining the target coordinate value from the candidate coordinate values ​​based on the energy difference of the sound signal generated by the reflection of the sound waves by the auricle, the low-frequency sound can be first filtered out and the high-frequency sound can be retained when calculating the root mean square value of the amplitude of the left sound signal and / or the root mean square value of the amplitude of the right sound signal. Then, the target coordinate value can be determined from the candidate coordinate values ​​by calculating the root mean square value of the amplitude of the high-frequency sound signal.

[0207] Based on this, in step S1400, the target coordinate value is selected from the candidate coordinate values ​​according to the root mean square value of the left sound signal amplitude and / or the root mean square value of the right sound signal amplitude as the first sound source orientation of the sound source of the external sound signal, including: step S1401 and step S1402.

[0208] Step S1401: performing a second bandpass filtering on the left sound signal and the right sound signal within a second set frequency range to obtain a second left filtered sound signal and a second right filtered sound signal.

[0209] In this embodiment, the second set frequency range may be, for example, 3000-6000 Hz, or may be other frequency ranges greater than 2000 Hz, which is not limited here.

[0210] In one embodiment, the lower limit value of the second set frequency range is greater than the upper limit value of the first set frequency range.

[0211] Step S1402, selecting a target coordinate value from the candidate coordinate values ​​as the first sound source position of the sound source of the external sound signal according to the root mean square value of the amplitude of the second left-side filtered sound signal and / or the root mean square value of the amplitude of the second right-side filtered sound signal.

[0212] By performing the second bandpass filtering on the left sound signal and the right sound signal, only the high frequency components in the left sound signal and the right sound signal can be retained for signal energy calculation, thereby improving the accuracy of determining the target coordinate value from the candidate coordinate values.

[0213] In other embodiments, Figure 8As shown, the earphone includes a first microphone array provided in the left earphone and a second microphone array provided in the right earphone, and the first sound source orientation of the sound source of the external sound signal is obtained including: step S3100 to step S3400.

[0214] Step S3100: Acquire a fourth sound signal obtained by collecting external sound signals using two symmetrical microphones at both ears.

[0215] In this embodiment, the two symmetrical microphones may be two microphones in the first microphone array and the second microphone array that are symmetrically arranged relative to the center of the wearer's head in terms of spatial layout. For ease of description, the two symmetrical microphones may be microphone A and microphone B, microphone A is included in the first microphone array, and microphone B is included in the second microphone array.

[0216] Step S3200: determining a target microphone array according to the fourth sound signal.

[0217] In this embodiment, the target microphone array is a microphone array that is closer to the sound source of the external sound signal. The target microphone array may be the first microphone array or the second microphone array.

[0218] In some embodiments, the fourth sound signal includes a left signal collected by a microphone provided in a left earphone and a right signal collected by a microphone provided in a right earphone, and step S3200 determines the target microphone array according to the fourth sound signal, including: step S3200.1.

[0219] Step S3200.1, determining a target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal.

[0220] In this embodiment, when the sound wave passes from the ear close to the sound source to the other ear, the head blocks and absorbs the sound, so there is an energy difference between the ears, that is, the interaural level difference (ILD). The interaural level difference will cause the signal size (that is, signal amplitude) of the left signal and the right signal to differ, so the target microphone array closer to the sound source can be determined based on the signal amplitude of the left signal and the signal amplitude of the right signal.

[0221] In an example, the microphone array corresponding to the signal with the larger signal amplitude between the signal amplitude of the left signal and the signal amplitude of the right signal may be used as the target microphone array.

[0222] In another example, the signal energy can also be directly represented by the root mean square value of the signal amplitude, and energy comparison can be performed to determine the target microphone array, that is, the root mean square value of the signal amplitude of the left signal is compared with the root mean square value of the signal amplitude of the right signal, and the microphone array corresponding to the signal with the larger root mean square value of the signal amplitude is determined as the target microphone array.

[0223] In another example, the signal amplitude of the left signal can be compared with the amplitude response of the head transfer function of the left ear at different sound source orientations obtained in advance to determine the approximate orientation of the sound source relative to the left ear, thereby determining the target microphone array. Alternatively, the signal amplitude of the right signal can be compared with the amplitude response of the head transfer function of the right ear obtained in advance to determine the approximate orientation of the sound source relative to the right ear, thereby determining the target microphone array.

[0224] It should be noted that the method of determining the target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal is not specifically limited here.

[0225] In some embodiments, determining the target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal in step S3200.1 includes: steps S3200.11 to S3200.13.

[0226] Step S3200.11: when the root mean square value of the signal amplitude of the left signal is greater than the root mean square value of the signal amplitude of the right signal, determine that the target microphone array is the first microphone array.

[0227] Step S3200.12: when the root mean square value of the signal amplitude of the left signal is smaller than the root mean square value of the signal amplitude of the right signal, determine that the target microphone array is the second microphone array.

[0228] Step S3200.13: when the root mean square value of the signal amplitude of the left signal is equal to the root mean square value of the signal amplitude of the right signal, determine that the target microphone array is the first microphone array or the second microphone array.

[0229] In this embodiment, the root mean square value of the signal amplitude of the sound signal collected by the microphone represents the energy of the external sound signal transmitted to the microphone position. Then, by calculating the root mean square value of the signal amplitude of the left signal and the root mean square value of the signal amplitude of the right signal, it is determined whether the sound source is closer to the left ear or the right ear, and then the target microphone array is determined.

[0230] When the RMS value of the signal amplitude of the left signal is greater than the RMS value of the signal amplitude of the right signal, the sound source is closer to the first microphone array at the left earphone position, and the first microphone array is used as the target microphone array.

[0231] When the RMS value of the signal amplitude of the left signal is smaller than the RMS value of the signal amplitude of the right signal, the sound source is closer to the second microphone array at the right earphone position, and the second microphone array is used as the target microphone array.

[0232] When the root mean square value of the signal amplitude of the left signal is equal to the root mean square value of the signal amplitude of the right signal, the first distance between the sound source and the first microphone array is close to the second distance between the sound source and the second microphone array, and the first microphone array or the second microphone array is used as the target microphone array.

[0233] It should be noted that the root mean square value of the signal amplitude of the left signal is equal to the root mean square value of the signal amplitude of the right signal and may be approximately equal.

[0234] According to an embodiment of the present application, by calculating the root mean square value of the signal amplitude of the left signal and the root mean square value of the signal amplitude of the right signal, the microphone array corresponding to the larger root mean square value is used as the target microphone array. Compared with the method of directly determining the target microphone array by the signal amplitude size, the target microphone array closer to the sound source can be determined more accurately, thereby improving the accuracy of sound source positioning.

[0235] Since the inter-aural level difference is frequency-dependent, that is, since low-frequency sound has a longer wavelength and is more likely to bypass the head, the inter-aural level difference is smaller and more difficult to compare, while high-frequency sound has a shorter wavelength and is more likely to be blocked by the head, resulting in a larger inter-aural level difference and easier to compare. Therefore, in order to improve the accuracy of determining the target microphone array based on the inter-aural level difference, when determining the target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal in step S3200.1, the low-frequency sound in the left signal and the right signal can be first filtered out, and the high-frequency sound is retained.

[0236] Based on this, in some embodiments, determining the target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal in step S3200.1 includes: step SB1 and step SB2.

[0237] Step SB1, performing a fourth bandpass filtering on the left signal and the right signal within a fourth set frequency range to obtain a filtered left signal and a filtered right signal.

[0238] In this embodiment, the fourth set frequency range may be, for example, 3000-6000 Hz, or may be other high frequency ranges with a frequency greater than 2000 Hz, which is not limited here.

[0239] In one embodiment, the fourth set frequency range is 3000-6000 Hz.

[0240] In this embodiment, the inventors found that when a 3000Hz sound signal is transmitted between the two ears, the level difference between the ears is more obvious, and the higher the frequency, the greater the level difference between the ears. However, since the wavelength of a sound signal greater than 6000Hz is small, it is easily interfered by other factors to produce interference cancellation. Therefore, for robustness, sound signals with frequencies greater than 6000Hz and sound signals with frequencies less than 3000Hz are not considered.

[0241] Step SB2, determining a target microphone array according to the signal amplitude of the filtered left signal and the signal amplitude of the filtered right signal.

[0242] In this embodiment, this step is the same as the step S3200.1 described above of determining the target microphone array according to the signal amplitude of the left signal and the signal amplitude of the right signal, and will not be repeated here.

[0243] Step S3300: Acquire a fifth sound signal obtained by the target microphone array collecting the external sound signal.

[0244] In this embodiment, the fifth sound signal obtained by the target microphone collecting the external sound signal can be directly obtained, or after the approximate sound source direction of the external sound signal (i.e., the second sound source direction) is determined through beamforming by the target microphone array, the fifth sound signal obtained by the target microphone array collecting the external sound signal after performing beamforming in the second sound source direction can be obtained. There is no limitation here.

[0245] In some implementations, in order to improve the signal-to-noise ratio of the fifth sound signal collected by the target microphone array, that is, to make the sound components in the fifth sound signal come more from the sound source of the external sound signal, the step S3200 of obtaining the fifth sound signal obtained by the target microphone array collecting the external sound signal includes:

[0246] A fifth sound signal is obtained by collecting the external sound signal after the target microphone array performs beamforming in the second sound source direction.

[0247] In this embodiment, when the sound source direction of the external sound signal is unknown, beamforming is performed through the target microphone array to determine the second sound source direction (i.e., the approximate sound source direction) of the sound source of the external sound signal relative to the target microphone array. That is, the second sound source direction is the sound source direction of the external sound signal determined by beamforming relative to the target microphone array.

[0248] The specific method of beamforming here may be initial scanning, fan-shaped or cone scanning, adaptive beamforming, multi-beam parallel processing, etc., which is not limited here.

[0249] The initial scan may be to perform beamforming in multiple directions in sequence to detect and identify the direction in which the sound source may exist, which is the second sound source orientation. The fan-shaped or conical scan may be to gradually narrow the search range until the orientation of the sound source signal is detected, that is, the second sound source orientation.

[0250] Those skilled in the art should understand that the specific method of beamforming involved above is well known in the art and will not be elaborated here.

[0251] After the second sound source orientation is determined, beamforming of the second sound source orientation is performed through the target microphone array, and a fifth sound signal of the external sound signal collected by the target microphone is obtained.

[0252] After the target microphone array performs beamforming for the second sound source orientation, the sound signal corresponding to the second sound source orientation in the fifth sound signal collected by the target microphone array will be enhanced, while the sound signals in other directions will be suppressed. Thus, the accuracy of determining the orientation of the sound source of the external sound signal relative to the first sound source of the target microphone array based on the fifth sound signal will be improved.

[0253] Step S3400: Determine, according to the fifth sound signal, the target sound source orientation of the sound source of the external sound signal relative to the target microphone array as the first sound source orientation of the sound source of the external sound signal.

[0254] In some embodiments, the target microphone array includes a first microphone and a second microphone, and the fifth sound signal includes a first sub-signal collected by the first microphone and a second sub-signal collected by the second microphone. In step S3400, the orientation of the sound source of the external sound signal relative to the first sound source of the target microphone array is determined based on the fifth sound signal, including: step S3400.1 to step S3400.2.

[0255] Step S3400.1: determine a third time difference according to the first divided signal and the second divided signal.

[0256] In this embodiment, the location of the sound source can be performed by taking the first sub-signal collected by the first microphone and the second sub-signal collected by the second microphone in the target microphone array as an example to illustrate step S3400 of the present application.

[0257] First, the time difference between when the external sound signal reaches the first microphone and when the external sound signal reaches the second microphone, ie, the third time difference, is determined according to the first split signal and the second split signal.

[0258] In some embodiments, determining the third time difference according to the first divided signal and the second divided signal in step S3400.1 includes: steps SC1 and SC2.

[0259] Step SC1, performing cross-correlation analysis on the first sub-signal and the second sub-signal to determine a fourth time difference.

[0260] This step is basically the same as the above step S1200 and will not be elaborated here.

[0261] Step SC2: determining the third time difference according to the fourth time difference and the sampling rate of the fifth sound signal.

[0262] This step is basically the same as the above step S1300.1 and will not be elaborated here.

[0263] In order to avoid affecting the accuracy of determining the third time difference due to the fact that the wavelength of the sound waves in the first sub-signal and the second sub-signal is smaller than the distance between the first microphone and the second microphone, in some embodiments, step SC1 performs cross-correlation analysis on the first sub-signal and the second sub-signal to determine the fourth time difference, including: step SC11 and step SC12.

[0264] Step SC11: performing a third bandpass filter on the first divided signal and the second divided signal within a third set frequency range to obtain a filtered first divided signal and a filtered second divided signal.

[0265] In this embodiment, the third set frequency range may be, for example, 100 Hz to 1500 Hz, or may be other frequency ranges less than 1500 Hz, which is not limited here.

[0266] In one embodiment, the third set frequency ranges from 100 Hz to 1500 Hz.

[0267] In this embodiment, the inventors have found that the wavelength of sound waves with a frequency above 1.5 kHz is smaller than the distance between the first microphone and the second microphone, which seriously affects the determination of the third time difference. Therefore, the third set frequency range can be set to 100 Hz to 1500 Hz.

[0268] In another embodiment, the lower limit value of the fourth set frequency range is greater than the upper limit value of the third set frequency range.

[0269] In this embodiment, the fourth set frequency range may be 3000-6000 Hz, and the third set frequency range may be 100 Hz to 1500 Hz.

[0270] Step SC12: performing cross-correlation analysis on the filtered first sub-signal and the filtered first sub-signal to determine a fourth time difference.

[0271] This step is the same as the above step SC1 and will not be repeated here.

[0272] Step S3400.2: determine the first sound source direction according to the third time difference and the target microphone distance.

[0273] In this embodiment, when the sound source is far away, it can be considered that the sound wave propagates in the form of a plane wave, and the direction of the plane wave is perpendicular to the direction of the wavefront. Fig. 9 As shown, let the angle between the incoming wave direction of the plane wave and the line connecting the two array elements (i.e., the line connecting the first microphone and the second microphone) be θ, that is, the first sound source orientation is θ, and the distance between the first microphone and the second microphone is d, that is, the target microphone spacing is d. The time difference between the wavefront being transmitted to the first microphone and the second microphone is the third time difference τ, and the wave speed is c. Then, the first sound source orientation θ can be calculated by the following formula (3):

[0274] cos(θ)=cτ / d (3)

[0275] According to the embodiment of the present application, beamforming is performed by the target microphone array to determine the second sound source orientation (i.e., the approximate sound source orientation), and then the third time difference is determined by the first sub-signal and the second sub-signal collected after beamforming of the second sound source orientation by the target microphone array, and the first sound source orientation is further determined based on the third time difference and the target microphone spacing. By determining the sound source orientation twice, the positioning accuracy of the sound source orientation can be improved.

[0276] <Device Example>

[0277] Fig.10 It is a structural diagram of a sound signal processing device according to an embodiment of the present application.

[0278] like Fig.10 As shown, the sound signal processing device 1000 includes a memory 1010 and a processor 1020, wherein the memory 1010 is used to store executable instructions; the processor 1020 is used to operate according to the control of the instructions to execute the method described in any of the above method embodiments.

[0279] How instructions control the operation of a processor is well known in the art and will not be described in detail here.

[0280] Fig.11 It is a schematic diagram of the structure of an earphone according to an embodiment of the present application.

[0281] like Fig.11 As shown, the earphone 110 includes at least one microphone 1101 and a sound signal processing device 1102 .

[0282] In some embodiments, the earphone includes a left earphone and a right earphone, and the left earphone and the right earphone are respectively provided with at least one microphone.

[0283] In this embodiment, the earphone may be a head-mounted earphone or an in-ear earphone.

[0284] In some embodiments, the earphone includes a first microphone array disposed on a left earphone and a second microphone array disposed on a right earphone.

[0285] In this embodiment, the earphone is a headset.

[0286] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0287] A computer-readable storage medium may be a tangible device that can hold and store instructions used by an instruction execution device. A computer-readable storage medium may be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples of computer-readable storage media (a non-exhaustive list) include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a raised structure in a groove on which instructions are stored, and any suitable combination of the foregoing. As used herein, a computer-readable storage medium is not to be interpreted as a transient signal per se, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (e.g., a light pulse through a fiber optic cable), or an electrical signal transmitted through a wire.

[0288] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0289] The computer program instructions for performing the operation of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed entirely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be personalized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present invention.

[0290] Various aspects of the present invention are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present application. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0291] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0292] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0293] The flow charts and block diagrams in the accompanying drawings show the possible architecture, functions and operations of the systems, methods and computer program products according to multiple embodiments of the present invention. In this regard, each box in the flow chart or block diagram can represent a part of a module, a program segment or an instruction, and a part of the module, a program segment or an instruction contains one or more executable instructions for realizing the specified logical function. In some alternative implementations, the functions marked in the box can also occur in a different order from the order marked in the accompanying drawings. For example, two consecutive boxes can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that it is equivalent to implement it by hardware, implement it by software, and implement it by combining software and hardware.

[0294] Embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the marketplace, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein. The scope of the present invention is defined by the appended claims.

Claims

1. A sound signal processing method, characterized in that: The sound signal processing method is applied to any earphone, wherein the earphone is provided with at least one microphone, and the method comprises: Acquire a first sound signal obtained by collecting an external sound signal by at least one microphone of the headset, and acquire a first sound source position of a sound source of the external sound signal; Performing convolution processing on the first sound signal according to a first head transfer function impulse response corresponding to the first sound source orientation to obtain a second sound signal; Determining amplitude information and phase information of the second sound signal according to the second sound signal; performing amplitude adjustment processing on the amplitude information of the second sound signal according to a first head transfer function corresponding to the first sound source orientation to obtain adjusted amplitude information; A target sound signal is obtained according to the adjusted amplitude information and the phase information.

2. The method according to claim 1, characterized in that The first head transfer function impulse response includes a first impulse response corresponding to a plurality of reference sampling points at a set sampling frequency, the first sound signal includes first sound data corresponding to a plurality of actual sampling points at the set sampling frequency, and the first sound signal is convolved according to the first head transfer function impulse response corresponding to the first sound source orientation to obtain a second sound signal, including: For any actual sampling point among the multiple actual sampling points, selecting a first impulse response corresponding to a reference sampling point matching the actual sampling point from the first head transfer function impulse response as the first impulse response corresponding to the actual sampling point; According to the first impulse response corresponding to the actual sampling point, convolution processing is performed on the first sound data corresponding to the actual sampling point to obtain convolution sound data corresponding to the actual sampling point; The second sound signal is determined according to the convolved sound data corresponding to each of the actual sampling points.

3. The method according to claim 1, characterized in that Determining amplitude information and phase information in the second sound signal according to the second sound signal includes: The second sound signal is subjected to time domain-frequency domain conversion processing to determine amplitude information and phase information in the second sound signal.

4. The method according to claim 1, characterized in that: The adjusted amplitude information is the adjusted frequency domain amplitude information, the phase information is the frequency domain phase information, and obtaining the target sound signal according to the adjusted amplitude information and the phase information of the second sound signal includes: Determining an adjusted frequency domain sound signal according to the adjusted frequency domain amplitude information and the frequency domain phase information; The adjusted frequency domain sound signal is subjected to frequency domain-time domain conversion processing to obtain a target sound signal.

5. The method according to claim 1, characterized in that The amplitude information includes measured amplitude values ​​of the second sound signal at multiple second frequencies, the first head transfer function includes reference amplitude values ​​at multiple first frequencies, and the amplitude adjustment processing is performed on the amplitude information of the second sound signal according to the first head transfer function corresponding to the first sound source orientation to obtain the adjusted amplitude information, including: For any second frequency, determining a measured amplitude value and a reference amplitude value corresponding to the second frequency according to the amplitude information and the first head transfer function; wherein the reference amplitude value corresponding to the second frequency is an amplitude value of the first head transfer function when the first frequency is equal to the second frequency; Determine the target amplitude value corresponding to the second frequency according to the measured amplitude value and the reference amplitude value corresponding to the second frequency, and obtain the target amplitude value corresponding to each second frequency; wherein the target amplitude value corresponding to the second frequency is the amplitude difference between the measured amplitude value and the reference amplitude value corresponding to the second frequency; The adjusted amplitude information is determined according to the target amplitude value corresponding to each second frequency.

6. The method according to claim 1, characterized in that The first head transfer function impulse response is obtained by performing frequency domain-time domain conversion processing on the first head transfer function, and the first head transfer function is obtained by performing time domain-frequency domain conversion processing on the first head transfer function impulse response.

7. The method according to claim 1, characterized in that The earphone comprises a left earphone and a right earphone, and the left earphone and the right earphone are respectively provided with at least one microphone, and the first sound source orientation of the sound source of the external sound signal is obtained, including: Acquire a third sound signal obtained by collecting external sound signals by two symmetrical microphones in each ear; wherein the third sound signal includes a left sound signal collected by the microphone arranged at the position of the left earphone and a right sound signal collected by the microphone arranged at the position of the right earphone; Performing cross-correlation analysis on the left sound signal and the right sound signal to determine a first time difference between the left sound signal and the right sound signal; Determine, according to the first time difference, candidate coordinate values ​​of the sound source of the external sound signal relative to a headphone coordinate system; wherein the headphone coordinate system is a coordinate system constructed based on the left headphone and the right headphone; According to the root mean square value of the amplitude of the left sound signal and / or the root mean square value of the amplitude of the right sound signal, a target coordinate value is selected from the candidate coordinate values ​​as the first sound source orientation of the sound source of the external sound signal.

8. The method according to claim 1, characterized in that: The earphone comprises a first microphone array provided in the left earphone and a second microphone array provided in the right earphone, and the first sound source orientation of the sound source for obtaining the external sound signal comprises: Acquire a fourth sound signal obtained by collecting external sound signals with two symmetrical microphones at each ear; Determine a target microphone array according to the fourth sound signal; wherein the target microphone array is one of the first microphone array and the second microphone array; Acquire a fifth sound signal obtained by collecting the external sound signal after the target microphone array performs beamforming in the second sound source orientation; According to the fifth sound signal, a first sound source orientation of a sound source of the external sound signal relative to the target microphone array is determined.

9. A sound signal processing device, characterized in that: The invention comprises a memory and a processor, wherein the memory is used to store executable instructions; and the processor is used to operate according to the control of the instructions to execute the method according to any one of claims 1 to 8.

10. An earphone, characterized in that: The earphone comprises at least one microphone and the sound signal processing device as claimed in claim 9.

Images