Sound signal processing method, device, headphone device and storage medium

Abstract

The present invention discloses a method and apparatus for processing sound signals, a headphone device, and a storage medium. The method for processing sound signals is applied to the headphone device, and a microphone array is provided in the headphone device. The microphone array includes at least one microphone respectively provided in the left earphone and the right earphone of the headphone device. The method for processing sound signals includes the following steps: obtaining an external sound signal through the microphone array, and determining a first sound source orientation of the sound source of the external sound signal relative to the headphone device; obtaining a target head-related transfer function corresponding to the first sound source orientation from head-related transfer functions of various orientations preset; and loading the target head-related transfer function into the external sound signal and outputting the signal through a speaker of the headphone device. The present invention realizes the directional enhancement of the sound signal, enabling the user to subjectively feel that the sound in this direction is clearer, which is beneficial for the user to judge the direction of the external sound.

Description

Technical Field

[0001] The present invention relates to the field of acoustic technologies, and particularly to a method and device for processing sound signals, a headphone device, and a storage medium. Background Art

[0002] In modern life, headphones are widely used as tools for entertainment and communication. When a user wears headphones, they are not sensitive to external sounds and cannot accurately judge the direction from which the sounds come. Therefore, the user cannot respond to external sounds in a timely manner.

[0003] The above content is only used to assist in understanding the technical solution of the present invention, and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of the present invention is to provide a method for processing sound signals, aiming to enhance the user's ability to judge the direction of external sounds when wearing headphones.

[0005] To achieve the above objective, the present invention provides a method for processing sound signals. The method for processing sound signals is applied to a headphone device. A microphone array is provided in the headphone device. The microphone array includes at least one microphone respectively provided in the left headphone and the right headphone of the headphone device. The method for processing sound signals includes the following steps:

[0006] Obtain an external sound signal through the microphone array, and determine a first sound source orientation of the sound source of the external sound signal relative to the headphone device;

[0007] Obtain a target head-related transfer function corresponding to the first sound source orientation from various head-related transfer functions in a preset orientation;

[0008] Load the target head-related transfer function into the external sound signal and output it through a speaker of the headphone device.

[0009] Optionally, the step of loading the target head-related transfer function into the external sound signal and outputting it through a speaker of the headphone device includes:

[0010] Load the first target head-related transfer function corresponding to the left headphone in the target head-related transfer function into the external sound signal corresponding to the left headphone, and then output it through the speaker of the left headphone;

[0011] Load the second target head-related transfer function corresponding to the right headphone in the target head-related transfer function into the external sound signal corresponding to the right headphone, and then output it through the speaker of the right headphone.

[0012] Optionally, the step of determining the first sound source orientation of the sound source of the external sound signal relative to the headphone device includes:

[0013] Determine the first angle of the sound source of the external sound signal relative to the coordinate origin in the first coordinate system, and use the first angle as the first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the first coordinate system is a coordinate system established based on the left earphone and the right earphone;

[0014] The steps of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset include:

[0015] Obtain the first target head-related transfer function of the left earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset;

[0016] Obtain the second target head-related transfer function of the right earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset.

[0017] Optionally, the steps of determining the first sound source orientation of the sound source of the external sound signal relative to the headphone device include:

[0018] Determine the second angle of the sound source of the external sound signal relative to the second coordinate system, determine the third angle of the sound source of the external sound signal relative to the third coordinate system, and use the second angle and the third angle as the first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the second coordinate system is a coordinate system established based on the left earphone, and the third coordinate system is a coordinate system established based on the right earphone;

[0019] The steps of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset include:

[0020] Obtain the first target head-related transfer function corresponding to the second angle from the head-related transfer functions of various orientations preset corresponding to the left earphone, and obtain the second target head-related transfer function corresponding to the third angle from the head-related transfer functions of various orientations preset corresponding to the right earphone.

[0021] Optionally, before the steps of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset, it further includes:

[0022] When detecting a signal input instruction triggered by the prompt information based on the output, pick up the sweep signal through the microphone array, where the prompt information is used to prompt the user to play the sweep signal through the user terminal;

[0023] Determine the second sound source orientation of the sound source of the sweep signal relative to the headphone device;

[0024] Calculate the personalized head-related transfer function corresponding to the azimuth of the second sound source according to the swept-frequency signal, and bind and store the personalized head-related transfer function with the azimuth of the second sound source.

[0025] Optionally, the step of loading the target head-related transfer function into the external sound signal and then outputting it through the speaker of the headphone device includes:

[0026] Analyze the external sound signal in the frequency domain to obtain the first frequency response curve of the external sound signal;

[0027] Load the second frequency response curve into the first frequency response curve and then output it through the speaker of the headphone device, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

[0028] Optionally, the step of loading the target head-related transfer function into the external sound signal and then outputting it through the speaker of the headphone device includes:

[0029] Extract the non-stationary noise signal from the external sound signal;

[0030] Load the target head-related transfer function into the non-stationary noise signal and then output it through the speaker of the headphone device.

[0031] To achieve the above object, the present invention also provides a sound signal processing device, which is deployed in a headphone device. A microphone array is provided in the headphone device. The microphone array includes at least one microphone respectively provided in the left earphone and the right earphone of the headphone device. The sound signal processing device includes:

[0032] A receiving module, configured to obtain an external sound signal through the microphone array and determine the first sound source azimuth of the sound source of the external sound signal relative to the headphone device;

[0033] An obtaining module, configured to obtain a target head-related transfer function corresponding to the first sound source azimuth from the head-related transfer functions of various azimuths preset;

[0034] A loading module, configured to load the target head-related transfer function into the external sound signal and then output it through the speaker of the headphone device.

[0035] To achieve the above object, the present invention also provides a headphone device, which includes: a memory, a processor, and a sound signal processing program stored on the memory and executable on the processor. When the sound signal processing program is executed by the processor, the steps of the above sound signal processing method are implemented.

[0036] In addition, to achieve the above object, the present invention also proposes a computer-readable storage medium, on which a sound signal processing program is stored. When the sound signal processing program is executed by the processor, the steps of the above sound signal processing method are implemented.

[0037] In the present invention, a microphone array is provided in the earphone device, and the microphone array includes at least one microphone provided in the left earphone and the right earphone of the earphone device respectively. The external sound signal is obtained through the microphone array, and the direction of the sound source of the external sound signal relative to the first sound source of the earphone device is determined. From the head-related transfer functions of various preset directions, the target head-related transfer function corresponding to the first sound source direction is obtained, and the target head-related transfer function is loaded into the external sound signal and then outputted by the speaker of the earphone device. The present invention realizes the direction enhancement of the sound signal, so that the user subjectively feels that the sound in the direction is clearer, which is conducive to the user to judge the direction of the external sound. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 It is a flowchart of a first embodiment of a sound signal processing method of the present invention;

[0039] Figure 2 A schematic diagram of a flow chart of a third embodiment of a sound signal processing method according to the present invention;

[0040] Figure 3 This is a schematic diagram of the functional modules of an embodiment of a sound signal processing device of the present invention;

[0041] Figure 4 The present invention is a schematic diagram of the configuration of a microphone array in a headset device according to an embodiment of the present invention.

[0042] The purpose, features and advantages of the present invention will be further described with reference to the accompanying drawings in conjunction with the embodiments. DETAILED DESCRIPTION

[0043] It should be understood that the specific embodiments described herein are only used to explain the present invention, and are not used to limit the present invention.

[0044] Reference Figure 1 , Figure 1 FIG. 4 is a flow chart of a first embodiment of a sound signal processing method according to the present invention.

[0045] The embodiment of the present invention provides a method for processing sound signals. It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described may be performed in a different order than here. The method for processing sound signals in the embodiment of the present invention is applied to a headphone device, and the headphone device may be a headset device, an ear-hook headphone device, an in-ear or semi-in-ear headphone device, etc., which is not specifically limited in this embodiment. A microphone array is provided in the headphone device, and the microphone array includes at least one microphone respectively provided in the left headphone and the right headphone of the headphone device. In this embodiment, the method for processing sound signals includes:

[0046] Step S10: Obtain an external sound signal through a microphone array, and determine a first sound source orientation of the sound source of the external sound signal relative to the headphone device.

[0047] In this embodiment, to solve the problem that the user's judgment of the sound source orientation of the external sound signal is inaccurate, a sound signal processing method is proposed. By loading the head-related transfer function corresponding to the orientation into the external sound signal, the external sound signal can be directionally enhanced, facilitating the user to judge the direction of the external sound.

[0048] Specifically, in this embodiment, the headphone device includes a left earphone and a right earphone. At least one microphone can be respectively arranged in the left earphone and the right earphone. For ease of description, the combination of at least one microphone arranged in the left earphone and the right earphone is hereinafter referred to as a microphone array. In the specific implementation manner, one microphone can be respectively arranged in the left earphone and the right earphone, or multiple microphones can be respectively arranged in the left earphone and the right earphone. These microphones can be the built-in feedforward microphones, feedback microphones or call microphones in the earphones, or additional microphones, which are not specifically limited in this embodiment.

[0049] The headphone device can obtain an external sound signal through each microphone in the microphone array to perform sound source localization based on the microphone array. In the sound source localization technology, based on the differences in the sound signals emitted by the same sound source picked up by multiple microphones in the microphone array on the headphone device from different positions, the orientation of the sound source relative to the headphone device can be determined. Therefore, in this embodiment, a microphone array composed of at least one microphone respectively arranged in the left earphone and the right earphone can be used to determine the orientation of the sound source of the external sound signal relative to the headphone device (hereinafter referred to as the first sound source orientation for distinction) by using the sound source localization technology.

[0050] It should be noted that the external sound signal obtained through the microphone array includes the external sound signals respectively obtained by each microphone in the microphone array. In the following steps, the operation of loading the head-related transfer function can be performed on the external sound signal obtained by one or more microphones according to specific implementation requirements.

[0051] Step S20: Obtain a target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset.

[0052] In the headphone device, head-related transfer functions (HRTFs) for various orientations can be preset. Here, the orientation refers to the orientation of the sound source of external sounds relative to the headphone device, and different orientations correspond to different HRTFs. The process of external sounds propagating from the sound source, being scattered by the human head, pinna, and torso, and reaching both ears can be regarded as an acoustic filtering system. The HRTF is the frequency-domain transfer function of this acoustic filtering system and can be represented by a frequency response curve. In a specific implementation, the HRTF can be obtained through laboratory tests or by the user themselves. Among them, the laboratory test process can be as follows: During the test, a test signal is played through a speaker, and a microphone set at the outer ear of the test subject picks up the test signal. Based on the picked-up test signal, an HRTF is calculated. The orientation corresponding to this HRTF is the orientation of the speaker relative to the microphone at the ear of the test subject. By changing the placement position of the speaker and repeating the test, HRTFs for different orientations can be obtained. It can be understood that when multiple microphones are set at different positions on the same ear or different ears of the test subject, an HRTF can be measured respectively based on the test signals picked up by each microphone. In a specific implementation, different HRTFs corresponding to different orientations can be measured based on only one microphone in the microphone array, that is, one orientation corresponds to only one HRTF, or different HRTFs corresponding to different orientations can be measured respectively based on multiple microphones in the microphone array, that is, one orientation corresponds to a set of HRTFs, and the set of HRTFs includes the HRTFs corresponding to multiple microphones respectively.

[0053] After determining the first sound source orientation, the headphone device can obtain the HRTF corresponding to the first sound source orientation (hereinafter referred to as the target HRTF for distinction) from the preset HRTFs for various orientations. The target HRTF is the HRTF corresponding to the first sound source orientation among the preset HRTFs for various orientations. In a specific implementation, the target HRTF can be an HRTF calculated based on the test signal picked up by one microphone in the microphone array, or a set of HRTFs calculated based on the test signals picked up by multiple microphones in the microphone array respectively.

[0054] Step S30: Load the target HRTF into the external sound signal and then output it through the speaker of the headphone device.

[0055] After the headphone device obtains the target head-related transfer function, it loads the target head-related transfer function into the external sound signal and then outputs it through the speaker of the headphone device. Among them, the method of loading the target head-related transfer function into the external sound signal can be by using the method of frequency response curve superposition. In a specific implementation, the frequency response curve of the target head-related transfer function can be directly superimposed on the frequency response curve of the external sound signal, or the external sound signal can be extracted with non-stationary noise signals and then superimposed on the frequency response curve of the target head-related transfer function. The external sound signal includes the sound signals picked up by multiple microphones respectively. In a specific implementation, the target head-related transfer function corresponding to the corresponding azimuth can be loaded into the external sound signal obtained by one microphone, or the target head-related transfer functions corresponding to the corresponding azimuths can be respectively loaded into the external sound signals obtained by multiple microphones. After loading the target head-related function into the external sound signal, it is output through the speaker corresponding to the microphone in the headphone device.

[0056] It should be noted that loading the target head-related transfer function into the external sound signal enhances the response amplitude of the external sound signal in the azimuth where its sound source is located (i.e., the first sound source azimuth) compared to not loading the target head-related transfer function. Therefore, when the external sound signal loaded with the target head-related transfer function is output, the user can subjectively feel the sound source azimuth of the external sound signal more clearly.

[0057] Furthermore, in an implementation, step S30 includes:

[0058] Step S301, analyze the external sound signal in the frequency domain to obtain the first frequency response curve of the external sound signal;

[0059] After the headphone device obtains the external sound signal, it can analyze the external sound signal in the frequency domain to obtain the frequency response curve of the external sound signal at the user's ear (hereinafter referred to as the first frequency response curve for distinction). In a specific implementation, if only the external sound signal picked up by one microphone is loaded with the head-related transfer function, it can be to analyze the external sound signal picked up by one microphone to obtain a frequency response curve of one microphone. If the external sound signals picked up by multiple microphones are respectively loaded with the head-related transfer function, it can be to analyze the external sound signals picked up by multiple microphones to respectively obtain a group of frequency response curves of multiple microphones, where multiple microphones respectively correspond to one frequency response curve in a group of frequency response curves.

[0060] Step S302, load the second frequency response curve into the first frequency response curve and then output it through the speaker of the headphone device, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

[0061] After the headphone device loads the frequency response curve of the target head-related transfer function (hereinafter referred to as the second frequency response curve for distinction) into the first frequency response curve, it then outputs through a speaker.

[0062] The first frequency response curve and the second frequency response curve are superimposed to obtain a new frequency response curve (hereinafter referred to as the third frequency response curve for distinction). In a specific implementation manner, it can be to superimpose the corresponding points of the first frequency response curve and the second frequency response curve at the same frequency. After converting the third frequency response curve into a sound signal in the time domain, it is output through the speaker of the headphone device.

[0063] It should be noted that compared with the first frequency response curve, the third frequency response curve obtained after superimposing the second frequency response curve has an increased response amplitude at the same frequency, and the user subjectively feels that the external sound signal is enhanced.

[0064] Furthermore, in an implementation manner, step S30 includes:

[0065] Step S303, extracting the non-steady-state noise signal from the external sound signal;

[0066] After the headphone device obtains the external sound signal, it can classify the external sound signal. Among the external sound signals obtained by the headphone device, there are steady-state noise signals and non-steady-state noise signals. The non-steady-state noise signal is extracted, where the non-steady-state noise is noise with a sound wave level greater than a certain value.

[0067] Step S304, loading the target head-related function into the non-steady-state noise signal and then outputting through the speaker of the headphone device.

[0068] The headphone device loads the target head-related function into the non-steady-state noise signal and then outputs through a speaker. In a specific implementation manner, the non-steady-state noise signal can be analyzed in the frequency domain to obtain a frequency response curve, the frequency response curve is superimposed with the second frequency response curve, and after converting the superimposed frequency response curve into a sound signal in the time domain, it is output through the speaker of the headphone device.

[0069] It should be noted that by extracting the non-steady-state noise signal from the external sound signal, loading the target head-related transfer function into the non-steady-state noise signal and then outputting through a speaker, compared with directly loading the head-related transfer function into the external sound signal and then outputting through a speaker, the interference signal in the external sound signal is reduced, which is more conducive to the user to judge the sound source direction of the external sound signal.

[0070] In this embodiment, a microphone array is arranged in the headphone device. The microphone array includes at least one microphone respectively arranged in the left earphone and the right earphone of the headphone device. An external sound signal is acquired through the microphone array, and a first sound source orientation of the sound source of the external sound signal relative to the headphone device is determined. A target head-related transfer function corresponding to the first sound source orientation is acquired from various preset head-related transfer functions in different orientations. A target head-related transfer function corresponding to the first sound source orientation is acquired from various preset head-related transfer functions in different orientations. This embodiment realizes directionally enhancing the sound signal, enabling the user to subjectively feel that the sound in this direction is clearer, which is beneficial for the user to judge the direction of the external sound.

[0071] Further, based on the above first embodiment, a second embodiment of the sound signal processing method of the present invention is proposed. In this embodiment, step S30 includes:

[0072] Step S305: After loading the first target head-related transfer function corresponding to the left earphone in the target head-related transfer function into the external sound signal corresponding to the left earphone, output it using the speaker of the left earphone.

[0073] The headphone device acquires the target head-related transfer function corresponding to the left earphone (hereinafter referred to as the first target head-related transfer function for distinction) from the target head-related transfer function, loads the first target head-related transfer function corresponding to the left earphone into the external sound signal acquired by the left earphone, and then outputs it using the speaker of the left earphone.

[0074] In an implementation manner, after the microphone arranged on the left earphone of the headphone device picks up an external sound signal, the frequency response curve of the external sound signal can be analyzed, that is, the first frequency response curve corresponding to the microphone arranged on the left earphone. The first target head-related transfer function corresponding to the microphone arranged on the left earphone is acquired from the target head-related transfer function, that is, the second frequency response curve corresponding to the microphone arranged on the left earphone. The second frequency response curve corresponding to the microphone arranged on the left earphone is superimposed on the first frequency response curve to obtain the third frequency response curve corresponding to the microphone arranged on the left earphone. After converting the third frequency response curve corresponding to the microphone arranged on the left earphone into a signal in the time domain, it is output using the speaker on the left earphone of the headphone device.

[0075] Step S306: After loading the second target head-related transfer function corresponding to the right earphone in the target head-related transfer function into the external sound signal corresponding to the right earphone, output it using the speaker of the right earphone.

[0076] The headphone device obtains the target head-related transfer function corresponding to the right headphone from the target head-related transfer function (hereinafter referred to as the second target head-related transfer function for distinction), loads the first target head-related transfer function corresponding to the right headphone into the external sound signal obtained by the right headphone, and then outputs it using the speaker of the right headphone.

[0077] In an embodiment, after the microphone provided on the right headphone in the headphone device picks up an external sound signal, the frequency response curve of the external sound signal can be analyzed, that is, the first frequency response curve corresponding to the microphone provided on the right headphone. The first target head-related transfer function corresponding to the microphone provided on the right headphone is obtained from the target head-related transfer function, that is, the second frequency response curve corresponding to the microphone provided on the right headphone. The second frequency response curve corresponding to the microphone provided on the right headphone is superimposed on the first frequency response curve to obtain the third frequency response curve corresponding to the microphone provided on the right headphone. After converting the third frequency response curve corresponding to the microphone provided on the right headphone into a signal in the time domain, it is output using the speaker on the right headphone in the headphone device.

[0078] In this embodiment, by loading the target head-related transfer function corresponding to the left headphone into the external sound signal obtained by the left headphone, and loading the target head-related transfer function corresponding to the right headphone into the external sound signal obtained by the right headphone, and then respectively outputting using the speakers corresponding to the left headphone and the right headphone, the external sound signals of the left headphone and the right headphone are enhanced specifically, making the user subjectively feel a stronger sense of external sound space, and enabling the user to more accurately judge the sound source direction of the external sound signal.

[0079] Further, in an embodiment, the step of determining the first sound source direction of the sound source of the external sound signal relative to the headphone device in step S10 includes:

[0080] Step S101, determining the first angle of the sound source of the external sound signal relative to the coordinate origin in the first coordinate system, and taking the first angle as the first sound source direction of the sound source of the external sound signal relative to the headphone device, where the first coordinate system is a coordinate system constructed based on the left headphone and the right headphone;

[0081] Among the head-related transfer functions of various preset directions, the head-related transfer function corresponding to one direction may specifically include the head-related transfer functions corresponding to the microphones in the left headphone and the right headphone respectively. In a specific embodiment, one direction, that is, the first sound source direction, can be represented by an angle.

[0082] Specifically, when testing the head-related transfer function, a coordinate system can be constructed based on the overall headphone device composed of the left earphone and the right earphone (hereinafter referred to as the first coordinate system for distinction). The angle of the sound source of the test signal relative to the coordinate origin in the first coordinate system is calculated through the test signal obtained by the microphone array, and this angle is used as a direction. The head-related transfer functions of the left earphone and the right earphone calculated respectively based on the test signals picked up by the microphones of the left earphone and the right earphone are used as the head-related transfer functions corresponding to this direction.

[0083] After obtaining the external sound signal through the microphone array, the headphone device can use the sound source localization technology to determine the angle of the external sound signal relative to the coordinate origin in the first coordinate system (hereinafter referred to as the first angle for distinction), and use the first angle as the first sound source direction.

[0084] In the specific implementation manner, the coordinate origin of the first coordinate system can be selected as needed. For example, the midpoint of the left earphone and the right earphone of the headphone device can be set as the coordinate origin of the first coordinate system, or the positive front of the central axis of the left and right earphones can be set as the coordinate origin of the first coordinate system, where the positive front refers to the front direction when the user wears the headphones. Using the sound source localization technology, based on the differences in the external sound signals picked up by the microphones arranged on the left earphone and the right earphone, the first angle of the sound source of the external sound signal relative to the coordinate origin in the first coordinate system can be obtained, and the first angle is used as the first sound source direction.

[0085] Step S20 includes:

[0086] Step S201, obtain the first target head-related transfer function of the left earphone corresponding to the first sound source direction from the head-related transfer functions of various directions preset;

[0087] Obtain the head-related transfer function of the left earphone corresponding to the first sound source direction (that is, the first angle), that is, the first target head-related transfer function, from the head-related transfer functions of various directions preset. In the specific implementation manner, if it is necessary to load the head-related transfer functions for multiple microphones in the left earphone, the external sound signals obtained by the multiple microphones in the left earphone can be respectively loaded with the first target head-related transfer functions corresponding to the microphones. If it is necessary to load the head-related transfer function for one microphone in the left earphone, the external sound signal obtained by this one microphone can be loaded with the corresponding first target head-related transfer function.

[0088] Step S202, obtain the second target head-related transfer function of the right earphone corresponding to the first sound source direction from the head-related transfer functions of various directions preset.

[0089] From the head-related transfer functions in various preset orientations, obtain the head-related transfer function of the right earphone corresponding to the first sound source orientation (i.e., the first angle), that is, the second target head-related transfer function. In a specific implementation, if it is necessary to load the head-related transfer function into multiple microphones in the right earphone, the external sound signals obtained by the multiple microphones in the right earphone can be respectively loaded with the second target head-related transfer function corresponding to each microphone. If it is necessary to load the head-related transfer function into one microphone in the right earphone, the external sound signal obtained by the one microphone can be loaded with the corresponding second target head-related transfer function.

[0090] Further, in an implementation, the step of determining the first sound source orientation of the sound source of the external sound signal relative to the earphone device in step S10 includes:

[0091] Step S102, determine the second angle of the sound source of the external sound signal relative to the second coordinate system, determine the third angle of the sound source of the external sound signal relative to the third coordinate system, and use the second angle and the third angle as the first sound source orientation of the sound source of the external sound signal relative to the earphone device, where the second coordinate system is a coordinate system established based on the left ear mechanism, and the third coordinate system is a coordinate system established based on the right ear mechanism;

[0092] Among the head-related transfer functions in various preset orientations, the head-related transfer function corresponding to one orientation may specifically include the head-related transfer functions corresponding to the microphones in the left earphone and the right earphone respectively. In a specific implementation, one orientation, that is, the first sound source orientation, can be represented by two angles.

[0093] Specifically, when testing the head-related transfer function, coordinate systems can be established based on the left earphone and the right earphone respectively. For the convenience of description, the coordinate system established based on the left ear mechanism is called the second coordinate system, and the coordinate system established based on the right ear mechanism is called the third coordinate system. The angles of the sound source of the test signal relative to the coordinate origin in the second coordinate system and the coordinate origin in the third coordinate system are respectively calculated through the test signals obtained by the microphone array. The two angles can be used as one orientation, and the head-related transfer functions of the left earphone and the right earphone respectively calculated from the test signals picked up by the microphones of the left earphone and the right earphone are used as the head-related transfer functions corresponding to this orientation.

[0094] After obtaining the external sound signal through the microphone array, the earphone device can use the sound source localization technology to determine the angle of the external sound signal relative to the coordinate origin in the second coordinate system (hereinafter referred to as the second angle for distinction), and the angle of the external sound signal relative to the coordinate origin in the third coordinate system (hereinafter referred to as the third angle for distinction), and use the second angle and the third angle as the first sound source orientation.

[0095] In the specific implementation manner, the coordinate origins of the second coordinate system and the third coordinate system can be selected as needed. For example, one of the microphones on the left earphone can be set as the coordinate origin of the second coordinate system, and one of the microphones on the right earphone can be set as the coordinate origin of the third coordinate system. Alternatively, a certain point on the outer shells of the left earphone and the right earphone can be respectively set as the coordinate origins of the corresponding coordinate systems. The earphone device utilizes the sound source localization technology. Based on the differences in the external sound signals picked up by the microphones arranged on the left earphone and the right earphone, the second angle of the sound source of the external sound signal relative to the coordinate origin in the second coordinate system and the third angle relative to the coordinate origin in the third coordinate system can be obtained. The second angle and the third angle are used as the first sound source orientation.

[0096] Step S20 includes:

[0097] Step S203, obtain the first target head-related transfer function corresponding to the second angle from the various azimuth head-related transfer functions preset for the left earphone, and obtain the second target head-related transfer function corresponding to the third angle from the various azimuth head-related transfer functions preset for the right earphone.

[0098] The various azimuth head-related transfer functions preset include the head-related transfer functions preset for various azimuths corresponding to the left earphone and the head-related transfer functions preset for various azimuths corresponding to the right earphone. Obtain the first target head-related transfer function corresponding to the second angle from the head-related transfer functions preset for various azimuths corresponding to the left earphone. Obtain the second target head-related transfer function corresponding to the third angle from the head-related transfer functions preset for various azimuths corresponding to the right earphone.

[0099] Further, based on the above first and / or second embodiments, a third embodiment of the sound signal processing method of the present invention is proposed. In this embodiment, refer to Figure 2 , before step S20, it further includes:

[0100] Step S40, when detecting a signal input instruction triggered by the output prompt information, pick up the sweep signal through the microphone array, where the prompt information is used to prompt the user to play the sweep signal through the user terminal;

[0101] A prompt message can be preset in the headphone device. By outputting this prompt message, the user can be prompted to input a head-related transfer function based on user characteristics (hereinafter referred to as personalized head-related transfer function for distinction) according to their needs. In the specific implementation, the content of the prompt message can be the content for prompting the user to play a swept-frequency signal through the user terminal, without specific limitation. The prompt message can be played by the headphone device through voice, or output through the display device of the user terminal after the headphones are connected to the user terminal, without specific limitation in this embodiment. After the headphone device outputs the prompt message, if a confirmation instruction from the user is received, this confirmation instruction is used as a signal input instruction. According to the signal input instruction, the headphone device picks up the swept-frequency signal played by the user using the user terminal through the microphone array. Among them, the swept-frequency signal can be a signal with continuously changing frequency.

[0102] Step S50: Determine the second sound source orientation of the sound source of the swept-frequency signal relative to the headphone device;

[0103] After the headphone device picks up the swept-frequency signal through each microphone in the microphone array, using sound source localization technology, based on the differences in the swept-frequency signals picked up by multiple microphones in the microphone array on the headphone device from different positions, the orientation of the user terminal that emits the swept-frequency signal relative to the headphone device (hereinafter referred to as the second sound source orientation for distinction) can be determined.

[0104] Step S60: Calculate the personalized head-related transfer function corresponding to the second sound source orientation according to the swept-frequency signal, and bind and store the personalized head-related transfer function with the second sound source orientation.

[0105] The headphone device calculates the user's personalized head-related transfer function according to the swept-frequency signal obtained by the microphone array. Bind the second sound source orientation with the personalized head-related transfer function and store them in the headphone device.

[0106] In a specific implementation, the headphone device calculates a personalized head-related transfer function based on the swept-frequency signals picked up by the microphones in the microphone array. The azimuth corresponding to the personalized head-related transfer function is the azimuth of the second sound source, that is, the azimuth of the user terminal relative to the microphones on the headphone device. By changing the placement position of the user terminal and repeating the test, personalized head-related transfer functions for different azimuths can be obtained. It can be understood that when multiple microphones are set at different positions on the same headphone of the headphone device or multiple microphones are set on different headphones, a personalized head-related transfer function can be measured respectively based on the test signals picked up by each microphone. In a specific implementation, a personalized head-related transfer function corresponding to different azimuths can be measured only based on one microphone in the microphone array, that is, one azimuth corresponds to only one personalized head-related transfer function, or a personalized head-related transfer function corresponding to different azimuths can be measured respectively based on multiple microphones in the microphone array, that is, one azimuth corresponds to a set of personalized head-related transfer functions, and the set of personalized head-related transfer functions includes the personalized head-related transfer functions corresponding to multiple microphones respectively.

[0107] Specifically, the headphone device can calculate the personalized head-related transfer function corresponding to the left headphone based on the swept-frequency signal picked up by the microphone on the left headphone, and calculate the personalized head-related transfer function corresponding to the right headphone based on the swept-frequency signal picked up by the microphone on the right headphone. The head-related transfer function corresponding to the left headphone and the personalized head-related transfer function corresponding to the right headphone can be bound and stored with the azimuth of the second sound source. By receiving the swept-frequency signals sent by the user terminal placed at multiple different angles and repeating the above test operations, personalized head-related transfer functions for various azimuths can be obtained. In a specific implementation, when it is detected that the personalized head-related transfer function corresponding to the current azimuth of the user terminal relative to the headphone device has been recorded, the user can be prompted to move the user terminal to obtain personalized head-related transfer functions for different azimuths.

[0108] In this embodiment, the headphone device obtains the personalized head-related transfer function based on the user characteristics by prompting the user to perform operations, realizes the headphone device to obtain and store the personalized head-related transfer function based on the user's physical characteristics, and the user has a better effect when using the headphone device, which is more conducive to the user to judge the azimuth of the external sound signals.

[0109] Further, in an implementation, among the head-related transfer functions for various preset azimuths, the head-related transfer function corresponding to one azimuth can specifically include the head-related transfer functions corresponding to the microphones in the left headphone and the right headphone respectively. In a specific implementation, one azimuth, that is, the azimuth of the second sound source, can be represented by one angle or two angles.

[0110] Specifically, in one embodiment, when testing the head-related transfer function, a coordinate system can be constructed based on the overall headphone device composed of the left earphone and the right earphone. The angle of the user terminal relative to the coordinate origin in the coordinate system constructed based on the overall headphone device is calculated through the swept-frequency signals obtained by the microphone array. This angle is used as an azimuth, and the head-related transfer functions of the left earphone and the right earphone calculated respectively based on the swept-frequency signals picked up by the microphones of the left earphone and the right earphone are used as the head-related transfer functions corresponding to this azimuth. In another embodiment, coordinate systems can also be constructed based on the left earphone and the right earphone respectively. Through the swept-frequency signals obtained by the microphone array, the angles of the user terminal relative to the coordinate origin in the coordinate system constructed based on the left earphone and the coordinate origin in the coordinate system constructed based on the right earphone are calculated respectively. The above two angles can be used as an azimuth, and the head-related transfer functions of the left earphone and the right earphone calculated respectively based on the swept-frequency signals picked up by the microphones of the left earphone and the right earphone are used as the head-related transfer functions corresponding to this azimuth.

[0111] The present invention also provides a sound signal processing device, which is deployed in a headphone device. A microphone array is provided in the headphone device. The microphone array includes at least one microphone respectively provided in the left earphone and the right earphone of the headphone device. Referring to Figure 3 , the sound signal processing device includes:

[0112] A receiving module 10, configured to obtain an external sound signal through the microphone array and determine a first sound source azimuth of the sound source of the external sound signal relative to the headphone device;

[0113] An obtaining module 20, configured to obtain a target head-related transfer function corresponding to the first sound source azimuth from various head-related transfer functions preset for different azimuths;

[0114] A loading module 30, configured to load the target head-related transfer function into the external sound signal and then output the signal through the speaker of the headphone device.

[0115] Further, the loading module 30 is configured to:

[0116] Load the first target head-related transfer function corresponding to the left earphone in the target head-related transfer function into the external sound signal corresponding to the left earphone, and then output the signal through the speaker of the left earphone;

[0117] Load the second target head-related transfer function corresponding to the right earphone in the target head-related transfer function into the external sound signal corresponding to the right earphone, and then output the signal through the speaker of the right earphone.

[0118] Further, the receiving module 10 is configured to:

[0119] Determine the first angle of the sound source of the external sound signal relative to the coordinate origin in the first coordinate system, and use the first angle as the first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the first coordinate system is a coordinate system established based on the left earphone and the right earphone;

[0120] The obtaining module 20 is configured to:

[0121] Obtain the first target head-related transfer function of the left earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset;

[0122] Obtain the second target head-related transfer function of the right earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset.

[0123] Further, the receiving module 10 is configured to:

[0124] Determine the second angle of the sound source of the external sound signal relative to the second coordinate system, determine the third angle of the sound source of the external sound signal relative to the third coordinate system, and use the second angle and the third angle as the first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the second coordinate system is a coordinate system established based on the left earphone, and the third coordinate system is a coordinate system established based on the right earphone;

[0125] The obtaining module 20 is configured to:

[0126] Obtain the first target head-related transfer function corresponding to the second angle from the head-related transfer functions of various orientations preset corresponding to the left earphone, and obtain the second target head-related transfer function corresponding to the third angle from the head-related transfer functions of various orientations preset corresponding to the right earphone.

[0127] Further, the sound signal processing device further includes:

[0128] A picking-up module, configured to pick up a sweep signal through a microphone array when detecting a signal input instruction triggered by the output prompt information, where the prompt information is used to prompt the user to play the sweep signal through the user terminal;

[0129] A determining module, configured to determine the second sound source orientation of the sound source of the sweep signal relative to the headphone device;

[0130] A calculating module, configured to calculate a personalized head-related transfer function corresponding to the second sound source orientation according to the sweep signal, and bind and store the personalized head-related transfer function with the second sound source orientation.

[0131] Further, the loading module 30 is configured to:

[0132] Analyze the external sound signal in the frequency domain to obtain the first frequency response curve of the external sound signal;

[0133] After loading the second frequency response curve into the first frequency response curve, it is output through the speaker of the headphone device, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

[0134] Furthermore, the loading module 30 is used for:

[0135] Extract the non-stationary noise signal from the external sound signal;

[0136] After loading the target head-related transfer function into the non-stationary noise signal, it is output through the speaker of the headphone device.

[0137] For each embodiment of the sound signal processing device of the present invention, reference can be made to each embodiment of the sound signal processing method of the present invention, which will not be elaborated here.

[0138] In addition, an embodiment of the present invention further provides a headphone device, which includes a structural housing, a communication module, a main control module (such as a microcontroller unit MCU), a speaker, a microphone array, a memory, etc. Refer to Figure 4 , the microphone array may include at least one microphone respectively arranged in the left earphone and the right earphone of the headphone device for picking up external sound signals. The main control module may include a microprocessor, an audio decoding unit, a power supply and a power management unit, sensors required by the system and other active or passive devices, etc. (which can be replaced, deleted or added according to actual functions) to implement the functions of receiving and playing audio. The headphone device can establish a communication connection with the user terminal through the communication module. A sound signal processing program may be stored in the memory of the headphone, and the microprocessor may be used to call the sound signal processing program stored in the memory and perform the following operations:

[0139] Obtain an external sound signal through the microphone array and determine the first sound source orientation of the sound source of the external sound signal relative to the headphone device;

[0140] Obtain the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset;

[0141] After loading the target head-related transfer function into the external sound signal, it is output through the speaker of the headphone device.

[0142] Furthermore, the operation of loading the target head-related transfer function into the external sound signal and then outputting it through the speaker of the headphone device includes:

[0143] After loading the first target head-related transfer function corresponding to the left earphone in the target head-related transfer function into the external sound signal corresponding to the left earphone, it is output through the speaker of the left earphone;

[0144] After loading the second target head-related transfer function corresponding to the right earphone in the target head-related transfer function into the external sound signal corresponding to the right earphone, it is output by the speaker of the right earphone.

[0145] Further, the operation of determining the first sound source orientation of the sound source of the external sound signal relative to the earphone device includes:

[0146] Determine the first angle of the sound source of the external sound signal relative to the coordinate origin in the first coordinate system, and use the first angle as the first sound source orientation of the sound source of the external sound signal relative to the earphone device, where the first coordinate system is a coordinate system constructed based on the left earphone and the right earphone;

[0147] The operation of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset includes:

[0148] Obtain the first target head-related transfer function of the left earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset;

[0149] Obtain the second target head-related transfer function of the right earphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset.

[0150] Further, the operation of determining the first sound source orientation of the sound source of the external sound signal relative to the earphone device includes:

[0151] Determine the second angle of the sound source of the external sound signal relative to the second coordinate system, determine the third angle of the sound source of the external sound signal relative to the third coordinate system, and use the second angle and the third angle as the first sound source orientation of the sound source of the external sound signal relative to the earphone device, where the second coordinate system is a coordinate system constructed based on the left earphone, and the third coordinate system is a coordinate system constructed based on the right earphone;

[0152] The operation of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset includes:

[0153] Obtain the first target head-related transfer function corresponding to the second angle from the head-related transfer functions of various orientations preset corresponding to the left earphone, and obtain the second target head-related transfer function corresponding to the third angle from the head-related transfer functions of various orientations preset corresponding to the right earphone.

[0154] Further, before the operation of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset, the microprocessor can also be used to call the sound signal processing program stored in the memory and perform the following operations:

[0155] When a signal input instruction triggered by an output-based prompt message is detected, a swept-frequency signal is picked up by a microphone array, where the prompt message is used to prompt the user to play the swept-frequency signal through a user terminal;

[0156] Determine the second sound source orientation of the sound source of the swept-frequency signal relative to the headphone device;

[0157] Calculate the personalized head-related transfer function corresponding to the second sound source orientation according to the swept-frequency signal, and bind and store the personalized head-related transfer function with the second sound source orientation.

[0158] Further, the operation of loading the target head-related transfer function into an external sound signal and then outputting it through the speaker of the headphone device includes:

[0159] Analyze the external sound signal in the frequency domain to obtain the first frequency response curve of the external sound signal;

[0160] Load the second frequency response curve into the first frequency response curve and then output it through the speaker of the headphone device, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

[0161] Further, the operation of loading the target head-related transfer function into an external sound signal and then outputting it through the speaker of the headphone device includes:

[0162] Extract the non-stationary noise signal from the external sound signal;

[0163] Load the target head-related transfer function into the non-stationary noise signal and then output it through the speaker of the headphone device.

[0164] For each embodiment of the headphone device of the present invention, reference can be made to each embodiment of the sound signal processing method of the present invention, which will not be elaborated here.

[0165] In addition, an embodiment of the present invention further provides a computer-readable storage medium, on which a sound signal processing program is stored. When the sound signal processing program is executed by a processor, the steps of the above sound signal processing method are implemented.

[0166] For each embodiment of the computer-readable storage medium of the present invention, reference can be made to each embodiment of the sound signal processing method of the present invention, which will not be elaborated here.

[0167] It should be noted that in this document, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, such that a process, method, article or system comprising a series of elements not only includes those elements but also other elements not expressly listed, or elements inherent to such process, method, article or system. Without further limitation, an element defined by the statement "comprising one..." does not exclude the presence of additional identical elements in the process, method, article or system comprising such element.

[0168] The serial numbers of the above embodiments of the present invention are only for description and do not represent the superiority or inferiority of the embodiments.

[0169] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-described embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium as described above (such as ROM / RAM, magnetic disk, optical disc) and includes several instructions to enable a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods described in various embodiments of the present invention.

[0170] The above are only the preferred embodiments of the present invention, and do not limit the patent scope of the present invention accordingly. Any equivalent structural or equivalent process transformation made by using the specification and drawings of the present invention, or directly or indirectly applied in other related technical fields, shall be equally included in the patent protection scope of the present invention.

Claims

1. A method for processing voice signals, characterized in that, The described sound signal processing method is applied to a headphone device, in which a microphone array is provided. The microphone array includes at least one microphone respectively provided in the left headphone and the right headphone of the headphone device. The sound signal processing method includes the following steps: Obtain an external sound signal through the microphone array, and determine a first sound source orientation of the sound source of the external sound signal relative to the headphone device; Obtain a target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset; Load the target head-related transfer function into the external sound signal and output it using the speaker of the headphone device. The steps include: Analyze the external sound signal in the frequency domain to obtain a first frequency response curve of the external sound signal; Load a second frequency response curve into the first frequency response curve and output it using the speaker of the headphone device to enhance the subjective feeling of sound source localization in the first sound source orientation, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

2. The sound signal processing method according to claim 1, characterized in that, The step of loading the target head-related transfer function into the external sound signal and outputting it using the speaker of the headphone device includes: After loading the first target head-related transfer function corresponding to the left headphone in the target head-related transfer function into the external sound signal corresponding to the left headphone, output it using the speaker of the left headphone; After loading the second target head-related transfer function corresponding to the right headphone in the target head-related transfer function into the external sound signal corresponding to the right headphone, output it using the speaker of the right headphone.

3. The sound signal processing method according to claim 2, wherein, The step of determining the first sound source orientation of the sound source of the external sound signal relative to the headphone device includes: Determine a first angle of the sound source of the external sound signal relative to the coordinate origin in a first coordinate system, and use the first angle as the first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the first coordinate system is a coordinate system established based on the left headphone and the right headphone; The step of obtaining the target head-related transfer function corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset includes: Obtain the first target head-related transfer function of the left headphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset; Obtain the second target head-related transfer function of the right headphone corresponding to the first sound source orientation from the head-related transfer functions of various orientations preset.

4. The voice signal processing method according to claim 2, wherein, The step of determining the first sound source orientation of the sound source of the external sound signal relative to the headphone device includes: Determine a second angle of the sound source of the external sound signal relative to a second coordinate system, determine a third angle of the sound source of the external sound signal relative to a third coordinate system, and use the second angle and the third angle as a first sound source orientation of the sound source of the external sound signal relative to the headphone device, where the second coordinate system is a coordinate system built based on the left ear, and the third coordinate system is a coordinate system built based on the right ear; The step of obtaining a target head-related transfer function corresponding to the first sound source orientation from head-related transfer functions of various orientations preset includes: Obtain a first target head-related transfer function corresponding to the second angle from head-related transfer functions of various orientations preset corresponding to the left headphone, and obtain a second target head-related transfer function corresponding to the third angle from head-related transfer functions of various orientations preset corresponding to the right headphone.

5. The sound signal processing method according to claim 1, wherein Before the step of obtaining a target head-related transfer function corresponding to the first sound source orientation from head-related transfer functions of various orientations preset, it further includes: When detecting a signal input instruction triggered by a prompt message based on an output, pick up a sweep signal through the microphone array, where the prompt message is used to prompt the user to play the sweep signal through a user terminal; Determine a second sound source orientation of the sound source of the sweep signal relative to the headphone device; Calculate a personalized head-related transfer function corresponding to the second sound source orientation according to the sweep signal, and bind and store the personalized head-related transfer function with the second sound source orientation.

6. The method for processing a sound signal according to any one of claims 1 to 5, characterized in that, The step of loading the target head-related transfer function into the external sound signal and then outputting it through the speaker of the headphone device includes: Extract the non-stationary noise signal in the external sound signal; Load the target head-related transfer function into the non-stationary noise signal and then output it through the speaker of the headphone device.

7. A voice signal processing device, characterized in that, The sound signal processing device is deployed in a headphone device, a microphone array is provided in the headphone device, the microphone array includes at least one microphone respectively provided in the left headphone and the right headphone of the headphone device, and the sound signal processing device includes: A receiving module, configured to obtain an external sound signal through the microphone array and determine a first sound source orientation of the sound source of the external sound signal relative to the headphone device; An obtaining module, configured to obtain a target head-related transfer function corresponding to the first sound source orientation from head-related transfer functions of various orientations preset; A loading module, configured to load the target head-related transfer function into the external sound signal and then output it through the speaker of the headphone device, and its steps include: Analyze the external sound signal in the frequency domain to obtain a first frequency response curve of the external sound signal; Load a second frequency response curve into the first frequency response curve and then output it through the speaker of the headphone device to enhance the subjective feeling of sound source localization in the first sound source orientation, where the second frequency response curve is the frequency response curve of the target head-related transfer function.

8. An earphone device, characterized in that, The device includes: a memory, a processor, and a sound signal processing program stored on the memory and executable on the processor, the sound signal processing program being configured to implement the steps of the sound signal processing method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that, A sound signal processing program is stored on the storage medium, and when the sound signal processing program is executed by a processor, the steps of the sound signal processing method according to any one of claims 1 to 6 are implemented.

Images