Audio processing method, device, wireless headset and computer readable medium

Abstract

The present application discloses an audio processing method, device, wireless headset and computer-readable medium, and relates to the field of headset technology. The method comprises: determining the spatial position parameters of the wireless headset based on the wireless signal sent by the sound source device, and the spatial position parameters are used to indicate the spatial position relationship between the wireless headset and the sound source device; determining the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain the target spatial audio parameters; determining the audio signal to be played according to the target spatial audio parameters and the audio signal output by the sound source device. The present application determines the spatial position between the wireless headset and the sound source device based on the wireless signal between the two. Compared with image sensors and motion sensors, not only does it not require additional hardware devices to be installed in the wireless headset, that is, it does not increase the cost of the wireless headset, but also the determined spatial position is more accurate.

Description

Technical Field

[0001] The present application relates to the field of earphone technology, and more specifically, to an audio processing method, device, wireless earphone and computer-readable medium. Background Art

[0002] Currently, when users wear headphones, they can combine head tracking technology and spatial sound rendering technology to enable users to feel the location and distance of the sound source device when using headphones, achieving better hearing effects. However, current head tracking technology often uses image sensors or motion sensors installed on the head, which are not very effective. Summary of the invention

[0003] The present application proposes an audio processing method, device, wireless headset and computer-readable medium to improve the above-mentioned defects.

[0004] In a first aspect, an embodiment of the present application provides an audio processing method applied to wireless headphones, the method comprising: determining spatial position parameters of the wireless headphones based on a wireless signal sent by a sound source device, the spatial position parameters being used to indicate the spatial position relationship between the wireless headphones and the sound source device; determining spatial audio parameters of the wireless headphones based on the spatial position parameters to obtain target spatial audio parameters; and determining an audio signal to be played based on the target spatial audio parameters and the audio signal output by the sound source device.

[0005] In a second aspect, an embodiment of the present application further provides an audio processing device, which is applied to wireless headphones, and the device includes: an acquisition unit, a determination unit, and a processing unit. The acquisition unit is used to determine the spatial position parameters of the wireless headphones based on the wireless signal sent by the sound source device, and the spatial position parameters are used to indicate the spatial position relationship between the wireless headphones and the sound source device. The determination unit is used to determine the spatial audio parameters of the wireless headphones based on the spatial position parameters to obtain the target spatial audio parameters. The processing unit is used to determine the audio signal to be played based on the target spatial audio parameters and the audio signal output by the sound source device.

[0006] In a third aspect, an embodiment of the present application also provides a wireless headset, comprising: an audio processing module and a speaker, the wireless communication module is connected to the audio processing module, the wireless communication module is used to obtain the wireless signal sent by the sound source device; the audio processing module is used to determine the audio signal to be played based on the above method.

[0007] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium, wherein the readable storage medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the above method.

[0008] The audio processing method, device, wireless headset and computer-readable medium provided by the present application obtain the audio signal sent by the sound source device based on the wireless communication link between the wireless headset and the sound source device, determine the spatial position parameters between the wireless headset and the sound source device, and then determine the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain the target spatial audio parameters, so that the spatial audio parameters of the wireless headset can be adjusted according to the spatial position between the wireless headset and the sound source device, and then the audio signal is played according to the target spatial audio parameters, so that when the wireless headset plays the audio signal, the audio characteristics corresponding to the audio signal can be related to the spatial position between the wireless headset and the sound source device, so as to achieve the spatial sound rendering effect. In addition, the present application determines the spatial position between the wireless headset and the sound source device based on the wireless signal between the two. Compared with image sensors and motion sensors, not only does it not install additional hardware devices in the wireless headset, that is, it does not increase the cost of the wireless headset, but also the determined spatial position is more accurate.

[0009] Other features and advantages of the embodiments of the present application will be described in the subsequent description, and partly become apparent from the description, or can be understood by practicing the embodiments of the present application. The purposes and other advantages of the embodiments of the present application can be realized and obtained by the structures specifically pointed out in the written description, claims, and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0011] Figure 1 A schematic diagram of the structure of a wireless headset provided in an embodiment of the present application is shown;

[0012] Figure 2 A schematic diagram of an audio circuit of a wireless headset provided in an embodiment of the present application is shown;

[0013] Figure 3 A schematic diagram of an audio processing module of a wireless headset provided in an embodiment of the present application is shown;

[0014] Figure 4 A schematic diagram of an audio processing module of a wireless headset provided in another embodiment of the present application is shown;

[0015] Figure 5 A schematic diagram of an audio processing module of a wireless headset provided in another embodiment of the present application is shown;

[0016] Figure 6 A method flow chart of an audio processing method provided by an embodiment of the present application is shown;

[0017] Figure 7 A schematic diagram of a sound source device provided in an embodiment of the present application is shown;

[0018] Figure 8 A method flow chart of an audio processing method provided by another embodiment of the present application is shown;

[0019] Fig. 9 A schematic diagram showing the time difference of sound reaching the left and right ears provided by an embodiment of the present application;

[0020] Fig.10 A method flow chart of an audio processing method provided by another embodiment of the present application is shown;

[0021] Fig.11 A schematic diagram showing the angle of arrival provided by an embodiment of the present application is shown;

[0022] Fig.12 A method flow chart of an audio processing method provided by yet another embodiment of the present application is shown;

[0023] Fig.13 A method flow chart of an audio processing method provided by yet another embodiment of the present application is shown;

[0024] Fig.14 A schematic diagram of a reverberant sound field provided by an embodiment of the present application is shown;

[0025] Fig.15 A module block diagram of an audio processing device provided in an embodiment of the present application is shown;

[0026] Fig.16 A storage unit provided in an embodiment of the present application and used to store or carry program codes for implementing an audio processing method according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0028] At present, when users wear headphones, the combination of head tracking technology and spatial sound rendering technology can enable users to feel the position and distance of the sound source device when using headphones, achieving better hearing effects.

[0029] For example, head tracking is implemented through an image sensor and a pre-created head-related transfer function (HRTF) HRTF database and filter are used to filter the 3D audio source for more realistic audio rendering. For another example, a head-based device (e.g., a digital gyroscope) is set on the headset, and the head tracking angle can be determined based on sensor data obtained from the digital gyroscope installed in the headphone assembly, and then a preset HRTF is selected to implement a binaural spatial sound filter to present a stable stereo image.

[0030] However, the inventors found in their research that current head tracking technology often uses image sensors or motion sensors installed on the head, which are not very effective. Specifically, the technology of using the camera of an electronic device to capture images of the surrounding environment and obtain the head position and posture information will first increase the power consumption of the electronic device and reduce the battery life; secondly, the accuracy of head rotation orientation recognition is affected by the camera clarity and image recognition algorithm. At the same time, the distance between the audio and video equipment and the headphone wearer cannot be calculated only by the camera and the orientation recognition algorithm. The above factors lead to poor rendering of spatial sound effects, affecting the user experience.

[0031] In addition, the method of using motion sensors to achieve head tracking, motion sensors mainly include accelerometers, gyroscopes and magnetic sensors, etc. These sensors have inherent disadvantages in motion tracking and angular direction. For example, the accelerometer provides a gravity vector, while the magnetometer is a compass. The information of these two sensors can be used to calculate the direction of the device. However, the output of these two sensors is not accurate and contains a lot of noise; while the gyroscope provides angular velocity along three axes. The information provided is very accurate and responds quickly, but drift errors will occur over a long period of time. The reason is that the angular velocity needs to be integrated to obtain the direction information, and the integration process will lead to small numerical errors. The accumulation of errors over a long period of time will form a more obvious drift.

[0032] Furthermore, when using headphones to listen to virtual surround sound, when the head rotates, the virtual surround sound in the headphones will rotate with the head, resulting in a different feeling when listening to music live. The virtual surround sound cannot make the user feel the distance between the person and the audio and video playback device, and the spatial sound rendering is not realistic enough.

[0033] Therefore, in order to overcome the above-mentioned defects, the embodiments of the present application provide an audio processing method, device, wireless headset and computer-readable medium, which can determine the spatial position between the wireless headset and the sound source device based on the wireless signal between the two. Compared with image sensors and motion sensors, not only no additional hardware devices are installed in the wireless headset, that is, the cost of the wireless headset is not increased, but the determined spatial position is more accurate.

[0034] Specifically, to facilitate the description of the method embodiments of the present application, the wireless headset provided in the embodiments of the present application is first introduced. The wireless headset can determine the spatial position parameters between the wireless headset and the sound source device, and can also realize spatial sound rendering to provide changes in sound when the user is in different spatial positions such as angles and distances from the sound source device when wearing the headset.

[0035] See also Figure 1 , Figure 1 A wireless headset 10 provided in an embodiment of the present application is shown, and the wireless headset 10 includes a housing 100 and an audio circuit 200 and a wireless communication module 300 located in the housing 100. As an implementation, the audio circuit 200 and the wireless communication module 300 are arranged in the housing 100, the audio circuit 200 is used to make a sound based on the audio data to be played to play the audio data, and the wireless communication module 300 is used to establish a wireless communication link between the wireless headset and other electronic devices supporting wireless communication, so that the wireless headset can exchange data with other electronic devices through the wireless communication link. As an implementation, the electronic device can be a device such as a smart phone, a tablet computer, an e-book, etc. that can run audio applications, and the electronic device can be a device that can play audio.

[0036] As an implementation method, Figure 2 As shown, the audio circuit 200 includes an audio processing module 210 , a memory 230 , a speaker 240 and a power supply circuit 220 , and the memory 230 , the speaker 240 and the power supply circuit 220 are all connected to the audio processing module 210 .

[0037] As an implementation mode, the audio processing module 210 is used to set audio parameters and control the speaker 240 to play audio. The audio parameters are used as parameters when the audio data is played. For example, the audio parameters may include volume, sound effect parameters, etc. Specifically, the audio parameters may include multiple sub-parameters, each sub-parameter corresponds to a component of the audio signal to be played, and each of the sub-parameters corresponds to a sound generation module; each of the sound generation modules is used to generate a sound signal based on the audio signal and the sub-parameter corresponding to the sound generation module, and the sound signal generated by each of the sound generation modules is used as the audio signal to be played.

[0038] In some embodiments, the audio signal to be played is composed of direct sound, reflected sound and reverberation sound, and the audio processing module 210 may include a direct sound module, a reflected sound module and a reverberation sound module, the direct sound module is used to output the direct sound based on the direct sound audio parameter; the reflected sound module is used to output the reflected sound based on the reflected sound audio parameter; the reverberation sound module is used to output the reverberation sound based on the reverberation sound audio parameter, and the direct sound, reflected sound and reverberation sound constitute the audio signal to be played. The audio processing module 210 may be a program module in a wireless headset, and the various functions of the audio processing module 210 may be implemented by the program module. For example, the audio processing module may be a program set in the memory of the wireless headset, and the program set can be called by the processor of the wireless headset to implement the functions of the audio processing module, that is, to implement the functions of the method embodiment of the present application.

[0039] In other embodiments, the audio processing module 210 may be a hardware module in a wireless headset, and each function in the audio processing module 210 may be implemented by a hardware circuit, for example, a direct sound module, a reflected sound module, a reverberation sound module, and other subsequent components may be hardware circuits. The audio processing module includes an audio adjuster and a processor, and the audio adjuster is connected to the processor; the processor is used to determine the spatial position parameters of the wireless headset based on the wireless signal sent by the sound source device received by the wireless communication module, and determine the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain the target spatial audio parameters; the audio adjuster is used to determine the audio signal to be played based on the target spatial audio parameters and the audio signal output by the sound source device.

[0040] Specifically, Figure 3 The audio processing module 210 includes: a processor 211, a direct sound module 212, a reflected sound module 213 and a reverberation sound module 214. The direct sound module 212, the reflected sound module 213 and the reverberation sound module 214 are all connected to the processor 211. The processor 211 is used to input the direct sound audio parameters to the direct sound module 212, the reflected sound audio parameters to the reflected sound module 213 and the reverberation sound audio parameters to the reverberation sound module 214. The direct sound module 212 is used to output the direct sound based on the direct sound audio parameters; the reflected sound module 213 is used to output the reflected sound based on the reflected sound audio parameters; and the reverberation sound module 214 is used to output the reverberation sound based on the reverberation sound audio parameters.

[0041] As an implementation, one or more applications may be stored in the memory 203 and configured to be executed by one or more processors 211, and the one or more programs are configured to execute the method described in the embodiments of the present application. For the specific implementation of the method, please refer to the subsequent embodiments.

[0042] The processor 211 may include one or more processing cores. The processor 211 uses various interfaces and lines to connect various parts of the entire electronic device 100, and executes various functions and processes data of the electronic device 100 by running or executing instructions, programs, code sets or instruction sets stored in the memory 203, and calling data stored in the memory 203. Optionally, the processor 211 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 211 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 211, and may be implemented separately through a communication chip.

[0043] The memory 203 may include a random access memory (RAM) or a read-only memory (ROM). The memory 203 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 203 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the terminal 100 during use.

[0044] As an implementation method, Figure 4 As shown, the audio processing module 210 also includes a first mixer 215, the direct sound module 212 includes a delay module 2121, the delay module 2121 is respectively connected to the input end of the reflected sound module 213 and the first input end of the first mixer 215, the output end of the reflected sound module 213 is respectively connected to the input end of the reverberation sound module 214 and the second input end of the first mixer 215, and the output end of the reverberation sound module 214 is connected to the third input end of the first mixer 215.

[0045] The delay module 2121 is used to delay the audio signal based on the direct sound audio parameters to obtain a direct sound signal, thereby simulating the direct sound at different distances and the difference between the direct sounds of the two ears; the reflected sound module 213 is used to adjust the volume and delay the full-frequency band part of the direct sound signal based on the reflected sound audio parameters to obtain a reflected sound signal; the reverberation sound module 214 is used to adjust the volume and delay the specified frequency band part of the reflected sound signal based on the reverberation sound audio parameters to obtain a reverberation sound signal; the first mixer 215 is used to mix the direct sound signal, the reflected sound signal and the reverberation sound signal and output the mixed spatial audio signal.

[0046] like Figure 5 As shown, the reflected sound module 213 includes a first filter group and a second mixer 2132, the first filter includes N parallel first all-pass filters 2131, each of the first all-pass filters 2131 is connected to an input end of the second mixer 2132, and the output end of the second mixer 2132 is respectively connected to the input end of the reverberation sound module 214 and the second input end of the first mixer 215, wherein N is a positive integer, and the first filter group is connected to the delay module 2121. The first all-pass filter 2131 can adjust the gain and delay of the input signal, so as to simulate the reflected sound after the signal output by the sound source device is reflected. The density of the reflected sound can be increased by multiple first all-pass filters 2131, that is, the reflected sound of multiple different reflection lengths and angles can be played. The direct sound output by the delay module 2121 forms a reflected sound after the volume adjustment and delay of the first all-pass filter 2131, and the reflected sound output by the multiple first all-pass filters 2131 is mixed by the second mixer 2132.

[0047] like Figure 5As shown in the figure, the reverberation module 214 includes a low-pass filter 2142 and a second filter bank. The second filter bank includes M second all-pass filters 2141 connected in series. The output end of the reflected sound module 213 is connected to the input end of the low-pass filter 2142 through the second filter bank, and the output end of the low-pass filter 2142 is connected to the third input end of the first mixer 215, where M is a positive integer. As an implementation manner, the output end of the second mixer 2132 is connected to the input end of the second all-pass filter 2141. The reflected sound output by the second mixer 2132 is input into the second filter bank. The second all-pass filter 2141 of the second filter bank is used to form reverberation. The low-pass filter simulates the attenuation of high frequencies in the air, that is, it is used to reduce the amplitude of the high-frequency part in the sound signal. The delay and gain of each all-pass filter can be set according to the situation. When the sampling rate is 44100 Hz, the delay value can be set to 200 - 2000 samples, and the gain range is 0 < g < 1; the delay of the low-pass filter is generally 1 sampling point, that is, a first-order low-pass filter can meet the requirements, and the gain range is 0 < g < 1.

[0048] As Figure 5 shown in the figure, the audio processing module 210 further includes an amplitude modulation module 216. The output end of the first mixer 215 is connected to the input end of the amplitude modulation module 216. The output end of the amplitude modulation module 216 is connected to a speaker, and is used to input the sound signal into the speaker for the speaker to play the sound.

[0049] As an implementation manner, there can be two such wireless earphones, namely a first earphone and a second earphone. For example, the first earphone can be an earphone for the user to wear on the left ear, and the second earphone can be an earphone for the user to wear on the right ear. Then both the first earphone and the second earphone include the above-mentioned hardware structure. The processor can separately adjust the audio parameters of the first earphone and the second earphone, that is, the parameters of the above-mentioned various hardware, so as to achieve the binaural spatial sound rendering effect. Among them, the delay module 2121 is used to simulate the binaural time, and the gain G is used to simulate the binaural sound pressure.

[0050] Specifically, for the implementation principle of the above wireless earphone, please refer to the subsequent method embodiments.

[0051] As Figure 6 shown in the figure, Figure 6 shows an audio processing method. This method is applied to the above wireless earphone. As an implementation manner, the execution subject of this method can be the above-mentioned processor. Specifically, this method includes: S601 to S603.

[0052] S601: Determine the spatial position parameters of the wireless earphone based on the wireless signal sent by the sound source device.

[0053] The spatial position parameter is used to indicate the spatial position relationship between the wireless headset and the sound source device. As an implementation, the sound source device may be an audio playback device, such as Figure 7 As shown, the audio playback device can be a smart phone 20, and the smart phone 20 is connected to the wireless headset via a wireless communication link. The smart phone 20 can send an audio signal to the wireless headset via the wireless communication link, and the wireless headset plays the audio signal, and the user listens to the audio signal through the wireless headset.

[0054] As another embodiment, the sound source device can be a virtual audio playback device. Specifically, a position point is set in the world coordinate system, and it is assumed that an audio playback device is set at the position point. In fact, there is no real audio playback device at the position point, but it is assumed that an audio playback device is set at the position point. When the user wears headphones, the method of the present application can feel that the position of the sound source device corresponding to the sound heard is pointed out at the position point. For example, in a virtual reality scene, a real world coordinate system is established based on the user's position point, and a position point is determined in the world coordinate system as the sound source device position point. Since there is a mapping relationship between the virtual world coordinate system of virtual reality and the real world coordinate system, the position of the sound source device in the virtual world coordinate system can be determined according to the mapping relationship between the sound source device position point and the virtual world coordinate system and the real world coordinate system, so that the user can feel the position of the sound source device in the virtual reality environment. In this embodiment, a positioning device can be set at the sound source device position point, and the positioning device can include a wireless communication device. The wireless headset can be connected to the wireless communication device of the positioning device through the wireless communication device in the wireless headset, thereby realizing the establishment of a wireless communication link between the wireless headset and the positioning device.

[0055] It should be noted that if the sound source device is an audio playback device, the wireless signal sent by the sound source device obtained by the wireless headset and the sound source device through the wireless communication link may be an audio signal or a wireless positioning signal. If the wireless signal is an audio signal, the sound played by the wireless device later is the audio signal. Figure 7 As shown, when the user wears headphones and watches a video played by the smart phone 20, the smart phone 20 sends the audio signal corresponding to the video to the wireless headphones through the wireless communication link between the smart phone 20 and the wireless headphones, so that the user wearing the wireless headphones can listen to the audio content corresponding to the video. The audio signal is not only used as the audio content to be played by the wireless headphones, but also can be used to determine the spatial position parameters between the wireless headphones and the sound source device. If the wireless signal is a wireless positioning signal, the wireless positioning signal can be any wireless signal, not limited to an audio signal.

[0056] As an implementation, the spatial position parameter may include at least one of a distance parameter and an angle of arrival. The strength of the wireless signal from the sound source device to the wireless headset is related to the distance between the two. For example, the greater the distance, the smaller the strength of the wireless signal. The angle of arrival can be determined by the phase difference and distance between wireless signals transmitted by different wireless communication devices (e.g., antennas). For specific acquisition methods, please refer to the subsequent embodiments.

[0057] As an implementation mode, the wireless communication device of the wireless headset may be a Bluetooth device, or of course, a WiFi or other device capable of sending wireless signals, and the wireless communication link between the wireless headset and the sound source device is a Bluetooth communication link.

[0058] S602: Determine spatial audio parameters of the wireless headset based on the spatial position parameters to obtain target spatial audio parameters.

[0059] As an implementation method, the spatial audio parameter includes a gain parameter and a delay length. Among them, the gain parameter is used to affect the playback volume of the wireless headset when playing audio content, that is, the wireless headset controls the playback volume of the wireless headset when playing audio content based on the gain parameter. As an implementation method, the spatial audio parameter can be a volume level, that is, a certain number of volume levels are pre-set, for example, level 1, level 2, level 3, level 4, etc., and the higher the level, the higher the volume. As another implementation method, the spatial audio parameter can be a volume percentage. The higher the volume percentage, the higher the volume. The volume percentage is a percentage of the maximum volume. For example, 80% means 80% of the maximum volume. As another implementation method, the gain parameter can be a sound pressure level. The higher the sound pressure level, the higher the volume.

[0060] The delay length is used to affect the playback time of the audio content played by the wireless headset, that is, the wireless headset determines the length of time to wait for playback based on the delay length, and then controls the wireless headset to play the audio content after waiting for the delay length. Different delay lengths correspond to different playback times of the audio content, and the longer the delay length, the later the playback time.

[0061] As an implementation mode, the spatial position parameter can reflect the distance and angle relationship between the wireless signal and the sound source device, and the distance and angle relationship can affect the volume and playback time of the audio played by the wireless headset. For example, the farther the user is from the sound source device, the smaller and later the sound heard. The adjustment strategy used to determine the spatial audio parameters of the wireless headset based on the spatial position parameter enables the audio content listened to by the user through the wireless headset to have the auditory effect of spatial sound after the sound emitted by the sound source device reaches the human ear after attenuation and delay in the space. For the specific adjustment method, please refer to the subsequent embodiments.

[0062] S603: Determine an audio signal to be played according to the target spatial audio parameter and the audio signal output by the sound source device.

[0063] As an implementation mode, if the sound source device is an audio playback device, the audio signal to be played may be an audio signal sent by the aforementioned sound source device. If the sound source device is a positioning device, the audio signal to be played may be audio data pre-stored in a wireless headset, or may be audio data sent to the wireless headset by other electronic devices. For example, in a virtual reality scenario, a user wears a head-mounted display device, the head-mounted display device includes a wireless headset, and the head-mounted display device is connected to an external terminal or has a video rendering device inside, then the head-mounted display device may store audio data or the head-mounted display device may obtain audio data from the terminal. A positioning device is provided in the real environment corresponding to the virtual reality, and the wireless headset adjusts the spatial audio parameters based on the spatial position between the wireless headset and the positioning device, and adjusts the audio signal based on the spatial audio parameters to obtain the audio data to be played. The audio data is used as the audio signal to be played, so as to simulate the spatial sound formed by the attenuation, reflection and reverberation of the sound emitted at the position of the positioning device and reaches the human ear and is listened to by the human ear.

[0064] Therefore, the spatial audio parameters of the wireless headset are determined by the spatial position parameters, so that when the wireless headset plays an audio signal, the audio characteristics corresponding to the audio signal can be related to the spatial position between the wireless headset and the sound source device, thereby achieving a spatial sound rendering effect. In addition, the present application determines the spatial position between the wireless headset and the sound source device based on the wireless signal between the two. Compared with image sensors and motion sensors, not only does it not require additional hardware devices to be installed in the wireless headset, that is, it does not increase the cost of the wireless headset, but also the determined spatial position is more accurate.

[0065] See also Figure 8 , Figure 8An audio processing method is shown, in which the spatial position parameter includes a distance parameter, and the spatial audio parameter includes a gain parameter and a delay length. As an implementation, the execution subject of the method may be the above-mentioned processor. Specifically, the method includes: S801 to S804.

[0066] S801: Determine a distance parameter between the wireless headset and the sound source device based on a wireless signal sent by the sound source device.

[0067] As an implementation mode, the signal strength of the wireless signal is obtained, and a distance parameter between the wireless headset and the sound source device is determined based on the signal strength, and the distance parameter may be a distance value. The higher the signal strength, the smaller the distance between the wireless headset and the sound source device, and the lower the signal strength, the larger the distance between the wireless headset and the sound source device, that is, the signal strength is negatively correlated with the distance.

[0068] As an implementation method, a multi-point positioning algorithm based on the received signal strength (RSSI) value calculates the distance between the Bluetooth signal transmitter and the receiver through a mathematical relationship according to the processed RSSI value and the signal attenuation model, thereby realizing the measurement of the signal strength into the distance. Specifically, the distance parameter is obtained according to the following formula:

[0069]

[0070] Wherein, d is the distance between the wireless headset and the sound source device, in meters, RSSI is the received signal strength of the wireless signal, abs(RSSI) is the absolute value of RSSI, A is the received signal strength of the receiving end when the Bluetooth transmitter and the receiver are 1 meter apart, n is the environmental attenuation factor, A and n are obtained by repeated experiments and comparison with the actual distance. According to the above formula (1), the distance between the sound source device and the wireless headset (i.e., the human ear) can be obtained for processing such as delay and volume adjustment of spatial sound rendering.

[0071] S802: Determine the gain parameter based on the negative correlation between the distance parameter and the gain parameter to obtain a target gain parameter.

[0072] The negative correlation between the distance parameter and the gain parameter means that the distance parameter is inversely proportional to the gain parameter. The distance parameter is the distance value, and the gain parameter is the volume value. The larger the distance value, the smaller the volume value, and the smaller the distance value, the larger the volume value.

[0073] As an implementation method, a first correspondence between a distance parameter and a gain parameter may be preset, and in the first correspondence, the distance parameter and the gain parameter are negatively correlated. After determining the distance parameter between the wireless headset and the sound source device, the distance parameter is used as a target distance parameter, and a gain parameter corresponding to the target distance parameter is searched in the first correspondence to obtain a target gain parameter.

[0074] As another implementation, a distance-volume relationship may be set to determine the gain parameter. In this relationship, the larger the distance parameter is, the smaller the spatial audio parameter is, that is, the distance parameter is negatively correlated with the gain parameter. As an implementation, a change rule between the distance and the gain may be predetermined, and the change rule includes a relationship between a distance change value and a gain change value. For example, for every increase in distance D, the gain decreases by g. Based on the change rule, the gain parameter corresponding to the current distance parameter may be determined.

[0075] In order to avoid the volume being too loud when the distance is too close, a distance threshold can be set. If the distance parameter is less than the distance threshold, the gain parameter determined based on the first corresponding relationship or the above distance-volume relationship is used as the initial gain parameter. Then, the initial gain parameter is reduced by a first specified value to obtain a target gain parameter. If the distance parameter is greater than the distance threshold, the gain parameter is determined based on the first corresponding relationship or the above distance-volume relationship as the target gain parameter. The distance threshold can be set based on experience. If the distance parameter is less than the distance threshold, it indicates that the distance between the wireless headset and the sound source device is too close, so that the sound source device can simulate the auditory effect that the sound source device will reduce the volume when the user is close to the sound source device. For example, when two users are communicating, the closer the distance between the two is, the lower the speaker's voice will be. In addition, it can also avoid that when the gain parameter is adjusted by distance, the gain parameter increases too much as the distance decreases, resulting in a poor user experience. The first specified value can be a pre-set empirical value. In addition, when the distance parameter is less than the distance threshold, the smaller the distance, the larger the first specified value, that is, the distance is negatively correlated with the first specified value.

[0076] As an implementation mode, the gain parameter may be a gain parameter of an amplitude modulation module in the above-mentioned audio processing circuit, and of course, may also include gain parameters of various filters in the above-mentioned audio processing circuit, wherein the gain parameter of the amplitude modulation module and the gain parameter of the all-pass filter can adjust the volume of the entire frequency band of the audio signal, and the gain parameter of the low-pass filter can adjust the volume of the high-frequency part of the audio signal. For example, adjusting the gain value of the low-pass filter can change the frequency response curve of the low-pass filter, thereby simulating the situation that high-frequency sound decays faster than low-frequency sound in the air, that is, high-frequency attenuation damping.

[0077] In addition, the gains of the filters in the reflected sound module 213 and the reverberation sound module 214 are also used to achieve the effects of reflected sound and reverberation sound, which will be specifically described in the subsequent embodiments.

[0078] S803: Determine the delay length based on the positive correlation between the distance parameter and the delay length to obtain a target delay length.

[0079] The positive correlation between the distance parameter and the delay length means that the distance parameter is proportional to the delay length, and the distance parameter is the distance value. The larger the distance value, the larger the delay length, and the smaller the distance value, the smaller the delay length. In other words, the smaller the distance, the earlier the sound can be heard.

[0080] As an implementation mode, a second correspondence between the distance parameter and the delay length may be preset, and in the second correspondence, the distance parameter and the delay length are positively correlated. After determining the distance parameter between the wireless headset and the sound source device, the distance parameter is used as the target distance parameter, and the delay length corresponding to the target distance parameter is searched in the second correspondence.

[0081] As another implementation, a preset relationship may be used to determine the delay length. The preset relationship is as follows:

[0082]

[0083] Among them, M is the delay length, d is the distance value, v is the sound propagation speed of 340m / s, fs is the signal processing sampling rate, the calculation of d can refer to the above content, and the delay length of M is measured by the number of sampling points. For example, if M is 2, it means 2 sampling points.

[0084] S804: Play the audio signal to be played according to the target spatial audio parameter.

[0085] The target gain parameter and the target delay length are used as the target spatial audio parameters.

[0086] As an implementation method, the number of wireless headphones can be 1, and the user can wear the wireless headphone on one ear. The wireless headphone adjusts the volume and playback time of the played audio signal according to the distance parameter. Therefore, when the user wears the headphone on one ear, he can also feel the volume and delayed auditory effects of the audio signal brought about by the change of the distance between the user and the sound source device.

[0087] As another embodiment, the number of the wireless earphones is two, namely the first earphone and the second earphone. The wireless earphone adjusts the spatial audio parameters of the first earphone according to the spatial position parameters corresponding to the first earphone to obtain the first target spatial audio parameters; adjusts the spatial audio parameters of the first earphone according to the spatial position parameters corresponding to the second earphone to obtain the second target spatial audio parameters; and controls the first earphone and the second earphone to play the audio signal based on the first target spatial audio parameters and the second target spatial audio parameters. Therefore, the first earphone and the second earphone can not only adjust the volume and delayed auditory effects of each earphone according to the distance value of each earphone, but also can achieve the binaural effect of binaural time difference and volume difference.

[0088] Specifically, people's judgment of the direction of sound is mainly affected by factors such as time difference, sound pressure difference, human body filtering effect and head rotation. The sound signal is transmitted from the sound source device to the comprehensive filtering process of the two ears, which includes air filtering, reverberation of the surrounding environment, and scattering and reflection of the human body (torso, head, auricle, etc.).

[0089] like Fig. 9 As shown, the distances between the audio playback device 20 and the left ear and the right ear of the user are different. Therefore, when the audio playback device 20 is played in public, the time lengths for the sound emitted by the audio playback device 20 to reach the left ear and the right ear are different. The right ear hears the sound before the left ear, that is, according to the different distances between the sound source device and the two ears, there will be a difference in the time for the sound to reach the left and right ears. This difference is called the time difference. In addition, the right ear is closer to the audio playback device 20 than the left ear, so the volume of the sound heard by the right ear should be higher than the volume of the sound heard by the left ear. It is assumed that there are two wireless headphones, namely the first headphone 201 and the second headphone 202. The user wears the first headphone 201 on the left ear and the second headphone 202 on the right ear. The distance parameter between the sound source device and the first headphone 201 is named the first distance value, and the distance parameter between the sound source device and the second headphone 202 is named the second distance value. The first distance value is greater than the second distance value. The first target spatial audio parameter corresponding to the first distance value includes a first gain parameter and a first delay length, and the second target spatial audio parameter corresponding to the second distance value includes a second gain parameter and a second delay length. Among them, the first gain parameter is smaller than the second spatial audio parameter, so the sound heard by the left ear is smaller than the sound heard by the right ear, thereby forming a volume difference between the two ears, that is, a sound level difference. The first delay length is greater than the second delay length, and the right ear hears the sound first than the left ear, thereby forming a time difference between the two ears.

[0090] See also Fig.10 , Fig.10An audio processing method is shown, in which the spatial position parameters include the arrival angle, and the spatial audio parameters include the gain parameter and the delay length. As an implementation, the execution subject of the method may be the above-mentioned processor. Specifically, the method includes: S1001 to S1003.

[0091] S1001: Determine an arrival angle between the wireless headset and the sound source device based on a wireless signal sent by the sound source device.

[0092] As an implementation mode, the wireless headset is provided with a first wireless communication device, and the sound source device is provided with a second wireless communication device. The communication connection between the first wireless communication device and the second wireless communication device can establish a wireless communication link between the wireless headset and the sound source device, thereby realizing wireless communication between the wireless headset and the sound source device. Wherein, the first wireless communication device includes a first antenna, and the second wireless communication device includes a second antenna. When there are multiple first antennas, for example, the number of the first antennas is at least two, the distances at which the wireless signal emitted by the second antenna reaches each first antenna are different, thereby generating a phase difference. Based on the phase difference, the angle of arrival of the sound source device to the wireless headset, that is, the angle of arrival between the wireless headset and the sound source device, can be calculated.

[0093] Specifically, assuming that the data vector of the audio signal is x(t), and assuming that the signal is to be phase-shifted and scaled as a sinusoidal (narrowband) signal, the following equation can be obtained:

[0094] x(t)=a(θ)s(t)+n(t) (3)

[0095] a(θ)=[1,e j2πd′sin(θ) / λ ,...,e j2π(m-1)d′sin(θ) / λ ] (4)

[0096] In the above equations (3) and (4), a(θ) is the mathematical model of the antenna array, namely the so-called array control vector, s(t) is the incident signal, n(t) is the noise signal, d' is the distance between adjacent antennas in the antenna array, and m is the number of antennas in the antenna array.

[0097] The covariance matrix is ​​obtained by the following formula (5):

[0098]

[0099] Using a(θ) and the covariance matrix Rxx to calculate the so-called spatial spectrum, we get the following:

[0100]

[0101] Find the maximum peak of the spatial spectrum, and θ corresponding to the maximum peak is the arrival angle.

[0102] As an embodiment, there are multiple first antennas in the wireless headset, so that an antenna array can be formed by the multiple first antennas, and the arrival angle is determined based on the phase difference of the wireless signal from the sound source device reaching each first antenna in the antenna array.

[0103] In addition, there may be one first antenna in the wireless headset, and multiple second antennas on the sound source device, and the distance between the multiple second antennas on the sound source device can be determined. Therefore, the arrival angle of the wireless signal transmitted from the first antenna to the second antenna can be determined, and then the arrival angle of the wireless signal from the sound source device to the wireless headset can be determined based on geometric principles.

[0104] S1002: Determine the gain parameter based on the negative correlation between the arrival angle and the gain parameter to obtain a target gain parameter.

[0105] The negative correlation between the arrival angle and the gain parameter means that the arrival angle is inversely proportional to the gain parameter. The gain parameter is the volume value. The larger the arrival angle, the smaller the volume value, and the smaller the arrival angle, the larger the volume value. Fig.11 As shown, θ1 and θ2 are the arrival angles of the second antenna of the sound source device to the two first antennas.

[0106] When the user wears two wireless earphones, for example, the user wears the first earphone on the left ear and the second earphone on the right ear, if the sound source device is directly in front of the user and in the middle position, the arrival angles between the sound source device and the first earphone and the second earphone are the same. When the user turns his head toward the left ear, the arrival angle between the sound source device and the first earphone is greater than the arrival angle between the sound source device and the second earphone, and when the user turns his head toward the right ear, the arrival angle between the sound source device and the first earphone is less than the arrival angle between the sound source device and the second earphone.

[0107] As an implementation manner, a third correspondence between the angle of arrival and the gain parameter may be preset, and in the third correspondence, the angle of arrival and the gain parameter are negatively correlated. After the angle of arrival between the wireless headset and the sound source device is determined, the angle of arrival is used as the target angle of arrival, and the gain parameter corresponding to the target angle of arrival is searched in the third correspondence to obtain the target gain parameter.

[0108] As another implementation, an angle-volume relationship may be set to determine the gain parameter. In this relationship, the larger the arrival angle, the smaller the spatial audio parameter, that is, the arrival angle is negatively correlated with the gain parameter. Specifically, the relationship between the gain parameter and the arrival angle is as follows:

[0109]

[0110] Among them, θ is the arrival angle, g is the correction gain factor, which is related to parameters such as the operational amplifier of the wireless headset sound system, speaker sensitivity, and the Bluetooth distance between the Bluetooth transmitter of the audio and video electronic equipment and the headset end. It can be determined according to usage requirements.

[0111] As an implementation method, taking into account that at some angles, the user's head will cause greater interference to the sound emitted by the sound source device, an angle threshold can be set. If the arrival angle is less than the angle threshold, the gain parameter determined based on the third correspondence or the above-mentioned angle-volume relationship is used as the initial gain parameter. Then, the initial gain parameter is reduced by a second specified value to obtain the target gain parameter. If the arrival angle is greater than the angle threshold, the gain parameter determined based on the third correspondence or the above-mentioned angle-volume relationship is used as the target gain parameter.

[0112] Sound source equipment

[0113] S1003: Play the audio signal to be played according to the target spatial audio parameter.

[0114] Specifically, the implementation of S1004 may refer to the aforementioned embodiment and will not be described in detail here.

[0115] See also Fig.12 , Fig.12 An audio processing method is shown, in which the spatial position parameters include distance parameters and arrival angles, and the spatial audio parameters include gain parameters and delay lengths. As an implementation, the executor of the method may be the above-mentioned processor. Specifically, the method includes: S1201 to S1204.

[0116] S1201: Determine a distance parameter and an arrival angle between the wireless headset and the sound source device based on a wireless signal sent by the sound source device.

[0117] S1202: Based on the negative correlation between the distance parameter and the gain parameter and the negative correlation between the arrival angle and the gain parameter, determine the gain parameter to obtain a target gain parameter.

[0118] As an implementation method, the gain parameter is determined based on the negative correlation between the distance parameter and the gain parameter to obtain a first gain parameter, wherein the implementation method of determining the first gain parameter can refer to the aforementioned embodiment and will not be described in detail here. Then, the gain parameter is determined based on the negative correlation between the angle of arrival and the gain parameter to obtain a second gain parameter, wherein the implementation method of determining the second gain parameter can refer to the aforementioned embodiment and will not be described in detail here.

[0119] The target gain parameter is obtained based on the first gain parameter and the second gain parameter. As an implementation mode, the average gain parameter of the first gain parameter and the second gain parameter can be obtained as the target gain parameter. Of course, the target gain parameter can also be obtained by weighted summing the first gain parameter and the second gain parameter. Specifically, the first weight and the second weight can be set to obtain the first product of the first weight and the first gain parameter, and the second product of the second weight and the second gain parameter, and the sum of the first product and the second product can be obtained as the target gain parameter. Among them, the first weight and the second weight can be set according to actual needs or experience, and the sum of the first weight and the second weight is 1. Specifically, the first weight represents the proportion of the first gain parameter in the target gain parameter, and the second weight represents the proportion of the second gain parameter in the target gain parameter.

[0120] As an implementation mode, considering that when the distance is relatively far, the change in sound volume attenuation caused by the change in distance will not be too large, therefore, after obtaining the distance parameter, it is determined whether the distance parameter is greater than the specified distance threshold. If it is greater, the first weight is set to the first value. If the distance parameter is less than or equal to the specified distance threshold, the first weight is set to the second value, wherein the first value is less than the second value, and the second weight is the difference between 1 and the first weight, that is, W2=1-W1, wherein W2 is the second weight and W1 is the first weight. Therefore, a decrease in the first weight will increase the second weight, that is, when the distance parameter is greater than the specified distance threshold, the proportion of the first gain parameter determined by the distance parameter is reduced, and the proportion of the second gain parameter determined by the angle of arrival is increased.

[0121] As another implementation mode, considering that when the arrival angle is relatively large, the human head has a greater occlusion on the sound emitted by the sound source device. Therefore, after obtaining the arrival angle, it is determined whether the arrival angle is greater than the specified angle threshold. If it is greater, the second weight is set to a third value, otherwise, the second weight is set to a fourth value, wherein the third value is greater than the fourth value. Similarly, the first weight is the difference between 1 and the second weight, that is, W1=1-W2, wherein W2 is the second weight and W1 is the first weight. Therefore, an increase in the second weight will reduce the first weight, that is, when the arrival angle is greater than the specified angle threshold, the proportion of the first gain parameter determined by the distance parameter is reduced, and the proportion of the second gain parameter determined by the arrival angle is increased, so that when the angle is larger, the proportion of the gain parameter determined by the arrival angle should be increased, because the arrival angle has a greater impact on the gain parameter.

[0122] S1203: Based on the positive correlation between the distance parameter and the delay length, determine the delay length to obtain a target delay length.

[0123] As an implementation manner, the delay length is determined based on the positive correlation between the distance parameter and the delay length. The implementation manner can refer to the aforementioned embodiment and will not be described in detail here.

[0124] Sound source device S1204: plays the audio signal to be played according to the target spatial audio parameter.

[0125] As an implementation manner, the wireless headset may be two, namely a first headset and a second headset, and the first headset and the second headset respectively determine their respective target spatial audio parameters based on the above method. For details, please refer to the aforementioned embodiment and will not be repeated here.

[0126] like Fig.13 As shown, Fig.13 An audio processing method is shown, which is applied to the above-mentioned wireless headset. As an implementation, the execution subject of the method may be the above-mentioned processor. Specifically, the method includes: S1301 to S1303.

[0127] S1301: Acquire a wireless signal sent by the sound source device based on a wireless communication link between the wireless headset and the sound source device, and determine a spatial position parameter between the wireless headset and the sound source device.

[0128] S1302: Adjust the direct sound space audio parameter, the reflected sound space audio parameter, and the reverberation sound space audio parameter based on the space position parameter to obtain the target space audio parameter.

[0129] like Fig.14 As shown, the reverberation sound field generated by the reflection of the surrounding environment has three components: direct sound 1401, early reflected sound 1402 and reverberation sound 1403. People's sense of space for sound is mainly based on early reflected sound and reverberation sound. First of all, the initial delay between the direct sound and the early reflected sound determines the user's perception of the size of the space. At the same time, the early reflected sound will come from all directions in the three-dimensional space. The sound is constantly reflected and attenuated in the space, forming a uniform and dense reverberation sound. The time and density of the reverberation reflect the acoustic characteristics of the entire space, and together with the direct sound and early reflected sound, it establishes the indoor sound field. The propagation of sound in space and the reverberation sound field formed are as follows Fig.14 Through the reverberant sound field, listeners perceive the different delays and loudness of early reflected sounds from different directions, which helps to determine the location and distance of the sound source device; in addition, it also allows listeners to perceive their own position in the space to a certain extent.

[0130] As an implementation method, since the spatial audio parameters include gain parameters and delay length, the direct sound spatial audio parameters include direct sound gain parameters and direct sound delay length, the reflected sound spatial audio parameters include reflected sound gain parameters and reflected sound delay length, and the reverberation sound spatial audio parameters include reverberation sound gain parameters and reverberation sound delay length.

[0131] As an implementation method, the direct sound space audio parameters, the reflected sound space audio parameters and the reverberation sound space audio parameters can all be determined by the above method, that is, according to the space position parameters. As another implementation method, Fig.14 As shown, since the propagation speed and number of reflections of direct sound, reflected sound and reverberation sound in space are different, the sound pressure levels and the length of time they take to reach the human ear are different. Specifically, the sound pressure levels of direct sound, reflected sound and reverberation sound decrease in sequence, and the length of time they take to reach the human ear increases in sequence. Therefore, the direct sound spatial audio parameters can be determined first, and then the reflected sound spatial audio parameters can be determined based on the direct sound spatial audio parameters, and then the reverberation spatial audio parameters can be determined based on the reflected sound spatial audio parameters.

[0132] As an implementation mode, the direct sound spatial audio parameter may be directly determined according to the above method embodiment, specifically, it may be determined according to the distance parameter, or according to the arrival angle, or according to both the distance parameter and the arrival angle. Figure 5 As shown, the delay parameter of the delay module 2121 is set according to the delay length of the direct sound, that is, the time length of the signal delay output of the delay module 2121, so as to set the time for the direct sound to reach the human ear. Figure 5 As shown, the amplitude modulation module 216 adjusts the gain parameters of the direct sound, reflected sound and reverberation sound as a whole, so as to adjust the playback volume of the direct sound, reflected sound and reverberation sound as a whole. Of course, since the sound output by the delay module can be regarded as the direct sound, and the direct sound is sequentially input into the reflected sound module and the reverberation sound module, the amplitude modulation module 216 can also be set after the delay module and before the reflected sound module, the reverberation sound module and the amplitude modulation module. Specifically, the delay module can delay the audio signal, and then set the gain parameter of the amplitude modulation module 216 based on the direct sound gain parameter, adjust the gain of the audio signal, and obtain the direct sound signal. Then, it is input into the emission sound module and the reverberation sound module. Specifically, the spatial audio parameters also include a specified gain parameter, and the direct sound signal, the reflected sound signal and the reverberation sound signal are mixed; the mixed audio signal is amplitude modulated based on the specified gain parameter to obtain the audio signal to be played.

[0133] Then, the reflected sound gain parameter is set based on the direct sound gain parameter. Specifically, the direct sound gain parameter can be reduced by the first specified gain parameter to obtain the reflected sound gain parameter. The reflected sound delay length is set based on the direct sound delay length. Specifically, the direct sound delay length can be increased by the first specified delay length to obtain the reflected sound delay length. Figure 5 As shown, the reflected sound can be realized by the first all-pass filter 2131, that is, the parameters of the first all-pass filter 2131 are adjusted based on the determined reflected sound gain parameters, for example, the delay length of the delay device and the gain value of the gain module in the first all-pass filter 2131. Different spatial audio parameters can be set for different first all-pass filters 2131, so as to realize the superposition of multiple different reflected sounds.

[0134] Then, the reverberation gain parameter is set based on the reflected sound gain parameter. Specifically, the reflected sound gain parameter can be reduced by the second specified gain parameter to obtain the reverberation gain parameter. The reverberation delay length is set based on the reflected sound delay length. Specifically, the reflected sound delay length can be increased by the second specified delay length to obtain the reverberation delay length. Figure 5 As shown, the reverberation sound can be realized by the second all-pass filter 2141, that is, the parameters of the second all-pass filter 2141 are adjusted based on the determined reverberation sound gain parameters, for example, the delay length of the delay device in the second all-pass filter 2141 and the gain value of the gain module. The density of the reverberation sound can be increased by connecting multiple second all-pass filters 2141 in series. In addition, the gain parameter of the low-pass filter 2142 can also be set to reduce the volume of the high-frequency part of the sound in the multiple second all-pass filters 2141 connected in series, thereby simulating high-frequency attenuation damping.

[0135] S1303: Determine an audio signal to be played according to the direct sound audio parameter, the reflected sound audio parameter, and the reverberation sound audio parameter.

[0136] Specifically, a direct sound signal corresponding to the audio signal is determined based on the direct sound audio parameters; a reflected sound signal corresponding to the audio signal is output based on the reflected sound audio parameters; a reverberation sound signal corresponding to the audio signal is output based on the reverberation sound audio parameters; and the direct sound signal, the reflected sound signal and the reverberation sound signal are mixed to obtain an audio signal to be played.

[0137] As described in the above-mentioned embodiment, the direct sound module is used to output the direct sound signal corresponding to the audio signal based on the direct sound audio parameters; the reflected sound module is used to output the reflected sound signal corresponding to the audio signal based on the reflected sound audio parameters; the reverberation sound module is used to output the reverberation sound signal corresponding to the audio signal based on the reverberation sound audio parameters; and the first mixer is used to mix the direct sound signal, the reflected sound signal and the reverberation sound signal into an audio signal to be played.

[0138] Specifically, the parameters of the direct sound module are set based on the direct sound audio parameters, the parameters of the reflected sound module are set based on the reflected sound audio parameters, and the parameters of the reverberation sound module are set based on the reverberation sound audio parameters. Specifically, the set parameters may include gain parameters and delay parameters of the module, which are specifically determined according to the spatial audio parameters corresponding to each module.

[0139] As an implementation mode, the direct sound audio parameter includes a direct sound delay length, the reflected sound audio parameter includes a reflected sound gain parameter and a reflected sound delay length, and the reverberation sound audio parameter includes a reverberation sound gain parameter and a reverberation sound delay length. The direct sound module delays the audio signal based on the direct sound delay length to obtain a direct sound signal, the reflected sound module adjusts the volume of the full-band portion of the direct sound signal based on the reflected sound gain parameter, and performs delay processing on the full-band portion of the direct sound signal based on the reflected sound delay length to obtain the reflected sound signal, and the reverberation sound module adjusts the volume of a designated frequency band portion of the reflected sound signal based on the reverberation sound gain parameter, and performs delay processing on the designated frequency band portion of the reflected sound signal based on the reverberation sound delay length to obtain a reverberation sound signal.

[0140] Specifically, Figure 5As shown, the delay module 2121 is used as a direct sound module, and the audio signal is input into the delay module 2121. The delay module 2121 delays the audio signal based on the direct sound delay length to obtain a direct sound signal. Then, the direct sound signal is divided into four paths and respectively input into the first mixer 215 and three first all-pass filters 2131. Each first all-pass filter 2131 performs volume adjustment and delay processing on the full-band part of the direct sound signal to obtain a reflected phonon signal. Multiple reflected phonon signals are mixed by the second mixer to form a reflected sound signal. The density and complexity of the reflected sound can be increased by setting multiple first all-pass filters. As an embodiment, the gain and delay parameters of each first all-pass filter can be different or the same. For example, they can all be reflected sound gain parameters and reflected sound delay lengths. Based on the reverberation sound audio parameters, the gain and delay parameters of M second all-pass filters 2141 are set; based on the low-pass filter, the high-frequency part of the reflected sound signal is filtered out and the low-frequency band is retained; based on the M second all-pass filters, the volume of the low-frequency part of the reflected sound signal is adjusted and the delay is processed in turn to obtain the reverberation sound signal.

[0141] The first mixer 215 mixes the direct sound signal, the reflected sound signal and the reverberation sound signal and inputs the mixed sound signal into the amplitude modulation module 216. The amplitude modulation module 216 modulates the amplitude of the mixed audio signal based on a specified gain parameter to obtain an audio signal to be played.

[0142] As an implementation method, after determining the spatial position parameters between the wireless headset and the sound source device, the specified gain parameter and the specified delay parameter are determined based on the above-mentioned implementation method, and the specified delay parameter is used as the direct sound delay length, that is, as the delay parameter of the delay module 2121, and the specified gain parameter is used as the gain parameter of the amplitude modulation module 216.

[0143] Then, the reflected sound audio parameters and the reverberation sound audio parameters are determined based on the direct sound delay length and the specified gain parameters, wherein the reflected sound gain parameter and the reverberation sound gain parameter are both obtained by further reducing the gain on the basis of the specified gain parameters, and the reflected sound delay length and the reverberation sound delay length are both obtained by further delaying on the basis of the specified delay parameters.

[0144] Specifically, the reflected sound gain parameter and the reverberation sound gain parameter may both be negative gains, so the reflected sound and the reverberation sound are further attenuated on the basis of the direct sound. As an implementation method, the reverberation sound gain parameter is less than the reflected sound gain parameter, that is, the reverberation sound is more severely attenuated than the reflected sound. The reflected sound delay length and the reverberation sound delay length are both positive numbers, so the reflected sound and the reverberation sound are further delayed on the basis of the direct sound. As an implementation method, the reverberation sound delay length is greater than the reflected sound delay length, that is, the reverberation sound is more severely delayed than the reflected sound. Specifically, the settings of the reflected sound gain parameter and the reverberation sound gain parameter as well as the reflected sound delay length and the reverberation sound delay length can be set according to the changes and requirements of the spatial audio in the environment in which the headphones are actually used, and are not limited here.

[0145] It should be noted that, for the parts of the above method that are not described in detail, reference can be made to the aforementioned embodiments and will not be repeated here.

[0146] See also Fig.15 , which shows a structural block diagram of an audio processing device provided in an embodiment of the present application. The audio processing device 1500 may include: an acquisition unit 1501, a determination unit 1502 and a playback unit 1503.

[0147] The acquisition unit 1501 is used to determine the spatial position parameter of the wireless headset based on the wireless signal sent by the sound source device, where the spatial position parameter is used to indicate the spatial position relationship between the wireless headset and the sound source device.

[0148] The determining unit 1502 is configured to determine the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain target spatial audio parameters.

[0149] As an implementation manner, the spatial position parameter includes at least one of a distance parameter and an arrival angle, and the spatial audio parameter includes a gain parameter and a delay length.

[0150] Furthermore, the determination unit 1502 is further configured to determine the gain parameter based on the negative correlation between the distance parameter and the gain parameter to obtain a target gain parameter; and determine the delay length based on the positive correlation between the distance parameter and the delay length to obtain a target delay length.

[0151] Furthermore, the determining unit 1502 is further configured to determine the gain parameter based on the negative correlation between the angle of arrival and the gain parameter, so as to obtain a target gain parameter.

[0152] Furthermore, the determination unit 1502 is also used to determine the gain parameter based on the negative correlation between the distance parameter and the gain parameter and the negative correlation between the arrival angle and the gain parameter to obtain the target gain parameter; based on the positive correlation between the distance parameter and the delay length, determine the delay length to obtain the target delay length.

[0153] Furthermore, the determining unit 1502 is further configured to adjust the direct sound spatial audio parameters, the reflected sound spatial audio parameters, and the reverberation sound spatial audio parameters based on the spatial position parameters to obtain target spatial audio parameters.

[0154] The processing unit 1503 is configured to determine the audio signal to be played according to the target spatial audio parameter and the audio signal output by the sound source device.

[0155] Furthermore, there are two wireless headphones, namely a first headphone and a second headphone, and the determination unit 1502 is further used to adjust the spatial audio parameters of the first headphone according to the spatial position parameters corresponding to the first headphone to obtain the first target spatial audio parameters; and adjust the spatial audio parameters of the first headphone according to the spatial position parameters corresponding to the second headphone to obtain the second target spatial audio parameters. The playback unit 1503 is also used to control the first headphone and the second headphone to play the audio signal based on the first target spatial audio parameters and the second target spatial audio parameters.

[0156] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here.

[0157] In several embodiments provided in the present application, the coupling between modules may be electrical, mechanical or other forms of coupling.

[0158] In addition, each functional module in each embodiment of the present application can be integrated into a processing module, or each module can exist physically separately, or two or more modules can be integrated into one module. The above integrated modules can be implemented in the form of hardware or software functional modules.

[0159] Please refer to Fig.16 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable medium 1600 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0160] The computer readable storage medium 1600 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer readable storage medium 1600 includes a non-transitory computer-readable storage medium. The computer readable storage medium 1600 has storage space for program code 1610 that performs any method steps in the above method. These program codes can be read from or written to one or more computer program products. The program code 1610 can be compressed, for example, in an appropriate form.

[0161] In summary, the audio processing method, device, wireless headset and computer-readable medium provided in the embodiments of the present application determine the spatial position between the wireless headset and the sound source device based on the wireless signal between the two. Compared with image sensors and motion sensors, not only no additional hardware equipment is installed in the wireless headset, that is, the cost of the wireless headset is not increased, but the determined spatial position is more accurate.

[0162] By measuring the distance and angle between the Bluetooth signal transmitter and the receiver in real time, the wireless headset and the sound source device are positioned, and the sound signal transmitted from the sound source device via Bluetooth is processed by binaural spatial sound rendering, thereby simulating an immersive auditory experience. By simulating the spatial sound scene in real time, each user can experience the best listening position at different locations, thus bringing the best immersive spatial sound experience; through spatial sound rendering, the head effect can be eliminated and the headset user experience can be improved; saving wireless headset storage space, this solution is different from the preset measured spatial binaural impulse response (BRIR) by adjusting the binaural spatial sound algorithm parameters in real time, which can save a lot of storage space and algorithm computing power; saving costs and power consumption, the binaural impulse response spatial sound rendering parameters are changed in real time through the Bluetooth positioning function, without incurring additional hardware costs and power consumption, while improving the battery life of the headset.

[0163] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An audio processing method, characterized in that: Applied to wireless headphones, the method comprises: Determining a spatial position parameter of the wireless headset based on a wireless signal sent by a sound source device, wherein the spatial position parameter is used to indicate a spatial position relationship between the wireless headset and the sound source device; Determine the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain target spatial audio parameters, wherein the spatial audio parameters include direct sound audio parameters, reflected sound audio parameters and reverberation sound audio parameters, the direct sound audio parameters include direct sound delay length, and the reflected sound audio parameters include reflected sound gain parameters and reflected sound delay length; Delaying the audio signal output by the sound source device based on the direct sound delay length to obtain a direct sound signal; Setting gain and delay parameters of N first all-pass filters based on the reflected sound audio parameters; Based on each of the first all-pass filters, volume adjustment and delay processing are performed on the full-band part of the direct sound signal to obtain a reflected phonon signal; Mixing the N reflected phonon signals processed by the first all-pass filters to obtain a reflected sound signal; Outputting a reverberation sound signal corresponding to the audio signal based on the reverberation sound audio parameter; The direct sound signal, the reflected sound signal and the reverberation sound signal are mixed to obtain an audio signal to be played.

2. The method according to claim 1, characterized in that The spatial position parameter includes at least one of a distance parameter and an arrival angle parameter, and the spatial audio parameter includes at least one of a gain parameter and a delay parameter.

3. The method according to claim 2, characterized in that The spatial position parameter includes a distance parameter, and determining the spatial audio parameter of the wireless headset based on the spatial position parameter includes: Determine the gain parameter based on the negative correlation between the distance parameter and the gain parameter to obtain a target gain parameter; The delay length is determined based on the positive correlation between the distance parameter and the delay length to obtain a target delay length.

4. The method according to claim 2, characterized in that: The spatial position parameter includes an angle of arrival, and determining the spatial audio parameter of the wireless headset based on the spatial position parameter includes: The gain parameter is determined based on the negative correlation between the arrival angle and the gain parameter to obtain a target gain parameter.

5. The method according to claim 2, characterized in that: The spatial position parameters include a distance parameter and an angle of arrival, and determining the spatial audio parameters of the wireless headset based on the spatial position parameters includes: Based on the negative correlation between the distance parameter and the gain parameter and the negative correlation between the arrival angle and the gain parameter, the gain parameter is determined to obtain a target gain parameter; Based on the positive correlation between the distance parameter and the delay length, the delay length is determined to obtain a target delay length.

6. The method according to claim 1, characterized in that The reverberation audio parameters include a reverberation gain parameter and a reverberation delay length; and determining the spatial audio parameters of the wireless headset based on the spatial position parameters to obtain target spatial audio parameters includes: Determining a specified gain parameter and a specified delay parameter based on the spatial position parameter; Using the specified delay parameter as a direct sound audio parameter; Obtaining a reflected sound gain parameter and a reverberation sound gain parameter based on the specified gain parameter; The reflected sound delay length and the reverberation sound delay length are obtained based on the specified delay parameters.

7. The method according to claim 1, characterized in that The reverberation audio parameters include a reverberation gain parameter and a reverberation delay length, and the step of outputting a reverberation signal corresponding to the audio signal based on the reverberation audio parameters includes: The volume of the designated frequency band portion of the reflected sound signal is adjusted based on the reverberation gain parameter, and the designated frequency band portion of the reflected sound signal is delayed based on the reverberation delay length to obtain a reverberation signal.

8. The method according to claim 7, characterized in that The step of adjusting the volume of the specified frequency band portion of the reflected sound signal based on the reverberation gain parameter, and performing delay processing on the specified frequency band portion of the reflected sound signal based on the reverberation delay length to obtain the reverberation signal comprises: Setting the gain and delay parameters of M second all-pass filters based on the reverberation sound audio parameters; Filter out the portion of the reflected sound signal outside the specified frequency band based on a low-pass filter; Based on the M second all-pass filters, volume adjustment and delay processing are performed on the designated frequency band part of the reflected sound signal in sequence to obtain a reverberation sound signal.

9. The method according to claim 8, characterized in that The designated frequency band is a low frequency band.

10. The method according to claim 7, characterized in that The spatial audio parameters also include a specified gain parameter, and the mixing and playing of the direct sound signal, the reflected sound signal, and the reverberation sound signal includes: Mixing the direct sound signal, the reflected sound signal and the reverberation sound signal; The mixed audio signal is amplitude modulated based on the specified gain parameter to obtain an audio signal to be played.

11. The method according to any one of claims 1 to 10, characterized in that: The sound source device signal is a Bluetooth signal.

12. An audio processing device, characterized in that: Applied to wireless headphones, the device comprises: An acquiring unit, configured to determine a spatial position parameter of the wireless headset based on a wireless signal sent by a sound source device, wherein the spatial position parameter is used to indicate a spatial position relationship between the wireless headset and the sound source device; a determining unit, configured to determine the spatial audio parameters of the wireless headset based on the spatial position parameters, and obtain target spatial audio parameters, wherein the spatial audio parameters include direct sound audio parameters, reflected sound audio parameters, and reverberation sound audio parameters, the direct sound audio parameters include direct sound delay length, and the reflected sound audio parameters include reflected sound gain parameters and reflected sound delay length; A processing unit is used to delay the audio signal output by the sound source device based on the direct sound delay length to obtain a direct sound signal; set the gain and delay parameters of N first all-pass filters based on the reflected sound audio parameters; perform volume adjustment and delay processing on the full-band part of the direct sound signal based on each of the first all-pass filters to obtain a reflected phonon signal; mix the reflected phonon signals processed by the N first all-pass filters to obtain a reflected sound signal; output a reverberation sound signal corresponding to the audio signal based on the reverberation sound audio parameters; and mix the direct sound signal, the reflected sound signal and the reverberation sound signal to obtain an audio signal to be played.

13. A wireless headset, characterized in that: include: An audio processing module and a speaker, wherein the wireless communication module is connected to the audio processing module; The wireless communication module is used to obtain the wireless signal sent by the sound source device; The audio processing module is used to determine the audio signal to be played based on the method according to any one of claims 1-11.

14. The wireless headset according to claim 13, characterized in that: The audio processing module includes an audio adjuster and a processor, and the audio adjuster is connected to the processor; The processor is used to determine the spatial position parameters of the wireless headset based on the wireless signal sent by the sound source device and received by the wireless communication module, determine the spatial audio parameters of the wireless headset based on the spatial position parameters, and obtain the target spatial audio parameters; The audio adjuster is used to determine the audio signal to be played based on the target space audio parameter and the audio signal output by the sound source device.

15. The wireless headset according to claim 14, characterized in that: The wireless headset further includes: a first mixer, the direct sound module, the reflected sound module and the reverberation sound module are all connected to the processor and the first mixer, and the first mixer is connected to the speaker; the spatial audio parameters include direct sound audio parameters, reflected sound audio parameters and reverberation sound audio parameters; The direct sound module is used to output a direct sound signal corresponding to the audio signal based on the direct sound audio parameter; The reflected sound module is used to output a reflected sound signal corresponding to the audio signal based on the reflected sound audio parameter; The reverberation sound module is used to output a reverberation sound signal corresponding to the audio signal based on the reverberation sound audio parameter; The first mixer is used to mix the direct sound signal, the reflected sound signal and the reverberation sound signal into an audio signal to be played.

16. The wireless headset according to claim 15, characterized in that: The direct sound module includes a delay module, and the delay module is connected to the input end of the reflected sound module and the first input end of the first mixer respectively; The delay module is used to delay the audio signal based on the direct sound audio parameter to obtain a direct sound signal; The reflected sound module is further used to adjust the volume and delay the full-band part of the direct sound signal based on the reflected sound audio parameters to obtain a reflected sound signal; The reverberation sound module is further used to perform volume adjustment and delay processing on a designated frequency band portion of the reflected sound signal based on the reverberation sound audio parameters to obtain a reverberation sound signal.

17. The earphone according to claim 16, characterized in that The reflected sound module includes a first filter group and a second mixer, the first filter group is connected to the delay module, the first filter includes N first all-pass filters connected in parallel, each of the first all-pass filters is connected to an input end of the second mixer, and the output end of the second mixer is respectively connected to the input end of the reverberation sound module and the second input end of the first mixer, wherein N is a positive integer; Each of the first all-pass filters is used to perform volume adjustment and delay processing on the full-band part of the direct sound signal based on the reflected sound audio parameter to obtain a reflected phonon signal; The second mixer is used to mix the reflected phonon signals output by each of the first all-pass filters to obtain a reflected sound signal.

18. The earphone according to claim 17, characterized in that The designated frequency band part is a low-frequency part, the reverberation sound module includes a low-pass filter and a second filter group, the second filter group includes M second all-pass filters connected in series, the output end of the reflected sound module is connected to the input end of the low-pass filter through the second filter group, and the output end of the low-pass filter is connected to the third input end of the first mixer, wherein M is a positive integer; The low-pass filter is used to filter out the high-frequency part of the reflected sound signal; The second all-pass filter is used to adjust the volume and delay the reflected sound signal of the low-frequency part output by the low-pass filter based on the reverberation sound audio parameters to obtain a reverberation sound signal.

19. The earphone according to claim 18, characterized in that It also includes an amplitude modulation module, the output end of the first mixer is connected to the input end of the amplitude modulation module, and the output end of the amplitude modulation module is connected to the speaker.

20. A computer readable medium, characterized in that The computer-readable medium stores a program code executable by a processor, and when the program code is executed by the processor, the processor executes the method according to any one of claims 1 to 11.

Images