Audio device, working method thereof and VR equipment

Abstract

The invention provides an audio device, a working method thereof and VR equipment, and belongs to the technical field of spatial audio. The audio device comprises an acoustic cavity; the at least one loudspeaker is positioned in the sound cavity; the acquisition unit is used for respectively acquiring audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data; and the calculation unit is used for processing the audio data according to the target HRTF, generating a target sound signal and outputting the target sound signal to the loudspeaker. According to the technical scheme, spatial stereo sound production can be simulated, and spatial immersion is created.

Description

Audio device and operating method thereof, VR equipment Technical Field

[0001] The present disclosure relates to the field of spatial audio technology, and in particular to an audio device and a working method thereof, and VR equipment. Background Art

[0002] When users use VR (Virtual Reality) and / or AR (Augmented Reality) devices to watch movies or play games, the display devices of VR and / or AR devices can provide users with an immersive feeling. However, the audio devices of existing VR and / or AR devices can only support two-channel surround sound, resulting in a weak sense of presence and failing to meet the user's requirements for an immersive 3D sound field experience.

[0003] Summary of the Invention

[0004] The technical problem to be solved by the present disclosure is to provide an audio device and a working method thereof, and a VR device, which can simulate spatial stereo sound and create a sense of spatial immersion.

[0005] To solve the above technical problems, the embodiments of the present disclosure provide the following technical solutions:

[0006] In one aspect, an audio device is provided, comprising:

[0007] vocal cavity;

[0008] at least one speaker located within the acoustic cavity;

[0009] An acquisition unit, configured to respectively acquire audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data;

[0010] A calculation unit is configured to process the audio data according to the target HRTF, generate a target sound signal, and output the target sound signal to the speaker.

[0011] In some embodiments, the audio device is a neck-hanging audio device, and the acoustic cavity is located in a neck-hanging portion of the neck-hanging audio device.

[0012] In some embodiments, the audio device further comprises:

[0013] a power supply located within the neck hanging portion;

[0014] The speaker is connected to the power supply via a wired manner.

[0015] In some embodiments, the audio device further comprises:

[0016] At least one low-frequency passive diaphragm and / or audio waveguide is located in the acoustic cavity.

[0017] In some embodiments, the low-frequency passive diaphragm is spaced apart from the loudspeaker; and / or

[0018] The audio waveguide is spaced apart from the speaker.

[0019] In some embodiments, the audio device is applied to a VR device, and the audio device further comprises:

[0020] a positioning unit, configured to locate the relative positional relationship between the speaker and the target part, wherein the target part includes the head and / or the ear;

[0021] The calculation unit is specifically used to obtain the first position of the virtual sound source in the spatial coordinate system of the VR device; obtain the second position of the target part in the spatial coordinate system, and determine the third position of the speaker in the spatial coordinate system based on the relative position relationship between the speaker and the target part; and generate the target sound signal based on the target HRTF at the first position, the HRTF at the third position and the audio data.

[0022] In some embodiments, the positioning unit includes at least one of the following:

[0023] a transmitting and receiving unit, comprising a transmitting unit located on the head-mounted display of the VR device and a receiving unit located on the neck-hanging portion, the transmitting unit being configured to transmit a target signal, the receiving unit being configured to determine a relative position between the head-mounted display and the speaker based on the received signal, and to determine a relative positional relationship between the speaker and the target portion based on the relative position between the head-mounted display and the speaker, wherein the target signal comprises at least one of the following: a non-sinusoidal narrow pulse, an ultrasonic wave, an optical signal, or an electromagnetic wave;

[0024] an image processing unit provided on the neck hanging portion, for acquiring an image of the target part, and determining a relative positional relationship between the speaker and the target part according to the image;

[0025] The inertial sensor IMU provided in the neck-hanging part is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck-hanging part, determine the relative position between the head-mounted display and the speaker based on the recorded relative rotation and displacement, and determine the relative position relationship between the speaker and the target part based on the relative position between the head-mounted display and the speaker.

[0026] In some embodiments, the audio device further comprises:

[0027] External interface for connecting to wired headphones.

[0028] In some embodiments, the audio device further comprises:

[0029] A microphone is provided on the neck hanging portion.

[0030] An embodiment of the present disclosure further provides a VR device, comprising a head-mounted display and the audio device as described above.

[0031] An embodiment of the present disclosure further provides a method for operating an audio device, which is applied to the audio device described above. The method includes:

[0032] Acquire audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data respectively;

[0033] The audio data is processed according to the target HRTF to generate a target sound signal, and the target sound signal is output to the speaker.

[0034] In some embodiments, the sound signal includes a right channel signal and a left channel signal, and before outputting the target sound signal to the speaker, the method further includes:

[0035] The crosstalk signal generated by the right channel signal is eliminated from the left channel signal, and the crosstalk signal generated by the left channel signal is eliminated from the right channel signal.

[0036] In some embodiments, the working method further comprises:

[0037] Locating the relative positional relationship between the speaker and the target part, wherein the target part includes the head and / or the ear;

[0038] The processing of the audio data according to the target HRTF to generate a target sound signal includes:

[0039] Obtain a first position of the virtual sound source in the spatial coordinate system of the VR device; obtain a second position of the target part in the spatial coordinate system, and determine a third position of the speaker in the spatial coordinate system based on the relative position relationship between the speaker and the target part; generate the target sound signal based on the target HRTF at the first position, the HRTF at the third position and the audio data.

[0040] In some embodiments, locating the relative position relationship between the speaker and the target part includes at least one of the following:

[0041] The transmitting unit transmits a target signal, the receiving unit determines a relative position between the head-mounted display and the speaker based on the received signal, and the relative position relationship between the speaker and the target part is determined based on the relative position between the head-mounted display and the speaker, wherein the target signal includes at least one of the following: a non-sinusoidal narrow pulse, an ultrasonic wave, an optical signal, and an electromagnetic wave;

[0042] acquiring an image of the target part using the image processing unit, and determining a relative positional relationship between the speaker and the target part according to the image;

[0043] The inertial sensor IMU is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck hanging part, and the relative position between the head-mounted display and the speaker is determined based on the recorded relative rotation and displacement. The relative position relationship between the speaker and the target part is determined based on the relative position between the head-mounted display and the speaker.

[0044] The embodiments of the present disclosure have the following beneficial effects:

[0045] In the above scheme, the target HRTF at the virtual sound source position corresponding to the audio data is obtained, the audio data is processed according to the target HRTF to generate a target sound signal, and the speaker is controlled to make sound according to the target sound signal, which can simulate spatial stereo sound and create a sense of spatial immersion. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] 1 to 4 are schematic structural diagrams of audio devices according to some embodiments of the present disclosure;

[0047] 5-6 are schematic diagrams of an audio device according to an embodiment of the present disclosure applied to a VR device;

[0048] FIG7 is a schematic structural diagram of an audio device according to another embodiment of the present disclosure;

[0049] 8 and 9 are flowcharts of a working method of the audio device according to an embodiment of the present disclosure.

[0050] Reference Signs 01 Audio device 02 Head-mounted display 03 Virtual sound source 011 Acoustic cavity 012 Loudspeaker 013 Low-frequency passive diaphragm 014 Audio waveguide 015 Microphone 016 Wired earphones DETAILED DESCRIPTION

[0051] In order to make the technical problems, technical solutions and advantages to be solved by the embodiments of the present disclosure more clear, they will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Embodiments of the present disclosure provide an audio device and a working method thereof, and a VR device, which can simulate spatial stereo sound and create a sense of spatial immersion.

[0053] An embodiment of the present disclosure provides an audio device, as shown in FIG1 to FIG4 , including:

[0054] Vocal cavity 011;

[0055] at least one loudspeaker 012 located in the acoustic cavity 011;

[0056] An acquisition unit, configured to respectively acquire audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data;

[0057] A calculation unit is used to process the audio data according to the target HRTF, generate a target sound signal, and output the target sound signal to the speaker 012.

[0058] In this embodiment, the target HRTF at the virtual sound source position corresponding to the audio data is obtained, the audio data is processed according to the target HRTF to generate a target sound signal, and the speaker is controlled to make sound according to the target sound signal, which can simulate spatial stereo sound and create a sense of spatial immersion.

[0059] HRTF (Head Related Transfer Function), also known as the ear spectrum curve, is a mathematical function used to measure and evaluate the transmission of sound through the human ear. It measures the spatial characteristics of sound as it passes through the eardrum and can simulate spatial stereo sound.

[0060] In the related art, whether wearing in-ear headphones or headphones for a long time, it is easy to affect the user's hearing ability. Frequent wearing of in-ear headphones can also easily induce problems such as ear canal inflammation, while wearing headphones can easily cause stuffiness and sweating. In this embodiment, the audio device can be a neck-mounted audio device. As shown in Figure 5, the audio device 01 can be worn around the user's neck, thus avoiding the hazards of in-ear headphones or headphones.

[0061] As shown in Figures 1-4, when audio device 01 is a neck-mounted audio device, acoustic cavity 011 can be located within the neck-mounted portion of the device. This allows audio device 01 to project sound around the user's head, providing a superior listening experience. Acoustic cavity 011 can contain one or more speakers 012. If multiple speakers 012 are built into acoustic cavity 011, they are evenly distributed within it. In this embodiment, to enhance the privacy of the audio device, speakers 012 can be selected with a small radiation angle.

[0062] In some embodiments, the audio device further comprises a power supply located within the neckband portion; this allows the power supply to be integrated within the neckband portion, simplifying the structure of the audio device. When the power supply is located within the neckband portion, the speaker 012 can be connected to the power supply via a wired connection.

[0063] In some embodiments, as shown in Figures 1 and 2, the audio device further includes: at least one low-frequency passive diaphragm 013 located within the acoustic cavity 011. The low-frequency passive diaphragm 013 can enhance the low-frequency performance of the audio device. In this embodiment, the number of speakers 012 and low-frequency passive diaphragms 013 can be combined in different quantities to achieve better low-frequency performance. For example, the ratio of speakers 012 to low-frequency passive diaphragms 013 can be 1:2, or 1:1, or 2:1. In one specific example, as shown in Figure 1, two speakers 011 and two low-frequency passive diaphragms 013 can be provided in the acoustic cavity 011; in another specific example, as shown in Figure 2, four speakers 011 and two low-frequency passive diaphragms 013 can be provided in the acoustic cavity 011.

[0064] In this embodiment, the speaker 012 and the low-frequency passive diaphragm 013 can be symmetrically arranged at both ends of the neck hanging part, so that the speaker 012 and the low-frequency passive diaphragm 013 can be arranged closer to the user's ears, providing the user with better sound effects.

[0065] To achieve a better low-frequency effect, when there are multiple speakers 012 and multiple low-frequency passive diaphragms 013, the speakers 012 and the low-frequency passive diaphragms 013 can be spaced apart. In a specific example, as shown in FIG2 , the speakers 012 can be placed on both sides of the low-frequency passive diaphragm 013 at the end of the neck hanging portion. Of course, when the ratio of the number of speakers 012 to the low-frequency passive diaphragm 013 is 1:2, the low-frequency passive diaphragm 013 can also be placed on both sides of the speaker 012 at the end of the neck hanging portion.

[0066] In some embodiments, as shown in Figures 3 and 4, the audio device further includes at least one audio waveguide 014 located within the acoustic cavity 011. The audio waveguide 014 can enhance the low-frequency performance of the audio device. In this embodiment, the number of speakers 012 and audio waveguides 014 can be combined in varying quantities to achieve better low-frequency performance. For example, the ratio of speakers 012 to audio waveguides 014 can be 4:1, or 1:1, or 2:1. In one specific example, as shown in Figure 3, two speakers 011 and one audio waveguide 014 can be provided in the acoustic cavity 011; in another specific example, as shown in Figure 4, four speakers 011 and one audio waveguide 014 can be provided in the acoustic cavity 011.

[0067] In this embodiment, the speaker 012 and the audio waveguide 014 can be symmetrically arranged at both ends of the neck hanging part, so that the speaker 012 and the audio waveguide 014 can be arranged closer to the user's ears, providing the user with better sound effects.

[0068] In order to achieve a better low-frequency effect, when there are multiple speakers 012 and multiple audio waveguides 014 , the speakers 012 and the audio waveguides 014 can be arranged at intervals.

[0069] In a specific example, as shown in FIG3 , a speaker 012 may be provided at each end of the neck-hanging portion, with the audio waveguide 014 located between the two speakers 012 ; or, as shown in FIG4 , two speakers 012 may be provided at each end of the neck-hanging portion, with the audio waveguide 014 located between the four speakers 012 .

[0070] The audio device of this embodiment can be used as an independent audio device, as shown in Figures 5 and 6. The audio device 01 of this embodiment can also be applied to VR devices, wherein 02 is a head-mounted display of the VR device, and 03 is a virtual sound source. In related technologies, when a user uses a VR device and the user's movements and behaviors change, the orientation of the sound source in the virtual reality has changed for the user, but the orientation of the sound source played in the audio playback device worn by the user (such as headphones) has not changed accordingly, which will affect the immersive feeling created by the virtual picture and reduce the user experience. In this embodiment, the audio device can process the audio data in real time to ensure that the position of the virtual sound source played by the audio device changes synchronously with the user's movements and behaviors, thereby creating a sense of spatial immersion and improving the user experience.

[0071] When the audio device of this embodiment is applied to a VR device, the audio device further includes:

[0072] a positioning unit, configured to locate the relative positional relationship between the speaker and the target part, wherein the target part includes the head and / or the ear;

[0073] The calculation unit is specifically used to obtain the first position of the virtual sound source in the spatial coordinate system of the VR device; obtain the second position of the target part in the spatial coordinate system, and determine the third position of the speaker in the spatial coordinate system based on the relative position relationship between the speaker and the target part; and generate the target sound signal based on the target HRTF at the first position, the HRTF at the third position and the audio data.

[0074] For example, the coordinates of the virtual sound source B in the spatial coordinate system of the VR device are: Among them, r B is the distance between the position of the virtual sound source B and the origin of the coordinate system, θ B is the azimuth, is the pitch angle, and the virtual sound source B emits sound S B , the sound signal heard by the human ear is The coordinates of speaker A in the spatial coordinate system of the VR device are Among them, r A is the distance between the position of speaker A and the origin of the coordinate system, θ A is the azimuth, is the pitch angle, and speaker A emits sound S A , the sound signal heard by the human ear is In order to make the speaker A simulate the sound of the virtual sound source B, it is necessary to make E A =E B ,but:

[0075]

[0076] Therefore, according to the audio data of the virtual sound source, the position of the virtual sound source B At the location of speaker A The speaker A can simulate the pronunciation of the virtual sound source B to create a sense of spatial immersion.

[0077] In this embodiment, there is no limitation on the method of obtaining HRTF. HRTFs at different positions can be obtained by searching a preset HRTF table; or, HRTFs at different positions can be obtained through software simulation; or, HRTFs at different positions can be obtained through actual testing.

[0078] In this embodiment, the first position of the virtual sound source in the spatial coordinate system of the VR device can be determined based on the acquired audio data; when the user wears a head-mounted display, the second position of the target part in the spatial coordinate system can be determined based on the position of the head-mounted display in the spatial coordinate system of the VR device (generally the coordinate origin or a known coordinate point) and the relative position relationship between the head-mounted display and the target part (which can be the ear or the head); the relative position relationship between the speaker and the target part can be located according to the positioning unit, and the third position of the speaker in the spatial coordinate system can be determined based on the second position and the relative position relationship between the speaker and the target part.

[0079] In some embodiments, the positioning unit may include a transmitting and receiving unit, which uses ultrasonic waves, UWB (non-sinusoidal narrow pulses), photoelectric positioning, electromagnetic positioning, etc. to determine the relative position relationship between the head-mounted display and the speaker, and determines the relative position relationship between the speaker and the target part based on the relative position between the head-mounted display and the speaker. Specifically, the transmitting and receiving unit includes a transmitting unit located on the head-mounted display of the VR device and a receiving unit located on the neck hanging part, the transmitting unit is used to transmit a target signal, the receiving unit is used to determine the relative position between the head-mounted display and the speaker based on the received signal, and determines the relative position relationship between the speaker and the target part based on the relative position between the head-mounted display and the speaker, and the target signal includes at least one of the following: non-sinusoidal narrow pulses, ultrasonic waves, optical signals, and electromagnetic waves.

[0080] In this embodiment, the transmitting and receiving units can determine the position of the neckband portion relative to the head-mounted display. Since the relative positions of the user's target area and the head-mounted display are essentially fixed, and the relative positions of the neckband portion and the speaker are essentially fixed, the relative position between the user's target area and the speaker can be calculated. In this embodiment, the receiving unit must be fixed to the neckband portion during positioning and cannot move or shake.

[0081] Of course, in this embodiment, the transmitting unit can also be set on the neck hanging part, and the receiving unit can be set on the head-mounted display of the VR device.

[0082] In some embodiments, the positioning unit may include an image processing unit provided in the neck hanging portion, and the image processing unit may acquire an image of the target part and determine the relative positional relationship between the speaker and the target part based on the image.

[0083] In this embodiment, the image processing unit can utilize a monocular camera, a binocular camera, a depth camera, or the like. When the audio device is capable of capturing an image of the user's target area, the image processing unit captures the image of the target area and, through coordinate system transformation, locates the relative position of the speaker to the target area. For example, if the target area is the ear, image processing can be used to segment the ear from the image, determine the location of the ear canal, and, through coordinate system transformation, locate the position of each speaker relative to the ear ( Ploudspeaker_j ).

[0084] In this embodiment, during positioning, the image processing unit needs to be fixed to the neck hanging part and cannot move or shake.

[0085] In some embodiments, the positioning unit may include an inertial sensor IMU arranged in the neck-hanging part, which is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck-hanging part, determine the relative position between the head-mounted display and the speaker based on the recorded relative rotation and displacement, and determine the relative position relationship between the speaker and the target part based on the relative position between the head-mounted display and the speaker.

[0086] The inertial sensor (IMU) can be combined with other absolute positioning methods to determine the relative position between the target part and the speaker. For example, after the image processing unit has been used to locate the relative position between the target part and the speaker, if the user rotates their head, preventing the image processing unit from capturing the ear image, the relative rotation and displacement between the head-mounted display and the neck hanging portion, recorded by the inertial sensor (IMU), can be used to determine the relative position of the ear relative to each speaker. In this embodiment, the inertial sensor must be fixed to the neck hanging portion during positioning and cannot move or shake.

[0087] In this embodiment, the positioning unit may include only one of the transmitting and receiving units, the image processing unit and the inertial sensor, or may include a combination of multiple transmitting and receiving units, the image processing unit and the inertial sensor. When the positioning unit adopts a combination of multiple transmitting and receiving units, the image processing unit and the inertial sensor, the positioning accuracy can be improved.

[0088] In some embodiments, as shown in FIG7 , the audio device further includes:

[0089] The external interface is used to connect to the wired headset 016. When the audio device is connected to the wired headset 016, it is only necessary to obtain the target HRTF at the location of the virtual sound source B, recorded as HRTFi, and convolve the audio data using the HRTFi at the location of the virtual sound source B to achieve spatial audio.

[0090] In some embodiments, as shown in FIG7 , the audio device further includes:

[0091] The microphone 015 is arranged on the neck hanging part, so that the voice input function can be realized by using the audio device.

[0092] In this embodiment, the sound signal emitted by the speaker includes a right channel signal and a left channel signal. Before the target sound signal is output to the speaker, the crosstalk signal generated by the right channel signal can be eliminated from the left channel signal, and the crosstalk signal generated by the left channel signal can be eliminated from the right channel signal. This can prevent the sounds of the left and right ears from crosstalking with each other and improve the user's auditory experience.

[0093] It is worth noting that the neck hanging part of this embodiment can also be a neck pillow part.

[0094] The above solution is introduced using VR equipment as an example. Similarly, the audio device of this embodiment can also be applied to AR equipment.

[0095] An embodiment of the present disclosure further provides a VR device, comprising a head-mounted display and the audio device as described above.

[0096] In related technologies, when a user uses a VR device and their movements change, the location of the sound source in the virtual reality changes for the user, but the location of the sound source played by the audio playback device (such as headphones) worn by the user does not change accordingly. This affects the immersive feeling created by the virtual image and reduces the user experience. In this embodiment, the audio device can process audio data in real time to ensure that the location of the virtual sound source played by the audio device changes synchronously with the user's movements, creating a sense of spatial immersion and improving the user experience.

[0097] An embodiment of the present disclosure further provides a method for operating an audio device, which is applied to the audio device described above. As shown in FIG8 , the method includes:

[0098] Step 101: Acquire audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data respectively;

[0099] Step 102: Process the audio data according to the target HRTF to generate a target sound signal, and output the target sound signal to the speaker.

[0100] In this embodiment, the target HRTF at the virtual sound source position corresponding to the audio data is obtained, the audio data is processed according to the target HRTF to generate a target sound signal, and the speaker is controlled to make sound according to the target sound signal, which can simulate spatial stereo sound and create a sense of spatial immersion.

[0101] HRTF stands for Head-Related Transfer Function, also known as the ear spectrum curve. HRTF is a mathematical function used to measure and evaluate the transmission of sound through the human ear. It measures the spatial characteristics of sound as it passes through the eardrum and can simulate spatial stereo sound.

[0102] In the related art, whether wearing in-ear headphones or headphones for a long time, it is easy to affect the user's hearing ability. Frequent wearing of in-ear headphones can also easily induce problems such as ear canal inflammation, while wearing headphones can easily cause stuffiness and sweating. In this embodiment, the audio device can be a neck-mounted audio device. As shown in Figure 5, the audio device 01 can be worn around the user's neck, thus avoiding the hazards of in-ear headphones or headphones.

[0103] In some embodiments, the sound signal includes a right channel signal and a left channel signal, and before outputting the target sound signal to the speaker, the method further includes:

[0104] The crosstalk signal generated by the right channel signal is eliminated from the left channel signal, and the crosstalk signal generated by the left channel signal is eliminated from the right channel signal. This can prevent the sounds of the left and right ears from crosstalking with each other and improve the user's auditory experience.

[0105] In some embodiments, the working method further comprises:

[0106] Locating the relative positional relationship between the speaker and the target part, wherein the target part includes the head and / or the ear;

[0107] The processing of the audio data according to the target HRTF to generate a target sound signal includes:

[0108] Obtain a first position of the virtual sound source in the spatial coordinate system of the VR device; obtain a second position of the target part in the spatial coordinate system, and determine a third position of the speaker in the spatial coordinate system based on the relative position relationship between the speaker and the target part; generate the target sound signal based on the target HRTF at the first position, the HRTF at the third position and the audio data.

[0109] For example, the coordinates of the virtual sound source B in the spatial coordinate system of the VR device are: Among them, r B is the distance between the position of the virtual sound source B and the origin of the coordinate system, θ B is the azimuth, is the pitch angle, and the virtual sound source B emits sound S B , the sound signal heard by the human ear is The coordinates of speaker A in the spatial coordinate system of the VR device are Among them, r A is the distance between the position of speaker A and the origin of the coordinate system, θ A is the azimuth, is the pitch angle, and speaker A emits sound S A , the sound signal heard by the human ear is In order to make the speaker A simulate the sound of the virtual sound source B, it is necessary to make E A =E B ,but:

[0110] Therefore, according to the audio data of the virtual sound source, the position of the virtual sound source B At the location of speaker A The speaker A can simulate the pronunciation of the virtual sound source B to create a sense of spatial immersion.

[0111] In this embodiment, there is no limitation on the method of obtaining HRTF. HRTFs at different positions can be obtained by searching a preset HRTF table; or, HRTFs at different positions can be obtained through software simulation; or, HRTFs at different positions can be obtained through actual testing.

[0112] In this embodiment, as shown in FIG9 , the first position of the virtual sound source in the spatial coordinate system of the VR device can be determined based on the acquired audio data, and the target HRTF at the first position can be obtained; when the user wears a head-mounted display, the second position of the target part in the spatial coordinate system can be determined based on the position of the head-mounted display in the spatial coordinate system of the VR device (generally the coordinate origin or a known coordinate point) and the relative position relationship between the head-mounted display and the target part (which can be the ear or the head); the relative position relationship between the loudspeaker and the target part can be obtained based on the positioning unit, and the third position of the loudspeaker in the spatial coordinate system can be determined based on the second position and the relative position relationship between the loudspeaker and the target part, and the HRTF at the third position can be obtained. The audio data is reconstructed according to the target HRTF at the first position and the HRTF at the third position to obtain a sound signal, and the loudspeaker is controlled to play the sound signal, thereby realizing a spatial sound field.

[0113] In some embodiments, the positioning unit may include a transmitting and receiving unit, and locating the relative position relationship between the speaker and the target part includes:

[0114] The transmitting unit is used to transmit a target signal, the receiving unit is used to determine the relative position between the head-mounted display and the speaker based on the received signal, and the relative position relationship between the speaker and the target part is determined based on the relative position between the head-mounted display and the speaker, wherein the target signal includes at least one of the following: a non-sinusoidal narrow pulse, an ultrasonic wave, a light signal, and an electromagnetic wave.

[0115] Specifically, the transmitting and receiving unit includes a transmitting unit located at the head-mounted display of the VR device and a receiving unit located at the neck-hanging part, the transmitting unit is used to transmit a target signal, and the receiving unit is used to determine the relative position between the head-mounted display and the speaker based on the received signal, and determine the relative position relationship between the speaker and the target part based on the relative position between the head-mounted display and the speaker, and the target signal includes at least one of the following: non-sinusoidal narrow pulse, ultrasonic wave, optical signal, electromagnetic wave.

[0116] In this embodiment, the transmitting and receiving units can locate the position of the neckband portion relative to the head-mounted display. Since the relative positions of the user's target area and the head-mounted display are essentially fixed, and the relative positions of the neckband portion and the speaker are essentially fixed, the relative position between the user's target area and the speaker can be calculated. In this embodiment, the receiving unit must be fixed to the neckband portion during positioning and cannot move or shake.

[0117] Of course, in this embodiment, the transmitting unit can also be set on the neck hanging part, and the receiving unit can be set on the head-mounted display of the VR device.

[0118] In some embodiments, the positioning unit may include an image processing unit, and locating the relative position relationship between the speaker and the target part includes:

[0119] The image processing unit is used to obtain an image of the target part, and the relative position relationship between the speaker and the target part is determined according to the image.

[0120] In this embodiment, the image processing unit can utilize a monocular camera, a binocular camera, a depth camera, or the like. When the audio device is capable of capturing an image of the user's target area, the image processing unit captures the image of the target area and, through coordinate system transformation, locates the relative position of the speaker to the target area. For example, if the target area is the ear, image processing can be used to segment the ear from the image, determine the location of the ear canal, and, through coordinate system transformation, locate the position of each speaker relative to the ear ( Ploudspeaker_j ).

[0121] In this embodiment, during positioning, the image processing unit needs to be fixed to the neck hanging part and cannot move or shake.

[0122] In some embodiments, the positioning unit may include an inertial sensor IMU, and the positioning of the relative position relationship between the speaker and the target part includes:

[0123] The inertial sensor IMU is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck hanging part, and the relative position between the head-mounted display and the speaker is determined based on the recorded relative rotation and displacement. The relative position relationship between the speaker and the target part is determined based on the relative position between the head-mounted display and the speaker.

[0124] The inertial sensor (IMU) can be combined with other absolute positioning methods to determine the relative position between the target part and the speaker. For example, after the image processing unit has been used to locate the relative position between the target part and the speaker, if the user rotates their head, preventing the image processing unit from capturing the ear image, the relative rotation and displacement between the head-mounted display and the neck hanging portion, recorded by the inertial sensor (IMU), can be used to determine the relative position of the ear relative to each speaker. In this embodiment, the inertial sensor must be fixed to the neck hanging portion during positioning and cannot move or shake.

[0125] In this embodiment, the positioning unit may include only one of the transmitting and receiving units, the image processing unit and the inertial sensor, or may include a combination of multiple transmitting and receiving units, the image processing unit and the inertial sensor. When the positioning unit adopts a combination of multiple transmitting and receiving units, the image processing unit and the inertial sensor, the positioning accuracy can be improved.

[0126] It should be noted that the various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences from other embodiments. In particular, since the embodiments are generally similar to the product embodiments, the description is relatively simple. For relevant parts, refer to the partial description of the product embodiments.

[0127] Unless otherwise defined, the technical or scientific terms used in this disclosure should have the usual meanings understood by persons of ordinary skill in the field to which this disclosure belongs. The words "first", "second" and similar terms used in this disclosure do not indicate any order, quantity or importance, but are only used to distinguish different components. Words such as "include" or "comprise" mean that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, without excluding other elements or objects. Words such as "connect" or "connected" are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. "Up", "down", "left", "right" and the like are only used to indicate relative positional relationships. When the absolute position of the object being described changes, the relative positional relationship may also change accordingly.

[0128] It will be understood that when an element such as a layer, film, region, or substrate is referred to as being “on” or “under” another element, it can be “directly on” or “under” the other element or intervening elements may be present.

[0129] In the description of the above embodiments, specific features, structures, materials or characteristics may be combined in an appropriate manner in any one or more embodiments or examples.

[0130] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An audio device, characterized in that: include: Vocal cavity; at least one speaker located within the acoustic cavity; An acquisition unit, used to respectively acquire audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data; A calculation unit is used to process the audio data according to the target HRTF, generate a target sound signal, and output the target sound signal to the speaker.

2. The audio device according to claim 1, characterized in that The audio device is a neck-hanging audio device, and the sound cavity is located in the neck-hanging part of the neck-hanging audio device.

3. The audio device according to claim 2, characterized in that The audio device further comprises: a power source located in the neck hanging portion; The speaker is connected to the power source via a wired manner.

4. The audio device according to claim 2, characterized in that The audio device further comprises: At least one low-frequency passive diaphragm and / or audio waveguide is located in the acoustic cavity.

5. The audio device according to claim 4, characterized in that The low-frequency passive diaphragm is spaced apart from the loudspeaker; and / or The audio waveguide is spaced apart from the speaker.

6. The audio device according to any one of claims 2 to 5, characterized in that: The audio device is applied to a VR device, and the audio device further comprises: A positioning unit, used to locate the relative position relationship between the speaker and the target part, wherein the target part includes the head and / or the ear; The calculation unit is specifically used to obtain a first position of the virtual sound source in the spatial coordinate system of the VR device; obtain a second position of the target part in the spatial coordinate system, and determine a third position of the speaker in the spatial coordinate system according to a relative position relationship between the speaker and the target part; and generate the target sound signal according to the target HRTF at the first position, the HRTF at the third position, and the audio data.

7. The audio device according to claim 6, characterized in that The positioning unit includes at least one of the following: A transmitting and receiving unit, comprising a transmitting unit located at the head-mounted display of the VR device and a receiving unit located at the neck hanging part, wherein the transmitting unit is used to transmit a target signal, and the receiving unit is used to determine the relative position between the head-mounted display and the speaker according to the received signal, and determine the relative position relationship between the speaker and the target part according to the relative position between the head-mounted display and the speaker, wherein the target signal comprises at least one of the following: a non-sinusoidal narrow pulse, an ultrasonic wave, an optical signal, and an electromagnetic wave; An image processing unit disposed on the neck hanging part is used to obtain an image of the target part and determine a relative position relationship between the speaker and the target part according to the image; The inertial sensor IMU arranged on the neck hanging part is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck hanging part, determine the relative position between the head-mounted display and the speaker according to the recorded relative rotation and displacement, and determine the relative position relationship between the speaker and the target part according to the relative position between the head-mounted display and the speaker.

8. The audio device according to claim 2, characterized in that The audio device further comprises: External interface for connecting to wired headphones.

9. The audio device according to claim 2, characterized in that The audio device further comprises: A microphone is arranged on the neck hanging part.

10. A VR device, characterized in that: The invention comprises a head mounted display and an audio device as claimed in any one of claims 1 to 9.

11. A method for operating an audio device, characterized in that: Applied to the audio device according to any one of claims 1 to 9, the working method comprises: Respectively obtain audio data and a target ear spectrum curve HRTF at a virtual sound source position corresponding to the audio data; The audio data is processed according to the target HRTF to generate a target sound signal, and the target sound signal is output to the speaker.

12. The operating method of the audio device according to claim 11, characterized in that: The sound signal includes a right channel signal and a left channel signal. Before outputting the target sound signal to the speaker, the method further includes: The crosstalk signal generated by the right channel signal is eliminated from the left channel signal, and the crosstalk signal generated by the left channel signal is eliminated from the right channel signal.

13. The operating method of the audio device according to claim 11, characterized in that: Applied to the audio device as claimed in claim 6, the working method further comprises: Locating the relative position relationship between the speaker and the target part, wherein the target part includes the head and / or the ear; The processing of the audio data according to the target HRTF to generate a target sound signal comprises: Acquire a first position of the virtual sound source in the spatial coordinate system of the VR device; acquire a second position of the target part in the spatial coordinate system, and determine a third position of the speaker in the spatial coordinate system according to a relative position relationship between the speaker and the target part; generate the target sound signal according to the target HRTF at the first position, the HRTF at the third position and the audio data.

14. The operating method of the audio device according to claim 13, characterized in that: Applied to the audio device as claimed in claim 7, the positioning of the relative position relationship between the speaker and the target part includes at least one of the following: The target signal is transmitted by the transmitting unit, and the relative position between the head mounted display and the speaker is determined by the receiving unit according to the received signal, and the relative position relationship between the speaker and the target part is determined according to the relative position between the head mounted display and the speaker, wherein the target signal includes at least one of the following: non-sinusoidal narrow pulse, ultrasonic wave, optical signal, electromagnetic wave; Acquiring an image of the target part by using the image processing unit, and determining a relative position relationship between the speaker and the target part according to the image; The inertial sensor IMU is used to record the relative rotation and displacement between the head-mounted display of the VR device and the neck hanging part, and the relative position between the head-mounted display and the speaker is determined according to the recorded relative rotation and displacement. The relative position between the speaker and the target part is determined by the relative position between them.

Images