Stereo audio signal adjusting method and device, head-mounted equipment and storage medium

Abstract

The invention relates to the technical field of virtuality, and discloses a stereo audio signal adjusting method and device, head-mounted equipment and a storage medium, which are used for improving the sound field space sense and user experience of the head-mounted equipment. The stereo audio signal adjusting method comprises the steps of obtaining motion parameters of a head-mounted device when it is detected that the pose of the head-mounted device changes, determining a reference position of a virtual sound source of the head-mounted device according to an initial stereo audio signal of the head-mounted device, determining a target position of the virtual sound source according to the motion parameters and the reference position of the virtual sound source, and adjusting the stereo audio signal according to the target position. And adjusting the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

Description

TECHNICAL FIELD

[0001] The present application relates to the virtual technical field, and particularly to a method and device for adjusting a stereo audio signal, a head-mounted device, and a storage medium. BACKGROUND

[0002] With the development of augmented reality technology, head-mounted devices such as VR glasses and AR glasses have appeared in the public eye. A head-mounted device includes a frame, a temple, and a display lens. At least two sound emitting devices are usually arranged on the head-mounted device. When a user watches a movie or plays a game through the head-mounted device, the sound emitting devices play stereo sound in coordination with the picture in the display lens to obtain an immersive experience.

[0003] In the prior art, the sound emitting devices play a stereo audio signal configured for a movie or a game. According to the sound source positioning effect, the position of a virtual sound source can be determined. Even if the position of the head of the wearer changes, the relative position between the virtual sound source and the head of the wearer remains unchanged, which causes the wearer to have a confused feeling about the sound field space and affects the immersive experience. SUMMARY

[0004] The main purpose of the present application is to provide a method and device for adjusting a stereo audio signal, a head-mounted device, and a storage medium, which are used to solve the problem of the relative position between a virtual sound source and a head-mounted device remaining unchanged, causing the wearer to have a confused feeling about the sound field space, and improve the sound field space feeling and user experience of the head-mounted device.

[0005] The first aspect of the present application provides a method for adjusting a stereo audio signal, comprising: obtaining a motion parameter of a head-mounted device when detecting that the pose of the head-mounted device changes; determining a reference position of a virtual sound source of the head-mounted device according to an initial stereo audio signal of the head-mounted device; determining a target position of the virtual sound source according to the motion parameter and the reference position of the virtual sound source; and adjusting the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0006] The second aspect of the present application provides a device for adjusting a stereo audio signal, comprising: an obtaining module configured to obtain a motion parameter of a head-mounted device when detecting that the pose of the head-mounted device changes; a first determining module configured to determine a reference position of a virtual sound source of the head-mounted device according to an initial stereo audio signal of the head-mounted device; a second determining module configured to determine a target position of the virtual sound source according to the motion parameter and the reference position of the virtual sound source; and an adjusting module configured to adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0007] The third aspect of the present application provides a head-mounted device, comprising a memory and at least one processor, the memory storing instructions; the at least one processor invokes the instructions in the memory to enable the head-mounted device to perform the above-mentioned adjustment method of the stereophonic audio signal.

[0008] The fourth aspect of the present application provides a computer-readable storage medium, which stores instructions, when running on a computer, enabling the computer to perform the above-mentioned adjustment method of the stereophonic audio signal.

[0009] In the technical solution provided by the present application, when the change of the pose of the head-mounted device is detected, the motion parameters of the head-mounted device are acquired, the reference position of the virtual sound source of the head-mounted device is determined according to the initial stereophonic audio signal of the head-mounted device, the target position of the virtual sound source is determined according to the motion parameters and the reference position of the virtual sound source, the initial stereophonic audio signal is adjusted based on the target position to obtain a target stereophonic audio signal. In the embodiment of the present application, the reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereophonic audio signal, the target position of the virtual sound source relative to the head-mounted device after the change of the pose of the head-mounted device is determined through the motion parameters of the head-mounted device, and the stereophonic audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of the motion of the head-mounted device, the feeling of the wearer to the sound field space is avoided from being confused, and the sound field space feeling and user experience of the head-mounted device are improved. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 An embodiment schematic diagram of the adjustment method of the stereophonic audio signal in the embodiment of the present application;

[0011] Figure 2 Another embodiment schematic diagram of the adjustment method of the stereophonic audio signal in the embodiment of the present application;

[0012] Figure 3 An embodiment schematic diagram of the adjustment device of the stereophonic audio signal in the embodiment of the present application;

[0013] Figure 4 Another embodiment schematic diagram of the adjustment device of the stereophonic audio signal in the embodiment of the present application;

[0014] Figure 5 An embodiment schematic diagram of the head-mounted device in the embodiment of the present application. DETAILED DESCRIPTION

[0015] The embodiment of the present application provides an adjustment method and device of a stereophonic audio signal, a head-mounted device and a storage medium, which are used to improve the sound field space feeling and user experience of the head-mounted device.

[0016] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above and below, if any, are used for distinguishing between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so construed herein can be interchanged, such that the embodiments described herein can be carried out in another order than the one described here without departing from the scope of the application. Also, the terms "comprising", "having", "including" and "containing" are to be construed open-ended, i.e., meaning "including, but not limited to", "comprising, but not limited to", "having, but not limited to", or "including, but not limited to", respectively, and allow for additional steps, elements, units, options, etc., apart from those specifically recited.

[0017] For the sake of understanding, the specific flow of the embodiments of the present application is described below. It can be understood that the execution subject of the present application can be a stereo audio signal adjustment device, and can also be a head-mounted device, which is not limited here. The embodiments of the present application are described by taking the head-mounted device as an example.

[0018] Referring to Figure 1 , one embodiment of the stereo audio signal adjustment method in the embodiments of the present application comprises:

[0019] 101. When it is detected that the pose of the head-mounted device changes, the motion parameters of the head-mounted device are acquired.

[0020] The head-mounted device is an AR (Augmented Reality) glass or a VR (Virtual Reality) glass. The AR glass superimposes a virtual picture in a real scene, and the VR glass constructs a virtual scene. The wearer can play a game, watch an immersive movie or interact with the virtual picture through the AR glass or the VR glass. At least one sound emitting device is arranged on each leg of the AR glass or the VR glass, and the sound emitting device is a loudspeaker.

[0021] An inertial sensor for detecting the pose of the head-mounted device is further arranged on the head-mounted device. The motion parameters of the head-mounted device are acquired through the inertial sensor on the head-mounted device. The inertial sensor comprises one or more of an accelerometer, a magnetometer and a gyroscope. The motion parameters of the head-mounted device are used to indicate the change in the pose of the head-mounted device. The motion parameters comprise one or more of a rotation direction, a rotation angle, a movement direction and a movement distance.

[0022] 102. The reference position of the virtual sound source of the head-mounted device is determined according to the initial stereo audio signal of the head-mounted device.

[0023] The initial stereo audio signal comprises at least a left channel audio signal and a right channel audio signal. The reference position of the virtual sound source is determined according to the phase difference and the amplitude difference between the left channel audio signal and the right channel audio signal.

[0024] The virtual sound source is a virtual sound source emitting the initial stereo audio signal, and the coordinate system in which the reference position is located is a world coordinate system centered on the head-mounted device.

[0025] 103. Determine the target position of the virtual sound source according to the motion parameters and the reference position of the virtual sound source.

[0026] No matter whether the head-mounted device moves or not, the reference position obtained according to the initial stereo audio signal is always unchanged, that is, the relative position relationship between the head-mounted device and the virtual sound source does not change, and the position of the virtual sound source in the sound field space should be fixed. When the head-mounted device moves, the relative position relationship between the head-mounted device and the virtual sound source should change, and for the head-mounted device, the position of the virtual sound source should change. The change direction of the position of the virtual sound source is opposite to the motion direction of the head-mounted device, the change distance of the position of the virtual sound source is the same as the motion distance of the head-mounted device. Based on the change direction of the position of the virtual sound source, the change distance of the position of the virtual sound source and the reference position, the target position can be determined. The target position is the correct position of the virtual sound source in the sound field space.

[0027] 104. Adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0028] Adjust the phase and amplitude of each channel audio signal in the initial stereo audio signal according to the direction and distance between the target position and the head-mounted device to obtain a target stereo audio signal.

[0029] In the embodiment of the application, the reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereo audio signal. The target position of the virtual sound source relative to the head-mounted device after the change of the pose of the head-mounted device is determined by the motion parameters of the head-mounted device. The stereo audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of movement of the head-mounted device, avoid the confusion of the wearer's feeling of the sound field space, and improve the sound field space feeling and user experience of the head-mounted device.

[0030] Please refer to Figure 2 Another embodiment of the adjustment method of the stereo audio signal in the embodiment of the application includes:

[0031] 201. When it is detected that the pose of the head-mounted device changes, obtain the motion parameters of the head-mounted device.

[0032] The head-mounted device is an AR (Augmented Reality) glass or a VR (Virtual Reality) glass, the AR glass superimposes a virtual picture on a real scene, and the VR glass constructs a virtual scene, a wearer can play a game, watch an immersive movie or interact with the virtual picture through the AR glass or the VR glass, at least one sound emitting device is arranged on each leg of the AR glass or the VR glass, the sound emitting device is a loudspeaker, and an inertial sensor for detecting a pose of the head-mounted device is further arranged on the head-mounted device.

[0033] Based on different application requirements and cost limitations, the types of sensors installed on different head-mounted devices are different, for example, a VR glass for watching an immersive movie is only installed with a gyroscope, and an AR glass for playing a parkour game is installed with a gyroscope and an acceleration sensor.

[0034] In the embodiment, the head-mounted device includes a gyroscope and / or a three-axis acceleration sensor, the gyroscope measures angular velocity change according to the inertial theorem, the three-axis acceleration sensor measures acceleration by using the principle of force balance, measures rotational acceleration and movement acceleration, and determines a movement direction and a movement distance according to rotational values and movement values accumulated in time by the rotational acceleration and the movement acceleration.

[0035] When it is detected that the pose of the head-mounted device changes, pose change information is collected through the gyroscope and / or the three-axis acceleration sensor on the head-mounted device, specifically, a moment when the pose of the head-mounted device changes is determined as an initial moment, information collected by the gyroscope is determined as attitude change information, the attitude change information is rotational information accumulated by the head-mounted device from the initial moment, the rotational information includes roll angular velocity, pitch angular velocity and yaw angular velocity, information collected by the three-axis acceleration sensor is determined as position change information, the position change information is movement information accumulated by the head-mounted device from the initial moment, the movement information includes a movement direction and a movement distance, and the attitude change information and / or the position change information are determined as motion parameters.

[0036] 202. Determine a reference position of a virtual sound source of the head-mounted device according to an initial stereo audio signal of the head-mounted device.

[0037] Decompose audio signals of left and right channels from the initial stereo audio signal of the head-mounted device to obtain a first audio signal and a second audio signal, and determine a candidate position of the virtual sound source according to the first audio signal and the second audio signal.

[0038] Affected by environmental factors, the transmission ability of sound in different scenes is different, and the change rate of sound is different, for example, the auditory effect of the same position sound source in an open scene and a closed scene is different, therefore, the transmission coefficient corresponding to the initial stereo audio signal is obtained, the transmission coefficient indicates the fast and slow of signal change in the sound scene corresponding to the initial stereo audio signal, the smaller the transmission coefficient represents the greater the environmental influence, the greater the transmission coefficient represents the greater the environmental influence, the transmission coefficient is obtained in the following two ways:

[0039] The first kind, the initial stereo audio signal is time-frequency converted to obtain a plurality of single frequency signals, the proportion of high frequency signals and the proportion of low frequency signals in the plurality of single frequency signals are determined, and the corresponding transmission coefficient is queried from the preset frequency coefficient table according to the proportion of high frequency signals and the proportion of low frequency signals.

[0040] The second kind, if the initial stereo audio signal is played together with the video, the video frames are collected from the video to obtain a video frame sequence, the transmission environment of the initial stereo audio signal is identified according to the video frame sequence, and the corresponding transmission coefficient is queried from the preset environment coefficient table according to the transmission environment.

[0041] The head-mounted device determines the reference position according to the transmission coefficient and the candidate position, and the position difference between the reference position and the candidate position is proportional to the transmission coefficient.

[0042] The head-mounted device determines the reference position according to the transmission coefficient and the candidate position, which excludes the influence of the sound scene on the virtual sound source position, and improves the accuracy of the reference position.

[0043] 203, analyze the motion parameters to obtain device rotation parameters and / or device movement parameters, the rotation parameters include the rotation direction and rotation angle of the head-mounted device, and the movement parameters include the movement direction and movement distance of the head-mounted device.

[0044] When the head-mounted device includes a gyroscope, the rotation direction and rotation angle of the roll angle, pitch angle and yaw angle are extracted from the motion parameters to obtain the rotation direction and rotation angle of the head-mounted device.

[0045] When the head-mounted device includes a three-axis acceleration sensor, the movement direction and movement distance are extracted from the motion parameters to obtain the movement direction and movement distance of the head-mounted device.

[0046] 204, the opposite direction of the rotation direction and the rotation angle are determined as the sound source rotation parameters.

[0047] For example, the head-mounted device determines that the reference position of the virtual sound source according to the initial stereo audio signal is in front of the head-mounted device. When the head-mounted device rotates 90° to the left, the virtual sound source should be in the position right of the head-mounted device, but at this time the reference position detected according to the initial stereo audio signal indicates that the virtual sound source is still in front of the head-mounted device, which is incorrect, and the correct sound emitting position is not obtained. The virtual sound source should be rotated 90° to the right based on the reference position.

[0048] Therefore, the rotation direction of the virtual sound source should be opposite to the rotation direction of the head-mounted device, and the rotation angle of the virtual sound source should be the same as the rotation angle of the head-mounted device. When the motion parameter includes the rotation direction and the rotation angle of the head-mounted device, the sound source rotation parameter can be determined according to the rotation direction and the rotation angle of the head-mounted device. The sound source rotation parameter includes a sound source rotation angle and a sound source rotation direction. The sound source rotation angle is the same as the rotation angle of the head-mounted device, and the sound source rotation direction is opposite to the rotation direction of the head-mounted device.

[0049] It should be noted that the rotation direction and the corresponding rotation angle detected by the gyroscope have three, which are the rotation direction and the rotation angle of the roll angle, the rotation direction and the rotation angle of the pitch angle, and the rotation direction and the rotation angle of the yaw angle. The rotation direction and the rotation angle of the head-mounted device have three, and therefore the sound source rotation parameter also includes three rotation directions and rotation angles.

[0050] 205、determine the opposite direction of the movement direction and the movement distance as the sound source movement parameter.

[0051] For example, when the head-mounted device determines that the sound emitting position of the virtual sound source according to the initial stereo audio signal is 10m to the left of the head-mounted device, the head-mounted device moves 5m to the right away from the virtual sound source, but at this time the reference position detected according to the initial stereo audio signal indicates that the virtual sound source is still 10m to the left of the head-mounted device, which is incorrect, and the correct sound emitting position is not obtained. The virtual sound source should move 5m to the left based on the reference position.

[0052] Therefore, the movement direction of the virtual sound source position is opposite to the movement direction of the head-mounted device, and the movement distance of the virtual sound source position is the same as the movement distance of the head-mounted device. When the motion parameter includes the movement direction and the movement distance, the sound source movement parameter can be determined according to the movement direction and the movement distance of the head-mounted device. The sound source movement parameter includes a sound source movement direction and a sound source movement distance. The sound source movement direction is opposite to the movement direction of the head-mounted device, and the sound source movement distance is the same as the movement distance of the head-mounted device.

[0053] 206、determine the target position of the virtual sound source according to at least one of the sound source rotation parameter and the sound source movement parameter, and the reference position of the virtual sound source.

[0054] In step 201, the types of sensors installed on different head-mounted devices are different, and the motion parameters of the head-mounted device obtained by the sensors are also different. At least one of the sound source rotation parameter and the sound source movement parameter is generated according to the motion parameters of the head-mounted device. In the present fact example, when the head-mounted device is installed with a gyroscope or a three-axis acceleration sensor, the target position of the virtual sound source is determined according to the sound source rotation parameter or the sound source movement parameter and the reference position of the virtual sound source; when the head-mounted device is installed with a gyroscope and a three-axis acceleration sensor, the target position of the virtual sound source is determined according to the sound source rotation parameter, the sound source movement parameter and the reference position of the virtual sound source.

[0055] 207. Adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0056] When the head-mounted device contains two or more sound emitting devices, the distribution of the two or more sound emitting devices on the head-mounted device is determined to obtain at least two distribution information. The distribution information is used to indicate the position distribution of the sound emitting devices on the head-mounted device. The phase modulation parameter and the amplitude modulation parameter corresponding to each sound emitting device are generated according to the target position and the at least two distribution information. The phase modulation parameter corresponding to each sound emitting device is determined as the target phase modulation parameter, and the amplitude modulation parameter corresponding to each sound emitting device is determined as the target amplitude modulation parameter.

[0057] On this basis, in order to improve the interaction with the real scene, the sound effect modulation parameter is determined based on the current environment scene of the head-mounted device. The sound effect modulation parameter is an environmental sound parameter matched with the environment scene. The candidate stereo audio signal is adjusted according to the sound effect modulation parameter to obtain the target stereo audio signal.

[0058] Optionally, the environmental information is collected by the environmental sensors on the head-mounted device. The environmental sensors include wind, light and rain sensors and noise sensors. The environmental sound factor is determined according to the environmental information. The environmental sound factor is a noise factor in the current environment scene, such as wind sound, car horn sound, etc. The sound effect modulation parameter matched with the current environment scene is queried from the preset sound effect parameter library according to the environmental sound factor. The candidate stereo audio signal is adjusted according to the sound effect modulation parameter to obtain the target stereo audio signal.

[0059] In the embodiment of the present application, the reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereo audio signal, the target position of the virtual sound source relative to the head-mounted device after the change of the pose of the head-mounted device is determined through the motion parameters of the head-mounted device, and the stereo audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of motion of the head-mounted device, avoid the confusion of the wearer's feeling of the sound field space, and improve the sound field space feeling of the head-mounted device. In addition, the sound effect modulation parameters matched with the current environment scene are obtained, the interaction with the real scene is improved, and the user experience is further improved.

[0060] The adjustment method of the stereo audio signal in the embodiment of the present application is described above, and the adjustment device of the stereo audio signal in the embodiment of the present application is described below. Please refer to Figure 3 The adjustment device of the stereo audio signal in the embodiment of the present application includes one embodiment:

[0061] The acquisition module 301 is configured to acquire the motion parameters of the head-mounted device when detecting the change of the pose of the head-mounted device.

[0062] The first determination module 302 is configured to determine the reference position of the virtual sound source of the head-mounted device according to the initial stereo audio signal of the head-mounted device.

[0063] The second determination module 303 is configured to determine the target position of the virtual sound source according to the motion parameters and the reference position of the virtual sound source.

[0064] The adjustment module 304 is configured to adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0065] In the embodiment of the present application, the reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereo audio signal, the target position of the virtual sound source relative to the head-mounted device after the change of the pose of the head-mounted device is determined through the motion parameters of the head-mounted device, and the stereo audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of motion of the head-mounted device, avoid the confusion of the wearer's feeling of the sound field space, and improve the sound field space feeling of the head-mounted device. In addition, the sound effect modulation parameters matched with the current environment scene are obtained, the interaction with the real scene is improved, and the user experience is further improved.

[0066] Please refer to Figure 4 Another embodiment of the adjustment device of the stereo audio signal in the embodiment of the present application includes:

[0067] The acquisition module 301 is configured to acquire the motion parameters of the head-mounted device when detecting the change of the pose of the head-mounted device.

[0068] The first determining module 302 is configured to determine a reference position of a virtual sound source of the head-mounted device according to an initial stereo audio signal of the head-mounted device.

[0069] The second determining module 303 is configured to determine a target position of the virtual sound source according to the motion parameter and the reference position of the virtual sound source.

[0070] The adjusting module 304 is configured to adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

[0071] Optionally, the obtaining module 301 is specifically configured to:

[0072] The moment when the pose of the head-mounted device changes is determined as an initial moment; information collected by the gyroscope is determined as the attitude change information, the attitude change information being rotation information accumulated by the head-mounted device since the initial moment; information collected by the three-axis acceleration sensor is determined as the position change information, the position change information being movement information accumulated by the head-mounted device since the initial moment; and the attitude change information and / or the position change information is determined as the motion parameter.

[0073] Optionally, the first determining module 302 is specifically configured to:

[0074] The audio signals of left and right channels are decomposed from the initial stereo audio signal of the head-mounted device to obtain a first audio signal and a second audio signal; a candidate position of the virtual sound source is determined according to the first audio signal and the second audio signal; a transmission coefficient corresponding to the initial stereo audio signal is obtained, the transmission coefficient indicating the speed of signal change in a sound scene corresponding to the initial stereo audio signal; and the reference position is determined according to the transmission coefficient and the candidate position.

[0075] Optionally, the second determining module 303 is specifically configured to:

[0076] The motion parameter is analyzed to obtain a device rotation parameter and / or a device movement parameter, the rotation parameter including a rotation direction and a rotation angle of the head-mounted device, and the movement parameter including a movement direction and a movement distance of the head-mounted device; an opposite direction of the rotation direction and the rotation angle are determined as a sound source rotation parameter; an opposite direction of the movement direction and the movement distance are determined as a sound source movement parameter; and the target position of the virtual sound source is determined according to at least one of the sound source rotation parameter and the sound source movement parameter and the reference position of the virtual sound source.

[0077] Optionally, the adjusting module 304 includes:

[0078] The first determination unit 3041 is configured to determine a target phase modulation parameter and a target amplitude modulation parameter based on the target position and the pose of the head-mounted device;

[0079] The first adjustment unit 3042 is configured to adjust the initial stereo audio signal according to the target phase modulation parameter and the target amplitude modulation parameter to obtain a candidate stereo audio signal.

[0080] The second determination unit 3043 is configured to determine an audio effect modulation parameter based on a current environmental scene of the head-mounted device, the audio effect modulation parameter being an environmental sound parameter matched with the environmental scene.

[0081] The second adjustment unit 3044 is configured to adjust the candidate stereo audio signal according to the audio effect modulation parameter to obtain a target stereo audio signal.

[0082] Optionally, the second determination unit 3043 is specifically configured to:

[0083] collect environmental information through an environmental sensor on the head-mounted device; determine an environmental sound factor according to the environmental information, the environmental sound factor being a noise factor in the current environmental scene; and generate the audio effect modulation parameter matched with the current environmental scene according to the environmental sound factor.

[0084] Optionally, the first adjustment unit 3042 is specifically configured to:

[0085] When the head-mounted device comprises two or more sound emitting devices, the first adjustment unit 3042 is specifically configured to: determine distribution information of the two or more sound emitting devices on the head-mounted device; generate a phase modulation parameter and an amplitude modulation parameter corresponding to each sound emitting device according to the target position and the distribution information; determine the phase modulation parameter corresponding to each sound emitting device as the target phase modulation parameter; and determine the amplitude modulation parameter corresponding to each sound emitting device as the target amplitude modulation parameter.

[0086] In the embodiment of the application, the reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereo audio signal, the target position of the virtual sound source relative to the head-mounted device after the pose of the head-mounted device changes is determined through the motion parameter of the head-mounted device, the stereo audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of movement of the head-mounted device, the feeling of the wearer to the sound field space is avoided from being confused, the sound field space feeling of the head-mounted device is improved, in addition, the audio effect modulation parameter matched with the current environmental scene is obtained, the interaction with the real scene is improved, and the user experience is further improved.

[0087] The above Figure 3 and Figure 4The adjustment device of the stereo audio signal in the embodiment of the application is described in detail from the perspective of the modular functional entity, and the head-mounted device in the embodiment of the application is described in detail from the perspective of hardware processing.

[0088] Referring to Figure 5 The head-mounted device includes a processor 500 and a memory 501, and the memory 501 stores machine executable instructions capable of being executed by the processor 500, and the processor 500 executes the machine executable instructions to implement the above-mentioned adjustment method of the stereo audio signal.

[0089] Further, Figure 5 The head-mounted device shown in the figure further includes a bus 502 and a communication interface 503, and the processor 500, the communication interface 503 and the memory 501 are connected through the bus 502.

[0090] The memory 501 can include a high-speed random access memory (RAM), and can also include a non-volatile memory, for example, at least one disk memory. The communication connection between the system network element and at least one other network element is realized through at least one communication interface 503 (which can be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. can be used. The bus 502 can be an ISA bus, a PCI bus or an EISA bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the convenience of representation, Figure 5 Only one bidirectional arrow is used in the figure to represent the bus, but it does not mean that there is only one bus or only one type of bus.

[0091] The processor 500 can be an integrated circuit chip with signal processing capability. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware in the processor 500 or the instruction in the form of software. The processor 500 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. Each method, step and logic block diagram disclosed in the embodiments of the present disclosure can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present disclosure can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read only memory, a programmable read only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 501, and the processor 500 reads the information in the memory 501, and combines the hardware to complete the method steps of the above embodiments.

[0092] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium, and can also be a volatile computer readable storage medium, and the computer readable storage medium stores instructions, when the instructions run on the computer, make the computer execute the steps of the adjusting method of the stereo audio signal, and the steps specifically include:

[0093] When the change in the pose of the head-mounted device is detected, the motion parameters of the head-mounted device are acquired, the reference position of the virtual sound source of the head-mounted device is determined according to the initial stereo audio signal of the head-mounted device, the target position of the virtual sound source is determined according to the motion parameters and the reference position of the virtual sound source, and the initial stereo audio signal is adjusted based on the target position to obtain a target stereo audio signal. The reference position of the virtual sound source relative to the head-mounted device is determined based on the initial stereo audio signal, the target position of the virtual sound source relative to the head-mounted device after the change in the pose of the head-mounted device is determined through the motion parameters of the head-mounted device, and the stereo audio signal is adjusted based on the target position, so that the wearer can perceive the position change of the virtual sound source in the case of movement of the head-mounted device, avoid confusion of the wearer in the perception of the sound field space, and improve the sound field space feeling and user experience of the head-mounted device.

[0094] The above head-mounted device includes a gyroscope and / or a three-axis acceleration sensor, and the motion parameters of the head-mounted device are acquired by: determining the moment when the pose of the head-mounted device changes as an initial moment; determining the information collected by the gyroscope as attitude change information, which is the rotation information accumulated by the head-mounted device from the initial moment; determining the information collected by the three-axis acceleration sensor as position change information, which is the movement information accumulated by the head-mounted device from the initial moment; and determining the attitude change information and / or the position change information as the motion parameters.

[0095] The above determination of the reference position of the virtual sound source of the head-mounted device according to the initial stereo audio signal of the head-mounted device includes: decomposing the audio signals of the left channel and the right channel from the initial stereo audio signal of the head-mounted device to obtain first and second audio signals; determining a candidate position of the virtual sound source according to the first and second audio signals; acquiring a transmission coefficient corresponding to the initial stereo audio signal, the transmission coefficient indicating the speed of signal change in the sound scene corresponding to the initial stereo audio signal; and determining the reference position according to the transmission coefficient and the candidate position.

[0096] The above determination of the target position of the virtual sound source according to the motion parameters and the reference position of the virtual sound source includes: analyzing the motion parameters to obtain device rotation parameters and / or device movement parameters, the rotation parameters including the rotation direction and rotation angle of the head-mounted device, and the movement parameters including the movement direction and movement distance of the head-mounted device; determining the opposite direction of the rotation direction and the rotation angle as sound source rotation parameters; determining the opposite direction of the movement direction and the movement distance as sound source movement parameters; and determining the target position of the virtual sound source according to at least one of the sound source rotation parameters and the sound source movement parameters and the reference position of the virtual sound source.

[0097] The adjusting the initial stereo audio signal based on the target position to obtain the target stereo audio signal includes: determining a target phase modulation parameter and a target amplitude modulation parameter based on the target position and a pose of the head-mounted device; adjusting the initial stereo audio signal according to the target phase modulation parameter and the target amplitude modulation parameter to obtain a candidate stereo audio signal; determining an audio effect modulation parameter based on a current environment scene of the head-mounted device, the audio effect modulation parameter being an environmental sound parameter matched with the environment scene; and adjusting the candidate stereo audio signal according to the audio effect modulation parameter to obtain the target stereo audio signal.

[0098] The determining the audio effect modulation parameter based on the current environment scene of the head-mounted device includes: collecting environment information through an environment sensor on the head-mounted device; determining an environmental sound factor based on the environment information, the environmental sound factor being a noise factor in the current environment scene; and generating the audio effect modulation parameter matched with the current environment scene according to the environmental sound factor.

[0099] The determining the target phase modulation parameter and the target amplitude modulation parameter based on the target position and the pose of the head-mounted device includes: when the head-mounted device includes two or more sound emitting devices, determining a distribution of the two or more sound emitting devices on the head-mounted device to obtain at least two distribution information; generating a phase modulation parameter and an amplitude modulation parameter corresponding to each sound emitting device according to the target position and the at least two distribution information; determining the phase modulation parameter corresponding to each sound emitting device as the target phase modulation parameter; and determining the amplitude modulation parameter corresponding to each sound emitting device as the target amplitude modulation parameter.

[0100] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein.

[0101] The integrated unit, if realized in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0102] The above-described embodiments are only used to illustrate the technical solutions of the present application, and are not intended to limit the present application; although the present application has been described in detail with reference to the foregoing embodiments, it should be understood by those skilled in the art that the technical solutions recorded in the foregoing embodiments can still be modified, or some technical features can be replaced by equivalent replacements; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for adjusting stereo audio signals, applied to head-mounted devices, characterized in that, The method for adjusting the stereo audio signal includes: When a change in the pose of the head-mounted device is detected, the motion parameters of the head-mounted device are acquired; The reference position of the virtual sound source of the head-mounted device is determined based on the initial stereo audio signal of the head-mounted device; The target position of the virtual sound source is determined based on the motion parameters and the reference position of the virtual sound source; The initial stereo audio signal is adjusted based on the target position to obtain the target stereo audio signal.

2. The method for adjusting stereo audio signals according to claim 1, characterized in that, The head-mounted device includes a gyroscope and / or a three-axis accelerometer, and acquiring the motion parameters of the head-mounted device includes: The moment when the pose of the head-mounted device changes is defined as the initial moment; The information collected by the gyroscope is determined as attitude change information, which is the rotation information accumulated by the head-mounted device from the initial moment. The information collected by the triaxial accelerometer is determined as position change information, which is the movement information accumulated by the head-mounted device from the initial moment. The attitude change information and / or the position change information are determined as motion parameters.

3. The method for adjusting stereo audio signals according to claim 1, characterized in that, Determining the reference position of the virtual sound source of the head-mounted device based on the initial stereo audio signal of the head-mounted device includes: The left and right channel audio signals are decomposed from the initial stereo audio signal of the head-mounted device to obtain the first audio signal and the second audio signal; The candidate positions of the virtual sound source are determined based on the first audio signal and the second audio signal; Obtain the transmission coefficient corresponding to the initial stereo audio signal, wherein the transmission coefficient indicates how fast the signal changes in the sound scene corresponding to the initial stereo audio signal; The reference position is determined based on the transmission coefficient and the candidate position.

4. The method for adjusting a stereo audio signal according to any one of claims 1-3, characterized in that, Determining the target position of the virtual sound source based on the motion parameters and the reference position of the virtual sound source includes: The motion parameters are analyzed to obtain device rotation parameters and / or device movement parameters. The rotation parameters include the rotation direction and rotation angle of the head-mounted device, and the movement parameters include the movement direction and movement distance of the head-mounted device. The opposite direction of the rotation direction and the rotation angle are determined as the sound source rotation parameters; The opposite direction of the moving direction and the moving distance are determined as the sound source moving parameters; The target position of the virtual sound source is determined based on at least one of the sound source rotation parameters and the sound source movement parameters, as well as the reference position of the virtual sound source.

5. The method for adjusting stereo audio signals according to claim 1, characterized in that, The adjustment of the initial stereo audio signal based on the target position to obtain the target stereo audio signal includes: Determine the target phase modulation parameters and target amplitude modulation parameters based on the target position and the pose of the head-mounted device; The initial stereo audio signal is adjusted according to the target phase modulation parameter and the target amplitude modulation parameter to obtain a candidate stereo audio signal; The sound effect modulation parameters are determined based on the current environmental scene of the head-mounted device, and the sound effect modulation parameters are environmental sound parameters that match the environmental scene. The candidate stereo audio signal is adjusted according to the sound effect modulation parameters to obtain the target stereo audio signal.

6. The method for adjusting stereo audio signals according to claim 5, characterized in that, The process of determining sound modulation parameters based on the current environmental scene of the head-mounted device includes: Environmental information is collected through environmental sensors on the head-mounted device; The environmental sound factor is determined based on the environmental information, and the environmental sound factor is the noise factor in the current environmental scene; Based on the ambient sound factors, sound effect modulation parameters that match the current environmental scene are generated.

7. The method for adjusting stereo audio signals according to claim 5, characterized in that, Determining the target phase modulation parameters and target amplitude modulation parameters based on the target position and the pose of the head-mounted device includes: When the head-mounted device contains two or more sound-generating devices, the distribution of the two or more sound-generating devices on the head-mounted device is determined to obtain at least two distribution information. Generate phase modulation parameters and amplitude modulation parameters for each sound-generating device based on the target location and the at least two distribution information. The phase modulation parameter corresponding to each sound-generating device is determined as the target phase modulation parameter; The amplitude modulation parameters corresponding to each sound-generating device are determined as the target amplitude modulation parameters.

8. A device for adjusting stereo audio signals, characterized in that, The stereo audio signal adjustment device includes: The acquisition module is used to acquire the motion parameters of the head-mounted device when a change in the pose of the head-mounted device is detected. The first determining module is used to determine the reference position of the virtual sound source of the head-mounted device based on the initial stereo audio signal of the head-mounted device; The second determining module is used to determine the target position of the virtual sound source based on the motion parameters and the reference position of the virtual sound source; An adjustment module is used to adjust the initial stereo audio signal based on the target position to obtain a target stereo audio signal.

9. A head-mounted device, characterized in that, The head-mounted device includes: a memory and at least one processor, wherein the memory stores instructions; The at least one processor invokes the instructions in the memory to cause the head-mounted device to perform the method for adjusting the stereo audio signal as described in any one of claims 1-7.

10. A computer-readable storage medium storing instructions thereon, characterized in that, When the instruction is executed by the processor, it implements the method for adjusting the stereo audio signal as described in any one of claims 1-7.

Images