Audio output method and device and storage medium

Abstract

The invention relates to an audio output method and device and a storage medium. The audio output method comprises the following steps: monitoring the posture of a terminal in response to a to-be-output audio collected by the terminal; in response to monitoring that the posture of the terminal changes, determining a deflection angle of the terminal relative to a first position, and determining a target audio output direction based on the deflection angle, the target audio output direction being an optimal direction in which a current user of the terminal perceives sound; the first position is a position where the terminal display screen directly faces a user; and outputting the audio in the target audio output direction based on the deflection angle and the to-be-output audio information. The technical scheme provided by the invention is an audio output scheme based on the terminal. According to the invention, when the terminal deflects at different angles, the terminal calculates the audio information in real time based on the deflection angle and determines the target audio output direction, so that a better audio playing effect is provided for a user.

Description

Technical Field

[0001] The present disclosure relates to the field of spatial audio technology, and in particular to an audio output method, device and storage medium. Background Art

[0002] Spatial Audio is an algorithm optimization based on Dolby Atmos. It uses a gyroscope to increase the sound direction when the head is swung, expands the sound field, and adds a sense of space to ordinary stereo through a transformation algorithm. In spatial audio, the virtual sound field provides much wider and richer details than stereo.

[0003] In the related art, when a user plays games or watches videos on a terminal, when the terminal shakes, the audio played through the speaker will also change accordingly, resulting in a poor audio listening experience and a poor user experience. Summary of the invention

[0004] In order to overcome the problems existing in the related art, the present disclosure provides an audio output method, device and storage medium.

[0005] According to a first aspect of an embodiment of the present disclosure, there is provided an audio output method, comprising: in response to a terminal collecting audio to be output, monitoring the posture of the terminal; in response to monitoring a change in the posture of the terminal, determining a deflection angle of the terminal relative to a first position, and determining a target audio output direction based on the deflection angle, the target audio output direction being an optimal direction in which a current user of the terminal perceives sound; the first position being a position when a display screen of the terminal is facing the user; and outputting audio in the target audio output direction based on the deflection angle and the audio information to be output.

[0006] In one embodiment, determining the target audio output direction based on the deflection angle includes: determining the sound source direction information of the audio to be output, and the number of speakers; in response to playing the audio to be output through the speaker of the terminal, determining the loudness of the audio to be output played by the speaker; and determining the target audio output direction based on the deflection angle and the loudness.

[0007] In one implementation, determining the target audio output direction based on the deflection angle and the loudness includes: determining a loudness difference between a first loudness and a second loudness, and a loudness sum between the first loudness and the second loudness, wherein the first loudness is a loudness of a first speaker playing the audio to be output, and the second loudness is a loudness of a second speaker playing the audio to be output, the first speaker is a speaker located on a first side of the terminal, the second speaker is a speaker located on a second side of the terminal, the first side and the second side are two opposite sides of the terminal, and the first speaker and the second speaker each include at least one speaker; determining the target audio output direction based on the loudness difference, the loudness sum, and the deflection angle of the terminal.

[0008] In one implementation, determining the target audio output direction based on the loudness difference between the first loudness and the second loudness, the loudness sum between the first loudness and the second loudness, and the deflection angle of the terminal includes: determining a ratio between the loudness difference between the first loudness and the second loudness and the loudness sum, and determining a sine angle of the deflection angle of the terminal; and determining a product of the ratio and the sine angle of the deflection angle of the terminal as the sine angle of the target audio output direction.

[0009] In one implementation, outputting audio in the target audio output direction based on the deflection angle and the audio information to be output includes: acquiring the first loudness and the second loudness; adjusting the first loudness and the second loudness based on the deflection angle of the terminal so that the first loudness and the second loudness are equal; and outputting audio in the target audio output direction in response to the first loudness and the second loudness being equal.

[0010] In one embodiment, adjusting the first loudness and the second loudness based on the deflection angle of the terminal includes: acquiring audio track information of the sound output by the first speaker and / or the second speaker; determining the audio track to be played first based on the audio track information of the sound, and setting the audio track to be played first to be delayed based on a preset time length; determining a speaker relatively far from the user among the first speaker and the second speaker, and shifting the sound output by the speaker relatively far from the user toward the direction of the speaker relatively close to the user, wherein the shifting distance is determined based on the deflection angle of the terminal; mixing the sound output by the speaker relatively far from the user with the sound output by the speaker relatively close to the user, and weakening the sound output by the speaker relatively close to the user, so that the first loudness and the second loudness are equal; the sound output by the speaker relatively far from the user has the first loudness, and the sound output by the speaker relatively close to the user has the second loudness.

[0011] In one embodiment, the translation distance is determined based on the deflection angle of the terminal in the following manner: obtaining the length information of the terminal and determining the cosine angle of the deflection angle of the terminal; determining the product between the length information of the terminal and the cosine angle of the deflection angle of the terminal as the translation distance of the sound output by the speaker.

[0012] According to a second aspect of an embodiment of the present disclosure, an audio output device is provided, comprising: a monitoring unit, used for a terminal to collect audio to be output and monitor the posture of the terminal; a determination unit, used to monitor changes in the posture of the terminal, determine a deflection angle of the terminal relative to a first position, and determine a target audio output direction based on the deflection angle, wherein the target audio output direction is an optimal direction in which a current user of the terminal perceives sound; the first position is a position when the terminal display screen is facing the user; and a processing unit, used to output audio in the target audio output direction based on the deflection angle and the audio information to be output.

[0013] In one embodiment, the determination unit determines the target audio output direction in the following manner: determining the sound source direction information of the audio to be output and the number of speakers; in response to playing the audio to be output through the speaker of the terminal, determining the loudness of the speaker playing the audio to be output; and determining the target audio output direction based on the deflection angle and the loudness.

[0014] In one implementation, the determination unit determines the target audio output direction in the following manner: determining a loudness difference between a first loudness and a second loudness, and a loudness sum between the first loudness and the second loudness, wherein the first loudness is a loudness of a first speaker playing the audio to be output, and the second loudness is a loudness of a second speaker playing the audio to be output, the first speaker is a speaker located on a first side of the terminal, the second speaker is a speaker located on a second side of the terminal, the first side and the second side are two opposite sides of the terminal, and the first speaker and the second speaker respectively include at least one speaker; determining the target audio output direction based on the loudness difference, the loudness sum, and a deflection angle of the terminal.

[0015] In one implementation, the determination unit determines the target audio output direction in the following manner: determining a ratio between a loudness difference between the first loudness and the second loudness and the loudness sum, and determining a sine angle of a deflection angle of the terminal; and determining a product of the ratio and the sine angle of the deflection angle of the terminal as the sine angle of the target audio output direction.

[0016] In one implementation, the processing unit outputs audio in the target audio output direction based on the deflection angle and the audio information to be output in the following manner: acquiring the first loudness and the second loudness; adjusting the first loudness and the second loudness based on the deflection angle of the terminal so that the first loudness and the second loudness are equal; in response to the first loudness and the second loudness being equal, outputting audio in the target audio output direction.

[0017] In one embodiment, adjusting the first loudness and the second loudness based on the deflection angle of the terminal includes: acquiring audio track information of the sound output by the first speaker and / or the second speaker; determining the audio track to be played first based on the audio track information of the sound, and setting the audio track to be played first to be delayed based on a preset time length; determining a speaker relatively far from the user among the first speaker and the second speaker, and shifting the sound output by the speaker relatively far from the user toward the direction of the speaker relatively close to the user, wherein the shifting distance is determined based on the deflection angle of the terminal; mixing the sound output by the speaker relatively far from the user with the sound output by the speaker relatively close to the user, and weakening the sound output by the speaker relatively close to the user, so that the first loudness and the second loudness are equal; the sound output by the speaker relatively far from the user has the first loudness, and the sound output by the speaker relatively close to the user has the second loudness.

[0018] In one embodiment, the translation distance is determined based on the deflection angle of the terminal in the following manner: obtaining the length information of the terminal and determining the cosine angle of the deflection angle of the terminal; determining the product between the length information of the terminal and the cosine angle of the deflection angle of the terminal as the translation distance of the sound output by the speaker.

[0019] According to a third aspect of an embodiment of the present disclosure, there is provided an audio output device, including:

[0020] a processor; a memory for storing instructions executable by the processor;

[0021] The processor is configured to call the instructions stored in the memory to execute the audio output method as described in the first aspect or any one of the embodiments of the first aspect.

[0022] According to a fourth aspect of an embodiment of the present disclosure, a storage medium is provided, in which instructions are stored. When the instructions in the storage medium are executed by a processor of a terminal, the terminal is enabled to execute the audio output method described in the first aspect or any one of the embodiments of the first aspect.

[0023] The technical solution provided by the embodiments of the present disclosure may include the following beneficial effects: when the terminal detects a change in the current posture without connecting any peripheral device, the terminal determines the deflection angle of the terminal relative to the first position. The target audio output direction is determined based on the deflection angle, and the audio is output in the target audio output direction, thereby achieving an audio output effect based on the terminal itself. When the terminal deflects at different angles, the terminal calculates the audio information in real time based on the deflection angle and determines the target audio output direction, thereby providing users with a better audio playback effect.

[0024] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present disclosure and, together with the description, serve to explain the principles of the present disclosure.

[0026] Figure 1 The figure is a flowchart of an audio output method according to an exemplary embodiment.

[0027] Figure 2 The diagram is a schematic diagram showing a terminal being deflected according to an exemplary embodiment.

[0028] Figure 3is a schematic diagram showing angle information of a terminal according to an exemplary embodiment.

[0029] Figure 4 The present invention is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment.

[0030] Figure 5 is a schematic diagram showing a simulated sound source direction according to an exemplary embodiment.

[0031] Figure 6 The present invention is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment.

[0032] Figure 7 The diagram is a schematic diagram showing a method of determining a target output direction based on four speakers according to an exemplary embodiment.

[0033] Figure 8 The present invention is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment.

[0034] Fig. 9 It is a schematic diagram showing an audio output according to an exemplary embodiment.

[0035] Fig.10 The figure is a flow chart of an audio output method according to an exemplary embodiment.

[0036] Fig.11 The figure is a flow chart of a loudness adjustment method according to an exemplary embodiment.

[0037] Fig.12 The figure is a flow chart of a method for determining a translation distance according to an exemplary embodiment.

[0038] Fig.13 The figure is a flow chart of an audio output method according to an exemplary embodiment.

[0039] Fig.14 The invention is a block diagram of an audio output device according to an exemplary embodiment.

[0040] Fig.15 The present invention is a block diagram of an audio output device according to an exemplary embodiment. DETAILED DESCRIPTION

[0041] Here, exemplary embodiments will be described in detail, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following exemplary embodiments do not represent all implementations consistent with the present disclosure.

[0042] Spatial audio is to place the surround sound channels precisely in the right position, so that users can feel an immersive surround sound experience. For example, when a user uses a terminal to play an orchestral performance, the user can clearly feel that the sound of the main instrument comes from different directions, the sound of the cello comes from the right front, the sound of the violin comes from the left front, and so on. Through spatial audio technology, the source direction and distance of various different sounds can be truly reproduced, making the user feel as if they are in the scene.

[0043] In the related art, when a user uses a terminal to play games or watch videos, the terminal may shake and deflect due to certain operations, causing the audio played through the speaker to change accordingly, resulting in poor audio listening and a bad user experience.

[0044] In view of this, the present disclosure provides an audio output method, which can monitor the deflection angle of the terminal relative to the first position in real time, determine the output direction of the target audio in real time based on the acquired deflection angle, and output audio in the target audio output direction, thereby providing users with better audio playback effects.

[0045] Figure 1 is a flowchart of an audio output method according to an exemplary embodiment. Figure 1 As shown, the following steps are included.

[0046] In step S11, in response to the terminal collecting the audio to be output, the posture of the terminal is monitored.

[0047] In the disclosed embodiments, the terminal collects audio to be output from an application, such as game audio, video playback audio, music application, etc.

[0048] In the disclosed embodiment, the terminal includes a motion detection device for measuring the motion data of an object in space, such as a gyroscope sensor and an accelerometer sensor, etc. The terminal monitors the current placement posture of the terminal through the motion detection device, such as the deflection angle of the terminal, the distance information between the terminal and the user, etc.

[0049] In step S12, in response to detecting a change in the posture of the terminal, a deflection angle of the terminal relative to the first position is determined, and a target audio output direction is determined based on the deflection angle.

[0050] In the disclosed embodiment, the terminal detects that the current placement posture has changed, that is, the terminal has deflected, and determines the deflection angle of the terminal relative to the first position through the motion detection device. The first position is the initial placement position of the terminal, for example, the first position can be the position when the terminal display screen is facing the user.

[0051] In the embodiment of the present disclosure, the terminal determines the target audio output direction of the output audio according to the deflection angle monitored in real time, wherein the target audio output direction is the best direction of the output sound perceived by the user.

[0052] For example, Figure 2 is a schematic diagram showing a terminal deflecting according to an exemplary embodiment, such as Figure 2 As shown, the first position is the position when the terminal display screen is facing the user, and the terminal is deflected to the right by an angle of θ relative to the first position.

[0053] Furthermore, the deflection of the terminal is not limited to unilateral deflection, for example, left-right deflection, up-down deflection, etc.

[0054] In the embodiment of the present disclosure, the terminal may represent the measured deflection angle through three angles, including: a directional angle (yaw), a pitch angle (pitch), and a rotation angle (roll).

[0055] Figure 3 is a schematic diagram showing terminal angle information according to an exemplary embodiment. Figure 3 As shown, the direction angle is the angle information of the terminal deflecting around the Z axis, the pitch angle is the angle information of the terminal deflecting around the X axis, and the rotation angle is the angle information of the terminal deflecting around the Y axis.

[0056] In the disclosed embodiment, the motion detection device configured in the terminal can calculate the angle value of the current terminal deflection in real time. The angle value is represented by the azimuth angle, the pitch angle, and the rotation angle. The motion detection device monitors the deflection angle information of the terminal at a rate of n times per second, and each angle value corresponds to a current timestamp (time). For example, the motion detection device monitors the deflection angle information of the terminal at a rate of 100 times per second, that is, 100 groups of angle values ​​can be obtained within 1 second, and the timestamps corresponding to these 100 groups of angle values ​​are [t1, t2, ..., t 99 , t 100 ] is used to represent it. By default, the angle information corresponding to the first position when the user turns on this function is the initial value, and its timestamp is represented by t, and the corresponding three angle values ​​are represented by y, p, and r respectively. Similarly, the three angle values ​​corresponding to the t1 timestamp are represented by: y1, p1, and r1 respectively. The three angle change values ​​obtained by subtracting the angle value at time t1 from the angle value at the initial time t are represented by θy, θp, and θr respectively, and their absolute values ​​are taken. The calculation formula is as follows:

[0057] |θy|=y1-y

[0058] |θp|=p1-p

[0059] |θr|=r1-r

[0060] In the disclosed embodiment, the three calculated angle change values ​​are compared with a preset threshold value θt. If it is determined that any of the angle change values ​​is greater than or equal to θt, it is determined that the posture of the terminal has changed, that is, it is determined that the terminal has deflected. The terminal determines the deflection angle of the terminal relative to the first position through a motion detection device, and then determines the target audio output direction based on the deflection angle.

[0061] In step S13, audio is output in a target audio output direction based on the deflection angle and the audio information to be output.

[0062] In the disclosed embodiment, the terminal outputs audio in a target audio output direction based on the acquired deflection angle information and the audio information to be output.

[0063] In the disclosed embodiment, the terminal monitors changes in the current posture, determines the deflection angle of the terminal relative to the first position, and outputs audio in the output direction of the target audio that changes based on the terminal deflection. This enables the terminal to calculate the audio information in real time based on the deflection angle and determine the target audio output direction when the terminal deflects at different angles during use, thereby providing users with better audio playback effects and a better user experience.

[0064] In the embodiment of the present disclosure, the audio information to be outputted collected by the terminal includes the channel configuration of the audio. The channel is the process of picking up sound through a microphone and playing it through a speaker. For example, the channel configuration of the audio may include: 2 channels, 4 channels, 5.1 channels, etc.

[0065] Figure 4 is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment. Figure 4 As shown, the following steps are included.

[0066] In step S21, the sound source direction information of the audio to be output and the number of speakers are determined.

[0067] In the disclosed embodiment, the terminal determines the sound source direction information of the audio by acquiring the channel configuration of the audio to be output, and simulates the placement position of each sound source.

[0068] For example, Figure 5 is a schematic diagram showing a simulated sound source direction according to an exemplary embodiment. Figure 5 As shown, the terminal obtains that the channel configuration of the audio to be output is 5.1 channels, and determines that the sound source direction of the audio is directly in front of the terminal, the left front, the right front, the left rear and the right rear.

[0069] In the embodiment of the present disclosure, it is also necessary to determine the number of speakers configured in the terminal, and the audio is played based on speakers of different numbers. For example, the terminal may be configured with two, four, or other different numbers of speakers.

[0070] In the embodiment of the present disclosure, after the terminal determines the sound source direction information of the audio to be output and the number of speakers, the audio to be output is played through the configured speakers. The terminal collects the sound information of the audio to be output through the configured analog microphone, and stores the collected sound information containing the sound source orientation information. Then, the audio is played through the speakers configured by the terminal.

[0071] For example, when the terminal is configured with two speakers and the channel of the audio to be output is configured as 5.1 channels, the terminal plays the audio to be output containing the sound source orientation information through the two speakers, and the user can feel that the sound comes from the front, left front, right front, left rear and right rear.

[0072] In step S22, in response to the audio to be output being played through the speaker of the terminal, the loudness of the audio to be output played by the speaker is determined.

[0073] In step S23, the target audio output direction is determined based on the deflection angle and the loudness.

[0074] In the embodiment of the present disclosure, the audio to be output is played through the speaker of the terminal, and the loudness information of the audio played by the speaker is determined. The loudness information of the audio can be understood as the sound volume perceived by the user's ears after the audio to be output is played.

[0075] In the disclosed embodiment, the terminal is detected to be deflected relative to the first position, and the deflection angle of the terminal is determined by the motion detection device. For example, when the user is driving, shooting, picking up, etc. during the game, the terminal will deflect accordingly, and the deflection angle of the terminal is determined by the motion detection device, and the target audio output direction of the output audio is determined in real time based on the loudness information of the speaker obtained by the terminal.

[0076] In the embodiment of the present disclosure, the terminal determines the target audio output direction based on the loudness difference between the first loudness and the second loudness, the loudness sum, and the deflection angle of the terminal.

[0077] Figure 6 is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment. Figure 6 As shown, the following steps are included.

[0078] In step S31 , a loudness difference between the first loudness and the second loudness, and a loudness sum between the first loudness and the second loudness are determined.

[0079] In step S32, the target audio output direction is determined based on the loudness difference, the loudness sum, and the deflection angle of the terminal.

[0080] The first loudness is the loudness of the audio to be output played by the first speaker, and the second loudness is the loudness of the audio to be output played by the second speaker.

[0081] Secondly, the first speaker is a speaker located on the first side of the terminal, and the second speaker is a speaker located on the second side of the terminal. The first side and the second side are two opposite sides of the terminal. For example, the first side can be the left side of the terminal, and the second side can be the right side of the terminal.

[0082] Furthermore, the first speaker and the second speaker each include at least one speaker. For example, the first speaker may include one speaker, two speakers, and the like.

[0083] For example, in the embodiment of the present disclosure, when it is determined that the first speaker and the second speaker both include one speaker, it is determined that the current terminal is configured with dual speakers. The loudness of the audio played by the first speaker is the first loudness, and the loudness of the audio played by the second speaker is the second loudness. The terminal can determine the target audio output direction based on the first loudness, the second loudness, and the deflection angle of the terminal.

[0084] For example, in the embodiment of the present disclosure, when it is determined that both the first speaker and the second speaker include two speakers, it is determined that the current terminal is configured with four speakers. The loudness of the played audio is determined to be a first loudness by the two speakers of the first speaker, and a first target output direction is generated. The loudness of the played audio is determined to be a second loudness by the two speakers of the second speaker, and a second target output direction is generated. The two target output directions are calculated to obtain a normalized total sound source direction, and the final target audio output direction is determined.

[0085] Figure 7 is a schematic diagram showing a method of determining a target output direction based on four speakers according to an exemplary embodiment. Figure 7 As shown, the terminal is equipped with four speakers, namely speaker 1, speaker 2, speaker 3 and speaker 4. When the terminal is detected to be deflected, speakers 1 and 2 generate a target output direction A, and speakers 3 and 4 generate a target output direction B. By calculating the two target output directions, a normalized total sound source direction C can be obtained for output, which is always output in the middle of the terminal.

[0086] In the embodiment of the present disclosure, the terminal can determine the target audio output direction by determining the loudness difference and the loudness sum between the first loudness of the audio to be output played by the first speaker and the second loudness of the audio to be output played by the second speaker, and based on the determined terminal deflection angle information.

[0087] Figure 8 is a flow chart of a method for determining a target audio output direction according to an exemplary embodiment. Figure 8 As shown, the following steps are included.

[0088] In step S41, a ratio between a loudness difference between the first loudness and the second loudness and a loudness sum is determined, and a sine angle of a deflection angle of the terminal is determined.

[0089] In step S42, the product of the ratio and the sine angle of the terminal's deflection angle is determined as the sine angle of the target audio output direction.

[0090] In the embodiment of the present disclosure, the terminal determines the output direction of the target audio by determining the product of the ratio of the loudness difference between the first loudness and the second loudness to the loudness sum and the sine angle of the deflection angle of the terminal to obtain the sine angle of the target audio output direction.

[0091] In the embodiment of the present disclosure, the sine angle of the target audio output direction and the deflection angle of the terminal satisfy the preset sine theorem calculation formula. The sine theorem calculation formula is as follows:

[0092]

[0093] Among them, θ is the angle of the target audio output direction, θ0 is the deflection angle of the terminal, and A L is the first loudness, A R Second loudness.

[0094] Fig. 9 is a schematic diagram showing an audio output according to an exemplary embodiment, such as Fig. 9 As shown in the figure, when the terminal is equipped with two speakers, the loudness of the audio played by the left speaker is A. L , the loudness of the audio played by the right speaker is A R The terminal detects through the motion detection device that the terminal is deflected by an angle θ0 relative to the first position, and obtains an angle θ of the target audio output direction according to the sine theorem calculation formula.

[0095] In the embodiment of the present disclosure, the terminal adjusts the first loudness and the second loudness to make the first loudness and the second loudness equal, thereby ensuring that the output direction of the target audio is in front of the user, thereby providing the user with a better playback experience.

[0096] Fig.10 is a flow chart of a method for audio output according to an exemplary embodiment. Fig.10 As shown, the following steps are included.

[0097] In step S51, a first loudness and a second loudness are obtained.

[0098] In the embodiment of the present disclosure, the terminal obtains in real time a first loudness of the audio played by the first speaker and a second loudness of the audio played by the second speaker.

[0099] In step S52, based on the deflection angle of the terminal, the first loudness and the second loudness are adjusted so that the first loudness and the second loudness are equal.

[0100] In step S53, in response to the first loudness and the second loudness being equal, the audio is output in the target audio output direction.

[0101] In the disclosed embodiment, the terminal detects that the current posture has changed, and determines the deflection angle of the terminal relative to the first position through the motion detection device. Based on the deflection angle of the terminal, the terminal adjusts the first loudness and the second loudness so that the first loudness and the second loudness are equal, so that the sound volume perceived by the user's ear of the currently played audio is equal, thereby ensuring that the audio sound is always transmitted from the front of the user, increasing the spatial sense of audio playback, and providing the user with a better audio playback effect.

[0102] In the embodiment of the present disclosure, the terminal can set a delay of a preset length for the sound being played relative to the first audio track played, and translate the sound output by a speaker that is relatively far away from the user toward a speaker that is relatively close to the user, so as to achieve mixing of the sound output by the speaker that is relatively close to the user, so that the first loudness and the second loudness are equal.

[0103] Fig.11 is a flow chart of a method for loudness adjustment according to an exemplary embodiment. Fig.11 As shown, the following steps are included.

[0104] In step S61, audio track information of the sound output by the first speaker and / or the second speaker is obtained.

[0105] In the disclosed embodiment, the terminal obtains the audio track information of the sound output by the first speaker and / or the second speaker. It can be understood that the sound has multiple audio tracks, one audio track corresponds to a part of the sound, and the audio track can record the audio data at a specific time position. For example, when the user plays music through the terminal, each audio track of the music can be defined as the performance of a musical instrument.

[0106] In step S62, based on the audio track information of the sound, the audio track to be played first is determined, and the audio track to be played first is set, and is delayed based on a preset duration.

[0107] In the disclosed embodiment, the terminal determines the audio track to be played first based on the audio track information of the sound, and sets a delay of a preset time length for the played sound relative to the audio track played first.

[0108] In step S63, the speaker relatively far from the user among the first speaker and the second speaker is determined, and the sound output by the speaker relatively far from the user is translated toward the direction of the speaker relatively close to the user.

[0109] In the embodiment of the present disclosure, the terminal determines the speaker between the first speaker and the second speaker which is relatively far away from the user, and translates the sound output by the speaker which is relatively far away from the user toward the direction of the speaker which is relatively close to the user, so as to achieve mixing of the sound output by the speaker which is relatively close to the user.

[0110] In step S64, the sound output by the speaker relatively far from the user and the sound output by the speaker relatively close to the user are mixed, and the sound output by the speaker relatively close to the user is weakened so that the first loudness is equal to the second loudness.

[0111] In the embodiment of the present disclosure, the sound output by the speaker relatively far from the user is shifted toward the direction of the speaker relatively close to the user, so as to mix the sound output by the speaker relatively far from the user and the sound output by the speaker relatively close to the user. This gives the effect that the sound is transmitted to the user from the output direction of the speaker relatively far from the user, and the sound output by the speaker relatively close to the user is weakened so that the first loudness and the second loudness are equal.

[0112] For example, when the terminal is equipped with two speakers, after the terminal is deflected, it is detected that the speaker on the left side of the terminal is a speaker relatively far from the user, and the speaker on the right side of the terminal is a speaker relatively close to the user. For the sound played from the left speaker of the terminal, a delay of a preset length is set relative to the first track played, and it is shifted to the right, so that the sound played from the left speaker is mixed to the right, showing the effect of sound being transmitted from the left side to the audience, weakening the sound played from the right speaker, so that the first loudness and the second loudness are equal.

[0113] In the disclosed embodiment, the terminal determines the distance over which the sound is translated through the length information of the terminal and the deflection angle of the terminal.

[0114] Fig.12 is a flow chart of a method for determining a translation distance according to an exemplary embodiment. Fig.12 As shown, the following steps are included.

[0115] In step S71, the length information of the terminal is obtained, and the cosine angle of the deflection angle of the terminal is determined.

[0116] In the embodiment of the present disclosure, the translation distance of the sound output by the speaker is determined by acquiring the length information of the terminal and calculating the cosine angle of the deflection angle of the terminal.

[0117] In step S72, the product of the length information of the terminal and the cosine angle of the deflection angle of the terminal is determined as the distance for the sound output by the speaker to be translated.

[0118] In the disclosed embodiment, the product of the length information of the terminal and the cosine angle of the terminal deflection angle is used as the distance for the sound output by the speaker to be translated.

[0119] For example, when the terminal is equipped with two speakers, the motion detection device of the terminal detects that the terminal is deflected by an angle of θ relative to the first position. The length information of the terminal is L, and the length information is multiplied by the cosine angle cosθ of the terminal deflection angle to obtain the translation distance of the sound output by the speaker.

[0120] Fig.13 is a flow chart of a method for audio output according to an exemplary embodiment. Fig.13 As shown, the following steps are included.

[0121] In step S81, the terminal detects that the current posture has changed and deflected.

[0122] In step S82, the terminal performs angle calculation through the configured motion detection device to determine the deflection angle of the terminal relative to the first position.

[0123] In the disclosed embodiment, the gyroscope sensor can be used to obtain the angular velocity of the terminal deflection, and the angle of the terminal reverse deflection can be obtained by integral calculation. The output specific force is obtained by using the accelerometer sensor, and the inertial navigation solution is performed in combination with the angular velocity information of the gyroscope sensor to obtain the terminal's attitude information, that is, the current terminal deflection angle.

[0124] In step S83, the terminal transmits the angle information to the audio system for separate calculation.

[0125] In the disclosed embodiment, when the terminal determines that the current posture has changed and deflects, the terminal determines the target audio output direction in real time based on the deflection angle information. The obtained deflection angle data is transmitted to the audio system through the system, and the audio system transmits the audio to the audio mapping unit after calculation.

[0126] In step S84, the audio system outputs the sound from the speakers after calculation.

[0127] In the disclosed embodiment, the terminal remaps the sound field according to the obtained deflection angle information, calculates the audio data through the audio system, and transmits it to the audio playback unit (speaker), and the speaker plays the audio in the target audio output direction.

[0128] In the disclosed embodiment, when the terminal detects a change in the current posture without connecting any peripheral device, the terminal determines the deflection angle of the terminal relative to the first position. The target audio output direction is determined based on the deflection angle, and the audio is output in the target audio output direction. When the terminal deflects at different angles, the terminal calculates the audio information in real time based on the deflection angle and determines the target audio output direction, thereby providing the user with a better audio playback effect.

[0129] Based on the same concept, an embodiment of the present disclosure also provides an audio output device.

[0130] It is understandable that the audio output device provided by the embodiment of the present disclosure includes hardware structures and / or software modules corresponding to the execution of each function in order to realize the above functions. In combination with the units and algorithm steps of each example disclosed in the embodiment of the present disclosure, the embodiment of the present disclosure can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art may use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the technical solution of the embodiment of the present disclosure.

[0131] Fig.14 is a block diagram of an audio output device 100 according to an exemplary embodiment. Fig.14 The device includes a monitoring unit 101, a determining unit 102 and a processing unit 103.

[0132] The monitoring unit 101 is used for the terminal to collect the audio to be output and monitor the posture of the terminal.

[0133] The determination unit 102 is used to monitor changes in the terminal's posture, determine a deflection angle of the terminal relative to the first position, and determine a target audio output direction based on the deflection angle, wherein the target audio output direction is an optimal direction for the current user of the terminal to perceive sound.

[0134] The first position is the position when the terminal display screen faces the user.

[0135] The processing unit 103 is configured to output audio in a target audio output direction based on the deflection angle and the audio information to be output.

[0136] In the embodiment of the present disclosure, the determination unit 102 determines the target audio output direction in the following manner: determining the sound source direction information of the audio to be output and the number of speakers. In response to playing the audio to be output through the speaker of the terminal, determining the loudness of the audio to be output played by the speaker. Based on the deflection angle and the loudness, determining the target audio output direction.

[0137] In the embodiment of the present disclosure, the determination unit 102 determines the target audio output direction in the following manner: determine the loudness difference between the first loudness and the second loudness, and the loudness sum between the first loudness and the second loudness, the first loudness is the loudness of the first speaker playing the audio to be output, the second loudness is the loudness of the second speaker playing the audio to be output, the first speaker is a speaker located on the first side of the terminal, the second speaker is a speaker located on the second side of the terminal, the first side and the second side are two opposite sides of the terminal, and the first speaker and the second speaker each include at least one speaker. Based on the loudness difference, the loudness sum, and the deflection angle of the terminal, the target audio output direction is determined.

[0138] In the embodiment of the present disclosure, the determination unit 102 determines the target audio output direction in the following manner: determining the ratio between the loudness difference between the first loudness and the second loudness and the loudness sum, and determining the sine angle of the deflection angle of the terminal. Determine the product of the ratio and the sine angle of the deflection angle of the terminal as the sine angle of the target audio output direction.

[0139] In the embodiment of the present disclosure, the processing unit 103 outputs audio in a target audio output direction based on the deflection angle and the audio information to be outputted in the following manner: obtaining a first loudness and a second loudness. Based on the deflection angle of the terminal, adjusting the first loudness and the second loudness so that the first loudness and the second loudness are equal. In response to the first loudness and the second loudness being equal, outputting audio in the target audio output direction.

[0140] In an embodiment of the present disclosure, adjusting the first loudness and the second loudness based on the deflection angle of the terminal includes: obtaining the track information of the sound output by the first speaker and / or the second speaker. Based on the track information of the sound, determining the track to be played first, setting the track to be played first, and delaying it based on a preset duration. Determining the speaker relatively far from the user among the first speaker and the second speaker, and shifting the sound output by the speaker relatively far from the user to the direction of the speaker relatively close to the user, and the shifting distance is determined based on the deflection angle of the terminal. Mixing the sound output by the speaker relatively far from the user with the sound output by the speaker relatively close to the user, and weakening the sound output by the speaker relatively close to the user, so that the first loudness and the second loudness are equal. The sound output by the speaker relatively far from the user is the first loudness, and the sound output by the speaker relatively close to the user is the second loudness.

[0141] In the embodiment of the present disclosure, the translation distance is determined based on the deflection angle of the terminal in the following manner: the length information of the terminal is obtained, and the cosine angle of the deflection angle of the terminal is determined. The product between the length information of the terminal and the cosine angle of the deflection angle of the terminal is determined as the translation distance of the sound output by the speaker.

[0142] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0143] Fig.15 2 is a block diagram of an audio output device 200 according to an exemplary embodiment. For example, the device 200 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0144] Reference Fig.15 , the device 200 may include one or more of the following components: a processing component 202 , a memory 204 , a power component 206 , a multimedia component 208 , an audio component 210 , an input / output (I / O) interface 212 , a sensor component 214 , and a communication component 216 .

[0145] The processing component 202 generally controls the overall operation of the device 200, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 202 may include one or more processors 220 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 202 may include one or more modules to facilitate interaction between the processing component 202 and other components. For example, the processing component 202 may include a multimedia module to facilitate interaction between the multimedia component 208 and the processing component 202.

[0146] The memory 204 is configured to store various types of data to support operations on the device 200. Examples of such data include instructions for any application or method operating on the device 200, contact data, phone book data, messages, pictures, videos, etc. The memory 204 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0147] The power component 206 provides power to the various components of the device 200. The power component 206 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device 200.

[0148] The multimedia component 208 includes a screen that provides an output interface between the device 200 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 208 includes a front camera and / or a rear camera. When the device 200 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0149] The audio component 210 is configured to output and / or input audio signals. For example, the audio component 210 includes a microphone (MIC), and when the device 200 is in an operation mode, such as a call mode, a recording mode, and a speech recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 204 or sent via the communication component 216. In some embodiments, the audio component 210 also includes a speaker for outputting audio signals.

[0150] I / O interface 212 provides an interface between processing component 202 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0151] The sensor assembly 214 includes one or more sensors for providing various aspects of the status assessment of the device 200. For example, the sensor assembly 214 can detect the open / closed state of the device 200, the relative positioning of components, such as the display and keypad of the device 200, the sensor assembly 214 can also detect the position change of the device 200 or a component of the device 200, the presence or absence of user contact with the device 200, the orientation or acceleration / deceleration of the device 200 and the temperature change of the device 200. The sensor assembly 214 can include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 214 can also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 214 can also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor or a temperature sensor.

[0152] The communication component 216 is configured to facilitate wired or wireless communication between the device 200 and other devices. The device 200 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, or a combination thereof. In an exemplary embodiment, the communication component 216 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 216 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0153] In an exemplary embodiment, the apparatus 200 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors or other electronic components to perform the above method.

[0154] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 204 including instructions, and the instructions can be executed by the processor 220 of the device 200 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0155] It is to be understood that in the present disclosure, "plurality" refers to two or more than two, and other quantifiers are similar. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The singular forms "a", "the" and "the" are also intended to include plural forms, unless the context clearly indicates other meanings.

[0156] It is further understood that the terms "first", "second", etc. are used to describe various information, but such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other, and do not indicate a specific order or degree of importance. In fact, the expressions "first", "second", etc. can be used interchangeably. For example, without departing from the scope of the present disclosure, the first information can also be referred to as the second information, and similarly, the second information can also be referred to as the first information.

[0157] It will be further understood that the terms “center”, “longitudinal”, “lateral”, “front”, “back”, “up”, “down”, “left”, “right”, “vertical”, “horizontal”, “top”, “bottom”, “inside”, “outside”, etc., indicating orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, are only for the convenience of describing the present embodiment and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation.

[0158] It can be further understood that, unless otherwise specified, “connection” includes a direct connection without other components between the two, and also includes an indirect connection with other components between the two.

[0159] It is further understood that, although the operations are described in a specific order in the drawings in the embodiments of the present disclosure, it should not be understood as requiring the operations to be performed in the specific order shown or in a serial order, or requiring the execution of all the operations shown to obtain the desired results. In certain environments, multitasking and parallel processing may be advantageous.

[0160] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modifications, uses or adaptations of the present disclosure, which follow the general principles of the present disclosure and include common knowledge or customary technical means in the art that are not disclosed in the present disclosure.

[0161] It should be understood that the present disclosure is not limited to the precise structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the scope of the appended claims.

Claims

1. An audio output method, characterized in that: include: In response to the terminal collecting the audio to be output, monitoring the posture of the terminal; In response to monitoring a change in the posture of the terminal, determining a deflection angle of the terminal relative to a first position, and determining a target audio output direction based on the deflection angle, the target audio output direction being an optimal direction in which a current user of the terminal perceives sound; The first position is the position when the terminal display screen is facing the user; Based on the deflection angle and the audio information to be output, audio is output in the target audio output direction.

2. The method according to claim 1, characterized in that: The determining the target audio output direction based on the deflection angle comprises: Determine the sound source direction information of the audio to be output, and the number of speakers; In response to playing the audio to be output through a speaker of the terminal, determining the loudness of the audio to be output played by the speaker; The target audio output direction is determined based on the deflection angle and the loudness.

3. The method according to claim 2, characterized in that The determining the target audio output direction based on the deflection angle and the loudness includes: Determine a loudness difference between a first loudness and a second loudness, and a loudness sum between the first loudness and the second loudness, wherein the first loudness is a loudness of a first speaker playing the audio to be output, and the second loudness is a loudness of a second speaker playing the audio to be output, the first speaker is a speaker located on a first side of the terminal, the second speaker is a speaker located on a second side of the terminal, the first side and the second side are two opposite sides of the terminal, and the first speaker and the second speaker each include at least one speaker; The target audio output direction is determined based on the loudness difference, the loudness sum, and the deflection angle of the terminal.

4. The method according to claim 3, characterized in that The determining the target audio output direction based on the loudness difference between the first loudness and the second loudness, the loudness sum between the first loudness and the second loudness, and the deflection angle of the terminal includes: determining a ratio of a loudness difference between the first loudness and the second loudness to the loudness sum, and determining a sine angle of a deflection angle of the terminal; The product of the ratio and the sine angle of the deflection angle of the terminal is determined as the sine angle of the target audio output direction.

5. The method according to any one of claims 1 to 4, characterized in that: The step of outputting the audio in the target audio output direction based on the deflection angle and the audio information to be output includes: obtaining the first loudness and the second loudness; Based on the deflection angle of the terminal, adjusting the first loudness and the second loudness so that the first loudness and the second loudness are equal; In response to the first loudness and the second loudness being equal, outputting audio in the target audio output direction.

6. The method according to claim 5, characterized in that The adjusting the first loudness and the second loudness based on the deflection angle of the terminal includes: Acquire audio track information of the sound output by the first speaker and / or the second speaker; Based on the audio track information of the sound, determine the audio track to be played first, and set the audio track to be played first to be delayed based on a preset time length; Determine a speaker that is relatively far from the user among the first speaker and the second speaker, and translate the sound output by the speaker that is relatively far from the user toward the direction of the speaker that is relatively close to the user, wherein the translation distance is determined based on the deflection angle of the terminal; Mixing the sound output by the speaker relatively far from the user and the sound output by the speaker relatively close to the user, and weakening the sound output by the speaker relatively close to the user, so that the first loudness and the second loudness are equal; The sound level output by the speaker that is relatively far from the user is the first loudness, and the sound level output by the speaker that is relatively close to the user is the second loudness.

7. The method according to claim 6, characterized in that The translation distance is determined based on the deflection angle of the terminal in the following manner: Acquire the length information of the terminal, and determine the cosine angle of the deflection angle of the terminal; The product of the length information of the terminal and the cosine angle of the deflection angle of the terminal is determined as the distance for translation of the sound output by the speaker.

8. An audio output device, characterized in that: include: A monitoring unit, used for the terminal to collect the audio to be output and monitor the posture of the terminal; a determination unit, configured to detect a change in the posture of the terminal, determine a deflection angle of the terminal relative to the first position, and determine a target audio output direction based on the deflection angle, wherein the target audio output direction is an optimal direction in which a current user of the terminal perceives sound; The first position is the position when the terminal display screen is facing the user; A processing unit is used to output audio in the target audio output direction based on the deflection angle and the audio information to be output.

9. The device according to claim 8, characterized in that The determination unit determines the target audio output direction in the following manner: Determine the sound source direction information of the audio to be output, and the number of speakers; In response to playing the audio to be output through a speaker of the terminal, determining the loudness of the audio to be output played by the speaker; The target audio output direction is determined based on the deflection angle and the loudness.

10. The device according to claim 9, characterized in that The determination unit determines the target audio output direction in the following manner: Determine a loudness difference between a first loudness and a second loudness, and a loudness sum between the first loudness and the second loudness, wherein the first loudness is a loudness of a first speaker playing the audio to be output, and the second loudness is a loudness of a second speaker playing the audio to be output, the first speaker is a speaker located on a first side of the terminal, the second speaker is a speaker located on a second side of the terminal, the first side and the second side are two opposite sides of the terminal, and the first speaker and the second speaker each include at least one speaker; The target audio output direction is determined based on the loudness difference, the loudness sum, and the deflection angle of the terminal.

11. The device according to claim 10, characterized in that The determination unit determines the target audio output direction in the following manner: determining a ratio of a loudness difference between the first loudness and the second loudness to the loudness sum, and determining a sine angle of a deflection angle of the terminal; The product of the ratio and the sine angle of the deflection angle of the terminal is determined as the sine angle of the target audio output direction.

12. The device according to any one of claims 8 to 11, characterized in that The processing unit outputs audio in the target audio output direction based on the deflection angle and the audio information to be output in the following manner: obtaining the first loudness and the second loudness; Based on the deflection angle of the terminal, adjusting the first loudness and the second loudness so that the first loudness and the second loudness are equal; In response to the first loudness and the second loudness being equal, outputting audio in the target audio output direction.

13. The device according to claim 12, characterized in that The adjusting the first loudness and the second loudness based on the deflection angle of the terminal includes: Acquire audio track information of the sound output by the first speaker and / or the second speaker; Based on the audio track information of the sound, determine the audio track to be played first, and set the audio track to be played first to be delayed based on a preset time length; Determine a speaker that is relatively far from the user among the first speaker and the second speaker, and translate the sound output by the speaker that is relatively far from the user toward the direction of the speaker that is relatively close to the user, wherein the translation distance is determined based on the deflection angle of the terminal; Mixing the sound output by the speaker relatively far from the user and the sound output by the speaker relatively close to the user, and weakening the sound output by the speaker relatively close to the user, so that the first loudness and the second loudness are equal; The sound level output by the speaker that is relatively far from the user is the first loudness, and the sound level output by the speaker that is relatively close to the user is the second loudness.

14. The device according to claim 13, characterized in that The translation distance is determined based on the deflection angle of the terminal in the following manner: Acquire the length information of the terminal, and determine the cosine angle of the deflection angle of the terminal; The product of the length information of the terminal and the cosine angle of the deflection angle of the terminal is determined as the distance for translation of the sound output by the speaker.

15. An audio output device, characterized in that: include: processor; a memory for storing processor-executable instructions; Wherein, the processor is configured to: The method is used to call the instructions stored in the memory to execute the audio output method according to any one of claims 1 to 7.

16. A storage medium, characterized in that: The storage medium stores instructions, and when the instructions in the storage medium are executed by a processor of the terminal, the terminal is enabled to execute the audio output method according to any one of claims 1 to 7.

Images