Audio playing method and device and electronic equipment

Abstract

The invention provides an audio playing method and device and electronic equipment, and the method comprises the steps: obtaining the relative position information of a target object relative to a first playing device and a second playing device in response to a playing instruction of a target audio; based on the relative position information, controlling the first playing device to play a first target audio channel, and controlling the second playing device to play a second target audio channel; wherein if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel; if the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel; the first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

Description

Technical Field

[0001] The present disclosure relates to the technical field of audio processing, and in particular to an audio playing method, device and electronic device. Background Art

[0002] When playing audio, in order to create a stereo effect, multiple speakers are used to play different sound signals respectively, so that the audience can feel a three-dimensional audio experience. However, currently only the sound image of one speaker can be adjusted, and it is impossible to accurately adjust the sound image of multiple speakers, resulting in the stereo effect being affected and the audio playback effect being poor. Summary of the invention

[0003] The present disclosure provides an audio playback method, apparatus, device and storage medium.

[0004] According to a first aspect of the present disclosure, there is provided an audio playback method, comprising:

[0005] In response to a target audio playback instruction, obtaining relative position information of the target object relative to the first playback device and the second playback device;

[0006] Based on the relative position information, controlling the first playback device to play a first target audio channel and controlling the second playback device to play a second target audio channel;

[0007] Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel;

[0008] If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel;

[0009] The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

[0010] According to a second aspect of the present disclosure, there is provided an audio playback device, the device comprising:

[0011] An acquisition module, configured to acquire relative position information of a target object relative to a first playback device and a second playback device in response to a playback instruction of a target audio;

[0012] A first processing module, configured to control the first playback device to play a first target audio channel and control the second playback device to play a second target audio channel based on the relative position information;

[0013] Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel;

[0014] If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel;

[0015] The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

[0016] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0017] at least one processor; and

[0018] a memory communicatively connected to the at least one processor; wherein,

[0019] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the present disclosure.

[0020] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The above and other objects, features and advantages of the exemplary embodiments of the present disclosure will become readily understood by reading the detailed description below with reference to the accompanying drawings. In the accompanying drawings, several embodiments of the present disclosure are shown in an exemplary and non-limiting manner, in which:

[0022] In the drawings, the same or corresponding reference numerals represent the same or corresponding parts.

[0023] Figure 1 A schematic diagram showing a flow chart of an audio playback method provided by an embodiment of the present disclosure;

[0024] Figure 2 A schematic diagram showing a flow chart of an audio playback method provided by an application embodiment of the present disclosure;

[0025] Figure 3 A schematic diagram of an audio-visual system provided by an application embodiment of the present disclosure is shown;

[0026] Figure 4 A schematic diagram showing another sound image provided by an application embodiment of the present disclosure is shown;

[0027] Figure 5A schematic diagram showing another sound image provided by an application embodiment of the present disclosure is shown;

[0028] Figure 6 A schematic diagram of the structure of an audio playback device provided by an embodiment of the present disclosure is shown;

[0029] Figure 7 A schematic structural diagram of an electronic device provided by an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0030] In order to make the purpose, features, and advantages of the present disclosure more obvious and easy to understand, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present disclosure.

[0031] First, some terms in the embodiments of the present disclosure are explained:

[0032] Audio channel: An audio channel is a path for audio signals to be transmitted. It can be divided into different types such as mono, stereo and multi-channel. A mono channel has only one audio signal path, i.e. one audio channel, and all sounds are emitted from the same location. A stereo channel usually consists of a left channel and a right channel, which can create a sense of direction and space for the sound.

[0033] Audio component: Audio components refer to the different frequency components that make up an audio signal. Audio signals can be divided into different components such as low frequency, medium frequency and high frequency. The transmission and processing methods of different audio components in the audio channel will affect the quality and characteristics of the sound. For example, by adjusting the volume balance of different frequency components in the audio channel, the timbre and style of the sound can be changed.

[0034] Sound image: Sound image refers to the position and direction of sound in the audio space. It is produced by the volume, phase and time difference of different channels in the audio channel.

[0035] Figure 1 A schematic diagram of a flow chart of an audio playback method provided by an embodiment of the present disclosure is shown, such as Figure 1 As shown, the audio playing method includes:

[0036] Step 101: In response to a target audio playback instruction, obtain relative position information of a target object relative to a first playback device and a second playback device;

[0037] Step 102: Based on the relative position information, control the first playback device to play a first target audio channel and control the second playback device to play a second target audio channel;

[0038] Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel;

[0039] If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel;

[0040] The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

[0041] In some embodiments, in addition to the first playback device and the second playback device, other playback devices may be provided in the embodiments of the present disclosure, and the number and location of the playback devices used are not limited. Accordingly, the operation of other playback devices is similar to that of the first playback device and the second playback device. For example, when constructing a three-dimensional coordinate system, other playback devices may be located, the audio channels and playback parameters required to be played by other playback devices may be determined, and the required audio channels may be played by other playback devices, etc., so as to better present the sound effects of the target audio.

[0042] In addition, although only the first audio channel, the second audio channel, the third audio channel, and the fourth audio channel are described above, in actual application, the number of audio channels obtained by separating the target audio is not limited, and other audio channels can also be separated; the processing and playback of other audio channels are similar to those of the first audio channel and the second audio channel. For example, corresponding playback devices are used to play other audio channels to better present the sound effects of the target audio.

[0043] The method can be applied to a control device, and the control device can also be used as an audio source device to control a playback device and instruct the playback device to play corresponding audio.

[0044] When used, the control device and the playback device (such as the first playback device and the second playback device) can be interconnected through wired connection or wireless connection (such as ultrasonic communication, Bluetooth communication) to perform one-way communication or two-way communication.

[0045] The control device may be a television, a computer, a laptop computer, a mobile phone terminal, a tablet computer or other electronic device.

[0046] The playback device may be a device for playing audio, such as a stereo, a speaker, a computer with a speaker, or a tablet computer with a speaker.

[0047] Here, according to different relative position information, the first playback device and the second playback device can be controlled to play different audio channels to obtain the sound image of each audio component in the target audio. Sound image refers to the sense of position of sound in space, such as front, back, left or right; by controlling the audio channels played by the playback device, different sound images can be obtained, and different sound images can make the sound seem to be emitted from a specific position, rather than just from the position of the playback device. In this way, by controlling each playback device to play different audio channels, when each audio component has a different sound image, a richer sound effect can be produced.

[0048] Here, each audio channel (such as the first audio channel, the second audio channel, the third audio channel, and the fourth audio channel) may include at least one audio component or a partial proportion of audio of at least one audio component.

[0049] In one example, there are playback device A and playback device B, and the target audio is separated to obtain audio component A, audio component B, audio component C, and audio component D;

[0050] According to the required sound effects, based on the relative position information, it is determined that audio component A and audio component B need to be played by the first playback device, and audio component C and audio component D need to be played by the second playback device; then the first target audio channel includes: audio component A, audio component B, and the second target audio channel includes: audio component C, audio component D.

[0051] Of course, according to the required sound effect, based on the relative position information, if it is determined that the first playback device needs to play audio component A, and the second playback device needs to play audio component B, audio component C, and audio component D; then the first target audio channel includes: audio component A, and the second target audio channel includes: audio component B, audio component C, and audio component D.

[0052] In this way, by controlling each playback device to play different audio channels, when each audio component has a different sound image, richer sound effects can be produced.

[0053] In yet another example, there are playback devices A and playback devices B, and the target audio is separated to obtain audio component A and audio component B. According to the required sound effect, based on the relative position information, it is determined that the first playback device needs to play the audio component A of the first proportion and the audio component B of the second proportion, and the second playback device needs to play the audio component A of the third proportion and the audio component B of the fourth proportion. Then, the first target audio channel includes: audio component A of the first proportion and audio component B of the second proportion, and the second target audio channel includes: audio component A of the third proportion and audio component B of the fourth proportion.

[0054] In this way, a certain proportion of the audio components played by each playback device can form a sound image of a specific audio component, producing a richer sound effect.

[0055] Here, the relative position information includes: relative position information between the first playback device and the target object, for example, the first playback device is at position A1, which is to the left of the target object;

[0056] The relative position information also includes: relative position information between the second playback device and the target object, for example, the second playback device is at position A2, which is to the right of the target object;

[0057] Assume that the sound position (i.e., sound image) of the first target audio channel is determined to be position B1, which is to the left of the target object, and the sound position of the second target audio channel is B2, which is to the right of the target object; then, a first playback device can be selected to play the first target audio channel, and a second playback device can be selected to play the second target audio channel, to form the sound image of the two audios.

[0058] In this way, by controlling each playback device to play different audio channels, audio channels with different sound emission positions (i.e., different sound and image effects) can be obtained. If each audio channel includes multiple audio components, audio components with different sound emission positions can be obtained, thereby producing richer sound effects.

[0059] In some embodiments, the method further comprises: constructing a three-dimensional coordinate system of the target space;

[0060] The three-dimensional coordinate system is marked with coordinates of the control device, coordinates of the target object, and coordinates of each playback device, and each playback device includes: a first playback device and a second playback device.

[0061] Specifically, the three-dimensional coordinate system may include: the first position coordinates of the target object in the target space, the second position coordinates of the first playback device in the target space, the third position coordinates of the second playback device in the target space, and the position coordinates of the control device in the target space.

[0062] The control device may be at the same location as the target object, for example, the target object is a user and the control device is a mobile phone terminal held by the user. Of course, the two may not be at the same location, which is not limited here.

[0063] The target space is the space where the control device, the target object, the first playback device, and the second playback device are located.

[0064] Here, any positioning method can be used to construct the three-dimensional coordinate system of the target space, for example, to collect an image of the target space, the image having the control device, each playback device and the target object in the target space; to recognize the image using the target model, and to calculate the relative position of each playback device and the target object relative to the control device according to the image recognition result; and to convert the obtained relative position data into a three-dimensional coordinate system with the control device as the origin. Here, the control device itself has a positioning function, that is, it can obtain its own positioning.

[0065] For another example, the control device receives first data from a first playback device and second data from a second playback device. Here, the first playback device and the second playback device respectively have positioning sensors, and the first data includes: the actual position of the first playback device; the second data includes: the actual position of the second playback device; and a three-dimensional coordinate system is constructed based on the actual position of the target object, the first data, and the second data.

[0066] It should be noted that the above-mentioned three-dimensional coordinate system can also be constructed by other positioning methods, for example, positioning based on GPS or other satellite positioning systems, network-based positioning, positioning based on Bluetooth or other short-range wireless communication technologies, visual positioning, sensor-based positioning, and sound-based positioning. For another example, a three-dimensional coordinate system can be established and positioned based on the results obtained by interconnecting and sharing various devices and measuring distances; or positioning can be performed based on offline or real-time marking of photos or videos; or based on the accelerometer of a device (such as a mobile phone), the position of the playback device can be moved to and the positioning can be recorded; or the user can manually modify the playback device position or add other playback devices (including the position and orientation of the playback device) based on the coordinate image displayed by the control device. There is no limitation on the specific method, and all playback devices and control devices can be established in the three-dimensional coordinate system.

[0067] In addition, the position of each playback device can actually be adjusted. When constructing a three-dimensional coordinate system, prompt information can be used to guide the user to move the playback device to a more appropriate position and angle to achieve a better sound field effect.

[0068] In some embodiments, the method further includes: controlling the first playback device and the second playback device to play the test audio respectively;

[0069] Based on the test audio collected by the audio collection device, determining a first distance between the target object and the first playback device and a second distance between the target object and the second playback device;

[0070] According to the first distance and the second distance, controlling the first playback device to play the first target audio channel's playback time, and controlling the second playback device to play the second target audio channel's playback time;

[0071] Wherein, the audio collection device is an audio collection device of a device carried by the target object.

[0072] Here, the control device is used to sequentially control the first playback device and the second playback device to play the test audio (standard ultrasonic audio may be used), and the distance from the audio acquisition device (the control device may also be used) to each playback device is known to be d0i according to the three-dimensional coordinate system, and the speed of sound is v. According to the time difference t0i=d0i / v+Δt0i when the audio acquisition device receives the test audio, the communication delay Δt0i=t0i-d0i / v of the audio acquisition device playing the audio is obtained, and the average value may be taken after multiple measurements during measurement;

[0073] According to the distance d1i from the listening position (i.e. the position of the target object) to each playback device i, the time difference t1i=d1i / v+Δt0i of the listening position is calculated, and the maximum time interval T=max(t1i), then the playback delay of each playback device i is T-t1i;

[0074] According to the playback delay T-t1i of each playback device i, the playback time of each playback device is adjusted so that the time difference from the playback audio to the listening position is consistent. In this way, phase misalignment or interference of the sound is avoided, the sound field effect at the listening position is improved, and problems such as poor sound quality caused by inconsistent playback are avoided.

[0075] Furthermore, the ultrasonic audio calculation method can automatically adjust when the positioning position changes without the user noticing, thereby ensuring the sound field effect at the listening position at all times.

[0076] In some embodiments, the method further comprises:

[0077] A sound source separation method is used to determine multiple audio channels of the target audio, where the multiple audio channels at least include: a first audio channel and a second audio channel; or a third audio channel and a fourth audio channel.

[0078] Here, the separation method can be sound source separation based on the component information of the audio itself (such as midi or mixing), or it can be a sound source separation algorithm for multiple audio channels, or it can be a sound source separation method based on an artificial intelligence model, which is not limited here. Through sound source separation, audio components such as human voices (such as one or more different human voices), audio components of musical instrument sounds (such as one or more different musical instruments), and audio components of ambient sounds are obtained.

[0079] The first audio channel and the second audio channel may respectively include at least one audio component or a partial proportion of audio of at least one audio component.

[0080] In this way, by separating different audio components into different audio channels and playing them respectively by the first playback device and the second playback device, sound images of different audio components can be generated, thereby improving the playback effect of the target audio.

[0081] In some embodiments, the relative position information of the target object relative to the first playback device and the second playback device includes:

[0082] The first position coordinates of the target object in the target space, the second position coordinates of the first playback device in the target space, and the third position coordinates of the second playback device in the target space;

[0083] The method further comprises:

[0084] Determining a first playback parameter of the first playback device based on the first position coordinates and the second position coordinates;

[0085] Determining a second playback parameter of the second playback device based on the first position coordinates and the third position coordinates;

[0086] The target space is the space where the target object, the first playback device, and the second playback device are located.

[0087] Here, the relative position information can be obtained based on the constructed three-dimensional coordinate system.

[0088] The playback parameters may include at least one of the following: playback time, equalizer, filter and other audio playback related parameters. These parameters may affect the effect of the sound in the indoor environment and ensure that the sound achieves the expected listening experience at different locations.

[0089] The equalizer (EQ) is used to adjust the volume of different frequencies to optimize the clarity and quality of the sound. For example, the low frequencies can be enhanced to improve the bass effect, or the mid- and high-frequency frequencies can be reduced to eliminate harsh sounds.

[0090] Filters are used to remove or reduce a specific frequency range of sound. For example, a low-pass filter removes high-frequency noise, and a high-pass filter removes low-frequency noise. Filters can clean up sounds and reduce interference.

[0091] Here, based on the preset rule, the first position coordinate and the second position coordinate are used to determine the first playback parameter of the first playback device; based on the first position coordinate and the third position coordinate, the second playback parameter of the second playback device is determined. The preset rule is used to indicate a method for determining playback parameters in various scenarios.

[0092] For example, the playing time can be calculated using the above-mentioned test method for the test audio.

[0093] The filter can be obtained by calculating the corresponding filter to eliminate the reverberation after the control device detects the effect of the reverberation generated by each playback device in the target space at the target object. For example, a device can be placed on the target object or at the location of the target object to control each playback device to send an ultrasonic carrier signal. According to the ultrasonic carrier signal sent by each playback device received by the device, the effect of the reverberation generated in the space at the target object is detected, and the filter to eliminate the reverberation is obtained by calculation. In this way, the echo or reverberation introduced in the environment of the target space can be removed by the filter, thereby improving the clarity and quality of the audio signal.

[0094] The equalizer can select the corresponding equalizer according to the type of each audio component based on preset rules, such as using different equalizers for male voices, female voices and different musical instruments. In this way, the equalizer can optimize the audio output, modify the timbre and improve the sound effect.

[0095] In this way, after determining the first playback parameter and the second playback parameter, the first playback device plays the first target audio channel based on the first playback parameter, and the second playback device plays the second target audio channel based on the second playback parameter, so that each audio component in the presented target audio has its own sound image, and here, the playback parameters are determined based on the first position coordinates of the target object, so that the playback effect of the presented target audio can achieve a better listening experience at the position of the target object.

[0096] In some embodiments, the method further comprises:

[0097] In response to a change in the first position coordinates of the target object, adjusting the first playback parameter and the second playback parameter; or

[0098] In response to a change in the second position coordinates of the first playback device, adjusting the first playback parameter; or

[0099] In response to a change in the third position coordinate of the second playback device, the second playback parameter is adjusted.

[0100] Here, considering that any one of the target object, the first playback device, and the second playback device may change position, which will also affect the playback effect of the target audio. Therefore, when it is determined that any one of them has changed, the playback parameters can be adjusted to improve the quality of the sound during the subsequent playback process.

[0101] Specifically, when the position of any one of the target object, the first playback device, and the second playback device changes, the coordinate information in the three-dimensional coordinate system is updated accordingly, and the relative position information is also updated accordingly, and the first playback parameter and / or the second playback parameter are re-determined based on the updated relative position information. The specific method can adopt the above solution, which will not be described in detail here.

[0102] In some embodiments, the relative position information includes relative position information of the first playback device and relative position information of the second playback device;

[0103] The first playback device relative position information is the relative position information between the first playback device and the target object;

[0104] The second playback device relative position information is the relative position information between the second playback device and the target object;

[0105] The method further comprises:

[0106] Obtaining sound position information corresponding to each audio channel included in the target audio;

[0107] Based on the sound position information of each of the audio channels, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device are acquired.

[0108] Here, the sound positions corresponding to the various audio channels may be different. By selecting a playback device that matches the sound positions corresponding to the various audio channels to play the corresponding audio channels, different sound images may be obtained, ultimately presenting better sound effects.

[0109] For example, the relative position information of the first playback device is the relative position information between the first playback device and the target object. For example, the first playback device is at position A1, which is to the left of the target object.

[0110] The relative position information of the second playback device is the relative position information between the second playback device and the target object. For example, the second playback device is at position A2, which is to the right of the target object.

[0111] Assuming that the sound position of the first target audio channel is determined to be position B1, which is to the left of the target object, and the sound position of the second target audio channel is determined to be position B2, which is to the right of the target object; then, a first playback device can be selected to play the first target audio channel, and a second playback device can be selected to play the second target audio channel to form the sound image of the two audios.

[0112] In some embodiments, if the target audio corresponds to video data, obtaining the sound position information corresponding to each audio channel included in the target audio includes:

[0113] Obtaining sound position information corresponding to each audio channel included in the target audio in the video data; or

[0114] The sound position configuration file corresponding to each audio channel included in the target audio is read to obtain the sound position information corresponding to each audio channel included in the target audio.

[0115] In an example, the target audio may correspond to video data, and obtaining the sound position information corresponding to each audio channel included in the target audio in the video data may include:

[0116] The image corresponding to each audio channel is determined according to the video data, the relative position relationship of each audio channel is determined according to the image corresponding to each audio channel, and then the sound position corresponding to each audio channel is determined (that is, the sound image required for each audio channel is determined).

[0117] Here, each audio channel can respectively include an audio component. For example, according to the image, it can be determined that the relative position relationship is that the position of instrument A (an audio component) is to the right of the position of instrument B (another audio component). Correspondingly, it can be determined that the sound position of the audio channel of instrument A should be to the right of the sound position of the audio channel of instrument B.

[0118] Here, each audio channel may include multiple audio components respectively, and the relative position relationship between each audio component can be determined according to the image corresponding to each audio component. For example, according to the image, it can be determined that the relative position relationship is that the position of instrument A is to the right of the positions of instruments B and C. Then, it can be determined that the first audio channel includes instrument A, and the second audio channel includes: instrument B and instrument C. The sound position of the first audio channel is to the right of the sound position of the second audio channel. In this way, by controlling each playback device to play different audio channels, each audio channel has a different sound image, which can produce richer sound effects.

[0119] In another example, the sound position profile may be used to indicate the sound positions corresponding to each audio channel or the relationship between the sound positions; the sound position profile may also indicate the arrangement method adopted (such as the arrangement orientation of the choir).

[0120] The sound position configuration file can be a file obtained according to the user's own settings. For example, the control device provides a configuration function, and the user can select the sound position of each audio channel in the three-dimensional coordinate system. The sound position configuration file is generated based on the sound position of each audio channel selected by the user.

[0121] The sound position profile can also be a file configured when recording the target audio. For example, when recording the target audio, the user sets the relative relationship of the sound positions of each audio channel, for example, audio channel A is in front of audio channel B, audio channel C is on the left side of audio channel B, etc.

[0122] In some embodiments, the method further comprises:

[0123] In response to a change in the sound position information of the audio channel, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device are acquired based on the sound position information of each audio channel.

[0124] Here, considering that the sound position information of each audio channel may change, for example, if the video data includes a walking person, as the person walks, the sound position should also change accordingly. Based on the changed sound position information of each audio channel, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device can be re-acquired; or, based on the sound position information of each audio channel, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device in different time periods can be acquired, wherein different time periods refer to different time periods within the target audio playback time.

[0125] In this way, the audio, video, and sound and image are matched in combination with the changes in the sound position, thereby providing users with better sound effects and experience.

[0126] In some embodiments, if there is a third target audio channel whose sound position information does not match the relative position information of each playback device, the method further includes:

[0127] Controlling the third playback device to play the third target audio channel with a third playback parameter, and controlling the fourth playback device to play the third target audio channel with a fourth playback parameter;

[0128] The third playback parameter and the fourth playback parameter are used to enable the third playback device and the fourth playback device to simulate the sound position corresponding to the third target audio channel.

[0129] Here, the third target audio channel may be any one of the first audio channel, the second audio channel, the third audio channel, and the fourth audio channel, or may be other audio components separated from the target audio, which is not limited here.

[0130] Consider the situation where there is a mismatch between the sound position information and the relative position information of each playback device, for example, the position of the playback device does not match the sound position in the video data, and for another example, during audio playback, the position of the sound object changes, causing a mismatch.

[0131] For an unmatched third target audio channel, a playback device may be used to virtually generate a desired sound emission position. For example, two playback devices may use their own playback parameters to simultaneously play the third target audio channel to virtually generate a sound effect of the third target audio channel at the corresponding sound emission position.

[0132] Here, the third playback parameter and the fourth playback parameter may include a stereo filter, and the stereo filter is used to adjust the playback ratio of the audio between the playback devices. By adjusting the ratio, the position of the sound in space can be set. Here, the audio channel is played based on the corresponding playback parameters according to the distance difference calculation based on the required sound emission position (i.e., the required sound image), the position of the target object, the position of the third playback device, the position of the fourth playback device, etc.

[0133] For example, there are playback devices A and playback devices B, and the target audio is separated into audio component A, audio component B, and audio component C. It is determined that audio component A must be played at actual position 1, but there is no playback device at actual position 1. The third playback parameter and the fourth playback parameter can be calculated based on the distance difference between the position of playback device A, the position of playback device B, and the actual position 1. The proportion of audio component A played by playback device A and the proportion of audio component A played by playback device B are adjusted respectively through the third playback parameter and the fourth playback parameter to obtain the desired sound and image, that is, the audio component A is virtualized to make a sound at actual position 1 (that is, the desired sound and image) through playback devices A and playback devices B.

[0134] It should be noted that the sound position information in the embodiments of the present disclosure is based on the target object. For example, the position of the aforementioned instrument A is to the right of the positions of instruments B and C. This is based on the target object. The arrangement of instruments A, B, and C is that the position of instrument A is to the right of the positions of instruments B and C, that is, the first audio channel includes instrument A, and the second audio channel includes instruments B and C, so that the sound position of the first audio channel perceived by the target object is to the right of the sound position of the second audio channel.

[0135] In this way, by virtually playing the effect of the third target audio channel at the corresponding sound position, the audio, video, and sound and image are matched, thereby providing users with better sound effects and experience.

[0136] Figure 2 A schematic diagram of a flow chart of an audio playback method provided by an application embodiment of the present disclosure is shown; Figure 2As shown, the method includes:

[0137] Step 201: multiple devices are positioned to construct a three-dimensional coordinate system;

[0138] Here, multiple devices include: a control device, a listening object (equivalent to a target object), and multiple speakers (equivalent to a playback device);

[0139] Step 202: performing sound source separation on the target audio to obtain multiple audio channels;

[0140] Here, sound source separation can be performed through the component information of the audio itself (such as midi or remix), or by using an AI sound source separation algorithm. Of course, traditional sound source separation methods can also be used, which is not limited here.

[0141] For example, after separation, audio components of different human voices, audio components of different musical instruments, and environmental sounds are obtained, which are denoted as Xi (i represents the identifier of each audio component).

[0142] Step 203: Play multiple audio channels through multiple speakers.

[0143] Here, playing multiple audio channels through multiple speakers may include:

[0144] According to the positions of multiple speakers and the position of the listening object, combined with the sound and image change strategy, the sound and image Xi of each audio component is determined, and then which part of the speakers j constitutes the sound and image Xi of the audio component is selected. The component of the sound and image Xi in the speaker j is Xij, and each speaker Xj=ΣXij is superimposed and output.

[0145] If the number of audio components changes or the sound image moves, the speaker Xj will change accordingly, and fade-in and fade-out will be used for smoothing.

[0146] Different audio components can have different sound images, such as Figure 3 As shown, there are sound image H1 and sound image H2. The sound image H1 is obtained through the two speakers on the left, and the sound image H2 is obtained through the two speakers on the right.

[0147] The sound image can change the sound position, such as Figure 4 As shown, the position of the sound object remains unchanged, the sound image corresponding to the vehicle at time t1 is on the lower left side, and the sound image corresponding to the vehicle at time t2 is on the upper right side.

[0148] Here, the sound image change strategy is used to indicate how to determine the sound image of each audio component based on the actual situation of each audio separation obtained by the three-dimensional coordinate system, and further determine how to construct the sound image Xi of the audio component through each speaker.

[0149] like Figure 5As shown in (a), a sound image of audio separation is determined, such as the sound image of a single instrument is directly in front of the listening object;

[0150] like Figure 5 As shown in (b), when adding instruments, the arrangement needs to be changed. It can be dynamically adjusted according to the arrangement of the symphony, or it can be dynamically adjusted according to the composition of the instruments, with instruments with high energy being moved to the front row and background instruments being moved to the back row;

[0151] like Figure 5 As shown in (c), based on the separated vocal audio components, dynamic adjustments can be made according to the arrangement of the choir;

[0152] like Figure 5 As shown in (d), if the target audio corresponds to video data, the audio component can be detected by comparing the video data and the sound and image position can be dynamically adjusted.

[0153] Of course, the user can also set the sound image of each audio component to a fixed position or a fixed mode of movement, such as a surround playback mode.

[0154] By separating the target audio, multiple audio components are obtained, and the sound image of each audio component is determined. Multiple speakers are used to play them respectively to present different audio images, so that the listening effect of the listening object will be more balanced and natural.

[0155] Playing the multiple audio channels through the multiple speakers may also include: determining a playback parameter of each speaker.

[0156] Here, the playback parameters may include at least one of the following: playback time, equalizer, filter and other audio playback related parameters. These parameters may affect the effect of the sound in the indoor environment and ensure that the sound achieves the expected listening experience at different locations.

[0157] For example, the playback time may be calculated based on the distance between the position coordinates of the listening object and the position coordinates of each speaker.

[0158] The filter can be obtained by calculating a corresponding reverberation elimination filter after the control device detects the effect of the reverberation generated by each loudspeaker in the target space at the listening object.

[0159] The equalizer can select a corresponding equalizer according to the type of each audio component based on a preset rule, for example, different equalizers are used for male voices, female voices and different musical instruments.

[0160] Of course, other methods can also be used to determine the playback parameters. Figure 1 The method is described in detail and will not be repeated here.

[0161] Figure 6A schematic diagram of the structure of an audio playback device provided by an embodiment of the present disclosure is shown; Figure 6 As shown, the audio playback device includes:

[0162] An acquisition module, configured to acquire relative position information of a target object relative to a first playback device and a second playback device in response to a playback instruction of a target audio;

[0163] A first processing module, configured to control the first playback device to play a first target audio channel and control the second playback device to play a second target audio channel based on the relative position information;

[0164] Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel;

[0165] If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel;

[0166] The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

[0167] In some embodiments, the relative position information of the target object relative to the first playback device and the second playback device includes:

[0168] The first position coordinates of the target object in the target space, the second position coordinates of the first playback device in the target space, and the third position coordinates of the second playback device in the target space;

[0169] The device further includes: a second processing module, configured to determine a first playback parameter of the first playback device based on the first position coordinate and the second position coordinate;

[0170] Determining a second playback parameter of the second playback device based on the first position coordinates and the third position coordinates;

[0171] The target space is the space where the target object, the first playback device, and the second playback device are located.

[0172] In some embodiments, the second processing module is further used to adjust the first playback parameter and the second playback parameter in response to a change in the first position coordinates of the target object; or

[0173] In response to a change in the second position coordinates of the first playback device, adjusting the first playback parameter; or

[0174] In response to a change in the third position coordinate of the second playback device, the second playback parameter is adjusted.

[0175] In some embodiments, the relative position information includes relative position information of a first playback device and relative position information of a second playback device; the relative position information of the first playback device is the relative position information between the first playback device and the target object; the relative position information of the second playback device is the relative position information between the second playback device and the target object;

[0176] The device further includes: a third processing module, configured to obtain sound position information corresponding to each audio channel included in the target audio;

[0177] Based on the sound position information of each of the audio channels, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device are acquired.

[0178] In some embodiments, if the target audio corresponds to video data, the third processing module is used to obtain the sound position information corresponding to each audio channel included in the target audio in the video data; or

[0179] The sound position configuration file corresponding to each audio channel included in the target audio is read to obtain the sound position information corresponding to each audio channel included in the target audio.

[0180] In some embodiments, the third processing module is further used to respond to changes in the sound position information of the audio channel and, based on the sound position information of each audio channel, obtain the first target audio channel that matches the relative position information of the first playback device and the second target audio channel that matches the relative position information of the second playback device.

[0181] In some embodiments, if there is a third target audio channel whose sound position information does not match the relative position information of each playback device, the first processing module is further used to control the third playback device to play the third target audio channel with a third playback parameter, and control the fourth playback device to play the third target audio channel with a fourth playback parameter;

[0182] The third playback parameter and the fourth playback parameter are used to enable the third playback device and the fourth playback device to simulate the sound position corresponding to the third target audio channel.

[0183] In some embodiments, the second processing module is further used to control the first playback device and the second playback device to play the test audio respectively;

[0184] Based on the test audio collected by the audio collection device, determining a first distance between the target object and the first playback device and a second distance between the target object and the second playback device;

[0185] According to the first distance and the second distance, controlling the first playback device to play the first target audio channel's playback time, and controlling the second playback device to play the second target audio channel's playback time;

[0186] Wherein, the audio collection device is an audio collection device of a device carried by the target object.

[0187] It is understandable that, when the audio playback device provided in the above embodiment implements the corresponding audio playback method, the above processing can be assigned to different program modules as needed to complete all or part of the above-described processing. In addition, the device provided in the above embodiment and the embodiment of the corresponding method belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0188] An embodiment of the present disclosure provides a computer-readable storage medium storing executable instructions, wherein the executable instructions are stored. When the executable instructions are executed by a processor, the processor will be triggered to execute the audio playback method provided by the embodiment of the present disclosure.

[0189] In some embodiments, the computer-readable storage medium may be a ferroelectric random access memory (FRAM), a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a flash memory, a magnetic surface memory, an optical disk, or a CD-ROM; it may also be various devices including one or any combination of the above memories.

[0190] In some embodiments, executable instructions may be in the form of a program, software, software module, script or code, written in any form of programming language (including compiled or interpreted languages, or declarative or procedural languages), and may be deployed in any form, including as a stand-alone program or as a module, model, subroutine or other unit suitable for use in a computing environment.

[0191] By way of example, executable instructions may be deployed to be executed on one computing device, or on multiple computing devices located at one site, or on multiple computing devices distributed across multiple sites and interconnected by a communication network.

[0192] An embodiment of the present disclosure provides a computer program product, which includes a computer program / instructions. When the computer program / instructions are executed by a processor, the audio playback method described in the present disclosure is implemented.

[0193] Figure 7 The structure diagram of an electronic device provided by the embodiment of the present disclosure is shown in FIG. Figure 7 As shown, the electronic device 70 includes: a processor 701 and a memory 702 for storing a computer program that can be run on the processor; when the processor 701 is used to run the computer program, it executes the audio playback method provided by the embodiment of the present disclosure.

[0194] In actual application, the electronic device 70 may further include: at least one network interface 703. The various components in the electronic device 70 are coupled together via a bus system 704. It is understood that the bus system 704 is used to achieve connection and communication between these components. In addition to the data bus, the bus system 704 also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, Figure 7 In the figure, various buses are marked as bus system 704. There may be at least one processor 701. The network interface 703 is used for wired or wireless communication between the electronic device 70 and other devices.

[0195] The memory 702 in the embodiment of the present disclosure is used to store various types of data to support the operation of the electronic device 70 .

[0196] The method disclosed in the above-mentioned embodiment of the present disclosure can be applied to the processor 701, or implemented by the processor 701. The processor 701 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above-mentioned method can be completed by the hardware integrated logic circuit in the processor 701 or the instruction in the form of software. The above-mentioned processor 701 can be a general-purpose processor, a digital signal processor (DSP, DiGital Signal Processor), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The processor 701 can implement or execute the disclosed methods, steps and logic block diagrams in the embodiment of the present disclosure. The general-purpose processor can be a microprocessor or any conventional processor, etc. The steps of the method disclosed in the embodiment of the present disclosure can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be combined to execute. The software module can be located in a storage medium, which is located in the memory 702. The processor 701 reads the information in the memory 702 and completes the steps of the above-mentioned method in combination with its hardware.

[0197] In some embodiments, the electronic device 70 can be implemented by one or more application specific integrated circuits (ASIC), DSP, programmable logic device (PLD), complex programmable logic device (CPLD), field programmable gate array (FPGA), general processor, controller, microcontroller (MCU), microprocessor, or other electronic components to execute the aforementioned method.

[0198] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0199] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0200] The above is only a specific embodiment of the present disclosure, but the protection scope of the present disclosure is not limited thereto. Any person skilled in the art who is familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present disclosure, which should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. An audio playback method, comprising: In response to a target audio playback instruction, obtaining relative position information of the target object relative to the first playback device and the second playback device; Based on the relative position information, controlling the first playback device to play a first target audio channel and controlling the second playback device to play a second target audio channel; Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel; If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel; The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

2. According to the method of claim 1, the relative position information of the target object relative to the first playback device and the second playback device comprises: The first position coordinates of the target object in the target space, the second position coordinates of the first playback device in the target space, and the third position coordinates of the second playback device in the target space; The method further comprises: Determining a first playback parameter of the first playback device based on the first position coordinates and the second position coordinates; Determining a second playback parameter of the second playback device based on the first position coordinates and the third position coordinates; The target space is the space where the target object, the first playback device, and the second playback device are located.

3. The method according to claim 2, further comprising: In response to a change in the first position coordinates of the target object, adjusting the first playback parameter and the second playback parameter; or In response to a change in the second position coordinates of the first playback device, adjusting the first playback parameter; or In response to a change in the third position coordinate of the second playback device, the second playback parameter is adjusted.

4. The method according to claim 1, wherein the relative position information comprises relative position information of a first playback device and relative position information of a second playback device; the relative position information of the first playback device is relative position information between the first playback device and the target object; The relative position information of the second playback device is the relative position information between the second playback device and the target object; the method further includes: Obtaining sound position information corresponding to each audio channel included in the target audio; Based on the sound position information of each of the audio channels, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device are acquired.

5. The method according to claim 4, if the target audio corresponds to video data, obtaining the sound position information corresponding to each audio channel contained in the target audio comprises: Obtaining sound position information corresponding to each audio channel included in the target audio in the video data; or The sound position configuration file corresponding to each audio channel included in the target audio is read to obtain the sound position information corresponding to each audio channel included in the target audio.

6. The method according to claim 4 or 5, further comprising: In response to a change in the sound position information of the audio channel, the first target audio channel matching the relative position information of the first playback device and the second target audio channel matching the relative position information of the second playback device are acquired based on the sound position information of each audio channel.

7. The method according to claim 4, if there is a third target audio channel whose sound position information does not match the relative position information of each playback device, the method further comprises: Controlling the third playback device to play the third target audio channel with a third playback parameter, and controlling the fourth playback device to play the third target audio channel with a fourth playback parameter; The third playback parameter and the fourth playback parameter are used to enable the third playback device and the fourth playback device to simulate the sound position corresponding to the third target audio channel.

8. The method according to claim 1, further comprising: Controlling the first playback device and the second playback device to play test audio respectively; Determine, based on the test audio collected by the audio collection device, a first distance between the target object and the first playback device and a second distance between the target object and the second playback device; According to the first distance and the second distance, controlling the first playback device to play the first target audio channel's playback time, and controlling the second playback device to play the second target audio channel's playback time; Wherein, the audio collection device is an audio collection device of a device carried by the target object.

9. An audio playback device, comprising: An acquisition module, configured to acquire relative position information of a target object relative to a first playback device and a second playback device in response to a playback instruction of a target audio; A first processing module, configured to control the first playback device to play a first target audio channel and control the second playback device to play a second target audio channel based on the relative position information; Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel; If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel; The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

10. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to execute: In response to a target audio playback instruction, obtaining relative position information of the target object relative to the first playback device and the second playback device; Based on the relative position information, controlling the first playback device to play a first target audio channel and controlling the second playback device to play a second target audio channel; Wherein, if the relative position information represents a first relative position, the first target audio channel is a first audio channel, and the second target audio channel is a second audio channel; If the relative position information represents a second relative position, the first target audio channel is a third audio channel, and the second target audio channel is a fourth audio channel; The first audio channel is different from the third audio channel, and / or the second audio channel is different from the fourth audio channel.

Images