Audio playing method, electronic equipment, storage medium and computer program product

Abstract

The invention discloses an audio playing method, electronic equipment, a storage medium and a computer program product, relates to the technical field of audio playing, and can improve a sound playing effect when the electronic equipment plays audio. More than two loudspeakers are arranged in the electronic equipment, and a hardware interface supporting multi-channel playing is arranged. Specifically, when a multi-channel sound source comprising N channels of sound channel data is played, the electronic equipment can adaptively create N transmission paths, so that each channel of sound channel data of the multi-channel sound source can be transmitted to a lower layer without being lost. Furthermore, the lower layer can perform sound effect processing related to a three-dimensional sound field for the more complete and more comprehensive sound channel data, generate audio data which is in one-to-one correspondence with the loudspeakers in the electronic equipment and has more space sense and more sound field information after the sound effect processing, and control the loudspeakers to play the corresponding audio data based on the hardware interface. And a three-dimensional sound field playing effect can be realized.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of audio playing, and in particular to an audio playing-out method, an electronic device, a storage medium and a computer program product. BACKGROUND

[0002] With the popularity of electronic devices such as smart tablets, most users will use electronic devices to play audio and video in daily life, for example, the sound in daily audio and video entertainment scenes is usually played in the form of playing-out, especially when using video APPs or game APPs, the quality of playing-out experience directly affects the overall audio experience of the electronic device.

[0003] Generally, electronic devices do not have the ability to support multi-channel playing, when playing multi-channel sound sources, multiple channel data are usually automatically reduced to stereo double channels for playing-out. In this way, part of the sound field information contained in the multi-channel is discarded, the sound played out cannot restore the multi-channel information, resulting in poor sound effect of playing-out and poor auditory experience. SUMMARY

[0004] Embodiments of the present application provide an audio playing-out method, an electronic device, a storage medium and a computer program product, which can improve the sound playing-out effect when the electronic device plays audio, and further improve the auditory experience of users.

[0005] To achieve the above-mentioned purpose, embodiments of the present application adopt the following technical solutions:

[0006] In a first aspect, an audio playing-out method is provided. The method is applied to an electronic device. The electronic device is provided with M loudspeakers, where M is greater than 2, and a hardware interface supporting multi-channel audio data (supporting multi-channel playing). Specifically, the electronic device receives a playing instruction of a first sound source. The first sound source includes first channel data of a first channel number N, where the first channel number N is greater than or equal to 3. In response to the playing instruction, N playing paths are created. That is, when playing a multi-channel sound source, a playing path is created for each channel of the multi-channel sound source. Each playing path is used to transmit target channel data, such as the first channel data or second channel data obtained by preprocessing the first channel data. Thus, the method is not limited to fixed two-channel two-path, and each channel data of the multi-channel sound source can be transmitted to the lower layer without being lost. Further, sound field widening and other audio effects can be performed based on the N target channel data transmitted by the N playing paths, and N third channel data with stronger spatial sense is obtained. The sound field width of each third channel data is not less than the sound field width of the corresponding target channel data, and the sound field width of at least one third channel data is greater than the sound field width of the corresponding target channel data. That is, at least part of the N target channel data is subjected to sound field widening, and the sound field widened channel data and the target channel data not subjected to sound field widening together constitute the N third channel data. Further, M audio data corresponding to the M loudspeakers is generated based on the N third channel data. Further, the M loudspeakers can be controlled to play the corresponding audio data based on the hardware interface supporting multi-channel playing.

[0007] In the above scheme, the software and hardware of the electronic device are improved. In the software aspect, playing paths matching the channel number of a multi-channel sound source are adaptively created to transmit the channel data of the sound source to the lower layer without loss. The lower layer can perform sound field widening and other audio effects based on more complete and comprehensive multi-channel data, and obtain channel data with stronger spatial sense. The audio data corresponding to each loudspeaker is generated. The audio data has more channel information of the sound source and stronger spatial sense than the two-channel data. In the hardware aspect, the hardware interface supporting multi-channel audio data is provided, and the audio data corresponding to each loudspeaker is transmitted to the loudspeaker for playing to realize a three-dimensional sound field playing effect (playing-out effect). That is, without the need for additional auxiliary equipment (without the need for purchasing additional multiple playing devices for networking), the three-dimensional sound field playing effect is realized by improving the software and hardware, which saves cost and improves the audio playing-out effect, thereby improving the user experience.

[0008] In addition, the playing channel can be adaptively created, so that the playing of multiple-channel sound sources such as 5.1 / 7.1 / 5.1.2 / 7.1.4 can be applied, the applicability is greatly improved, and the user experience is further improved.

[0009] Further, when multiple playing devices are purchased to form a network (i.e., a home theater device), interaction transmission between the devices is involved, and there is a certain delay problem, and the placement direction of the devices also affects the audio playing effect. The scheme is implemented in the electronic device, has relatively better stability, has smaller delay problem, and the speaker direction is fixed and will not be affected by the use process of the user, so that the optimal audio playing effect is ensured.

[0010] In a possible implementation manner of the first aspect, the electronic device includes an application layer, an application framework layer, and a hardware abstraction layer. The application layer receives a playing instruction of a first sound source; the application framework layer identifies a first channel number of the first sound source, to select a playing link supporting multi-channel playing based on the first channel number; and the hardware abstraction layer creates a playing channel of the first channel number based on the selected playing link.

[0011] In the scheme, the application layer, the application framework layer, and the hardware abstraction layer are improved, so that the application layer, the application framework layer, and the hardware abstraction layer can cooperate with each other, and a playing channel matched with the channel number of the multi-channel sound source can be conveniently and adaptively created.

[0012] In a possible implementation manner of the first aspect, the application framework layer can select a native playing link supporting multi-channel playing based on the first channel number of the multi-channel sound source, or select a customized playing link supporting multi-channel playing. The native playing link refers to a playing link provided by the software system of the electronic device, for example, direct or offload, which are playing links supporting multi-channel playing provided by the software system. The customized playing link refers to a playing link supporting multi-channel playing customized by the manufacturer of the electronic device.

[0013] In the scheme, the native playing link supporting multi-channel playing is innovatively applied to the scene of three-dimensional sound field playing of the multi-channel sound source, that is, the application framework layer and the hardware abstraction layer are connected and called with the native playing link, and together form the audio playing scheme for realizing the three-dimensional sound field playing effect. That is, a new technical scheme for realizing the three-dimensional sound field playing by reasonably using the native playing link is provided, and the resource utilization rate is improved.

[0014] In addition, a custom playback link supporting multi-channel playback can also be implemented, so that the playback link is not limited to the playback link provided by the software system, but can be extended more flexibly according to actual needs, can adapt to three-dimensional sound field playback of any multi-channel sound source, greatly improves the applicability, and improves the user experience.

[0015] In a possible implementation of the first aspect, the electronic device further includes a digital signal processor; the digital signal processor generates corresponding third channel data for each target channel data transmitted by each playback channel; and the digital signal processor generates audio data corresponding to each speaker based on the third channel data, and controls each speaker to play the corresponding audio data through the hardware interface.

[0016] In the above scheme, the digital signal processor implements multi-channel sound effect processing and multi-channel data management to generate audio data, which decouples the upper layer, is more flexible and has higher efficiency.

[0017] In a possible implementation of the first aspect, for each speaker, the audio data corresponding to the speaker is generated based on third channel data in the third channel data of the first channel number, which matches the orientation of the speaker. Wherein, the orientation of the speaker refers to the relative orientation of the speaker and the user; the electronic device is placed in different attitudes relative to the user, and the relative orientation of the same speaker and the user is different.

[0018] In the above scheme, the orientation matching principle can more accurately allocate channel data to each speaker, and then more accurately synthesize the audio data corresponding to the speaker, so that the multiple speakers play the audio data to form a more accurate three-dimensional sound field playback effect, thereby improving the user experience.

[0019] In a possible implementation of the first aspect, the third channel data of the first channel number includes center channel data and non-center channel data. The electronic device can split the center channel data into at least two sub-center channel data and then assign them to the speakers. It should be noted that, in the current attitude of the electronic device relative to the user, the speakers assigned to the sub-center channel data are located on the left and right sides of the electronic device. For the speaker assigned to the sub-center channel data, the electronic device can combine the non-center channel data and the sub-center channel data assigned to the speaker to generate the audio data corresponding to the speaker.

[0020] For example, the speaker to which the sub-center channel data is assigned can be all the speakers, i.e., the center channel data can be assigned to all the speakers, and then for each speaker, the electronic device can combine the sub-center channel data and the non-center channel data assigned to the speaker to obtain the audio data corresponding to the speaker. It should be understood that the non-center channel data assigned to the speaker can be the third channel data whose channel orientation matches the orientation of the speaker.

[0021] For another example, the speaker to which the sub-center channel data is assigned can be part of the speakers. Then, for the speaker to which the sub-center channel data is not assigned, the electronic device can combine the non-center channel data assigned to the speaker to generate the audio data corresponding to the speaker.

[0022] In the above scheme, the center channel data is assigned to the speakers on the left and right sides, and is combined with other channel data assigned to the speakers on the left and right sides to be played, thereby forming the effect that the center sound is played at the middle position of the electronic device.

[0023] In a possible implementation of the first aspect, the third channel data of the first number of channels includes center channel data and non-center channel data, and the speakers set by the electronic device include first type speakers and second type speakers. In the current placement posture of the electronic device relative to the user, the position of the first type speakers is higher than the position of the second type speakers. The center channel data is split into sub-center channel data corresponding to each speaker in the first type speakers, i.e., the center sound data is assigned to the speakers with relatively higher positions. For each speaker in the first type speakers, the electronic device can combine the sub-center channel data corresponding to the speaker and the non-center channel data whose channel orientation matches the orientation of the speaker to obtain the audio data corresponding to the speaker. For each speaker in the second type speakers, the electronic device can combine the non-center channel data whose channel orientation matches the orientation of the speaker to obtain the audio data corresponding to the speaker.

[0024] In the above scheme, when the user uses the electronic device, in many cases, the ear is higher than the center position of the electronic device. If the center sound is played at the center or near the center of the electronic device, the user will feel uncomfortable and the auditory experience of the user will be slightly poor. Therefore, assigning the center channel data to the speakers with relatively higher positions or relatively higher positions from the current perspective of the user can form the effect that the center sound is emitted at the center position of the electronic device, so that the position of the center sound (e.g., the center vocal) is more accurate and closer to the ear, thereby improving the auditory experience of the user.

[0025] In a possible implementation manner of the first aspect, the first sound source is a 7.1.4 channel sound source; the first type of speaker includes a first speaker and a second speaker, and the second type of speaker includes a third speaker and a fourth speaker; in a current placement posture of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left corner and the upper right corner of the electronic device respectively, and the third speaker and the fourth speaker are located at the lower left corner and the lower right corner of the electronic device respectively. The electronic device can combine front left channel data, top front left channel data and top back left channel data in third channel data of the first channel number with sub-center channel data corresponding to the first speaker to obtain audio data corresponding to the first speaker. The electronic device can combine front right channel data, top front right channel data and top back right channel data in third channel data of the first channel number with sub-center channel data corresponding to the second speaker to obtain audio data corresponding to the second speaker. The electronic device can combine back left channel data and back left surround channel data in third channel data of the first channel number to obtain audio data corresponding to the third speaker. The electronic device can combine back right channel data and back right surround channel data in third channel data of the first channel number to obtain audio data corresponding to the fourth speaker.

[0026] In the above scheme, the first sound source is a 7.1.4 channel sound source, and there are four speakers. When data is allocated to the speakers, the center channel data can be allocated to the first speaker at the upper left corner and the second speaker at the upper right corner. In addition, based on the position matching principle, channel data close to the upper left in the 7.1.4 channel sound source can be allocated to the first speaker currently located at the upper left corner relative to the user, channel data close to the upper right can be allocated to the second speaker currently located at the upper right corner relative to the user, channel data close to the lower left can be allocated to the third speaker currently located at the lower left corner relative to the user, and channel data close to the lower right can be allocated to the fourth speaker currently located at the lower right corner relative to the user. Therefore, the accuracy of data allocation is improved, the three-dimensional sound field playback effect of the 7.1.4 channel sound source can be better achieved, and the user's auditory experience is improved. Moreover, the effect of the center sound being emitted from a position above the center of the electronic device can be formed, and the user's experience is further improved.

[0027] In a possible implementation manner of the first aspect, before generating, for each loudspeaker, audio data corresponding to the loudspeaker based on third channel data in the first number of channel data whose channel orientation matches the orientation of the loudspeaker, the orientation of each loudspeaker relative to the user in a real-time placement posture of the electronic device can be acquired. Thus, based on the current orientation of each loudspeaker relative to the user in the real-time placement posture, the channel data with the matched orientation is allocated to each loudspeaker, thereby avoiding the problem of inaccurate allocation of channel data caused by the change of the orientation of the loudspeaker relative to the user due to the rotation of the electronic device, and further improving the subsequent three-dimensional sound field playback effect.

[0028] In a possible implementation manner of the first aspect, in the concurrent playback scene, fourth channel data of a second sound source is acquired, the second sound source being a sound source played concurrently with the first sound source. For each loudspeaker, the electronic device can combine the third channel data whose channel orientation matches the orientation of the loudspeaker and the fourth channel data whose channel orientation matches the orientation of the loudspeaker, to obtain audio data corresponding to the loudspeaker.

[0029] In the above scheme, in the concurrent playback scene, the channel data whose channel orientation matches the orientation of the same loudspeaker is combined first according to the orientation matching principle, to obtain audio data corresponding to the loudspeaker, which is then played by the loudspeaker. Thus, each loudspeaker can reasonably and accurately play the sound of multiple sound sources in the concurrent playback scene, thereby meeting the scene requirement of concurrent playback. Moreover, the concurrent playback through the above scheme is not simply playing the sound of multiple sound sources, but reasonably allocating channel data to accurately and harmoniously play the sound of two sound sources. In addition, the spatial effect of the three-dimensional sound field playback formed for the multi-channel sound source is not affected in the concurrent playback scene, thereby greatly improving the user experience.

[0030] In a possible implementation manner of the first aspect, the fourth channel data of the second sound source includes left channel data and right channel data; the electronic device includes a first loudspeaker, a second loudspeaker, a third loudspeaker and a fourth loudspeaker; and in a current placement posture of the electronic device relative to the user, the first loudspeaker and the second loudspeaker are located at the upper left corner and the upper right corner of the electronic device respectively, and the third loudspeaker and the fourth loudspeaker are located at the lower left corner and the lower right corner of the electronic device respectively; the left channel data is allocated to the first loudspeaker and the third loudspeaker respectively; and the right channel data is allocated to the second loudspeaker and the fourth loudspeaker respectively.

[0031] In the above scheme, when the multi-channel sound source and the stereo double-channel sound source are played concurrently, the left and right channel data of the stereo double-channel sound source can be accurately combined with the channel data in the multi-channel sound source based on the channel orientation, thereby achieving a good audio externalization effect in the concurrent playback scene.

[0032] In a possible implementation of the first aspect, each speaker has a corresponding power amplifier PA. For each speaker, the electronic device can transmit, through a hardware interface supporting multi-channel playback, audio data corresponding to the speaker to a PA corresponding to the speaker, input the audio data to the speaker through the PA for audio playback; when the speaker is playing audio, the current and voltage of the speaker are detected in real time, and the parameters of the PA are adjusted in real time according to the current and voltage, so that the speaker is kept in a target working state.

[0033] In the above scheme, in the audio external playback scheme for forming a three-dimensional sound field playback effect, the current and voltage of the speaker can be detected in real time, and the parameters of the PA are adjusted in real time based thereon, and the audio data corresponding to the speaker is processed by power amplification based on the adjusted PA parameters, so as to control the audio playback of the speaker. Thus, the speaker is always in an optimal working state, and the audio external playback effect of the three-dimensional sound field is realized.

[0034] In a second aspect, the present application provides an electronic device, which at least includes: more than two speakers, a memory and one or more processors; the memory, the more than two speakers and the processor are coupled; the more than two speakers are used for external audio, the memory stores computer program code, and the computer program code includes computer instructions, when the one or more processors execute the computer instructions, the electronic device executes the method of any one of the above first aspect.

[0035] In a third aspect, the present application provides a chip system applied to an electronic device, the chip system includes one or more processors, and the processor is used to call computer instructions, so that the electronic device executes the method of any one of the above first aspect.

[0036] In a fourth aspect, the present application provides a computer readable storage medium, which includes computer instructions, when the computer instructions run on an electronic device, the electronic device executes the method of any one of the above first aspect.

[0037] In a fifth aspect, the present application provides a computer program product, when the computer program product runs on a computer, the computer executes the method of any one of the above first aspect. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 An effect schematic diagram of a cinema-level sound field experience provided by an embodiment of the present application;

[0039] Figure 2 A channel position schematic diagram of a 5.1 channel system provided by an embodiment of the present application;

[0040] Figure 3 A channel position schematic diagram of a 7.1 channel system provided for an embodiment of the present application;

[0041] Figure 4 A channel position schematic diagram of a 7.1.4 channel system provided for an embodiment of the present application;

[0042] Figure 5 A principle diagram of a related technology limited to dual-channel playback provided for an embodiment of the present application;

[0043] Figure 6 A hardware framework schematic diagram of an embodiment of the present application for implementing multi-channel playback;

[0044] Figure 7 A hardware framework schematic diagram of another embodiment of the present application for implementing multi-channel playback;

[0045] Figure 8 A hardware structure block diagram of an electronic device provided for an embodiment of the present application;

[0046] Figure 9A A software structure block diagram of an electronic device provided for an embodiment of the present application;

[0047] Figure 9B Another software structure block diagram of an electronic device provided for an embodiment of the present application;

[0048] Figure 10 A timing diagram of an audio external playing method provided for an embodiment of the present application;

[0049] Figure 11 A method schematic diagram of another software structure block diagram of an electronic device provided for an embodiment of the present application;

[0050] Figure 12 A timing diagram of another audio external playing method provided for an embodiment of the present application;

[0051] Figure 13 A sound field widening effect schematic diagram provided for an embodiment of the present application;

[0052] Figure 14 A schematic diagram of a relative position between a loudspeaker and a user provided for an embodiment of the present application;

[0053] Figures 15A-15C Schematic diagrams of three different mixing processing provided for an embodiment of the present application;

[0054] Figure 16 A mixing processing schematic diagram in a changing posture placement scenario provided for an embodiment of the present application;

[0055] Figure 17A mixing processing schematic diagram in a concurrent playing scene is provided in the embodiments of the present application. DETAILED DESCRIPTION

[0056] In this specification, reference to "one embodiment" or "some embodiments" means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the application. The appearances of the phrase "in one embodiment" or "in some embodiments" in various places in the specification are not necessarily all referring to the same embodiment, although it can. The terms "comprising," "including," "having" and their variations, as used in this specification, mean "including but not limited to," unless expressly specified otherwise. The term "coupled" means either a direct connection or an indirect connection through one or more intervening wires or components, unless otherwise specifically noted.

[0057] In the embodiments of the present application, the word "exemplary" or "for example" is used to mean serving as an example, instance, or illustration. Any embodiment or design described herein as "exemplary" or "for example" is not necessarily to be construed as preferred or advantageous over other embodiments or designs. Rather, use of the word "exemplary" or "for example" is intended to present concepts in a concrete manner.

[0058] In the following, the terms "first", "second", "third", and "fourth" are used only for descriptive purposes and are not to be construed as indicating or implying relative importance or a specific number of technical features indicated. Thus, the features defined with "first", "second", "third", and "fourth" can include one or more of the features explicitly or implicitly. In the description of the embodiments, the meaning of "a plurality of" is two or more, unless otherwise specified.

[0059] The embodiments of the present application provide an audio playing-out method, which can be applied to an electronic device. The electronic device is provided with a plurality of (more than two) loudspeakers. In an audio playing-out scene, without the aid of other auxiliary devices, the electronic device can control the loudspeakers to play multi-channel audio data by executing the audio playing-out method in the embodiments of the present application, so as to enable a user to experience the sound field space information conveyed by the multi-channel audio data, provide the user with a cinema-level sound field experience, and not simply a stereo effect, thereby improving the audio playing-out effect. For details, please refer to Figure 1 , Figure 1 In the embodiments of the present application, the electronic device is taken as a tablet computer as an example. In response to a user playing audio data by using the tablet computer, the user can experience the effect that the loudspeakers seem to play in a three-dimensional space, and achieve a three-dimensional sound field experience close to a cinema level.

[0060] It should be understood that the source of the audio data played in the audio playback scene can include at least one of a single audio file, audio data in a video file, audio data in an application or webpage, etc., without limitation.

[0061] The multi-channel audio data refers to audio data under a multi-channel system, which includes channel data of multiple channels. The multi-channel system refers to a channel system with more than two channels. For example, the multi-channel system can include 2.1 channels, 3.0 channels, 5.1 channels, 5.1.2 channels, 7.1 channels, 7.1.2 channels, or 7.1.4 channels, etc.

[0062] It should be understood that different multi-channel systems have different channel position layouts. The channel position refers to the sound direction and position effect that the channel data is intended to present when recording or playing. For example, the channel position of the left channel data is intended to present the effect that the sound is played on the left side of the user's ear, the channel position of the right channel data is intended to present the effect that the sound is played on the right side of the user's ear, and the channel position of the sky channel data (or called the top channel data) is intended to present the effect that the sound is played above the user.

[0063] In the conventional method, if a user wants to experience a cinema-level three-dimensional sound field effect like Figure 1 , it is usually necessary to purchase a set of home theater equipment required by the multi-channel system and place each home theater equipment at the corresponding position according to the channel position layout of the multi-channel system. The channel positions of different channel systems and the placement positions of the home theater equipment will be described below. Figures 2-4 .

[0064] Figure 2 is a channel position diagram of a 5.1 channel system. The 5.1 channel system includes a front left channel FL, a front right channel FR, a center channel C, a left surround channel SL, a right surround channel SR, and a subwoofer channel SW. As can be seen from Figure 2 , if the sound field effect of the 5.1 channel system is to be realized, an audio box needs to be purchased for each channel position, i.e., at least six audio boxes need to be purchased and placed according to the channel position layout of the 5.1 channel system.

[0065] Figure 3 is a channel position diagram of a 7.1 channel system. The 7.1 channel system adds two rear surround channels, i.e., left and right rear surround channels, based on the 5.1 channel system. Specifically, the 7.1 channel system includes a front left channel FL, a front right channel FR, a center channel C, a left surround channel SL, a right surround channel SR, a left rear surround LB, a right rear surround RB, and a subwoofer channel SW. As can be seen from Figure 3It can be seen that if the sound field effect of the 7.1 channel system is to be achieved, at least 8 speakers need to be purchased and arranged according to the channel position layout of the 7.1 channel system.

[0066] Figure 4 is a schematic diagram of the channel position of the 7.1.4 channel system. The 7.1.4 channel system adds 4 sky channels to the 7.1 channel system, i.e., a top front left channel TFL, a top front right channel TFR, a top rear left channel TRL, and a top rear right channel TRR. Specifically, the 7.1.4 channel system includes a front left channel FL, a front right channel FR, a center channel C, a left surround channel SL, a right surround channel SR, a left back surround channel LB, a right back surround channel RB, a subwoofer channel SW, the top front left channel TFL, the top front right channel TFR, the top rear left channel TRL, and the top rear right channel TRR. From Figure 4 It can be seen that if the sound field effect of the 7.1.4 channel system is to be achieved, at least 12 speakers need to be purchased and arranged according to the channel position layout of the 7.1.4 channel system.

[0067] From Figures 2-4 It can be found that the cost will be very high by purchasing a set of home theater equipment to assist each other to achieve a better audio external effect.

[0068] With the rapid development of terminal technology, many electronic devices have multiple (more than 2) speakers. For example, many electronic devices have 4, 6, 8 or even more speakers. If not assisted by other devices, the three-dimensional sound field audio external effect will be greatly reduced by using an electronic device with multiple speakers.

[0069] However, in the related art, some electronic devices have multiple speakers but do not have the ability to support multi-channel playback. Then, even if the electronic device plays a multi-channel sound source (i.e., multi-channel audio data), it will eventually still reduce the stereo double-channel playback, and the multiple speakers can only increase the playback volume or playback loudness of the left and right double channels, and cannot achieve the three-dimensional sound field audio external effect. The following will combine Figure 5 The above related art is schematically described.

[0070] Please refer to Figure 5 For example, in the scenario of playing a multi-channel sound source on a tablet, 8 speakers are provided in the tablet. Since the tablet does not have the ability to support multi-channel playback, when playing a 5.1 / 7.1 multi-channel sound source in the tablet, the tablet will first downmix the multi-channel sound source into stereo left and right channel data. For example, Figure 5 The tablet will downmix the 6-channel data of the 5.1 channel sound source into 2-channel data of left channel data and right channel dataFigure 5 Each arrow in the figure represents a sound signal, a channel of sound data or a channel of audio data essentially belongs to a sound signal. Even if some audio processing algorithms are set in the panel (such as sound effect processing), these algorithms will only act on the left and right channel data subsequently, that is, the panel will only use these algorithms to process the left and right channel data, and then the left and right channel data can only be obtained after the algorithm processing. Finally, only the left and right channel data is output in the 8 speakers, for example, the processed left channel data is played through the 4 speakers on the left side of the panel, and the processed right channel data is played through the 4 speakers on the right side of the panel. It should be understood that when the multi-channel sound source is down-mixed into stereo two-channel data, part of the sound field information in the multi-channel sound source is discarded, resulting in that the sound played by the panel cannot accurately restore the expression of the multi-channel data, so that the three-dimensional sound field playing effect cannot be realized, and only stereo playing can be realized.

[0071] In addition, in some other related technologies, signal conversion and other processing are performed after the channel data is distributed to the speakers. This scheme still belongs to the soft and hardware channels that are not connected inside the device, that is, in the process of transmitting the channel data to the speakers, it is still limited by the two-channel channel, resulting in that part of the useful sound field information is discarded. Finally, the channel data distributed to the speakers is already the data after the loss of the sound field, and then the signal conversion and other processing based on the data after the loss of the sound field is performed, and the external playing effect is limited.

[0072] In view of the above problems, in the scheme of the present application, the software and hardware levels of the electronic device are improved, which involves the cooperation of the software and hardware system (that is, supporting multi-channel channels is realized at the software and hardware levels), and the electronic device is given the ability to support multi-channel playing, and combined with an audio external playing method proposed in the embodiments of the present application, the three-dimensional sound field playing effect can be achieved through the speakers of the electronic device.

[0073] Specifically, when playing a multi-channel sound source, at the software level, the electronic device can adaptively create a playing channel matching the number of channels of the multi-channel sound source, so that each channel of the multi-channel sound source can be transmitted to the lower layer without being lost. Further, the sound effect processing related to the three-dimensional sound field is performed on the more complete and more comprehensive channel data, and audio data corresponding to each speaker of the electronic device is generated after the sound effect processing. At the hardware level, a hardware interface supporting multi-channel audio data is provided in the electronic device, and through the hardware interface, the audio data corresponding to each speaker can be transmitted to the corresponding speaker for playing, so as to realize the output of the three-dimensional sound field.

[0074] It should be understood that the sound channel widening described in the embodiments of the present application does not limit all sound channel data of the multi-channel sound source to be sound field widened, and only a part of the sound channel data (i.e., the sound channel data that can improve the three-dimensional space effect after sound field widening) can be sound field widened, and then the sound channel data after sound field widening and the sound channel data of other sound channels without sound field widening are mixed to generate audio data corresponding to each loudspeaker in the electronic device.

[0075] The scheme in the embodiments of the present application can realize the three-dimensional sound field playing effect without additional auxiliary equipment, thereby greatly reducing the hardware cost. In addition, compared with the multi-channel sound source in the prior art, the multi-channel sound source in the embodiments of the present application can fully play the role of the multiple loudspeakers and improve the utilization rate of the hardware resources of the electronic device itself. Figure 5

[0076] For ease of understanding, the embodiments of the present application will be described in detail below with reference to the accompanying drawings. Figure 6 Figure 6 In the embodiment, eight loudspeakers (i.e., SPK1 to SPK8) are arranged in the electronic device. The electronic device is provided with a hardware interface supporting multi-channel audio data, and the hardware interface is used to transmit the audio data to the eight loudspeakers. Figure 6 In the embodiment, only the 5.1 sound source is taken as an example for description, and the present application is not limited to the 5.1 sound source. Other multi-channel sound sources such as 7.1, 7.1.2, 7.1.4, etc. can also be played in the electronic device. Figure 6 In the embodiment, each arrow represents a sound signal (for example, a sound channel data or audio data. It should be understood that the sound channel data and the audio data in the embodiments of the present application are essentially sound signals, and the two are used to distinguish the sound signals at different stages).

[0077] ​​5.1 channel sound source can include 6 channels of data, i.e., front left channel FL, front right channel FR, center channel C, left surround channel SL, right surround channel SR, and subwoofer channel SW. Thus, the electronic device can adaptively create 6 playback channels, each of which is used to transmit 1 channel of data, so that the 6 channels of data of the 5.1 channel sound source are not lost and are all transmitted downward for subsequent multi-channel sound source sound effect processing. In the subsequent multi-channel sound source sound effect processing, all or part of the 6 channels of data can be subjected to three-dimensional sound field widening and other sound effect processing. Assuming that the sound field widening is performed on the channel data of the front left channel FL, the front right channel FR, the left surround channel SL, and the right surround channel SR, and the sound field widening is not performed on the channel data of the center channel C and the subwoofer channel SW. After the sound effect processing, there are still 6 channels of data, i.e., the front left channel FL', the front right channel FR', the left surround channel SL', the right surround channel SR', and the center channel C and the subwoofer channel SW without sound field widening. The electronic device can then perform multi-channel data management on the 6 channels of data after the sound effect processing, for example, the electronic device can mix the 6 channels of data after the sound effect processing according to the respective channel positions to generate 8 channels of audio data corresponding to or mapped to the 8 speakers one by one. The electronic device can transmit the 8 channels of audio data to the corresponding speakers for playback through a hardware interface supporting multi-channel audio data, thereby realizing the three-dimensional sound field playback effect.

[0078] In some embodiments, each speaker in the electronic device has a corresponding power amplifier PA. Referring to Figure 7 , the 8 speakers each have a corresponding PA, i.e., PA1 to PA8. After mixing to generate 8 channels of audio data corresponding to or mapped to the 8 speakers one by one, the electronic device can transmit each channel of audio data to the corresponding PA through a hardware interface supporting multi-channel audio data, and input the audio data to the corresponding speaker for audio playback through the PA. In this scheme, the audio data assigned to the speaker is power amplified through the power amplifier PA and then input to the speaker for playback, which can improve the audio playback effect to some extent.

[0079] It should be understood that Figure 6 and Figure 7The hardware interface in the electronic device supports a time division multiplexing (TDM) data delivery mode. Therefore, the electronic device can deliver corresponding audio data to the loudspeakers through the hardware interface directly or indirectly via the PA in a time division multiplexing manner. The time division multiplexing manner can multiplex the hardware interface to deliver each channel of audio data to the corresponding loudspeaker in a very short time slice, so as to achieve the effect of playing audio in each loudspeaker at the same time or almost at the same time. The above scheme multiplexes one hardware interface, improves the resource utilization rate, and saves the communication resources.

[0080] In other embodiments, the hardware interface of the electronic device can also be improved, so that each channel of loudspeaker corresponds to one hardware interface, that is, the hardware interface corresponds to the loudspeaker one by one, so that the corresponding audio data can be transmitted to the loudspeaker based on the hardware interface corresponding to each loudspeaker. The embodiments of the present application do not limit the specific implementation of the hardware interface supporting multi-channel playback.

[0081] For example, the electronic device can be a tablet computer, a television (also referred to as a smart television, a smart screen, or a large screen device), a notebook computer, a personal computer (PC), an ultra-mobile personal computer (UMPC), a handheld computer, a wearable electronic device (for example, a smart watch, a smart bracelet, smart glasses), a vehicle-mounted device, a virtual reality device, and the like, which has an audio playback function and is provided with multiple loudspeakers. The embodiments of the present application do not specially limit the specific form of the electronic device.

[0082] In some embodiments, the loudspeakers of the electronic device can be 4, 6, 8, 12, or even more. The layout of the loudspeakers of the electronic device satisfies the following conditions:

[0083] The multiple loudspeakers in the electronic device are symmetrically distributed on both sides of the electronic device, that is, the number of loudspeakers on both sides is the same, and the loudspeakers on both sides are symmetrically distributed with respect to the axis center line of the electronic device. For example, Figure 7 As shown in FIG. 1, the 8 loudspeakers are symmetrically distributed on both sides of the electronic device, that is, there are 4 loudspeakers on each side, and the 4 loudspeakers on both sides are symmetrically distributed with respect to the axis center line of the electronic device. Figure 15A As shown in FIG. 2, the 4 loudspeakers are symmetrically distributed on both sides of the electronic device, that is, there are 2 loudspeakers on each side, and the loudspeakers on both sides are symmetrically distributed with respect to the axis center line of the electronic device.

[0084] For ease of understanding, the internal structure of the electronic device of the embodiments of the present application will be described below. Figure 8 The internal structure of the electronic device of the embodiments of the present application will be described below.

[0085] AsFigure 8 As shown, the electronic device 800 can include a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (USB) interface 830, a charge management module 840, a power management module 841, a battery 842, an antenna 1, an antenna 2, a mobile communication module 850, a wireless communication module 860, an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, a headset interface 870D, a sensor module 880, a key 190, a motor 891, an indicator 892, a camera 893, a display screen 894, and a subscriber identification module (SIM) card interface 895, etc. The sensor module 880 can include a pressure sensor, a gyroscope sensor, a barometric sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, etc.

[0086] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 800. In other embodiments of the present application, the electronic device 800 can include more or fewer components than shown, or combine certain components, or split certain components, or different arrangement of components. The components shown can be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 810 can include one or more processing units, for example: the processor 810 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units can be independent devices, or can be integrated in one or more processors.

[0088] The controller can be the nerve center and command center of the electronic device 800. The controller can generate operation control signals according to instruction operation codes and timing signals, and complete the control of fetching and executing instructions.

[0089] A memory can also be provided in the processor 810 for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory. The memory can hold instructions or data that the processor 810 has just used or has used before. If the processor 810 needs to use the instructions or data again, it can be called directly from the memory. This avoids repeated access and reduces the waiting time of the processor 810, thereby improving the efficiency of the system.

[0090] The modem processor can include a modulator and a demodulator. The modulator is configured to modulate a low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is configured to demodulate a received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to a loudspeaker 870A, a receiver 870B, etc.). In some embodiments, the modem processor can be a separate device. In other embodiments, the modem processor can be independent of the processor 810 and be arranged in the same device as the mobile communication module 850 or other functional modules.

[0091] The digital signal processor is configured to process digital signals, such as digital audio signals.

[0092] The external memory interface 820 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 800. The external memory card communicates with the processor 810 through the external memory interface 820 to implement data storage functions. For example, audio-related files such as music and videos are stored in the external memory card.

[0093] The internal memory 821 can be used to store computer executable program codes, which include instructions. The processor 810 executes various functional applications and data processing of the electronic device 800 by running the instructions stored in the internal memory 821. The internal memory 821 can include a program storage area and a data storage area. The program storage area can store an operating system, at least one application program required by a function (such as a sound playing function), etc. The data storage area can store data created during the use of the electronic device 800 (such as audio data, a phonebook, etc.). In addition, the internal memory 821 can include a high-speed random access memory, and can also include a non-volatile memory, such as at least one magnetic disk storage device, a flash memory device, a universal flash memory (UFS), etc.

[0094] The electronic device 800 can implement audio functions through an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, a headphone interface 870D, and an application processor, etc. For example, music playing, inputting voice instructions, recording, etc.

[0095] The audio module 870 is configured to convert a digital audio signal into an analog audio signal for output, and to convert an analog audio input into a digital audio signal. The audio module 870 can also be configured to encode and decode audio signals. In some embodiments, the audio module 870 can be disposed in the processor 810, or some functional modules of the audio module 870 can be disposed in the processor 810. For example, all or part of the functional modules of the audio module 870 can be disposed in a digital signal processor.

[0096] In some embodiments of the present application, the audio module can include a multi-channel data processing module, and thus, the multi-channel data processing module can be disposed in a digital signal processor, so that the digital signal processor can perform sound field widening, multi-channel data merging management, etc.

[0097] The speaker 870A, also known as a "loudspeaker", is configured to convert an audio electrical signal into an acoustic signal. The electronic device 800 can listen to music or listen to a hands-free call through the speaker 870A. In embodiments of the present application, the electronic device can include multiple speakers 870A, which are configured to implement a three-dimensional sound field playing effect and provide a three-dimensional sound field experience for a user.

[0098] The receiver 870B, also known as an "earpiece", is configured to convert an audio electrical signal into an acoustic signal. When the electronic device 800 answers a call or a voice message, the receiver 870B can be held close to a human ear to listen to the voice.

[0099] The microphone 870C, also known as a "microphone", "sound transducer", is configured to convert an acoustic signal into an electrical signal. When making a call or sending a voice message, a user can speak into the microphone 870C close to the human mouth to input an acoustic signal into the microphone 870C. The electronic device 800 can be provided with at least one microphone 870C. In other embodiments, the electronic device 800 can be provided with two microphones 870C, which are configured to not only collect acoustic signals, but also implement a noise reduction function. In other embodiments, the electronic device 800 can be provided with three, four or more microphones 870C, which are configured to collect acoustic signals, reduce noise, and identify a sound source, implement a directional recording function, etc.

[0100] The pressure sensor 880A is configured to sense a pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 880A can be disposed on the display screen 894. When a touch operation is performed on the display screen 894, the electronic device 800 detects the intensity of the touch operation according to the pressure sensor 880A. The electronic device 800 can also calculate the position of the touch according to the detection signal of the pressure sensor 880A.

[0101] In some embodiments of the present application, the electronic device 800 can further analyze the placement posture of the electronic device relative to the user according to the position of the touch detected by the pressure sensor 880A, and determine the current relative position of each speaker in the electronic device relative to the user.

[0102] The gyroscope sensor can be configured to determine the motion posture of the electronic device 800. The magnetic sensor includes a Hall sensor. The acceleration sensor can detect the acceleration of the electronic device 800 in each direction (generally three axes). When the electronic device 800 is stationary, the acceleration sensor can detect the size and direction of gravity. The acceleration sensor can also be used to identify the posture of the electronic device, and applied to screen switching, pedometer, and other applications.

[0103] In some embodiments of the present application, the electronic device 800 can analyze the placement posture of the electronic device relative to the user in real time according to at least one of the gyroscope sensor, the magnetic sensor, or the acceleration sensor, and determine the current relative position of each speaker in the electronic device relative to the user.

[0104] The touch sensor, also referred to as a “touch panel”. The touch sensor can be disposed on the display screen 894, and the touch sensor and the display screen 894 form a touch screen, also referred to as a “touch screen”. The touch sensor is configured to detect a touch operation performed on or near the touch sensor. In some embodiments, the touch sensor can also be disposed on the surface of the electronic device 800, which is different from the position of the display screen 894.

[0105] In some embodiments of the present application, the electronic device 800 can further analyze the placement posture of the electronic device relative to the user according to the touch position detected by the touch sensor, and determine the current relative position of each speaker in the electronic device relative to the user.

[0106] The methods in the embodiments of the present application can be implemented in the electronic device 800 with the hardware structure described above.

[0107] The software system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-kernel architecture, a micro-service architecture, or a cloud architecture. The embodiments of the present application do not make specific limitations in this regard. It can be understood that the software system of the layered architecture can divide the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces.

[0108] Please refer to Figure 9A , the software system includes, from top to bottom, an application layer, an application framework layer (Framework layer), and a hardware abstract layer (HAL) layer.

[0109] The application layer (Application) can include a series of application packages. The application layer can include multiple application packages. For example, the application layer can include an audio application (not shown in the figure), which can play audio data, such as for playing a multi-channel sound source. It should be understood that the audio application can be a music APP or a video APP or any other application that can play audio data. In some examples, the module for implementing the process of "multi-channel sound source playback" can be the audio application in the application layer or a partial functional module in the audio application, which can be denoted as a first module. Figure 9A The module for implementing the process of "multi-channel sound source playback" can be the audio application in the application layer or a partial functional module in the audio application, which can be denoted as a first module.

[0110] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the application programs in the application layer. The application framework layer includes some pre-defined functions and algorithms.

[0111] In some embodiments of the present application, the module for implementing the process of "multi-channel sound source playback" can transmit the channel data (i.e., audio data) of the sound source that needs to be output externally to the application framework layer. The application framework layer can include an algorithm library or algorithm package, which has a module (denoted as a second module) for implementing "playback link selection", i.e., for selecting a playback link that supports playing a multi-channel sound source.

[0112] The HAL layer is used to connect the application framework layer and the underlying layer.

[0113] In the audio playback scenario related to the embodiments of the present application, the HAL layer can create a playback path corresponding to the number of channels of the multi-channel sound source based on the ability of the multi-channel playback supported by the playback link selected by the application framework layer. It should be understood that the module in the HAL layer that creates the playback path can be denoted as a third module.

[0114] Further, after creating multiple playback paths, the HAL layer can pass the channel data transmitted by the application framework layer to a digital signal processor (DSP) in the underlying layer, and the DSP can perform sound field widening and multi-channel merging management on the channel data, thereby obtaining audio data corresponding to each speaker.

[0115] Each speaker is connected to a power amplifier (PA), which is used to transmit the audio data input by the digital signal processor (DSP) to the corresponding speaker for playback. It should be understood that Figure 9AThe eight speakers and eight power amplifiers are merely examples and do not limit the number of power amplifiers and speakers.

[0116] The method in the embodiments of the present application will be described in detail below in combination with the hardware structure and the software system structure.

[0117] 1. In response to a playing instruction for a multi-channel sound source, a playing path corresponding to the number of channels of the multi-channel sound source is created.

[0118] In some examples, the electronic device can include a first module in the application layer, a second module in the application framework layer, and a third module in the hardware abstraction layer, which cooperate together to create a playing path corresponding to the number of channels of the multi-channel sound source, which can be simply referred to as multi-module cooperative creation of a playing path. The first module is used for playing a sound source, the second module is used for selecting a playing link, and the third module is used for creating a playing path. The specific names of the modules are not limited. It should be understood that the first module, the second module, and the third module can also have other functions and are not limited to the above functions.

[0119] Now, the multi-module cooperative creation of a playing path will be introduced. Figure 9A As shown in FIG. 1, Figure 9A Each arrow in FIG. 1 represents a sound signal (i.e., a channel signal or audio data), and the multi-channel sound source includes 8-channel data. The module (i.e., the first module, the name of which is not shown in FIG. 1) in the application layer for implementing multi-channel sound source playing can respond to a playing instruction to play the multi-channel sound source and deliver the 8-channel data to the application framework layer. The module (i.e., the second module, the name of which is not shown in FIG. 1) in the application framework layer for implementing playing link selection can select a playing link supporting the playing of 8-channel data based on the delivered 8-channel data. The third module (not shown in the figure) in the hardware abstraction layer creates an 8-channel playing path based on the selected playing link. Figure 9A Figure 9A It should be noted that the above is only a brief introduction to the multi-module cooperative creation of a playing path. More specific processing of the multi-module cooperative creation of a playing path can be found in the following description of steps S101-S105 in FIG. 1. Figure 9A

[0120] It should be noted that the above is only a brief introduction to the multi-module cooperative creation of a playing path. More specific processing of the multi-module cooperative creation of a playing path can be found in the following description of steps S101-S105 in FIG. 1. Figure 9A Figure 10 2. The electronic device delivers each channel data corresponding to the multi-channel sound source based on the playing path.

[0121] It should be noted that the electronic device can deliver the original channel data of the multi-channel sound source to the lower layer, or can deliver the preprocessed data of the original channel data, which is not limited.

[0122] It should be noted that the electronic device can deliver the original channel data of the multi-channel sound source to the lower layer, or can deliver the preprocessed data of the original channel data, which is not limited. ​​​

[0123] Exemplarily, as shown in Figure 9A The hardware abstraction layer in the electronic device creates each playback channel and can deliver each channel data corresponding to the multi-channel sound source to the digital signal processor based on each playback channel to realize the transparent transmission of the multi-channel data. For example, Figure 9A In the example, the hardware abstraction layer can transparently transmit 8-channel data downward.

[0124] 3. The electronic device performs three-dimensional sound field widening processing on all or part of the channel data delivered based on the playback channel, and generates audio data corresponding to each speaker in the electronic device based on the multi-channel data after the sound field widening processing.

[0125] Exemplarily, as shown in Figure 9A The digital signal processor performs multi-channel sound effect processing and multi-channel data management on the channel data delivered by the playback channel to generate audio data corresponding to each speaker.

[0126] The multi-channel sound effect processing refers to processing for improving the sound externalization effect, which can specifically include sound field widening processing of multi-channel data. In some embodiments, in addition to the sound field widening processing, other sound effect processing can also be included, such as optimizing and fine-tuning the data of different frequency bands (such as low frequency, medium frequency and high frequency) in the channel data, which is not limited. The multi-channel data management refers to allocating the multi-channel data output by the module for implementing the multi-channel sound effect processing to the speakers to generate audio data corresponding to each speaker. For example, Figure 9A In the example, there are 8 speakers in total, and the module for implementing the multi-channel data management is used to generate 8-channel audio data, each channel of which corresponds to a speaker. It should be understood that the multi-channel data management can include mixing processing, and in addition, can also include channel data splitting processing (such as splitting the center channel data C into multiple sub-center channel data) or channel data discarding processing (such as discarding the subwoofer channel data SW), which is not limited.

[0127] In some examples, the multi-channel sound effect processing and the multi-channel data management can be performed by one of the modules in the digital signal processor, or by at least two modules. For example, Figure 9A In the example, the module for implementing the multi-channel sound effect processing and the module for implementing the multi-channel data management.

[0128] It should be understood that here only the digital signal processor is exemplified. In fact, other processing chips can also perform multi-channel sound effect processing and multi-channel data management on the channel data delivered by the playback channel to generate audio data corresponding to each speaker, which is not limited.

[0129] 4. The electronic device controls each speaker to play corresponding audio data through a hardware interface supporting multi-channel playback.

[0130] For example, each speaker is connected with a power amplifier, such as, Figure 9A In this embodiment, there are 8 speakers in total, so there are 8 PAs. The digital signal processor can transmit the audio data corresponding to each speaker to each power amplifier through the hardware interface supporting multi-channel playback, and input the audio data to each speaker for playing after processing by each power amplifier, thereby improving the audio playback effect.

[0131] In some embodiments, the PA not only plays a role of power amplification, but also can be used to adjust the working state of the speaker.

[0132] Specifically, the digital signal processor includes a module for implementing PA data processing (i.e., a Smart module for implementing PA data processing / an Intelligent module for implementing PA data processing), such as Figure 9B As shown in the figure. After the module for implementing multi-channel data management generates 8 pieces of audio data corresponding to the 8 speakers, the 8 pieces of audio data can be input to the module for implementing PA data processing. Through the IV feedback processing of the module for implementing PA data processing, the current and voltage of the speaker are detected in real time, and the parameters of the PA are adjusted in real time according to the current and voltage, so that the speaker always maintains a target working state. The target working state is the best working state, which is equivalent to the state of the speaker working at the maximum limit. It can be understood that in the target working state, the function of the speaker can be maximized or maximized, and the speaker will not be abnormal due to the maximum function.

[0133] It should be understood that, Figure 9B In this embodiment, only one module for implementing PA data processing is shown. In some examples, each PA can correspond to a module for implementing PA data processing, and this is not limited.

[0134] Next, the method of the present application will be further described in combination with the timing diagram and the software and hardware framework diagram.

[0135] In some embodiments, please refer to Figure 10 , a timing diagram of an audio playback method is provided, which is applied to the electronic device in the embodiments of the present application, and the electronic device includes at least two speakers. The method specifically includes the following steps:

[0136] S101, a first module in the application layer acquires a playing instruction of a multi-channel sound source.

[0137] Exemplarily, the playing instruction can be generated by a first module triggered by a user, such as a music application or a video application, or can be automatically generated by the first module in the electronic device, for example, in a scenario where a short video application automatically plays continuously, the playing instruction can be automatically generated.

[0138] In S102, the first module in the application layer delivers N-channel first channel data of the multi-channel sound source to the application framework layer.

[0139] The first channel data is the original channel data of the multi-channel sound source. The first channel number of the multi-channel sound source is N, and N is greater than or equal to 3, so the playing instruction indicates that the multi-channel sound source to be played includes N-channel data.

[0140] In S103, the second module in the application framework layer can select a playing link supporting the first channel number N.

[0141] In S104, the second module in the application framework layer notifies the hardware abstraction layer of the selected playing link and N-channel target channel data.

[0142] The above step S104 can include at least two parts of processing, that is, playing link selection processing and data delivery processing, which will be described in more detail.

[0143] I. Playing link selection processing:

[0144] The software system of the electronic device is pre-configured with a playing link. Exemplarily, the provider of the playing link pre-configures or encapsulates the playing link with audio playing capability based on some resource information or parameter information required for implementing audio playing. It should be understood that the playing link can provide relevant resource information or other support information required for implementing audio playing, such as memory buffer, supported sampling rate, bit width, etc., so the playing link has the capability to support audio playing. Different playing links have different capabilities to support audio playing. If some resource information or parameter information possessed by some playing links cannot support playing of a multi-channel sound source, it means that the playing link does not have the capability to support playing of the multi-channel sound source.

[0145] The capability of the playing link to support audio playing is reflected in the number of channels that the playing link can support to play to some extent. For example, some playing links only support playing of double-channel sound sources, and some playing links have stronger capability to support audio playing, which can support 6-channel, 8-channel, 12-channel or even more channel numbers of sound sources.

[0146] Therefore, after receiving the channel data of the multi-channel sound source delivered by the application layer, the channel number identification module in the application framework layer can parse the channel data in the multi-channel sound source to identify the channel number of the multi-channel sound source. Further, the second module in the application framework layer can select a playback link that matches the channel number, i.e., a playback link that supports multi-channel playback. In some examples, the channel number identification module can also be integrated in the second module, so that the second module also has the ability to identify the channel number.

[0147] In some examples, after identifying the channel number of the multi-channel sound source, the application framework layer can select a playback link that supports multi-channel playback in any of the following ways:

[0148] (1) From a plurality of playback links set in advance, select a playback link that matches the channel number.

[0149] For example, a plurality of playback links are set in advance in the electronic device, and the number of channels supported by the playback links can be different, for example, some playback links support playing 2-channel data, and some playback links support playing 6-channel data. The application framework layer can select a playback link that matches the channel number of the currently played multi-channel sound source from the plurality of playback links.

[0150] In some examples, a configuration file is set in the electronic device, and the configuration file records the correspondence between the channel number and the playback link identifier. The second module of the application framework layer can select a playback link identifier that matches the first channel number of the first sound source based on the configuration file.

[0151] Further, if there are multiple playback link identifiers that match the channel number of the currently played multi-channel sound source, one of the playback link identifiers can be selected at random or according to a preset rule.

[0152] (2) Select a default playback link.

[0153] For example, a default playback link can be specified in advance in the electronic device. The default playback link supports multi-channel playback. After confirming that the currently played sound source is a multi-channel sound source based on the identified channel number, the application framework layer can select the default playback link.

[0154] In some examples, the playback link that supports multi-channel playback can include a native playback link that supports multi-channel playback, and / or a custom playback link that supports multi-channel playback.

[0155] The native playback link refers to a playback link that is self-provided by the software system of the electronic device, such as direct or offload, and the like. It should be understood that the scheme of the present application is to innovatively apply the native playback link supporting multi-channel playback to the scenario of three-dimensional sound field playback of a multi-channel sound source. In the scheme of the present application, the docking and calling of the native playback link are implemented in the application framework layer and the hardware abstraction layer, which together form the audio externalization scheme for implementing the three-dimensional sound field playback effect in the embodiment of the present application, and is the result of creative labor.

[0156] The custom playback link is a playback link customized by the manufacturer of the electronic device. Specifically, the manufacturer of the electronic device pre-disposes or encapsulates a playback link supporting audio playback based on some resource information or parameter information required for implementing audio playback, and the playback link is recorded as a custom playback link.

[0157] It should be noted that, at the data processing level, the second module in the application framework layer selects a playback link identifier. The application framework layer transmits the playback link identifier to the hardware abstraction layer to achieve the purpose of notifying the selected playback link.

[0158] II. Data delivery processing

[0159] The target channel data delivered by the application framework layer to the hardware abstraction layer can be the original first channel data in the multi-channel sound source or the second channel data pre-processed from the first channel data. The target channel data is also N-way.

[0160] In some examples, the pre-processing can include volume processing. For example, the second module in the application framework layer can increase the volume of each channel data through volume processing, so that the subsequent processing of the lower layer is based on the channel data after volume increase. In this way, the sound loudness of the final output of the loudspeaker can be improved to some extent, and the sound externalization effect can be optimized. For details, please refer to Figure 11 As shown in Figure 11 , the module in the application framework layer can perform playback link selection and volume processing, and deliver the channel data after volume processing (second channel data) to the hardware abstraction layer.

[0161] In other examples, the pre-processing can not be limited to volume processing, but can also include other pre-processing such as noise reduction, and the like, which is not limited.

[0162] S105, the third module in the hardware abstraction layer can create N playback paths based on the selected playback link.

[0163] It should be understood that the playback link has the capability of supporting multi-channel playback. Therefore, after the application framework layer informs the hardware abstraction layer of the selected playback link, the third module in the hardware abstraction layer can invoke the playback link interface corresponding to the playback link to create a playback path matching the first number of channels using the resource data or parameters supported by the playback link.

[0164] For example, the hardware abstraction layer can allocate a memory buffer supporting the transmission of 8-channel data, determine a sampling rate and bit width supporting the transmission of 8-channel data, and the like, to achieve the purpose of creating an 8-channel playback path with the support of the playback link. Figure 9A

[0165] S106, the third module in the hardware abstraction layer issues N pieces of target channel data to the multi-channel data processing module in the digital signal processor based on the N playback paths.

[0166] The multi-channel data processing module is a module that implements multi-channel sound effect processing and multi-channel data management. It should be understood that the naming is not limited to the multi-channel data processing module, as long as it can implement multi-channel sound effect processing and multi-channel data management, it belongs to the multi-channel data processing module mentioned in the embodiments of the present application.

[0167] S107, the multi-channel data processing module in the digital signal processor performs three-dimensional sound field widening on at least part of the target channel data to obtain N pieces of third channel data.

[0168] It should be understood that, through the adaptive creation of the playback path in the embodiments of the present application, the number of channels of the target channel data received by the digital signal processor is the same as the first number of channels N of the multi-channel sound source. For example, the multi-channel sound source has 8-channel data, and then the target channel data received is also 8-channel. Further, the digital signal processor can perform three-dimensional sound field widening processing on all or part of the target channel data to obtain N pieces of third channel data.

[0169] As can be seen from the above, it is not limited to performing sound field widening on each piece of target channel data. That is, only part of the target channel data under the N pieces of target channel data needs to be three-dimensional sound field widened, and the target channel data under other channels does not need to be three-dimensional sound field widened. Therefore, the channel data after three-dimensional sound field widening and the target channel data without sound field widening can be taken together as third channel data, that is, N pieces of third channel data are obtained. Subsequently, M pieces of audio data corresponding to M loudspeakers are generated based on the N pieces of third channel data.

[0170] ​It should be noted that the sound field width of each third sound channel data is not less than the sound field width of the corresponding target sound channel data, and the sound field width of at least one third sound channel data is greater than the sound field width of the corresponding target sound channel data. That is, if the target sound channel data is subjected to sound field widening, the third sound channel data after the sound field widening is necessarily greater than the target sound channel data before the sound field widening. If the target sound channel data is not subjected to sound field widening, and it is taken as one third sound channel data, the sound field width is unchanged.

[0171] For example, the hardware abstraction layer downwardly transmits 8 target sound channel data (i.e., N=8). It is assumed that 2 of the 8 target sound channel data do not need to be subjected to sound field widening, and are denoted as target sound channel data 1 and target sound channel data 2, and the other 6 target sound channel data need to be subjected to sound field widening. Then, after the processing of S107, 8 third sound channel data can be obtained, which can include 6 sound channel data after the sound field widening and the target sound channel data 1 and target sound channel data 2 which are not subjected to sound field widening. Figure 9A

[0172] In addition, the foregoing 5.1 sound channel source is taken as an example for further description. It is assumed that the sound channel data of the front left sound channel FL, the front right sound channel FR, the left surround sound channel SL and the right surround sound channel SR are subjected to sound field widening, and the sound channel data of the center sound channel C and the subwoofer sound channel SW do not need to be subjected to sound field widening. Then, the 6 third sound channel data include the front left sound channel FL', the front right sound channel FR', the left surround sound channel SL', the right surround sound channel SR', and the center sound channel C and the subwoofer sound channel SW which are not subjected to sound field widening.

[0173] In some examples, the digital signal processor can determine the target sound channel data which needs to be subjected to sound field widening and the target sound channel data which does not need to be subjected to sound field widening based on the sound channel positions corresponding to the target sound channel data. For example, the target sound channel data under the center sound channel or the subwoofer sound channel does not need to be subjected to sound field widening. The target sound channel data under the left and right front sound channels, the left and right surround sound channels, the top sound channel and the like need to be subjected to sound field widening. In other embodiments, the module for implementing the multi-channel sound effect processing can also subject the target sound channel data under all or a part of the sound channel positions to sound field widening according to a default rule, and this is not limited.

[0174] It should be noted that in other examples, all the target sound channel data can be subjected to three-dimensional sound field widening, and this is not limited.

[0175] The specific processing of how to perform the three-dimensional sound field widening will be described in detail in the following Figure 12 Step S126.

[0176] ​S108, the multi-channel data processing module in the digital signal processor generates M pieces of audio data corresponding to the M loudspeakers in the electronic device based on the N pieces of third channel data.

[0177] In some examples, the digital signal processor can adopt a position matching principle to generate the audio data corresponding to each loudspeaker based on the mixing of the N pieces of third channel data. Specifically, the digital signal processor generates the audio data of the loudspeaker based on the third channel data whose channel position matches the position of the loudspeaker. The following will be described in detail in the step S128 in the method shown in Figure 12

[0178] In other examples, the channel system and the channels thereof known in the industry are relatively clear, so that the position identification and matching can be omitted, and a binding relationship between the channels and the loudspeakers can be set in advance. Based on the binding relationship, the third channel data corresponding to each loudspeaker is determined from the N pieces of third channel data, and the audio data corresponding to each loudspeaker is generated based on the third channel data corresponding to the loudspeaker. Further, the binding relationship can be a dynamic binding relationship, which can be dynamically updated according to the relative position between the loudspeaker and the user.

[0179] S109, the multi-channel data processing module in the digital signal processor delivers the M pieces of audio data based on the hardware interface supporting multi-channel playback.

[0180] S1010, the M loudspeakers play the corresponding audio data.

[0181] In the above scheme, the application layer, the application framework layer, the hardware abstraction layer, and the digital signal processor are all improved accordingly. Through the cooperation of the application layer, the application framework layer, the hardware abstraction layer, and the digital signal processor, the playback effect of the three-dimensional sound field is achieved without the need for additional special cinema-level hardware devices.

[0182] In some embodiments, please refer to Figure 12 , another timing diagram of an audio external playing method is provided, which is also applied to the electronic device in the embodiments of the present application, and the electronic device includes more than two loudspeakers. The method specifically includes the following steps:

[0183] S120, the first module in the application layer acquires a playing instruction of a multi-channel sound source.

[0184] S121, the first module in the application layer delivers N pieces of first channel data of the multi-channel sound source to the application framework layer.

[0185] ​S122, the second module in the application framework layer can select a playback link supporting the first number N of sound channels to support multi-channel playback, and perform volume processing on the N first sound channel data to obtain N second sound channel data.

[0186] Among them, the second sound channel data is greater than the first sound channel data before volume processing.

[0187] S123, the second module in the application framework layer notifies the hardware abstraction layer of the selected playback link and the N second sound channel data.

[0188] S124, the third module in the hardware abstraction layer creates N playback paths based on the selected playback link.

[0189] S125, the third module issues the N second sound channel data to the module for implementing multi-channel sound effect processing in the digital signal processor based on the N playback paths.

[0190] In this embodiment, the digital signal processor includes a module for implementing multi-channel sound effect processing and a module for implementing multi-channel data management, i.e. the module for implementing multi-channel sound effect processing and the module for implementing multi-channel data management can be two sub-modules of the multi-channel data processing module in the hardware abstraction layer. The hardware abstraction layer issues the N second sound channel data to the module for implementing multi-channel sound effect processing based on the N playback paths. Figure 10

[0191] S126, the module for implementing multi-channel sound effect processing performs sound field widening on at least part of the second sound channel data based on the sound channel orientation to obtain N third sound channel data.

[0192] As can be seen from the above, it is not limited to performing sound field widening on each second sound channel data. The sound channel data after sound field widening from the second sound channel data and the second sound channel data without sound field widening can be taken together as third sound channel data, i.e. N third sound channel data is obtained. It should be understood that in other examples, all second sound channel data can also be three-dimensionally sound field widened, which is not limited.

[0193] For example, the module for implementing multi-channel sound effect processing can determine the second sound channel data that needs to be sound field widened and the second sound channel data that does not need to be sound field widened based on the sound channel orientation corresponding to the second sound channel data. For example, the second sound channel data under the center channel or subwoofer channel does not need to be sound field widened. The second sound channel data under the left and right front channels, left and right surround channels, top channels, etc. needs to be sound field widened. In other embodiments, the module for implementing multi-channel sound effect processing can also perform sound field widening on all or part of the second sound channel data under the specified sound channel orientation according to the default rule, which is not limited.

[0194] ​For the second channel data requiring sound field widening, the digital signal processor can perform horizontal sound field widening and / or vertical sound field widening according to the channel direction of the second channel data, to obtain third channel data corresponding to the second channel data.

[0195] The horizontal sound field widening refers to sound field widening in the horizontal direction. The third channel data after horizontal sound field widening is farther away from the electronic device in the horizontal direction compared with the corresponding second channel data. The vertical sound field widening refers to sound field widening in the vertical direction. The third channel data after vertical sound field widening is farther away from the electronic device in the vertical direction compared with the corresponding second channel data. It should be understood that the horizontal direction and the vertical direction in the embodiments of the present application can be determined with reference to the world coordinate system.

[0196] For the convenience of understanding, taking third channel data requiring sound field widening including left channel data and right channel data, and sky channel data as an example, the difference before and after sound field widening is shown in the following Figure 13 Figure 13 The dashed line in the figure represents the output position of the speaker (i.e. the output position of the sound signal) of the channel data. The output position of the left channel data before sound field widening is S1, and the output position after sound field widening is S11. The position S11 is farther away from the electronic device in the horizontal direction compared with the position S1, i.e. it becomes more left. The output position of the right channel data before sound field widening is S2, and the output position after sound field widening is S22. The position S22 is farther away from the electronic device in the horizontal direction compared with the position S2, i.e. it becomes more right. Thus, the sound wrapping effect on the left and right sides of the user is stronger, and the spatial sense of the sound on the left and right sides of the user is enhanced. The output position of the sky channel data before sound field widening is S3, and the output position after sound field widening is S33. The position S33 becomes more up compared with the position S3, thus it is closer to the effect of sound coming from above the user's head.

[0197] In some examples, the sound field widening in different directions such as horizontal sound field and / or vertical sound field can be realized through a head transfer function corresponding to each sound field widening direction. For example, the head transfer function matrix can be convolved with each channel data to realize sound field rendering in different directions, so as to realize the purpose of three-dimensional sound field widening.

[0198] S127, the module for realizing multi-channel sound effect processing delivers N pieces of third channel data to the module for realizing multi-channel data management in the digital signal processor.

[0199] S128, for each speaker, the module for realizing multi-channel data management selects third channel data with matching direction, and generates audio data corresponding to the speaker based on the third channel data with matching direction.

[0200] ​It should be noted that before introducing how to generate the audio data for each speaker, the following will first introduce the channel orientation and speaker orientation:

[0201] As mentioned above, in the embodiments of this application, channel orientation refers to the desired sound direction and position when playing channel data. For example, the channel orientation of the left channel data is intended to present the effect of sound playing on the left side of the user's ear, and the channel orientation of the right channel data is intended to present the effect of sound playing on the right side of the user's ear.

[0202] The speaker's orientation refers to its current relative position to the user, or, more specifically, its relative position to the user given the current orientation of the electronic device relative to the user. It should be understood that speakers are generally fixed within electronic devices, and the relative orientation of the electronic device to the user will change regardless of the device's orientation.

[0203] The following example uses an electronic device with four speakers, combined with... Figure 14 This illustrates different placement positions and the relative positions of the speaker and the user in each placement position.

[0204] Please see Figure 14 In (a), the electronic device is positioned in its first orientation from the user's perspective. The camera on the electronic device is facing upwards, meaning that from the user's viewpoint, the electronic device is currently positioned in front with its camera facing upwards.

[0205] Please see Figure 14 (b) shows the second orientation of the electronic device from the user's position as the observation point. In this orientation, the camera of the electronic device is facing downwards, meaning that from the user's perspective, the electronic device is currently positioned in front with the camera facing downwards.

[0206] like Figure 14 As shown in (a), with the user's position as the observation point, in the first placement posture, speaker SPK1 is located at the upper left corner of the electronic device, speaker SPK2 is located at the upper right corner of the electronic device, speaker SPK3 is located at the lower left corner of the electronic device, and speaker SPK4 is located at the lower right corner of the electronic device.

[0207] like Figure 14 As shown in (b), with the user's position as the observation point, in the second placement posture, the orientation of each speaker relative to the user (i.e., the relative orientation with respect to the user) compared to... Figure 14 (a) has changed. Specifically, in the second orientation, speaker SPK1 is located at the lower right corner of the electronic device, speaker SPK2 is located at the lower left corner of the electronic device, speaker SPK3 is located at the upper right corner of the electronic device, and speaker SPK4 is located at the upper left corner of the electronic device.

[0208] The following describes in detail how to generate the audio data corresponding to each speaker, specifically including the following steps (1) to (2):

[0209] (1) For each speaker, select the third channel data whose channel orientation matches the orientation of the speaker.

[0210] Specifically, for each speaker, the module for multi-channel data management can select third-channel data from the third-channel data of the first number of channels. The selected third-channel data must have a channel orientation that matches the speaker's orientation or meets a preset orientation proximity condition. The selected third-channel data is then matched to the speaker's orientation. Subsequently, the module for multi-channel data management can generate audio data corresponding to that speaker based on the matched third-channel data.

[0211] The following combination Figure 14 Example of the speaker's position relative to the user in (a) is shown below.

[0212] In the first placement position, the speaker SPK1 is located in the upper left corner of the electronic device. Therefore, the channel data of the upper left channels, such as the front left channel, the top front left channel, and the top rear left channel, are matched with the position of SPK1 in the upper left corner.

[0213] Since the speaker SPK2 is located in the upper right corner of the electronic device, the channel data of the upper right channels, such as the front right channel, the top front right channel, and the top rear right channel, are matched with the position of SPK2 in the upper right corner.

[0214] Since the speaker SPK3 is located in the lower left corner of the electronic device, the channel data for the left surround channel, the left rear surround channel, and other channels located on the lower left or rear left side are matched with the position of the SPK3 in the lower left corner.

[0215] Since the speaker SPK4 is located in the lower right corner of the electronic device, the channel data for the right surround channel, the right rear surround channel, and other channels located on the lower or rear right side are matched with the position of the SPK4 in the lower right corner.

[0216] (2) For each speaker, mix the audio based on the matching third channel data to generate the audio data corresponding to that speaker.

[0217] Specifically, for each speaker, after selecting the third channel data matching the orientation of the speaker, the third channel data can be mixed, such as up-mixed or down-mixed, to generate the audio data corresponding to the speaker. It should be noted that the up-mixing or down-mixing processing refers to the processing in the case where there is not a unique third channel data matching each speaker. If there is a unique third channel data matching each speaker, no mixing is needed, and the unique matching third channel data can be directly controlled to be played by each speaker.

[0218] Next, the up-mixing and down-mixing processing will be described in conjunction with Examples 1 and 2.

[0219] Example 1, more than one third channel data matching the speaker, down-mixing.

[0220] Down-mixing refers to reducing the number of channels, i.e., mixing channel data with a larger number of channels into channel data with a smaller number of channels. For example, mixing 6-channel data into 4-channel data is down-mixing.

[0221] Specifically, for any speaker, if there is more than one third channel data matching the speaker (i.e., multiple third channel data), the multiple third channel data matching the speaker need to be combined into one by down-mixing to obtain the audio data corresponding to the speaker.

[0222] Example 2, some speakers do not have matching third channel data, up-mixing.

[0223] Up-mixing refers to increasing the number of channels, i.e., mixing channel data with a smaller number of channels into channel data with a larger number of channels. For example, mixing 6-channel data into 8-channel data is up-mixing.

[0224] Specifically, for some speakers, there can be a lack of third channel data matching the orientation of the speaker. Therefore, for these speakers lacking matching third channel data, the digital signal processor can generate channel data with a channel orientation matching the orientation of the speaker based on the existing third channel data, and determine the audio data corresponding to the speaker based on the newly generated channel data. For example, the newly generated channel data can be directly determined as the audio data corresponding to the speaker, or the newly generated channel data can be combined with other third channel data (such as center channel data) to generate the audio data corresponding to the speaker.

[0225] Similarly, the audio data corresponding to the speaker is not necessarily generated based only on the third channel data matching the position of the speaker, but the third channel data matching the position of the speaker can also be combined with other third channel data (such as center channel data) to generate the audio data corresponding to the speaker.

[0226] S129, the module for implementing multi-channel data management distributes corresponding audio data to the M speakers based on a hardware interface supporting multi-channel playback.

[0227] S1210, the M speakers play corresponding audio data.

[0228] In the above scheme, the application layer, the application framework layer, the hardware abstraction layer, and the digital signal processor are all improved accordingly. Through the cooperation of the application layer, the application framework layer, the hardware abstraction layer, and the digital signal processor, the three-dimensional sound field playback effect is achieved without the need for additional specialized cinema-level hardware devices. In addition, through the interaction between the above software and hardware modules, more specific processing such as volume processing, sound field widening based on sound field position, and sound mixing based on position matching principle to allocate channel data to speakers is achieved, which can improve the accuracy and spatial sense of the audio data generated by each speaker to a certain extent, and achieve a good three-dimensional sound field playback effect.

[0229] In some embodiments, the multi-channel sound source includes center channel data. Then, through the adaptive creation of the playback channel in each of the above embodiments without data loss, the third channel data of the first channel quantity obtained also includes center channel data and non-center channel data. The non-center channel data refers to the third channel data in the third channel data of the first channel quantity except for the center channel data. As described above, the module for implementing multi-channel data management can mix the center channel data when generating audio data corresponding to the speaker based on the third channel data. Next, how to mix the third channel data matching the speaker with the center channel data to generate the audio data corresponding to the speaker will be described in the following. Figures 15A-15C

[0230] Specifically, after obtaining the first channel quantity of third channel data after sound effect processing, the electronic device (such as the digital signal processor in the electronic device) can split the center channel data into at least two sub-center channel data. Then, the split sub-center channel data is distributed to the speakers. It should be understood that the electronic device can split the center channel data to all speakers or only to part of the speakers.

[0231] ​For the speaker to which the sub-center channel data is allocated, the electronic device can combine the non-center channel data and the sub-center channel data allocated to the speaker, and generate the audio data corresponding to the speaker by mixing. For example, the allocated non-center channel data and sub-center channel data can be combined by weighting according to the corresponding weights to obtain the audio data corresponding to the speaker. In other examples, the allocated non-center channel data and sub-center channel data can be directly combined without considering the weights. This is not limited.

[0232] In addition, for the speaker to which the sub-center channel data is not allocated, the electronic device can combine the non-center channel data allocated to the speaker by weighting or directly, and generate the audio data corresponding to the speaker by mixing. As known from the above, the non-center channel data allocated to the speaker refers to the non-center channel data whose channel orientation matches the orientation of the speaker.

[0233] Next, how to mix will be described in more detail in combination with two different splitting modes of splitting the center channel data to all speakers or only to part of the speakers.

[0234] Splitting mode 1: splitting the center channel data to all speakers.

[0235] Specifically, the electronic device can split the center channel data evenly to each speaker, that is, the volume of the sub-center channel data allocated to each speaker is the same. The electronic device can also not split evenly, but allocate sub-center channel data with larger volume to part of the speakers, and allocate sub-center channel data with smaller volume to another part of the speakers. For example, in combination with (a) in Figure 14 , the speaker SKP1 at the upper left corner and the speaker SKP2 at the upper right corner can be allocated sub-center channel data with larger volume, and the speaker SKP3 at the lower left corner and the speaker SKP4 at the lower right corner can be allocated sub-center channel data with smaller volume, so as to form the effect of the center sound being emitted from the position above the center of the electronic device.

[0236] Please refer to Figure 15A , taking an example of the electronic device having 4 speakers and the multi-channel sound source being a 7.1.4 channel sound source, to illustrate the case of mixing by using splitting mode 1:

[0237] It should be understood that Figures 15A-15C and Figure 16 focus on describing how to mix to generate the audio data corresponding to each speaker, and the scheme in the embodiments of the present application is to transparently transmit by adaptively creating a playback path matching the number of channels of the multi-channel sound source, so the structure of the application framework layer and the HAL layer is not shown.

[0238] As Figure 15AAs shown, the 7.1.4 sound source includes 12 channels of sound data. It should be noted that each channel of sound data of the multi-channel sound source can be pre-processed (such as volume processing) before being sent to the digital signal processor, or can be sent without pre-processing. For brevity of expression, Figure 15A In the present embodiment, the 12 channels of sound data are represented as "front left channel data FL, front right channel data FR, center channel data C, left surround channel data SL, right surround channel data SR, left back surround channel data LB, right back surround channel data RB, subwoofer channel data SW, top front left channel data TFL, top front right channel data TFR, top back left channel data TRL, and top back right channel data TRR".

[0239] In fact, if each channel of sound data of the multi-channel sound source is pre-processed before being sent to the digital signal processor, then Figure 15A "FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR" in the present embodiment represent the pre-processed sound data (i.e., second sound data). If not pre-processed, but each channel of sound data of the multi-channel sound source is sent directly to the digital signal processor, then Figure 15A "FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR" in the present embodiment represent the original sound data of the multi-channel sound source (i.e., first sound data). It should be understood that Figure 15B Figure 15C and Figure 16 The same applies to the following, which will not be described again.

[0240] As shown in FIG. 1, Figure 15A The 4 speakers include SPK1 to SPK4. In the placement posture of the electronic device shown in FIG. 1, Figure 15A SPK1 is located at the upper left corner of the electronic device, SPK2 is located at the upper right corner of the electronic device, SPK3 is located at the lower left corner of the electronic device, and SPK4 is located at the lower right corner of the electronic device. The modules for implementing multi-channel data management include data management modules corresponding to the 4 speakers respectively, i.e., a module for implementing SPK1 data management to a module for implementing SPK4 data management, each data management module being responsible for generating audio data corresponding to one speaker. Each speaker is connected to a power amplifier PA, so there are PA1 to PA4.

[0241] ​In the multi-channel sound effect processing stage, the FR and FL can be input to the front channel processing module, the SL, SR, LB and RB can be input to the rear channel processing module, and the TFL, TFR, TRL and TRR can be input to the sky channel processing module for sound effect processing or sound field expansion to obtain FR', FL', SL', SR', LB', RB', TFL', TFR', TRL' and TRR'.

[0242] In the multi-channel data management stage, since there is no separate subwoofer speaker in the electronic device, the SW is input to the low frequency channel processing module for discarding. The module for implementing multi-channel data management can input C to the middle channel processing module for processing to split C into multiple sub-middle channel data, for example, into 4 sub-middle channel data to obtain C1 to C4. The module for implementing multi-channel data management can respectively assign the 4 sub-middle channel data to 4 speakers. For example, C1 is assigned to SPK1, C2 is assigned to SPK2, C3 is assigned to SPK3, and C4 is assigned to SPK4. It should be understood that the 4 sub-middle channel data C1 to C4 can be equal (i.e., evenly divided for the middle channel data C) or not equal (i.e., unevenly divided for the middle channel data C), which is not limited.

[0243] For SPK1, since the position of SPK1 is located at the upper left corner of the electronic device, which is relatively located at the upper left of the user, in addition to assigning the sub-middle channel data C1, the module for implementing SPK1 data management can also assign channel data close to the upper left or the upper left corner (such as FL', TFL' and TRL') to the module for implementing SPK1 data management. Further, the module for implementing SPK1 data management can combine FL', TFL', TRL' and C1 to generate audio data corresponding to the speaker SPK1. The module for implementing SPK1 data management can input the audio data corresponding to the speaker SPK1 to PA1 to control the speaker SPK1 to play the audio data through PA1.

[0244] Similarly, for SPK2, since the position of SPK2 is located at the upper right corner of the electronic device, which is relatively located at the upper right of the user, in addition to assigning the sub-middle channel data C2, the module for implementing SPK1 data management can also assign channel data close to the upper right or the upper right corner (such as FR', TFR' and TRR') to the module for implementing SPK1 data management, and further, FR', TFR', TRR' and C2 can be combined to generate audio data corresponding to the speaker SPK2. The module for implementing SPK2 data management can input the audio data corresponding to the speaker SPK2 to PA2 to control the speaker SPK2 to play the audio data through PA2.

[0245] For SPK3, since the orientation of SPK3 is at the lower left corner of the electronic device, which is relatively at the lower left of the user, in addition to assigning the sub-center channel data C3, the channel data with an orientation close to the lower left or the lower left corner (such as SL' and LB') can also be assigned to the module for implementing SPK3 data management, and then C3, SL' and LB' can be combined to generate audio data corresponding to the speaker SPK3. The module for implementing SPK3 data management can input the audio data corresponding to the speaker SPK3 to PA3 to control the speaker SPK3 to play the audio data through PA3.

[0246] For SPK4, since the orientation of SPK4 is at the lower right corner of the electronic device, which is relatively at the lower right of the user, in addition to assigning the sub-center channel data C4, the channel data with an orientation close to the lower right or the lower right corner (such as SR' and RB') can also be assigned to the module for implementing SPK1 data management, and then C4, SR' and RB' can be combined to generate audio data corresponding to the speaker SPK4. The module for implementing SPK4 data management can input the audio data corresponding to the speaker SPK4 to PA4 to control the speaker SPK4 to play the audio data through PA4.

[0247] In some examples, as described above, the digital signal processor can also include a module for implementing PA data processing, and each PA can correspond to a module for implementing PA data processing, respectively. For details, please refer to Figure 15B , 4 speakers are connected to 1 PA, and each PA can correspond to a module for implementing PA data processing, respectively, that is, there are PA1 data processing module to PA4 data processing module. As Figure 15B indicated, the module for implementing SPK data management corresponding to each speaker can input the corresponding 4-way audio data to the corresponding module for implementing PA data processing for PA control, and then input to each PA for transmission to the speaker through PA. Taking SPK1 as an example, after the module for implementing SPK1 data management generates one-way audio data corresponding to SPK1, the module for implementing SPK1 data management can input the generated one-way audio data to the PA1 data processing module. After the PA1 data processing module adjusts the parameters of PA1 based on the IV feedback mechanism, the audio data is input to the speaker SPK1 for playing after power amplification processing by PA1.

[0248] Taking the weighted combination of the channel data assigned to the speaker as an example, in combination with Figure 15A or Figure 15B , the channel data assignment and combination are as follows:

[0249] SPK1: FL' + 0.707 * CI + 0.707 * TFL' + 0.707 * TRL'

[0250] SPK2: FR' + 0.707 * C2 + 0.707 * TFR' + 0.707 * TRR'

[0251] SPK3: SL' + BL' + 0.707 * C3

[0252] SPK4: SR' + BR' + 0.707 * C4

[0253] Wherein, 0.707 is the weight, it should be noted that, here, only 0.707 is taken as an example to illustrate the weight, and the weight value should not be limited.

[0254] Splitting mode 2: split the middle channel data to part of the speakers.

[0255] It should be understood that, in the current placement posture of the electronic device relative to the user, the speakers allocated to the sub-middle channel data are located on the left and right sides of the electronic device, so as to form the effect that the sound of the middle channel is located in the middle position of the electronic device, such as the effect of the vocal middle. Specifically, the part of the speakers allocated to the sub-middle channel data can be the speakers with relatively high positions (i.e. the first type of speakers) in the current placement posture of the electronic device relative to the user. It should be understood that the speakers with relatively low positions belong to the second type of speakers.

[0256] In combination with Figure 15B For example, the positions of the upper left and upper right speakers SPK1 and SPK2 are higher than those of the lower left and lower right speakers SPK3 and SPK4, so the electronic device can split the middle channel data into two sub-middle channel data, and then allocate each sub-middle channel data to the upper left speaker SPK1 and the upper right speaker SPK2 respectively, so as to form the effect that the middle sound is emitted from the position above the center of the electronic device. It should be understood that the electronic device can split the middle channel data equally or unequally to part of the speakers, which is not limited.

[0257] For ease of understanding, the electronic device has four speakers, and the sound source of 7.1.4 channels is taken as an example for illustrative description.

[0258] Please refer to Figure 15C , Figure 15C The placement posture of the electronic device relative to the user is consistent with the above Figure 15B , therefore, the positions of the speakers are also consistent with the above Figure 15B , which will not be described again.

[0259] In the multi-channel sound effect processing stage, the processing of SW, FR, FL, SL, SR, LB, RB, TFL, TFR, TRL and TRR is also the same as the above processing, thus, after the sound effect processing or sound field widening, FR', FL', SL', SR', LB', RB', TFL', TFR', TRL' and TRR' can be obtained. Figure 15B

[0260] In the multi-channel data management stage, the difference from the above is that, Figure 15B Figure 15C The C is input to the middle channel processing module to split into 2 sub-middle channel data, denoted as C5 and C6. That is, only the sub-middle channel data is allocated to the two loudspeakers SPK1 and SPK2 located above. For example, only C5 is allocated to SPK1, and C6 is allocated to SPK2, and no sub-middle channel data is allocated to SPK3 and SPK4. It should be understood that the volumes of the 2 sub-middle channel data C5 and C6 can be equal or unequal.

[0261] For SPK1, in addition to the allocation of sub-middle channel data C5, channel data with a channel orientation close to the upper left or upper left corner (such as FL', TFL' and TRL') can also be allocated to the module for implementing SPK1 data management, and then FL', TFL', TRL' and C5 can be combined to generate audio data corresponding to loudspeaker SPK1. The module for implementing SPK1 data management can input the audio data corresponding to loudspeaker SPK1 to PA1 to control loudspeaker SPK1 to play the audio data through PA1.

[0262] For SPK2, in addition to the allocation of sub-middle channel data C6, channel data with a channel orientation close to the upper right or upper right corner (such as FR', TFR' and TRR') can also be allocated to the module for implementing SPK1 data management, and then FR', TFR', TRR' and C6 can be combined to generate audio data corresponding to loudspeaker SPK2. The module for implementing SPK2 data management can input the audio data corresponding to loudspeaker SPK2 to PA2 to control loudspeaker SPK2 to play the audio data through PA2.

[0263] For SPK3, only channel data with a channel orientation close to the lower left or lower left corner (such as SL' and LB') needs to be allocated to the module for implementing SPK3 data management. Then, SL' and LB' are combined to generate audio data corresponding to loudspeaker SPK3. The module for implementing SPK3 data management can input the audio data corresponding to loudspeaker SPK3 to PA3 to control loudspeaker SPK3 to play the audio data through PA3.

[0264] ​​For SPK4, only the channel data with the channel position close to the lower right or the lower right corner (e.g., SR' and RB') need to be assigned to the module for implementing the SPK1 data management. Then, SR' and RB' are combined to generate the audio data corresponding to the speaker SPK4. The module for implementing the SPK4 data management can input the audio data corresponding to the speaker SPK4 to PA4 to control the speaker SPK4 to play the audio data through PA4.

[0265] Similarly, taking the example of the weighted combination of the channel data assigned to the speakers, in combination with Figure 15C , the channel data assignment and combination are as follows:

[0266] SPK1: FL' + 0.707 * C5 + 0.707 * TFL' + 0.707 * TRL'

[0267] SPK2: FR' + 0.707 * C6 + 0.707 * TFR' + 0.707 * TRR'

[0268] SPK3: SL' + BL'

[0269] SPK4: SR' + BR'

[0270] As described above, 0.707 is the weight.

[0271] The inventors of the present application have found through careful research that the electronic device is usually rotated by the user during use, so the placement posture of the electronic device relative to the user changes constantly, thereby causing the position of the speaker relative to the user to change. If the audio data input into each speaker is determined based only on the placement posture of the electronic device relative to the user at the beginning of playing the sound source, the audio data played by each speaker when the electronic device is rotated will not be accurate enough.

[0272] The problem caused by the change in the placement posture of the electronic device will be described in combination with Figure 15B and Figure 16

[0273] From the foregoing, Figure 15B , the electronic device is in a first placement posture relative to the user. Figure 16 , the first placement posture is based on Figure 15B , the second placement posture is obtained by rotating the electronic device 180° based on the first placement posture. From Figure 15B , it can be seen that the 4-channel data under the sky channel are input to the speakers SPK1 and SPK2. Assuming that the 4-channel data under the sky channel are the flying sound of an airplane, the user hears the sound effect of the airplane flying from left to right in the sky. After the electronic device is rotated 180°, as shown in Figure 16 ​As shown, SPK1 and SPK2 will be rotated from the upper left corner and the upper right corner to the lower left corner and the lower right corner. If the channel data of the flight of the plane is continuously input to SPK1 and SPK2, the sound effect of the plane flying in the opposite direction along the ground will be generated, which is obviously wrong.

[0274] Therefore, in order to avoid the above problems, the inventors of the present application propose the following scheme: the electronic device can detect the real-time placement posture of the electronic device relative to the user, and obtain the real-time positions of each loudspeaker under the real-time placement posture, that is, the real-time positions of each loudspeaker relative to the user. Further, for each loudspeaker, the electronic device can combine the third channel data whose channel position matches the real-time position of the loudspeaker as the audio data corresponding to the loudspeaker. In this way, even if the electronic device is rotated, the three-dimensional sound field playback effect can still be accurately realized.

[0275] As shown, Figure 16 With the scheme of the present application, after SPK1 and SPK2 will be rotated from the upper left corner and the upper right corner to the lower left corner and the lower right corner, the 4-channel data of the sky channel will no longer be continuously input to SPK1 and SPK2, but will be accurately input to SPK4 at the upper left corner and SPK3 at the upper right corner under the current placement posture of the electronic device according to the current placement posture of the electronic device. The sound effect of the plane flying from left to right in the sky is still played.

[0276] For example, the electronic device can detect the real-time placement posture of the electronic device relative to the user in real time through one of a gyroscope sensor, an acceleration sensor, a magnetic sensor, a touch sensor, or a pressure sensor.

[0277] In addition, the inventors of the present application have found through repeated experiments that in some cases, concurrent playback scenarios can also exist. That is, in the case of playing a first sound source, a second sound source can be concurrently played. For example, during the playback of music, a phone call or a voice prompt may suddenly be received, and the phone call or the voice prompt is the second sound source played concurrently. For another example, a device supporting concurrent playback can concurrently play videos or audios of two platforms. The concurrent playback scenario in the embodiments of the present application is not limited, and any scenario in which two sound sources are played simultaneously belongs to the scope of the concurrent playback scenario mentioned in the present application.

[0278] In the concurrent playback scenario, the electronic device supports playing the sound information of the first sound source and the sound information of the second sound source simultaneously when playing out through the loudspeaker. Specifically, the electronic device needs to mix and combine the channel data of the first sound source and the channel data of the second sound source to obtain the audio data corresponding to each loudspeaker, and then play the corresponding audio data through the loudspeaker. In this way, the audio data played by the loudspeaker contains the sound information of multiple sound sources.

[0279] In some embodiments, in the concurrent playing scenario, the first sound source includes first channel data of a first channel number, and after the digital signal processor performs multi-channel sound effect processing, third channel data of the first channel number is obtained. The second sound source indicating playing includes fourth channel data of a second channel number. Then, the electronic device can assign the fourth channel data to the loudspeakers whose positions match the channel positions of the fourth channel data, and for each of the loudspeakers, the electronic device can combine the third channel data and the fourth channel data assigned to the loudspeaker to obtain audio data corresponding to the loudspeaker. As known from the above, the third channel data assigned to the loudspeaker refers to the third channel data whose channel position matches the position of the loudspeaker.

[0280] In the following, taking an example that the electronic device has 4 loudspeakers, the first sound source in the concurrent playing scenario is a 7.1.4 channel sound source (including FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR), and the second sound source is a stereo sound source (including left channel data L and right channel data R), and combining Figure 15B and Figure 17 , the processing in the concurrent playing scenario is described in more detail.

[0281] Please refer to Figure 17 , Figure 17 , the electronic device in the above Figure 15B is consistent with the above Figure 15B , and thus the positions of the loudspeakers are also consistent with the above

[0282] In addition, the assignment of the first sound source is the same as Figure 15B . Based on the position matching principle, the left channel data L included in the second sound source is assigned to the SPK1 and SPK3 located on the left side, and the right channel data R included in the second sound source is assigned to the SPK2 and SPK4 located on the right side.

[0283] The final channel data assigned to the 4 loudspeakers are as follows:

[0284] SPK1: FL', C1, TFL', TRL', L

[0285] SPK2: FR', C2, TFR', TRR', R

[0286] SPK3: SL', BL', L

[0287] SPK4: SR', BR', R

[0288] Similarly, taking an example that the channel data assigned to the loudspeakers is weighted and combined, combining Figure 17 , the channel data assignment and combination are as follows:

[0289] SPK1: FL' + 0.707 * CI + 0.707 * TFL' + 0.707 * TRL' + L

[0290] SPK2: FR' + 0.707 * C2 + 0.707 * TFR' + 0.707 * TRR' + R

[0291] SPK3: SL' + BL' + L

[0292] SPK4: SR' + BR' + R

[0293] It should be understood that the above only for example can assign the left channel data L to SPK1 and SPK3. In fact, part of the left channel data L can be assigned to SPK1 and the other part to SPK3, without limiting both speakers SPK1 and SPK3 to be assigned to complete left channel data L.

[0294] It should be noted that when playing the second sound source, the electronic device identifies the second channel quantity of the second sound source when playing the second sound source, for example, the electronic device identifies the second channel data through the application framework layer. Further, according to the second channel quantity, it is identified whether the second sound source is a multi-channel sound source, and then different processing is performed according to whether the second sound source is a multi-channel sound source, which is specifically described in the following two cases:

[0295] Case one: the second sound source is a multi-channel sound source.

[0296] Specifically, if the second sound source is identified as a multi-channel sound source based on the second channel quantity, all processing steps before multi-channel data management (i.e., before mixing into speaker audio data) can be performed. Then, in the multi-channel data management stage, the channel data corresponding to the first sound source and the second sound source respectively transmitted to the first sound source and the second sound source are mixed according to the channel orientation to generate audio data corresponding to each speaker respectively.

[0297] Case two: the second sound source is not a multi-channel sound source.

[0298] Specifically, if the second audio source is not a multi-channel audio source, the application framework layer also selects a playback link corresponding to the second number of channels of the second audio source. However, the application framework layer can not need to perform preprocessing such as volume adjustment on the second audio source, and can directly transmit the channel data of the second audio source to the hardware abstraction layer, and notify the hardware abstraction layer of the selected playback link. The hardware abstraction layer can also create a playback path corresponding to the selected playback link, and directly transmit the channel data of the second audio source downward, but the playback path is not multiple, but one or two playback paths. Further, the digital signal processor can also not need to perform multi-channel audio effect processing (such as three-dimensional sound field widening processing) on the second audio source, but can directly mix the audio effect processed channel data (such as third channel data) of the first audio source and the original channel data of the second audio source according to the channel position.

[0299] It should be understood that if the second audio source is not a multi-channel audio source, it is largely not a main audio source, but a temporary audio source played concurrently, so that the sound playing effect thereof does not need a three-dimensional sound field cinema-level effect. Therefore, in the above scheme, another set of simplified processing logic can be performed on the second audio source which is not a multi-channel audio source, so as to omit steps such as volume preprocessing and audio effect processing, thereby effectively saving system resources without affecting or only slightly affecting the user experience.

[0300] The above only takes the second audio source as a stereo sound source as an example to illustrate the concurrent playing scene. It should be understood that the concurrent playing scene of the second audio source as a single-channel, stereo, 5.1-channel, 7.1-channel, 5.1.2-channel and 7.1.4-channel can also be performed according to similar mixing logic, and essentially, the channel data of multiple audio sources can be mixed according to the channel position in the last multi-channel data management stage, and no limitation is made thereto.

[0301] The embodiment of the present application also provides a chip system applied to an electronic device, the chip system comprising one or more processors, and the processor is configured to call computer instructions to enable the electronic device to perform various functions or steps in the above method embodiments.

[0302] The embodiment of the present application also provides a computer readable storage medium comprising computer instructions, when the computer instructions are run on the above electronic device, the computer instructions enable the electronic device to perform various functions or steps in the above method embodiments.

[0303] The embodiment of the present application also provides a computer program product, when the computer program product is run on a computer, the computer program product enables the computer to perform various functions or steps in the above method embodiments.

[0304] Through the description of the above embodiments, those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above functional modules is exemplified, and in actual application, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above.

[0305] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only illustrative, for example, the division of the modules or units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another device, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed units can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0306] The units described as separate components can or can not be physically separated, and the components shown as units can be one physical unit or multiple physical units, that is, they can be located in one place or distributed to multiple different places. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0307] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0308] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present application essentially or the parts that make contributions to the prior art or all or part of the technical solutions can be embodied in the form of a software product, which is stored in a storage medium and includes a plurality of instructions for causing an apparatus (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various storage medium that can store program codes.

[0309] The above merely provides the specific implementation of the present application, but the protection scope of the present application is not limited to this. Any change or replacement within the technical scope disclosed by the present application should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. An audio playback method, characterized by, The method is applied to an electronic device, the electronic device is provided with more than two loudspeakers, and the electronic device is provided with a hardware interface supporting multi-channel playback; the method comprises: receiving a playing instruction of a first sound source; the first sound source comprises first channel data of a first channel number, and the first channel number is greater than or equal to 3; in response to the playing instruction, creating a playing path of the first channel number; each playing path is used for transmitting target channel data; the target channel data is the first channel data or second channel data; the second channel data is obtained by preprocessing the first channel data; generating corresponding third channel data for each target channel data transmitted by each playing path; wherein the sound field width of each third channel data is not less than the sound field width of the corresponding target channel data, and the sound field width of at least one third channel data is greater than the sound field width of the corresponding target channel data; generating audio data corresponding to each loudspeaker based on the third channel data, and controlling each loudspeaker to play the corresponding audio data through the hardware interface.

2. The method of claim 1, wherein, The electronic device comprises an application layer, an application framework layer and a hardware abstraction layer; the receiving of the playing instruction of the first sound source comprises: the application layer receives the playing instruction of the first sound source; the response to the playing instruction to create the playing path of the first channel number comprises: the application framework layer selects a playing link supporting multi-channel playback based on the first channel number; the hardware abstraction layer creates the playing path of the first channel number based on the selected playing link.

3. The method according to claim 1 or 2, characterized in that, The application framework layer selects a playing link supporting multi-channel playback based on the first channel number, which comprises: the application framework layer selects a native playing link supporting multi-channel playback or a custom playing link supporting multi-channel playback based on the first channel number; wherein the native playing link refers to a playing link provided by a software system of the electronic device.

4. The method according to any one of claims 1-3, characterized in that, The electronic device further comprises a digital signal processor; the generation of the corresponding third channel data for each target channel data transmitted by each playing path comprises: the digital signal processor generates the corresponding third channel data for each target channel data transmitted by each playing path; the generation of the audio data corresponding to each loudspeaker based on the third channel data comprises: the digital signal processor generates the audio data corresponding to each loudspeaker based on the third channel data, and controls each loudspeaker to play the corresponding audio data through the hardware interface.

5. The method according to any one of claims 1-4, characterized in that, The generation of the audio data corresponding to each loudspeaker based on the third channel data comprises: for each loudspeaker, generating the audio data corresponding to the loudspeaker based on third channel data in the third channel data of the first channel number, the orientation of which matches the orientation of the loudspeaker; The orientation of the speaker refers to a current relative orientation of the speaker to the user; the electronic device has different placement postures relative to the user, and the current relative orientation of the same speaker to the user is different.

6. The method of claim 5, wherein, The third channel data of the first channel quantity includes center channel data and non-center channel data; the speaker set by the electronic device includes a first type of speaker and a second type of speaker; wherein, in the current placement posture of the electronic device relative to the user, the position of the first type of speaker is higher than that of the second type of speaker; The method further comprises: Splitting the center channel data into sub-center channel data corresponding to each speaker in the first type of speaker; The method further comprises: For each speaker in the first type of speaker, the sub-center channel data corresponding to the speaker and the non-center channel data with the channel orientation matching the orientation of the speaker are merged to obtain the audio data corresponding to the speaker; For each speaker in the second type of speaker, the non-center channel data with the channel orientation matching the orientation of the speaker is merged to obtain the audio data corresponding to the speaker.

7. The method of claim 6, wherein, The first sound source is a 7.1.4 channel sound source; the first type of speaker includes a first speaker and a second speaker, and the second type of speaker includes a third speaker and a fourth speaker; in the current placement posture of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left corner and the upper right corner of the electronic device respectively, and the third speaker and the fourth speaker are located at the lower left corner and the lower right corner of the electronic device respectively; The method further comprises: The front left channel data, the top front left channel data and the top rear left channel data in the third channel data of the first channel quantity are merged with the sub-center channel data corresponding to the first speaker to obtain the audio data corresponding to the first speaker; the front right channel data, the top front right channel data and the top rear right channel data in the third channel data of the first channel quantity are merged with the sub-center channel data corresponding to the second speaker to obtain the audio data corresponding to the second speaker; The method further comprises: For each speaker in the second type of speaker, the non-center channel data with the channel orientation matching the orientation of the speaker is merged to obtain the audio data corresponding to the speaker. merge the back left channel data and the back left surround channel data in the third channel data of the first channel number to obtain audio data corresponding to the third speaker; and merge the back right channel data and the back right surround channel data in the third channel data of the first channel number to obtain audio data corresponding to the fourth speaker.

8. The method of claim 5, wherein, Before the step of generating, for each of the speakers, audio data corresponding to the speaker based on third channel data of the first channel number that has a channel orientation matching an orientation of the speaker, the method further includes: obtaining orientations of each of the speakers in a real-time placement posture of the electronic device relative to the user.

9. The method of claim 5, wherein, The method further includes: obtaining fourth channel data of a second sound source; the second sound source is a sound source played concurrently with the first sound source; The step of generating, for each of the speakers, audio data corresponding to the speaker based on third channel data of the first channel number that has a channel orientation matching an orientation of the speaker, includes: For each of the speakers, merging third channel data that has a channel orientation matching an orientation of the speaker and fourth channel data that has a channel orientation matching an orientation of the speaker to obtain audio data corresponding to the speaker.

10. The method of claim 9, wherein, The fourth channel data of the second sound source includes left channel data and right channel data; the electronic device includes a first speaker, a second speaker, a third speaker, and a fourth speaker; wherein, in a current placement posture of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left corner and the upper right corner of the electronic device respectively, and the third speaker and the fourth speaker are located at the lower left corner and the lower right corner of the electronic device respectively; the left channel data is respectively assigned to the first speaker and the third speaker; and the right channel data is respectively assigned to the second speaker and the fourth speaker.

11. The method according to any one of claims 1-10, characterized in that, Each speaker has a corresponding power amplifier (PA); The step of controlling, through the hardware interface, each of the speakers to play corresponding audio data includes: For each of the speakers, transmitting, through the hardware interface, audio data corresponding to the speaker to a PA corresponding to the speaker, and inputting the audio data into the speaker through the PA for audio playback; When the speaker is playing audio, the current of the speaker and the voltage are detected in real time, and the parameters of the PA are adjusted in real time according to the current and the voltage, so as to keep the speaker in a target working state.

12. An electronic device, comprising: The electronic device includes a memory, more than two speakers, and one or more processors; the memory, the more than two speakers, and the processor are coupled; wherein the more than two speakers are used for playing audio, the memory stores computer program code, and the computer program code includes computer instructions, when the computer instructions are executed by the processor, the electronic device executes the method of any one of claims 1-11.

13. A chip system, characterized by The chip system is applied to an electronic device, and the chip system comprises one or more processors configured to invoke computer instructions to cause the electronic device to perform the method according to any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, The computer program product comprises computer instructions which, when executed on an electronic device, cause the electronic device to perform the method according to any one of claims 1-11.

15. A computer program product, characterised in that, The computer program product, when executed on a computer, causes the computer to perform the method according to any one of claims 1-11.

Images