Audio playing-out method, electronic device, storage medium and computer program product

Abstract

An audio external playing method, an electronic device, a storage medium and a computer program product, relate to the technical field of audio playing, and can improve the sound external playing effect when the electronic device plays audio. More than two loudspeakers are arranged in the electronic device, and a hardware interface supporting multi-channel playing is arranged. Specifically, when a multi-channel sound source including N-channel data is played, the electronic device can adaptively create N transmission channels, so that each channel data of the multi-channel sound source can be transmitted to the lower layer without being lost. Further, the lower layer can perform three-dimensional sound field related sound effect processing on more complete and more comprehensive channel data, and generate audio data corresponding to each loudspeaker in the electronic device after the sound effect processing, which has more spatial sense and more sound field information. Based on the above hardware interface, the loudspeaker plays the corresponding audio data, that is, the three-dimensional sound field playing effect can be realized.

Description

Technical Field

[0001] This application relates to the field of audio playback technology, and in particular to an audio playback method, electronic device, storage medium, and computer program product. Background Technology

[0002] With the widespread use of smart tablets and other electronic devices, most users use these devices to play audio and video on a daily basis. For example, they often play the sound from everyday audio-visual entertainment scenarios through external speakers. Especially when using movie or game apps, the quality of the external speaker experience directly affects the user's overall audio experience with the electronic device.

[0003] Most electronic devices do not support multi-channel playback. When playing multi-channel audio sources, they automatically downgrade the data from multiple channels to stereo for external amplification. As a result, some sound field information contained in the multi-channel audio is discarded, and the externally amplified sound cannot reproduce the multi-channel information, leading to poor sound quality and a bad listening experience. Summary of the Invention

[0004] This application provides an audio playback method, electronic device, storage medium, and computer program product, which can improve the sound playback effect when playing audio on an electronic device, thereby enhancing the user's auditory experience.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] Firstly, an audio playback method is provided, applied to an electronic device equipped with M speakers, where M is greater than 2, and a hardware interface supporting multi-channel audio data (multi-channel playback). Specifically, the electronic device receives a playback command from a first audio source, which includes first channel data of N channels, where N is greater than or equal to 3. In response to the playback command, N playback paths are created. That is, when playing a multi-channel audio source, a playback path is adaptively created for each channel of the multi-channel audio source. Each playback path is used to transmit one target channel data, for example, directly transmitting first channel data or second channel data after preprocessing the first channel data. This avoids being limited to two fixed two-channel paths, allowing all channel data of the multi-channel audio source to be transmitted to the next layer without loss. Furthermore, based on the more complete and comprehensive N target channel data transmitted through the N playback paths, sound field widening and other sound effect processing can be performed to obtain N third channel data with a more spatial feel. In this system, the sound field width of each third channel data is not less than the sound field width of the corresponding target channel data, and the sound field width of at least one third channel data is greater than the sound field width of the corresponding target channel data. That is, at least a portion of the N target channel data undergoes sound field widening. The widened channel data and the target channel data that do not require sound field widening together constitute the N third channel data. Further, based on the N third channel data, M audio data corresponding to each of the speakers are generated, which constitutes the M audio data. Subsequently, the M speakers can be controlled to play their respective audio data based on the aforementioned hardware interface supporting multi-channel playback.

[0007] The above solution improves the hardware and software of the electronic device. At the software level, it adaptively creates playback paths matching the number of channels in the multi-channel audio source, transmitting the source's channel data to the lower layer without loss. The lower layer can then use this more complete and comprehensive multi-channel data as a basis for sound field widening and other sound effect processing, resulting in channel data with a stronger sense of space. This generates audio data corresponding to each speaker. Compared to two-channel data, this audio data contains more channel information from the audio source and has a greater sense of space. At the hardware level, it features a hardware interface supporting multi-channel audio data, transmitting the audio data corresponding to each speaker individually to the speaker for playback, achieving a three-dimensional sound field playback effect (external playback effect). In other words, without requiring additional auxiliary equipment (no need to purchase multiple additional playback devices for networking), the hardware and software improvements enable the electronic device to achieve a three-dimensional sound field playback effect, saving costs and improving the external audio playback effect, thereby enhancing the user experience.

[0008] In addition, it can adaptively create playback paths, thus being compatible with various multi-channel audio sources such as 5.1 / 7.1 / 5.1.2 / 7.1.4, greatly improving its applicability and further enhancing the user experience.

[0009] Furthermore, purchasing multiple playback devices to form a network (i.e., a home theater system) involves data transmission between devices, which can cause latency issues. Additionally, the placement of the devices can affect the audio output. This solution, however, is implemented internally within the electronic device, resulting in better stability, less latency, and a fixed speaker position that is unaffected by user operation, thus ensuring superior audio output.

[0010] In one possible implementation of the first aspect, the electronic device includes an application layer, an application framework layer, and a hardware abstraction layer. The application layer receives a playback instruction from a first audio source; the application framework layer identifies the number of first channels of the first audio source to select a playback link that supports multi-channel playback based on the number of first channels; and the hardware abstraction layer creates a playback path for the number of first channels based on the selected playback link.

[0011] The above solution makes corresponding improvements to the application layer, application framework layer, and hardware abstraction layer, enabling them to collaborate and conveniently and adaptively create playback paths that match the number of channels of a multi-channel audio source.

[0012] In one possible implementation of the first aspect, the application framework layer can select either a native playback link that supports multi-channel playback based on the number of first channels of the multi-channel audio source, or a custom playback link that supports multi-channel playback. A native playback link refers to a playback link built into the software system of the electronic device, such as direct or offload, which are built-in multi-channel playback links. A custom playback link is a playback link defined by the manufacturer of the electronic device that supports multi-channel playback.

[0013] The above solution innovatively applies the native playback link that supports multi-channel playback to the scenario of 3D sound field playback of multi-channel audio sources. Specifically, it achieves interface and invocation with the native playback link at both the application framework layer and the hardware abstraction layer, together forming the aforementioned audio external playback solution that realizes the 3D sound field playback effect. This novel technical solution effectively utilizes the native playback link to achieve 3D sound field playback, improving resource utilization.

[0014] In addition, it can also realize the customization of playback links that support multi-channel playback, so that it is not limited to the playback links that come with the software system. It can expand the playback links more flexibly according to actual needs, and can adapt to the three-dimensional sound field playback of any multi-channel audio source, which greatly improves the applicability and enhances the user experience.

[0015] In one possible implementation of the first aspect, the electronic device further includes a digital signal processor; the digital signal processor generates corresponding third channel data for each target channel data transmitted by each playback path; and the digital signal processor generates audio data corresponding to each of the speakers based on the third channel data, and controls each of the speakers to play the corresponding audio data through the hardware interface.

[0016] In the above solution, the digital signal processor is used to perform multi-channel audio effect processing and multi-channel data management, and generate audio data. This achieves decoupling from the upper layer, making it more flexible and efficient.

[0017] In one possible implementation of the first aspect, for each speaker, audio data corresponding to the speaker is generated based on the third channel data in the third channel data, which matches the channel orientation with the speaker orientation. Here, the speaker orientation refers to the speaker's current relative position to the user; the relative position of the same speaker to the user varies depending on the placement of the electronic device relative to the user.

[0018] In the above scheme, the orientation matching principle can more accurately allocate channel data to each speaker, thereby more accurately synthesizing the audio data corresponding to the speaker. This results in a more accurate three-dimensional sound field playback effect when multiple speakers play audio data, thus improving the user experience.

[0019] In one possible implementation of the first aspect, the third channel data of the first number of channels includes center channel data and non-center channel data. The electronic device can split the center channel data into at least two sub-center channel data and allocate them to the speakers respectively. It should be noted that, in the current orientation of the electronic device relative to the user, the speakers allocated the sub-center channel data are located on the left and right sides of the electronic device. For the speakers allocated sub-center channel data, the electronic device can merge the non-center channel data and sub-center channel data allocated to that speaker, mixing them to generate the corresponding audio data for that speaker.

[0020] For example, the speaker assigned to the sub-center channel data could be any of the speakers, meaning the center channel data could be assigned to all speakers. Then, for each speaker, the electronic device could combine the sub-center channel data and non-center channel data assigned to that speaker to obtain the corresponding audio data for that speaker. It should be understood that the non-center channel data assigned to that speaker could be the third channel data whose channel orientation matches the speaker's orientation.

[0021] For example, a speaker assigned sub-center channel data could be a partial speaker. Then, for speakers not assigned sub-center channel data, the electronic device can merge the non-center channel data allocated to that speaker and mix it to generate the corresponding audio data for that speaker.

[0022] In the above scheme, the center channel data is allocated to the left and right speakers, and then combined with the other channel data allocated to the left and right speakers for playback, thereby creating the effect of the center channel sound playing in the middle of the electronic device.

[0023] In one possible implementation of the first aspect, the third channel data of the first number of channels includes center channel data and non-center channel data; the speakers set up by the electronic device include first-type speakers and second-type speakers. Wherein, in the current placement posture of the electronic device relative to the user, the position of the first-type speakers is higher than the position of the second-type speakers. The center channel data is split into sub-center channel data corresponding one-to-one with each speaker in the first-type speakers, that is, the center channel data is assigned to the speakers with relatively higher positions. For each speaker in the first-type speakers, the electronic device can merge the sub-center channel data corresponding to the speaker and the non-center channel data whose channel orientation matches the speaker orientation to obtain the audio data corresponding to the speaker. For each speaker in the second-type speakers, the electronic device can merge the non-center channel data whose channel orientation matches the speaker orientation to obtain the audio data corresponding to the speaker.

[0024] In the above solution, since users' ears are often higher than the center of the electronic device when using it, placing the center channel sound in the exact center or near the center of the device would cause users to have to tilt their heads down to hear it clearly, resulting in a slightly less than ideal listening experience. Therefore, allocating the center channel data to a speaker that is positioned relatively higher or slightly higher from the user's current perspective, creating the effect of the center channel sound emanating from slightly above the center of the electronic device, allows the center channel sound (such as center vocals) to be positioned more accurately and closer to the user's ears, thus improving the user's listening experience.

[0025] In one possible implementation of the first aspect, the first sound source is a 7.1.4 channel sound source; the first type of loudspeakers includes a first loudspeaker and a second loudspeaker, and the second type of loudspeakers includes a third loudspeaker and a fourth loudspeaker; in the current orientation of the electronic device relative to the user, the first loudspeaker and the second loudspeaker are located at the upper left and upper right corners of the electronic device, respectively, and the third loudspeaker and the fourth loudspeaker are located at the lower left and lower right corners of the electronic device, respectively. The electronic device can merge the front left channel data, top front left channel data, and top rear left channel data from the third channel data (number of first channels) with the sub-center channel data corresponding to the first loudspeaker to obtain audio data corresponding to the first loudspeaker. The electronic device can merge the front right channel data, top front right channel data, and top rear right channel data from the third channel data (number of first channels) with the sub-center channel data corresponding to the second loudspeaker to obtain audio data corresponding to the second loudspeaker. The electronic device can merge the rear left channel data and rear left surround channel data from the third channel data (number of first channels) to obtain audio data corresponding to the third loudspeaker. The electronic device can combine the rear right channel data and the rear right surround channel data from the third channel data of the first channel quantity to obtain the audio data corresponding to the fourth speaker.

[0026] In the above scheme, the first audio source is a 7.1.4 channel audio source with four speakers. When allocating data to the speakers, the center channel data can be assigned to the first speaker in the upper left corner and the second speaker in the upper right corner, which are positioned slightly above the center. Furthermore, based on the principle of orientation matching, channel data with an orientation close to the upper left corner can be assigned to the first speaker currently positioned in the upper left corner relative to the user; channel data with an orientation close to the upper right corner can be assigned to the second speaker currently positioned in the upper right corner; channel data with an orientation close to the lower left corner can be assigned to the third speaker currently positioned in the lower left corner; and channel data with an orientation close to the lower right corner can be assigned to the fourth speaker currently positioned in the lower right corner. This improves the accuracy of data allocation, better achieves the three-dimensional sound field playback effect of the 7.1.4 channel audio source, and enhances the user's auditory experience. Moreover, it creates the effect of the center channel sound emanating from a position slightly above the center of the electronic device, further improving the user experience.

[0027] In one possible implementation of the first aspect, before generating the audio data corresponding to each speaker from the third channel data (based on the number of first channels) where the channel orientation matches the speaker orientation, the orientation of each speaker relative to the user in real time can be obtained. Thus, based on the current orientation of each speaker relative to the user in real time, channel data matching the orientation is assigned to each speaker, avoiding the problem of inaccurate channel data allocation caused by changes in the speaker orientation relative to the user due to rotation of the electronic device, further improving the subsequent three-dimensional sound field playback effect.

[0028] In one possible implementation of the first aspect, in a concurrent playback scenario, the fourth channel data of a second audio source is acquired; the second audio source is an audio source played concurrently with the first audio source. For each speaker, the electronic device can merge the third channel data whose channel orientation matches the speaker orientation and the fourth channel data whose channel orientation matches the speaker orientation to obtain the audio data corresponding to the speaker.

[0029] In the above solution, under concurrent playback scenarios, the channel data of multiple concurrent sound sources that match the location of the same speaker are first merged according to the location matching principle to obtain the audio data corresponding to the speaker, which is then played by the speaker. This allows each speaker to reasonably and accurately play the sound of multiple sound sources in concurrent playback scenarios, meeting the requirements of concurrent playback. Moreover, concurrent playback using the above solution is not simply about playing the sound of multiple sound sources, but about reasonably allocating channel data so that the sound of two sound sources can be played accurately and harmoniously. In addition, the spatial effect of the three-dimensional sound field playback formed by multiple sound sources is not affected in concurrent playback scenarios, greatly improving the user experience.

[0030] In one possible implementation of the first aspect, the fourth channel data of the second audio source includes left channel data and right channel data; the electronic device includes a first speaker, a second speaker, a third speaker, and a fourth speaker; wherein, in the current orientation of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left and upper right corners of the electronic device, respectively, and the third speaker and the fourth speaker are located at the lower left and lower right corners of the electronic device, respectively; the left channel data is assigned to the first speaker and the third speaker; and the right channel data is assigned to the second speaker and the fourth speaker.

[0031] In the above solution, when multi-channel sound sources and stereo two-channel sound sources are played concurrently, the left and right channel data of the stereo two-channel sound source can be accurately merged with the channel data of the multi-channel sound source based on the channel orientation, thus achieving a better audio playback effect in this concurrent playback scenario.

[0032] In one possible implementation of the first aspect, each speaker has a one-to-one corresponding power amplifier (PA). For each speaker, the electronic device can transmit the audio data corresponding to the speaker to the corresponding PA through a hardware interface that supports multi-channel playback. The PA then inputs the audio data into the speaker for audio playback. While the speaker is playing audio, the current and voltage of the speaker are detected in real time, and the parameters of the PA are adjusted in real time according to the current and voltage to keep the speaker in the target operating state.

[0033] In the above solution, the audio playback scheme that creates a three-dimensional sound field effect can detect the speaker's current and voltage in real time and adjust the PA parameters accordingly. Based on the adjusted PA parameters, the corresponding audio data of the speaker is amplified to control the speaker's audio playback. This ensures that the speaker is always in optimal working condition, achieving the audio playback effect of a three-dimensional sound field.

[0034] In a second aspect, this application provides an electronic device comprising at least: more than two speakers, a memory, and one or more processors; the memory, the more than two speakers, and the processors are coupled; the more than two speakers are used for external audio playback; the memory stores computer program code, the computer program code including computer instructions, which, when executed by one or more processors, cause the electronic device to perform the method as described in any of the first aspects above.

[0035] Thirdly, this application provides a chip system applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the methods described in any of the first aspects above.

[0036] Fourthly, this application provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method described in any of the first aspects above.

[0037] Fifthly, this application provides a computer program product that, when run on a computer, causes the computer to perform any of the methods described in the first aspect. Attached Figure Description

[0038] Figure 1 This application provides a schematic diagram illustrating the effect of a cinema-level sound field experience in an embodiment of the present application.

[0039] Figure 2 A schematic diagram of the channel orientation of a 5.1 channel system provided in an embodiment of this application;

[0040] Figure 3 A schematic diagram of the channel orientation of a 7.1 channel system provided in an embodiment of this application;

[0041] Figure 4 A schematic diagram of the channel orientation of a 7.1.4 channel system provided for embodiments of this application;

[0042] Figure 5 A schematic diagram of a related technology limited to dual-channel playback is provided for an embodiment of this application;

[0043] Figure 6 A schematic diagram of a hardware framework for implementing multi-channel playback is provided in an embodiment of this application;

[0044] Figure 7 A schematic diagram of another hardware framework for implementing multi-channel playback provided in an embodiment of this application;

[0045] Figure 8 A hardware structure block diagram of an electronic device provided in an embodiment of this application;

[0046] Figure 9A A software structure block diagram of an electronic device provided in an embodiment of this application;

[0047] Figure 9B Another software structure block diagram for an electronic device provided in an embodiment of this application;

[0048] Figure 10 A timing diagram of an audio playback method provided in an embodiment of this application;

[0049] Figure 11 A schematic diagram of another method for combining a software structure block diagram of an electronic device, provided as an embodiment of this application;

[0050] Figure 12 A timing diagram of another audio playback method provided in an embodiment of this application;

[0051] Figure 13 A schematic diagram of a sound field broadening effect provided in an embodiment of this application;

[0052] Figure 14 A schematic diagram illustrating the relative position of a speaker and a user, provided for an embodiment of this application;

[0053] Figures 15A to 15C Schematic diagrams of three different audio mixing processes provided in the embodiments of this application;

[0054] Figure 16 This is a schematic diagram of audio mixing processing in a scenario where the placement posture changes, provided by an embodiment of this application.

[0055] Figure 17This is a schematic diagram of audio mixing in a concurrent playback scenario provided in an embodiment of this application. Detailed Implementation

[0056] References to "one embodiment" or "some embodiments" as used in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "in one embodiment," "in some embodiments," "in other embodiments," "in still other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized. The term "connection" includes both direct and indirect connections, unless otherwise stated.

[0057] In the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is an example, illustration, or description. Any embodiment or design that is described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design. Specifically, the use of the terms "exemplary" or "for example" is intended to present the relevant concepts in a specific manner.

[0058] Hereinafter, the terms "first," "second," "third," and "fourth" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first," "second," "third," and "fourth" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0059] This application provides an audio playback method applicable to electronic devices equipped with multiple (more than two) speakers. In audio playback scenarios, no additional auxiliary equipment is required. By executing the audio playback method described in this application, the electronic device controls the speakers to play multi-channel audio data, allowing the user to experience the spatial information of the sound field conveyed by the multi-channel audio data. This provides a cinematic sound field experience, not just a simple stereo effect, thus improving the audio playback quality. See details... Figure 1 , Figure 1 Taking tablets as an example of electronic devices, when users play audio data on a tablet, they can experience the effect of the speakers seemingly playing in a three-dimensional space, achieving a three-dimensional sound field experience similar to that of a movie theater.

[0060] It should be understood that the source of audio data played in an audio playback scenario can include at least one of the following: a separate audio file, audio data in a video file, audio data in an application or webpage, etc., without limitation.

[0061] Multichannel audio data refers to audio data from a multichannel system, which includes channel data for multiple channels. A multichannel system is a system with more than two channels. For example, a multichannel system can include 2.1 channels, 3.0 channels, 5.1 channels, 5.1.2 channels, 7.1 channels, 7.1.2 channels, or 7.1.4 channels, etc.

[0062] It should be understood that different multichannel systems have different channel orientation layouts. Channel orientation refers to the desired direction and position of sound when the channel data is recorded or played back. For example, the left channel data aims to present the sound as if it's playing to the left of the user's ear, the right channel data aims to present the sound as if it's playing to the right of the user's ear, and the sky channel data (or top channel data) aims to present the sound as if it's playing above the user.

[0063] In traditional methods, if the goal is to allow users to experience something like... Figure 1 The cinema-quality 3D sound field effect shown typically requires purchasing a home theater system with multiple channels and placing each device in its designated position according to the channel orientation. The following will combine... Figures 2 to 4 This diagram illustrates the channel orientation of different audio systems and the placement of home theater equipment.

[0064] Figure 2 This is a diagram showing the channel orientation of a 5.1 channel system. A 5.1 channel system includes the front left channel (FL), front right channel (FR), center channel (C), left surround channel (SL), right surround channel (SR), and subwoofer channel (SW). From... Figure 2 As can be seen, to achieve the sound field effect of a 5.1 channel system, one speaker needs to be purchased for each channel, that is, at least 6 speakers need to be purchased and placed according to the channel orientation layout of a 5.1 channel system.

[0065] Figure 3 This is a channel layout diagram for a 7.1 channel system. The 7.1 channel system adds two rear surround channels to the 5.1 channel system: left and right rear surround channels. Specifically, the 7.1 channel system includes the front left channel (FL), front right channel (FR), center channel (C), left surround channel (SL), right surround channel (SR), left rear surround channel (LB), right rear surround channel (RB), and subwoofer channel (SW). From... Figure 3It is known that to achieve the sound field effect of a 7.1 channel system, at least 8 speakers need to be purchased and arranged according to the channel orientation layout of a 7.1 channel system.

[0066] Figure 4 This is a channel orientation diagram of a 7.1.4 channel system. The 7.1.4 channel system adds four overhead channels to the 7.1 channel system: Top Front Left Channel (TFL), Top Front Right Channel (TFR), Top Rear Left Channel (TRL), and Top Rear Right Channel (TRR). Specifically, the 7.1.4 channel system includes the front left channel (FL), front right channel (FR), center channel (C), left surround channel (SL), right surround channel (SR), left rear surround channel (LB), right rear surround channel (RB), subwoofer channel (SW), top front left channel (TFL), top front right channel (TFR), top rear left channel (TRL), and top rear right channel (TRR). Figure 4 It is known that to achieve the sound field effect of a 7.1.4 channel system, at least 12 speakers need to be purchased and arranged according to the channel orientation layout of the 7.1.4 channel system.

[0067] from Figures 2 to 4 It's easy to see that purchasing a home theater system to complement each other and achieve good audio playback results would be very expensive.

[0068] With the rapid development of terminal technology, many electronic devices now have multiple (more than two) speakers. For example, many electronic devices have four, six, eight, or even more speakers. Achieving a three-dimensional sound field audio output effect through electronic devices with multiple speakers without the aid of other equipment would significantly reduce costs.

[0069] However, current technologies often fail to support multi-channel playback in electronic devices, even when equipped with multiple speakers. Therefore, even if an electronic device plays multi-channel audio sources (i.e., multi-channel audio data), it will ultimately be played in stereo (two-channel) format. Multiple speakers can only increase the volume or loudness of the left and right channels, failing to achieve a three-dimensional sound field. The following will combine... Figure 5 The above-mentioned related technologies are illustrated.

[0070] Please see Figure 5 Taking a tablet playing multi-channel audio sources as an example, the tablet has eight speakers. Since tablets do not support multi-channel playback, when playing 5.1 / 7.1 multi-channel audio sources, the tablet first mixes the multi-channel audio source into the left and right channels of stereo. For example, Figure 5 The middle channel will downmix the 6-channel data from the 5.1 channel audio source into 2 channels: left channel data and right channel data. Figure 5 Each arrow in the diagram represents a single audio signal (one channel of data or one audio data point is essentially one audio signal). Even if the tablet has some audio processing algorithms (such as sound effect processing), these algorithms will only apply to the left and right channel data. That is, the tablet will only use these algorithms to process the left and right channel data, resulting in only two channel data points after processing. Finally, the output from the eight speakers will only be the data from the left and right channels. For example, the processed left channel data will be played through the four speakers on the left side of the tablet, and the processed right channel data will be played through the four speakers on the right side. It should be understood that when mixing multi-channel audio sources with stereo two-channel data, some sound field information from the multi-channel audio sources will be discarded. This causes the sound played by the tablet to be unable to accurately reproduce the expression of the multi-channel data, thus failing to achieve a three-dimensional sound field playback effect and only achieving stereo playback.

[0071] Furthermore, some related technologies perform signal conversion and other processing after the channel data is allocated to the speaker. This approach still fails to fully integrate the internal hardware and software pathways of the device. During the transmission of channel data to the speaker, it is still limited by the constraints of a two-channel path, resulting in the loss of some useful sound field information. The channel data ultimately allocated to the speaker is already data with sound field loss; therefore, performing signal conversion and other processing based on this data has limited effectiveness in achieving optimal external playback.

[0072] To address the aforementioned issues, the solution in this application improves both the hardware and software aspects of the electronic device. This involves the coordination of hardware and software systems (i.e., supporting multi-channel audio at both the software and hardware levels), enabling the electronic device to support multi-channel playback. Furthermore, combined with an audio playback method proposed in the embodiments of this application, a three-dimensional sound field playback effect can be achieved through the electronic device's speaker.

[0073] Specifically, when playing multi-channel audio sources, at the software level, the electronic device can adaptively create playback paths that match the number of channels in the multi-channel audio source, ensuring that all channel data from the multi-channel audio source can be transmitted to the next layer without loss. Furthermore, it performs three-dimensional sound field-related sound effect processing on the more complete and comprehensive channel data, generating audio data corresponding to each speaker in the electronic device after the sound effect processing. At the hardware level, the electronic device is equipped with a hardware interface that supports multi-channel audio data. Through this hardware interface, the audio data corresponding to each speaker can be transmitted to the corresponding speaker for playback, thereby achieving the output of a three-dimensional sound field.

[0074] It should be understood that the channel widening described in the embodiments of this application is not limited to widening the sound field of all channel data of a multi-channel audio source. It can be widened only for a portion of the channel data (i.e., the channel data that can improve the three-dimensional spatial effect after widening the sound field). Then, the channel data after widening the sound field is mixed with the channel data of other channels that have not been widened to generate audio data that corresponds one-to-one with each speaker in the electronic device.

[0075] The solution in this embodiment of the application significantly reduces hardware costs because it achieves a three-dimensional sound field playback effect without requiring additional auxiliary equipment. Furthermore, compared to... Figure 5 Compared to the previous method of achieving stereo playback through multiple speakers, this method fully utilizes the capabilities of multiple speakers and improves the utilization rate of the hardware resources of the electronic device itself.

[0076] To facilitate understanding, we will now combine Figure 6 A diagram is provided. Please refer to [link / reference]. Figure 6 The electronic device has a total of 8 speakers (SPK1 to SPK8). It also has a hardware interface that supports multi-channel audio data transmission, used to transfer audio data to the 8 speakers. Figure 6 The example shown is based on a 5.1 channel audio source, but it is not limited to 5.1 channels. Electronic devices can also play other multi-channel audio sources such as 7.1, 7.1.2, and 7.1.4. Figure 6 Each arrow represents a sound signal (e.g., a channel data or audio data; it should be understood that channel data and audio data in the embodiments of this application are essentially sound signals, and are used to differentiate and represent sound signals at different stages).

[0077] A 5.1-channel audio source can include six channels: front left channel (FL), front right channel (FR), center channel (C), left surround channel (SL), right surround channel (SR), and subwoofer channel (SW). Therefore, the electronic device can adaptively create six playback paths, each transmitting one channel of data. This ensures that all six channels of the 5.1-channel audio source are transmitted without loss for subsequent multi-channel audio processing. In this multi-channel audio processing, all or part of these six channels can be individually processed with effects such as 3D sound field widening. For example, sound field widening might be applied to the front left channel (FL), front right channel (FR), left surround channel (SL), and right surround channel (SR), while sound field widening is not required for the center channel (C) and subwoofer channel (SW). After audio processing, the audio remains as 6-channel data: the front left channel FL', front right channel FR', left surround channel SL', right surround channel SR' (after sound field expansion), and the center channel C and subwoofer channel SW (without sound field expansion). Electronic devices can then manage this 6-channel data in a multi-channel manner. For example, they can mix the processed 6-channel data according to their respective channel orientations, generating 8 audio channels that correspond one-to-one with or are mapped to the 8 speakers. These 8 audio channels can then be distributed to their respective speakers via a hardware interface that supports multi-channel audio data, achieving a three-dimensional sound field playback effect.

[0078] In some embodiments, each speaker in the electronic device has a corresponding power amplifier (PA). See also Figure 7 Each of the eight speakers has a corresponding PA, namely PA1 to PA8. After mixing to generate eight audio data channels that correspond one-to-one with or are mapped to the eight speakers, the electronic device can transmit each audio data channel to its corresponding PA via a hardware interface that supports multi-channel audio data. The PA then inputs the audio data to the corresponding speaker for audio playback. In this scheme, the audio data allocated to the speakers is amplified by the power amplifier PA before being input to the speaker for playback, which can improve the audio playback effect to a certain extent.

[0079] It should be understood that Figure 6 and Figure 7The hardware interface supports Time Division Multiplexing (TDM) for data transmission. Therefore, electronic devices can use TDM to directly or indirectly transmit audio data to speakers via the PA through this hardware interface. TDM allows the hardware interface to be reused within a very short time slice to transmit audio data from various channels to their corresponding speakers, achieving simultaneous or near-simultaneous audio playback on all speakers. This solution reuses a single hardware interface, improving resource utilization and saving communication resources.

[0080] In other embodiments, the hardware interface of the electronic device can be improved so that each speaker corresponds to one hardware interface, that is, the hardware interface corresponds one-to-one with the speaker, so that corresponding audio data can be transmitted to the speaker based on the hardware interface corresponding to each speaker. The embodiments of this application do not limit the specific implementation of the hardware interface supporting multi-channel playback.

[0081] For example, the aforementioned electronic devices may specifically be tablet computers, televisions (also known as smart TVs, smart screens, or large-screen devices), laptop computers, personal computers (PCs), ultra-mobile personal computers (UMPCs), handheld computers, wearable electronic devices (e.g., smartwatches, smart bracelets, smart glasses), in-vehicle devices, virtual reality devices, and other electronic devices with audio playback functions and equipped with multiple speakers. The embodiments of this application do not impose special limitations on the specific form of the electronic devices.

[0082] In some embodiments, the electronic device may have 4, 6, 8, 12, or even more speakers. The speaker layout of the electronic device satisfies the following conditions:

[0083] In an electronic device, multiple speakers are symmetrically distributed on both sides of the device; that is, the number of speakers on both sides is the same, and the speakers on both sides are symmetrically distributed about the center line of the electronic device's axis. For example, Figure 7 The diagram shows eight speakers, four on each side, symmetrically arranged about the center line of the electronic device. For example... Figure 15A The four speakers shown are distributed in pairs on both sides of the electronic device, with two speakers on each side. Furthermore, the speaker axes on both sides are symmetrically distributed with respect to the center line of the electronic device's axis.

[0084] To facilitate understanding, we will now combine... Figure 8 The internal structure of the electronic devices in various embodiments of this application will be illustrated by examples.

[0085] like Figure 8 As shown, the electronic device 800 may include a processor 810, an external memory interface 820, an internal memory 821, a universal serial bus (USB) interface 830, a charging management module 840, a power management module 841, a battery 842, antenna 1, antenna 2, a mobile communication module 850, a wireless communication module 860, an audio module 870, a speaker 870A, a receiver 870B, a microphone 870C, a headphone jack 870D, a sensor module 880, a button 190, a motor 891, an indicator 892, a camera 893, a display screen 894, and a subscriber identification module (SIM) card interface 895, etc. The sensor module 880 may include pressure sensors, gyroscope sensors, barometric pressure sensors, magnetic sensors, accelerometers, distance sensors, proximity sensors, fingerprint sensors, temperature sensors, touch sensors, ambient light sensors, bone conduction sensors, etc.

[0086] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 800. In other embodiments of this application, the electronic device 800 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0087] The processor 810 may include one or more processing units, such as an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0088] The controller can be the nerve center and command center of the electronic device 800. The controller can generate operation control signals based on the instruction opcode and timing signals to control the fetching and execution of instructions.

[0089] The processor 810 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 810 is a cache memory. This memory can store instructions or data that the processor 810 has just used or that are used repeatedly. If the processor 810 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 810, and thus improves the efficiency of the system.

[0090] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs an audio signal through an audio device (not limited to speaker 870A, receiver 870B, etc.). In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 810 and may be housed in the same device as the mobile communication module 850 or other functional modules.

[0091] Digital signal processors are used to process digital signals, such as digital audio signals.

[0092] The external storage interface 820 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the electronic device 800. The external storage card communicates with the processor 810 through the external storage interface 820 to perform data storage functions. For example, audio-related files such as music and video can be saved on the external storage card.

[0093] Internal memory 821 can be used to store computer executable program code, which includes instructions. Processor 810 executes various functional applications and data processing of electronic device 800 by running the instructions stored in internal memory 821. Internal memory 821 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as a sound playback function), etc. The data storage area may store data created during the use of electronic device 800 (such as audio data, phonebook, etc.). Furthermore, internal memory 821 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0094] Electronic device 800 can implement audio functions through audio module 870, speaker 870A, receiver 870B, microphone 870C, headphone jack 870D, and application processor, such as music playback, voice command input, and recording.

[0095] The audio module 870 is used to convert digital audio signals into analog audio signals for output, and also to convert analog audio inputs into digital audio signals. The audio module 870 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 870 can be located in the processor 810, or some functional modules of the audio module 870 can be located in the processor 810. For example, all or some functional modules of the audio module 870 can be located in a digital signal processor.

[0096] In some embodiments of this application, the audio module may include a multi-channel data processing module. Therefore, the multi-channel data processing module can be set in the digital signal processor, so that the digital signal processor can perform processing such as sound field widening and multi-channel data merging management.

[0097] The speaker 870A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The electronic device 800 can listen to music or make hands-free calls through the speaker 870A. In this embodiment, the electronic device may include multiple speakers 870A to achieve a three-dimensional sound field playback effect, providing the user with a three-dimensional sound field experience.

[0098] The receiver 870B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 800 answers a telephone call or voice message, the receiver 870B can be brought close to the listener's ear to hear the voice.

[0099] Microphone 870C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 870C, inputting the sound signal into microphone 870C. Electronic device 800 may have at least one microphone 870C. In some embodiments, electronic device 800 may have two microphones 870C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 800 may have three, four, or more microphones 870C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0100] Pressure sensor 880A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, pressure sensor 880A may be disposed on display screen 894. When a touch operation is applied to display screen 894, electronic device 800 detects the intensity of the touch operation based on pressure sensor 880A. Electronic device 800 may also calculate the touch position based on the detection signal from pressure sensor 880A.

[0101] In some embodiments of this application, the electronic device 800 can further analyze the placement posture of the electronic device relative to the user based on the touch position detected by the pressure sensor 880A, thereby determining the current relative position of each speaker in the electronic device to the user.

[0102] Gyroscope sensors can be used to determine the motion attitude of electronic device 800. Magnetic sensors include Hall effect sensors. Accelerometers can detect the magnitude of acceleration of electronic device 800 in various directions (typically three axes). When electronic device 800 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the attitude of electronic device, and is applied to applications such as screen orientation switching and pedometers.

[0103] In some embodiments of this application, the electronic device 800 can analyze the placement posture of the electronic device relative to the user in real time based on at least one of a gyroscope sensor, a magnetic sensor, or an accelerometer, thereby determining the current relative position of each speaker in the electronic device to the user.

[0104] A touch sensor, also known as a "touch panel," can be located on a display screen 894. The touch sensor and display screen 894 together form a touchscreen, also called a "touch display." The touch sensor detects touch operations applied to or near it. In some embodiments, the touch sensor may also be located on the surface of the electronic device 800, in a different position than the display screen 894.

[0105] In some embodiments of this application, the electronic device 800 can further analyze the placement posture of the electronic device relative to the user based on the touch position detected by the touch sensor, thereby determining the current relative position of each speaker in the electronic device to the user.

[0106] The methods in each embodiment of this application can be implemented in an electronic device 800 having the above-described hardware structure.

[0107] The software system of an electronic device can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This application does not specifically limit this approach. It is understood that a layered architecture software system can divide the software into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces.

[0108] Please see Figure 9A The software system consists of the application layer, the application framework layer, and the hardware abstraction layer (HAL) from top to bottom.

[0109] The application layer can include a series of application packages. This application layer may include multiple application packages. For example, the application layer may include an audio application (not shown in the figure), which can play audio data, such as multi-channel audio sources. It should be understood that the audio application can be a music app or a video app, or other application capable of playing audio data. In some examples, it is used to implement... Figure 9A The module for processing "multi-channel audio source playback" can be an audio application in the application layer or a functional module within an audio application, and can be referred to as the first module.

[0110] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications within the application layer. The application framework layer includes predefined functions and algorithms.

[0111] In some embodiments of this application, the module for implementing multi-channel audio source playback can transmit the channel data (i.e., audio data) of the audio source that needs to be output externally to the application framework layer. The application framework layer may include an algorithm library or algorithm package, which has a module for implementing "playback link selection" (which may be referred to as the second module), that is, for selecting the playback link that supports the playback of multi-channel audio sources.

[0112] The HAL layer is used to connect the application framework layer and the underlying layer.

[0113] In the audio playback scenario described in this application embodiment, the HAL layer can create a playback path corresponding to the number of channels of the multi-channel audio source based on the multi-channel playback capability supported by the playback link selected by the application framework layer. It should be understood that the module in the HAL layer that creates the playback path can be referred to as the third module.

[0114] Furthermore, after creating multiple playback paths, the HAL layer can pass the channel data transmitted by the application framework layer to the underlying digital signal processor (DSP). The DSP then performs sound field widening and multi-channel merging management on the channel data to obtain the audio data corresponding to each speaker.

[0115] Each speaker is connected to a power amplifier (PA), which is used to transmit the audio data input from the digital signal processor (DSP) to the corresponding speaker for playback. It should be understood that... Figure 9AThe example given is only 8 speakers and 8 power amplifiers, and is not intended to limit the number of power amplifiers and speakers.

[0116] The method in the embodiments of this application will be described in detail below, in conjunction with the above hardware structure and software system structure:

[0117] 1. Respond to playback commands for multi-channel audio sources and create playback paths corresponding to the number of channels of the multi-channel audio source.

[0118] In some examples, an electronic device may include a first module in the application layer, a second module in the application framework layer, and a third module in the hardware abstraction layer. These multiple modules collaborate to create playback paths corresponding to the number of channels in a multi-channel audio source; this can be referred to as multi-module collaborative playback path creation. The first module is used to play the audio source, the second module is used to select the playback path, and the third module is used to create the playback path. The specific naming of each module is not limited. It should be understood that the first, second, and third modules may also have other functions, and are not limited to the functions mentioned above.

[0119] Now combined Figure 9A This section introduces the process of creating playback paths through multi-module collaboration. For example... Figure 9A As shown, Figure 9A Each arrow in the diagram represents a single audio signal (i.e., one channel signal or one audio data channel). A multi-channel audio source includes 8 channels of data. The module in the application layer used to implement multi-channel audio source playback (i.e., the first module) Figure 9A (The name is not shown in the text) can respond to playback commands to play multi-channel audio sources and pass these 8 channels of audio data to the application framework layer. The module in the application framework layer that implements playback link selection (i.e., the second module) Figure 9A The third module (not shown in the diagram) in the hardware abstraction layer can select a playback link that supports playing 8-channel data based on the transmitted 8-channel data. The third module in the hardware abstraction layer creates 8 playback paths based on the selected playback link.

[0120] It should be noted that the above is only a combination of... Figure 9A This section provides a brief introduction to the process of creating playback paths collaboratively across multiple modules. For a more detailed explanation of this process, please refer to the following... Figure 10 The relevant descriptions of steps S101-S105 in the text.

[0121] 2. Electronic devices send audio data for each channel of the multi-channel audio source based on the playback path.

[0122] It should be noted that electronic devices can send the original channel data of multi-channel audio sources to lower layers, or they can send data after preprocessing the original channel data; there is no limitation on this.

[0123] For example, such as Figure 9A As shown, the hardware abstraction layer in an electronic device creates each playback path and can send the channel data corresponding to the multi-channel audio source to the digital signal processor based on each playback path, realizing the pass-through of multi-channel data. For example, Figure 9A In the middle, the hardware abstraction layer can pass down 8-channel audio data.

[0124] 3. The electronic device performs three-dimensional sound field widening processing on all or part of the audio data sent from the playback path, and generates audio data corresponding to each speaker in the electronic device based on the multi-channel data after sound field widening processing.

[0125] For example, such as Figure 9A As shown, the digital signal processor performs multi-channel audio processing and multi-channel data management on the channel data sent from the playback path to generate audio data corresponding to each speaker.

[0126] Multi-channel audio processing refers to processing methods used to improve the sound output effect, specifically including sound field widening processing of multi-channel data. In some embodiments, in addition to sound field widening processing, other audio processing may be included, such as optimizing and fine-tuning data of different frequency bands (e.g., low frequency, mid frequency, and high frequency) in the channel data; this is not limited. Multi-channel data management refers to distributing the multi-channel data output by the module used to implement multi-channel audio processing to the speakers to generate audio data corresponding to each speaker. For example, Figure 9A There are a total of 8 speakers. The module for multi-channel data management generates 8 audio data streams, with each audio data stream corresponding to one speaker. It should be understood that multi-channel data management can include mixing processing, as well as channel data splitting processing (e.g., splitting the center channel data C into multiple sub-center channel data streams), or channel data discarding processing (e.g., discarding the subwoofer channel data SW), etc., without limitation.

[0127] In some examples, multi-channel audio processing and multi-channel data management can be performed by one module of the digital signal processor, or by at least two modules. For example, Figure 9A The system consists of two modules: one for multi-channel audio processing and the other for multi-channel data management.

[0128] It should be understood that this is only an example using a digital signal processor. In reality, other processing chips can also perform multi-channel audio processing and multi-channel data management on the channel data sent from the playback path to generate audio data corresponding to each speaker, and this is not limited to this.

[0129] 4. Electronic devices control each speaker to play corresponding audio data through a hardware interface that supports multi-channel playback.

[0130] For example, each speaker is connected to a power amplifier, such as... Figure 9A There are a total of 8 speakers, so there are 8 corresponding PAs. The digital signal processor can transmit the audio data corresponding to each speaker to the respective power amplifier through a hardware interface that supports multi-channel playback. After processing by each power amplifier, the audio data is input back to the respective speakers for playback, improving the audio output effect.

[0131] In some embodiments, the PA not only serves as a power amplifier but can also be used to adjust the operating status of the speaker.

[0132] Specifically, the digital signal processor includes modules for implementing PA data processing (i.e., Smart modules for implementing PA data processing / Intelligent modules for implementing PA data processing), such as... Figure 9B As shown. After the module for multi-channel data management generates eight audio data channels corresponding one-to-one with the eight speakers, these eight audio data channels can be input to the module for PA data processing. This PA data processing module performs IV feedback processing, that is, it detects the current and voltage of the speakers in real time, and adjusts the PA parameters in real time based on the current and voltage, ensuring the speakers always maintain the target operating state. The target operating state is the optimal operating state, equivalent to the state in which the speakers operate normally to their maximum extent. It can be understood that in the target operating state, the speaker's function can be maximized or even further enhanced, and the speaker will not malfunction due to maximizing its function.

[0133] It should be understood that Figure 9B This example only illustrates a single module for implementing PA data processing. In some examples, each PA may correspond to a separate module for implementing PA data processing; this is not a limitation.

[0134] The method described in this application will be further described below, using timing diagrams and hardware / software framework diagrams.

[0135] In some embodiments, please refer to Figure 10 A timing diagram of an audio playback method is provided. This method is applied to an electronic device in the embodiments of this application. The electronic device includes at least two speakers. The method specifically includes the following steps:

[0136] S101, the first module in the application layer obtains the playback instructions for the multi-channel audio source.

[0137] For example, the playback command can be generated by the user-triggered first module such as a music application or a video application, or it can be actively generated by the first module in the electronic device. For instance, in a scenario where a short video application plays automatically and continuously, the playback command can be actively generated.

[0138] S102, the first module in the application layer transmits N channels of first channel data of the multi-channel audio source to the application framework layer.

[0139] The first channel data is the original channel data from the multi-channel audio source. The number of first channels in a multi-channel audio source is N, where N is greater than or equal to 3. Therefore, the playback command indicates that the multi-channel audio source to be played includes N channels of audio data.

[0140] S103, the second module in the application framework layer can be selected to support playback links with N first channels.

[0141] S104, the second module in the application framework layer notifies the hardware abstraction layer of the selected playback link and N target channel data.

[0142] The above step S104 may include at least two parts of processing: playback link selection processing and data transmission processing. The following will describe these two parts of processing in more detail.

[0143] I. Playback Link Selection Processing:

[0144] Playback links are pre-configured in the software system of electronic devices. For example, the provider of the playback link pre-configures or encapsulates a playback link with audio playback capabilities based on resource information or parameter information required for audio playback. It should be understood that the playback link can provide relevant resource information or other supporting information required for audio playback, such as memory buffer, supported sampling rate, bit width, etc.; therefore, the playback link has the capability to support audio playback. Different playback links support different audio playback capabilities. If the resource information or parameter information of some playback links cannot support the playback of multi-channel audio sources, it means that the playback link does not have the capability to support the playback of multi-channel audio sources.

[0145] The ability of a playback link to support audio playback is, to some extent, reflected in the number of audio channels it can support. For example, some playback links only support two-channel audio sources, while others have stronger audio playback capabilities, supporting 6, 8, 12, or even more audio sources.

[0146] Therefore, after receiving the channel data of the multi-channel audio source from the application layer, the channel number recognition module in the application framework layer can parse the channel data in the multi-channel audio source to identify the number of channels in the multi-channel audio source. Furthermore, the second module in the application framework layer can select a playback link that matches the number of channels, i.e., a playback link that supports multi-channel playback. In some examples, the channel number recognition module can also be integrated into the second module, enabling the second module to also have the ability to identify the number of channels.

[0147] In some examples, after identifying the number of channels in a multi-channel audio source, the application framework layer can select the playback path that supports multi-channel playback in any of the following ways:

[0148] Method (1): Select a playback link that matches the number of channels from a set of pre-configured playback links.

[0149] For example, an electronic device may have multiple playback links pre-configured, and these playback links may support different numbers of audio channels. For instance, some playback links may support 2-channel data playback, while others may support 6-channel upscaling data playback. The application framework layer can then select a playback link that matches the number of channels of the currently playing multi-channel audio source.

[0150] In some examples, the electronic device has a configuration file that records the correspondence between the number of audio channels and playback link identifiers. The second module of the application framework layer can select a playback link identifier that matches the number of first audio channels of the first audio source based on the configuration file.

[0151] Furthermore, if there are multiple playback link identifiers that match the number of channels of the currently playing multi-channel audio source, then any playback link identifier can be selected from them or selected according to preset rules.

[0152] Method (2): Select the default playback link.

[0153] For example, a default playback path can be pre-defined in an electronic device. This default playback path supports multi-channel playback. After confirming that the currently playing audio source is a multi-channel audio source based on the identified number of channels, the application framework layer can select the default playback path.

[0154] In some examples, a playback link that supports multi-channel playback may include a native playback link that supports multi-channel playback, and / or a custom playback link that supports multi-channel playback.

[0155] The native playback link refers to the playback link built into the software system of the electronic device, such as direct or offload, which are playback links that support multi-channel playback built into the software system. It should be understood that the solution in this application innovatively applies the native playback link that supports multi-channel playback to the scenario of three-dimensional sound field playback of multi-channel audio sources. In the solution of this application, the interface and invocation with the native playback link are implemented at both the application framework layer and the hardware abstraction layer, together forming the audio external playback solution that achieves the three-dimensional sound field playback effect in the embodiments of this application. This is the result of creative effort.

[0156] A custom playback link is a playback link defined by the manufacturer of an electronic device. Specifically, the manufacturer pre-configures or encapsulates a playback link that supports audio playback capabilities based on some resource information or parameter information required to achieve audio playback; this playback link is referred to as a custom playback link.

[0157] It should be noted that, at the data processing level, the second module in the application framework layer selects the playback link identifier. The application framework layer notifies the hardware abstraction layer of the selected playback link by passing the playback link identifier.

[0158] II. Data Distribution and Processing:

[0159] The target channel data sent from the application framework layer to the hardware abstraction layer can be the original first channel data from a multi-channel audio source or the second channel data after preprocessing the first channel data. The target channel data is also N-channel.

[0160] In some examples, preprocessing may include volume control. For instance, a second module in the application framework layer can increase the volume of each channel's data through volume control, so that subsequent processing in lower layers is based on the increased channel data. This can, to some extent, improve the loudness of the final speaker output and optimize the sound output effect. See [link to relevant documentation] for details. Figure 11 .like Figure 11 As shown, the modules in the application framework layer can select playback links and process volume, and send the processed channel data (second channel data) to the hardware abstraction layer.

[0161] In other examples, preprocessing may not be limited to volume processing; it may also include other preprocessing such as noise reduction. There are no restrictions on this.

[0162] S105, the third module in the hardware abstraction layer, can create N playback paths based on the selected playback link.

[0163] It should be understood that the playback link has the capability to support multi-channel playback. Therefore, after the application framework layer notifies the hardware abstraction layer of the selected playback link, the third module in the hardware abstraction layer can call the playback link interface corresponding to that playback link to create a playback path that matches the number of the first channel using the resource data or parameters supported by the playback link.

[0164] by Figure 9A For example, the hardware abstraction layer can use the support provided by the playback link to allocate a memory buffer that supports the transmission of 8 channels of audio data, determine the sampling rate and bit width that support the transmission of 8 channels of audio data, and thus achieve the purpose of creating 8 playback paths.

[0165] S106, the third module in the hardware abstraction layer sends N target channel data to the multi-channel data processing module in the digital signal processor based on N playback paths.

[0166] The multi-channel data processing module is the module that implements multi-channel audio effect processing and multi-channel data management. It should be understood that the name is not limited to "multi-channel data processing module". As long as it can implement multi-channel audio effect processing and multi-channel data management, it belongs to the multi-channel data processing module mentioned in the embodiments of this application.

[0167] S107, the multi-channel data processing module in the digital signal processor performs three-dimensional sound field broadening on at least a portion of the target channel data to obtain N-channel third channel data.

[0168] It should be understood that, through the adaptive creation of playback paths in this embodiment, the number of target channel data received by the digital signal processor is the same as the number N of the first channel of the multi-channel audio source. For example, if the multi-channel audio source has 8 channels of data, then the received target channel data will also be 8 channels. Furthermore, the digital signal processor can perform three-dimensional sound field widening processing on all or part of the target channel data to obtain N channels of third channel data.

[0169] As mentioned above, it is not required to perform sound field widening on every single target channel data. That is, only some of the target channel data in the N target channel data may require three-dimensional sound field widening, while the target channel data in other channels may not. Therefore, the channel data after three-dimensional sound field widening, along with the target channel data without sound field widening, can be used together as the third channel data, resulting in N third channel data. Subsequently, based on this N third channel data, M audio data corresponding to M speakers are generated.

[0170] It should be noted that the sound field width of each third channel data is not less than the sound field width of the corresponding target channel data, and at least one third channel data has a sound field width greater than the corresponding target channel data. That is, if the target channel data has undergone sound field widening, then the third channel data after sound field widening will necessarily be larger than the target channel data before sound field widening. If the target channel data has not undergone sound field widening, then treating it as a third channel data will result in an unchanged sound field width.

[0171] by Figure 9A Taking the example, the hardware abstraction layer transmits 8 target channel data (i.e., N=8) downwards. Assume that 2 of these 8 target channel data do not require sound field widening, denoted as target channel data 1 and target channel data 2, while the other 6 target channel data require sound field widening. Then, after processing by S107, 8 third channel data can be obtained. These 8 third channel data can include 6 channel data after sound field widening, as well as target channel data 1 and target channel data 2 without sound field widening.

[0172] Furthermore, let's take the 5.1 channel audio source mentioned earlier as an example. Assume that the channel data for the front left channel (FL), front right channel (FR), left surround channel (SL), and right surround channel (SR) are widened, while the channel data for the center channel (C) and subwoofer channel (SW) do not require widening. Then, the 6-channel third-channel data would include the widened front left channel (FL'), front right channel (FR'), left surround channel (SL'), and right surround channel (SR'), as well as the unwidened center channel (C) and subwoofer channel (SW).

[0173] In some examples, the digital signal processor can determine which target channel data requires sound field widening and which does not, based on the channel orientation corresponding to the target channel data. For example, target channel data under the center channel or subwoofer channel does not require sound field widening. Target channel data under the left and right front channels, left and right surround channels, and top channels require sound field widening. In other embodiments, the module used to implement multi-channel audio processing can also perform sound field widening on all or pre-specified target channel data under certain channel orientations according to default rules; this is not limited.

[0174] It should be noted that in other examples, three-dimensional sound field broadening can also be performed on all target channel data, and there is no limitation on this.

[0175] The specific processing methods for three-dimensional sound field broadening will be discussed later. Figure 12 Step S126 will be described in detail.

[0176] S108, the multi-channel data processing module in the digital signal processor generates M channels of audio data corresponding one-to-one with M speakers in the electronic device based on N channels of third channel data.

[0177] In some examples, the digital signal processor (DSP) can employ a location-matching principle, generating audio data corresponding to each speaker based on N-channel third-channel data mixing. Specifically, the DSP mixes the third-channel data, matching the channel orientation with the speaker orientation, to generate the speaker's audio data. The following will... Figure 12 Step S128 in the text describes how to generate audio data for each speaker based on the orientation matching principle.

[0178] In other examples, the channel systems currently known in the industry and their individual channels are relatively well-defined. Therefore, location identification and matching can be omitted. Instead, a pre-defined binding relationship between the channels and speakers can be established. Based on this binding relationship, the third channel data corresponding to each speaker is determined from N third channel data, thereby generating the audio data corresponding to each speaker based on its corresponding third channel data. Furthermore, this binding relationship can be dynamic, dynamically updated according to the relative position between the speaker and the user.

[0179] S109, the multi-channel data processing module in the digital signal processor sends M channels of audio data based on a hardware interface that supports multi-channel playback.

[0180] S1010, M speakers play the corresponding audio data.

[0181] The above solution makes corresponding improvements to the application layer, application framework layer, hardware abstraction layer, and digital signal processor. Through the cooperation of the application layer, application framework layer, hardware abstraction layer, and digital signal processor, a three-dimensional sound field playback effect is achieved without the need for additional dedicated cinema-grade hardware.

[0182] In some embodiments, please refer to Figure 12 The document provides a timing diagram for another audio playback method, which is also applied to the electronic device in the embodiments of this application. The electronic device includes at least two speakers, and the method specifically includes the following steps:

[0183] S120, the first module in the application layer obtains the playback instructions for the multi-channel audio source.

[0184] S121, the first module in the application layer transmits N channels of first channel data of the multi-channel audio source to the application framework layer.

[0185] S122, the second module in the application framework layer can select a playback link that supports the number N first channels and supports multi-channel playback, and perform volume processing on the N first channel data to obtain N second channel data.

[0186] The data for the second channel is greater than the data for the first channel before volume processing.

[0187] S123, the second module in the application framework layer notifies the hardware abstraction layer of the selected playback link and N second channel data.

[0188] S124, the third module in the hardware abstraction layer, creates N playback paths based on the selected playback link.

[0189] S125, the third module, based on N playback paths, sends N second-channel data to the module in the digital signal processor used to implement multi-channel audio effects processing.

[0190] In this embodiment, the digital signal processor includes a module for implementing multi-channel audio effect processing and a module for implementing multi-channel data management. Specifically, the modules for multi-channel audio effect processing and multi-channel data management can be... Figure 10 The multi-channel data processing module has two sub-modules. The hardware abstraction layer sends N second-channel data to the module used to implement multi-channel audio effects processing based on N playback paths.

[0191] S126 is a module for implementing multi-channel audio processing. Based on the channel orientation, it widens the sound field of at least a portion of the second channel data to obtain N-channel third channel data.

[0192] As mentioned above, it is not limited to performing sound field widening on every second channel data. The channel data after sound field widening of the second channel data, along with the second channel data without sound field widening, can be combined as the third channel data, resulting in N channels of third channel data. It should be understood that in other examples, three-dimensional sound field widening can also be performed on all second channel data; this is not a limitation.

[0193] For example, the module for implementing multi-channel audio processing can determine which second-channel data requires sound field widening and which does not, based on the channel orientation corresponding to the second-channel data. For instance, the second-channel data under the center channel or subwoofer channel does not require sound field widening. The second-channel data under the left and right front channels, left and right surround channels, and top channels require sound field widening. In other embodiments, the module for implementing multi-channel audio processing can also widen the sound field of all or a pre-specified portion of the second-channel data according to default rules; this is not limited.

[0194] For second channel data that requires sound field widening, the digital signal processor can perform horizontal and / or vertical sound field widening based on the channel orientation of the second channel data to obtain the third channel data corresponding to the second channel data.

[0195] Horizontal sound field widening refers to the widening of the sound field in the horizontal direction. After horizontal sound field widening, the output position of the third channel data is farther from the electronic device in the horizontal direction compared to the corresponding second channel data. Vertical sound field widening refers to the widening of the sound field in the vertical direction. After vertical sound field widening, the output position of the third channel data is farther from the electronic device in the vertical direction compared to the corresponding second channel data. It should be understood that both the horizontal and vertical directions in the embodiments of this application can be determined with reference to the world coordinate system.

[0196] For ease of understanding, let's take the third channel data, which requires sound field widening and includes the left and right channel data, as well as the sky channel data, as an example. Figure 13 The diagram illustrates the difference before and after sound field widening. Please refer to [link / reference]. Figure 13 The dashed lines in the diagram represent the speaker output positions (i.e., the output positions of the sound signals) for the channel data. The left channel data output position before sound field widening is S1, and after widening, it outputs at S11. Position S11 is further from the electronic device horizontally than position S1, meaning it's further to the left. The right channel data output position before sound field widening is S2, and after widening, it outputs at S22. Position S22 is further from the electronic device horizontally than position S2, meaning it's further to the right. This creates a stronger sound envelope effect on the left and right sides of the user, enhancing the spatial sense of the sound on both sides. The sky channel data output position before sound field widening is S3, and after widening, it outputs at S33. Position S33 is higher than position S3, thus more closely resembling the effect of sound emanating from above the user's head.

[0197] In some examples, sound field widening in different directions, such as horizontal and / or vertical sound fields, can be achieved using head transfer functions corresponding to each sound field widening direction. For instance, the head transfer function matrix can be convolved with the data from each channel to achieve sound field rendering in different directions, thereby achieving the goal of three-dimensional sound field widening.

[0198] S127, the module for implementing multi-channel audio processing, transmits N channels of third-channel data to the module in the digital signal processor for implementing multi-channel data management.

[0199] S128, for each speaker, the module for multi-channel data management selects the third channel data that matches the orientation, and generates the audio data corresponding to that speaker based on the matching third channel data.

[0200] It should be noted that before introducing how to generate the audio data for each speaker, the following will first introduce the channel orientation and speaker orientation:

[0201] As mentioned above, in the embodiments of this application, channel orientation refers to the desired sound direction and position when playing channel data. For example, the channel orientation of the left channel data is intended to present the effect of sound playing on the left side of the user's ear, and the channel orientation of the right channel data is intended to present the effect of sound playing on the right side of the user's ear.

[0202] The speaker's orientation refers to its current relative position to the user, or, more specifically, its relative position to the user given the current orientation of the electronic device relative to the user. It should be understood that speakers are generally fixed within electronic devices, and the relative orientation of the electronic device to the user will change regardless of the device's orientation.

[0203] The following example uses an electronic device with four speakers, combined with... Figure 14 This illustrates different placement positions and the relative positions of the speaker and the user in each placement position.

[0204] Please see Figure 14 In (a), the electronic device is positioned in its first orientation from the user's perspective. The camera on the electronic device is facing upwards, meaning that from the user's viewpoint, the electronic device is currently positioned in front with its camera facing upwards.

[0205] Please see Figure 14 (b) shows the second orientation of the electronic device from the user's position as the observation point. In this orientation, the camera of the electronic device is facing downwards, meaning that from the user's perspective, the electronic device is currently positioned in front with the camera facing downwards.

[0206] like Figure 14 As shown in (a), with the user's position as the observation point, in the first placement posture, speaker SPK1 is located at the upper left corner of the electronic device, speaker SPK2 is located at the upper right corner of the electronic device, speaker SPK3 is located at the lower left corner of the electronic device, and speaker SPK4 is located at the lower right corner of the electronic device.

[0207] like Figure 14 As shown in (b), with the user's position as the observation point, in the second placement posture, the orientation of each speaker relative to the user (i.e., the relative orientation with respect to the user) compared to... Figure 14 (a) has changed. Specifically, in the second orientation, speaker SPK1 is located at the lower right corner of the electronic device, speaker SPK2 is located at the lower left corner of the electronic device, speaker SPK3 is located at the upper right corner of the electronic device, and speaker SPK4 is located at the upper left corner of the electronic device.

[0208] The following describes in detail how to generate the audio data corresponding to each speaker, specifically including the following steps (1) to (2):

[0209] (1) For each speaker, select the third channel data whose channel orientation matches the orientation of the speaker.

[0210] Specifically, for each speaker, the module for multi-channel data management can select third-channel data from the third-channel data of the first number of channels. The selected third-channel data must have a channel orientation that matches the speaker's orientation or meets a preset orientation proximity condition. The selected third-channel data is then matched to the speaker's orientation. Subsequently, the module for multi-channel data management can generate audio data corresponding to that speaker based on the matched third-channel data.

[0211] The following combination Figure 14 Example of the speaker's position relative to the user in (a) is shown below.

[0212] In the first placement position, the speaker SPK1 is located in the upper left corner of the electronic device. Therefore, the channel data of the upper left channels, such as the front left channel, the top front left channel, and the top rear left channel, are matched with the position of SPK1 in the upper left corner.

[0213] Since the speaker SPK2 is located in the upper right corner of the electronic device, the channel data of the upper right channels, such as the front right channel, the top front right channel, and the top rear right channel, are matched with the position of SPK2 in the upper right corner.

[0214] Since the speaker SPK3 is located in the lower left corner of the electronic device, the channel data for the left surround channel, the left rear surround channel, and other channels located on the lower left or rear left side are matched with the position of the SPK3 in the lower left corner.

[0215] Since the speaker SPK4 is located in the lower right corner of the electronic device, the channel data for the right surround channel, the right rear surround channel, and other channels located on the lower or rear right side are matched with the position of the SPK4 in the lower right corner.

[0216] (2) For each speaker, mix the audio based on the matching third channel data to generate the audio data corresponding to that speaker.

[0217] Specifically, for each speaker, after selecting the third channel data that matches the speaker's location, the third channel data can be mixed using methods such as upmixing or downmixing to generate the corresponding audio data for that speaker. It should be noted that upmixing or downmixing is performed when there isn't a uniquely matching third channel data for each speaker. If each speaker has a uniquely matching third channel data, then no mixing is needed, and each speaker can be directly controlled to play the uniquely matching third channel data.

[0218] The following examples, 1 and 2, will illustrate the overmixing and undermixing processes.

[0219] Example 1: More than one third channel of data is matched with the speaker, downmixed.

[0220] Down-mixing refers to reducing the number of channels, that is, mixing data with a large number of channels into data with a small number of channels. For example, mixing 6-channel data into 4-channel data is down-mixing.

[0221] Specifically, for any given speaker, if there are more than one (i.e. multiple) third-channel data channels that match that speaker, then it is necessary to use downmixing to merge the multiple third-channel data channels that match it into one channel, thus obtaining the audio data corresponding to that speaker.

[0222] Example 2: Some speakers do not have matching third channel data, resulting in upmixing.

[0223] Up-mixing refers to increasing the number of channels, that is, mixing data with fewer channels into data with more channels. For example, mixing 6-channel data into 8-channel data is up-mixing.

[0224] Specifically, some loudspeakers may lack third-channel data that matches their location. Therefore, for these loudspeakers lacking matching third-channel data, the digital signal processor can generate channel data with a location matching the loudspeaker's location based on existing third-channel data, and determine the corresponding audio data for the loudspeaker based on the newly generated channel data. For example, the newly generated channel data can be directly identified as the audio data corresponding to the loudspeaker, or it can be merged with other third-channel data (such as center channel data) to generate the audio data corresponding to the loudspeaker.

[0225] Similarly, it is not limited to generating the audio data corresponding to the speaker based solely on the third channel data that matches the speaker's orientation. Alternatively, the matching third channel data can be combined with other third channel data (such as center channel data) to generate the audio data corresponding to the speaker.

[0226] S129, a module for implementing multi-channel data management, sends corresponding audio data to M speakers based on a hardware interface that supports multi-channel playback.

[0227] S1210, M speakers play the corresponding audio data.

[0228] The above solution incorporates improvements to the application layer, application framework layer, hardware abstraction layer, and digital signal processor. Through the collaboration of these layers, a three-dimensional sound field playback effect is achieved without requiring additional specialized cinema-grade hardware. Furthermore, the interaction between these hardware and software modules enables more specific processing, such as volume control, sound field widening based on sound field orientation, and channel data allocation to speakers for mixing based on orientation matching principles. These improvements enhance the accuracy and spatial sense of the audio data generated by each speaker, resulting in a superior three-dimensional sound field playback effect.

[0229] In some embodiments, the multi-channel audio source includes center channel data. Therefore, the third channel data obtained using the adaptive playback path creation pass-through method described in the above embodiments, which does not lose data, also includes center channel data and non-center channel data. Here, non-center channel data refers to the third channel data other than the center channel data within the first number of third channel data. As mentioned above, the module used for multi-channel data management can combine center channel data with the audio data generated from the third channel data when mixing to produce the corresponding audio data for the speakers. This will be discussed further below. Figures 15A to 15C This describes how to mix the third channel data, which is matched with the speaker, with the center channel data to generate the audio data corresponding to the speaker.

[0230] Specifically, after obtaining the third channel data (the number of first channels after audio processing), the electronic device (e.g., a digital signal processor within the electronic device) can split the center channel data into at least two sub-center channel data. The split sub-center channel data is then distributed to the speakers. It should be understood that the electronic device can distribute the center channel data to all speakers or only to a subset of the speakers.

[0231] For a speaker allocated sub-center channel data, the electronic device can merge the non-center channel data and sub-center channel data allocated to that speaker to generate the corresponding audio data for that speaker. For example, the allocated non-center channel data and sub-center channel data can be weighted and merged according to appropriate weights to obtain the audio data corresponding to that speaker. In other examples, the allocated non-center channel data and sub-center channel data can be directly added and merged without considering weights. This is not limited to this approach.

[0232] Furthermore, for speakers that have not been assigned sub-center channel data, the electronic device can weighted merge or directly merge the non-center channel data assigned to that speaker to generate the corresponding audio data for that speaker. As mentioned above, the non-center channel data assigned to a speaker refers to the non-center channel data whose channel orientation matches the orientation of the speaker.

[0233] Next, we will explain in more detail how to mix the audio by combining two different splitting methods: splitting the center channel data to all speakers or splitting it to only some speakers.

[0234] Split Method 1: Split the center channel data to all speakers.

[0235] Specifically, the electronic device can evenly distribute the center channel data to each speaker, meaning each speaker receives the same volume of sub-center channel data. Alternatively, the electronic device can not distribute it evenly, but instead allocate higher-volume sub-center channel data to some speakers and lower-volume sub-center channel data to others. For example, combining... Figure 14 As described in (a), a larger volume sub-center channel data can be assigned to the upper left speaker SKP1 and the upper right speaker SKP2, while a smaller volume sub-center channel data can be assigned to the lower left speaker SKP3 and the lower right speaker SKP4, thereby creating the effect of the center sound being emitted from a position slightly above the center of the electronic device.

[0236] Please see Figure 15A Taking an electronic device with four speakers and a 7.1.4 channel audio source as an example, the following example illustrates the use of splitting method 1 for mixing:

[0237] It should be understood that Figures 15A to 15C as well as Figure 16 The main focus is on describing how to mix and generate audio data corresponding to each speaker. The solution in this application embodiment is to pass through by adaptively creating a playback path that matches the number of channels of the multi-channel audio source. Therefore, the structure of the application framework layer and the HAL layer is not shown.

[0238] like Figure 15AAs shown, the 7.1.4-channel audio source includes 12 channels of data. It should be noted that the channel data of a multi-channel audio source can be pre-processed (e.g., volume control) before being sent to the digital signal processor, or it can be sent without pre-processing. For the sake of brevity, Figure 15A In this context, the data is represented by the following 12 channels: Front Left Channel (FL), Front Right Channel (FR), Center Channel (C), Left Surround Channel (SL), Right Surround Channel (SR), Left Rear Surround Channel (LB), Right Rear Surround Channel (RB), Subwoofer Channel (SW), Top Front Left Channel (TFL), Top Front Right Channel (TFR), Top Rear Left Channel (TRL), and Top Rear Right Channel (TRR).

[0239] In fact, if the data from each channel of a multi-channel audio source is preprocessed before being sent to a digital signal processor, then... Figure 15A The terms "FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR" represent pre-processed channel data (i.e., second channel data). If no pre-processing is performed and the channel data from the multi-channel audio source is directly sent to the digital signal processor, then... Figure 15A The terms "FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR" represent the original channel data (i.e., the first channel data) of a multi-channel audio source. It should be understood that... Figure 15B , Figure 15C and Figure 16 That is also the case, and will not be elaborated further below.

[0240] like Figure 15A As shown, the four speakers include SPK1 to SPK4. Figure 15A In the arrangement of the electronic devices shown, speaker SPK1 is located in the upper left corner, speaker SPK2 in the upper right corner, speaker SPK3 in the lower left corner, and speaker SPK4 in the lower right corner. The module for multi-channel data management includes data management modules for each of the four speakers, namely modules for managing SPK1 to SPK4. Each data management module is responsible for generating audio data for one speaker. Each speaker is connected to a power amplifier PA, resulting in PA1 to PA4.

[0241] During the multi-channel audio processing stage, FR and FL can be input to the front channel processing module, SL, SR, LB and RB can be input to the rear channel processing module, and TFL, TFR, TRL and TRR can be input to the overhead channel processing module for audio processing or sound field widening, resulting in FR', FL', SL', SR', LB', RB', TFL', TFR', TRL' and TRR'.

[0242] During the multi-channel data management phase, since the electronic device does not have a separate subwoofer speaker, the input SW is discarded by the low-frequency channel processing module. The module for multi-channel data management can input C to the center channel processing module for processing, splitting C into multiple sub-center channel data, for example, splitting it into four sub-center channel data, resulting in C1 to C4. This module can then allocate the four sub-center channel data to the four speakers. For example, C1 can be allocated to SPK1, C2 to SPK2, C3 to SPK3, and C4 to SPK4. It should be understood that the four center channel data (C1 to C4) can be equal (i.e., the center channel data C is divided evenly) or unequal (i.e., the center channel data C is not evenly divided); this is not limited.

[0243] For SPK1, since SPK1 is located at the upper left corner of the electronic device, which is also relatively to the upper left of the user, in addition to allocating the center channel data C1, the module for multi-channel data management can also allocate channel data (e.g., FL', TFL', and TRL') that are close to the upper left or upper left corner to the module for SPK1 data management. Furthermore, the module for SPK1 data management can combine FL', TFL', TRL', and C1 to generate the audio data corresponding to speaker SPK1. The module for SPK1 data management can then input the audio data corresponding to speaker SPK1 to PA1, so that PA1 can control speaker SPK1 to play the audio data.

[0244] Similarly, for SPK2, since SPK2 is located in the upper right corner of the electronic device, which is relatively to the upper right of the user, in addition to allocating the center channel data C2, channel data located close to the upper right or upper right corner (such as FR', TFR', and TRR') can also be allocated to the module used to implement SPK1 data management. Then, FR', TFR', TRR', and C2 can be combined to generate the audio data corresponding to speaker SPK2. The module used to implement SPK2 data management can input the audio data corresponding to speaker SPK2 to PA2, so that PA2 can control speaker SPK2 to play the audio data.

[0245] For SPK3, since it is located in the lower left corner of the electronic device, which is also relatively to the lower left of the user, in addition to allocating the center channel data C3, channel data located close to the lower left or lower left corner (such as SL' and LB') can also be allocated to the module for SPK3 data management. Then, C3, SL', and LB' can be combined to generate the audio data corresponding to the speaker SPK3. The module for SPK3 data management can input the audio data corresponding to the speaker SPK3 to PA3, so that PA3 can control the speaker SPK3 to play the audio data.

[0246] For SPK4, since it is located in the lower right corner of the electronic device, which is also relatively to the lower right of the user, in addition to allocating the center channel data C4, channel data located close to the lower right or lower right corner (such as SR' and RB') can also be allocated to the module used for SPK1 data management. Then, C4, SR', and RB' can be combined to generate the audio data corresponding to speaker SPK4. The module used for SPK4 data management can input the audio data corresponding to speaker SPK4 to PA4, so that PA4 can control speaker SPK4 to play the audio data.

[0247] In some examples, as mentioned above, the digital signal processor may also include a module for implementing PA data processing, with each PA corresponding to a separate module for PA data processing. See details in [link to documentation]. Figure 15B Each of the four speakers is connected to one PA (Power Amplifier). Each PA can correspond to a module for PA data processing, i.e., there are PA1 data processing module to PA4 data processing modules. Figure 15B As shown, the module for SPK data management corresponding to each speaker can input the corresponding four audio data channels to the corresponding module for PA data processing for PA control, and then input them to each PA for transmission to the speaker. Taking SPK1 as an example, after the module for SPK1 data management generates one audio data channel corresponding to SPK1, it can input this generated audio data channel to the PA1 data processing module. After the PA1 data processing module adjusts the parameters of PA1 based on the IV feedback mechanism, PA1 amplifies the audio data and then inputs it to the speaker SPK1 for playback.

[0248] Taking the weighted merging of channel data allocated to loudspeakers as an example, combined with Figure 15A or Figure 15B The following diagram illustrates the allocation and merging of audio channel data:

[0249] SPK1:FL'+0.707*C1+0.707*TFL'+0.707*TRL'

[0250] SPK2:FR'+0.707*C2+0.707*TFR'+0.707*TRR'

[0251] SPK3: SL'+BL'+0.707*C3

[0252] SPK4: SR'+BR'+0.707*C4

[0253] Here, 0.707 is the weight. It should be noted that 0.707 is used here only as an example to illustrate the weight, and should not be used to limit the value of the weight.

[0254] Splitting Method 2: Split the center channel data to some speakers.

[0255] It should be understood that, given the current orientation of the electronic device relative to the user, the speakers allocated to the sub-center channel data must be located on the left and right sides of the electronic device to create the effect of the center channel sound being positioned in the middle of the electronic device, such as a centered vocal effect. Specifically, some of the speakers allocated to the sub-center channel data can be speakers positioned relatively high relative to the user (i.e., the first type of speakers). It should be understood that speakers positioned relatively low belong to the second type of speakers.

[0256] Combination Figure 15B For example, the speakers SPK1 and SPK2 in the upper left and upper right corners are positioned higher than the speakers SPK3 and SPK4 in the lower left and lower right corners. Therefore, the electronic device can split the center channel data into two sub-center channel data streams, and then distribute each sub-center channel data stream to the upper left speaker SPK1 and the upper right speaker SPK2 respectively, thus creating the effect of the center channel sound emanating from a position slightly above the center of the electronic device. It should be understood that the electronic device can distribute the center channel data evenly or unevenly among the speakers; there is no limitation on this.

[0257] To make it easier to understand, we will use an electronic device with four speakers playing a 7.1.4 channel audio source as an example for illustration.

[0258] Please see Figure 15C , Figure 15C The placement of electronic devices relative to the user's posture is as described above. Figure 15B Therefore, the orientation of each speaker is also consistent with the above. Figure 15B The above is consistent with the central theme, so I will not elaborate further.

[0259] In the multi-channel audio processing stage, the processing of SW, FR, FL, SL, SR, LB, RB, TFL, TFR, TRL, and TRR is the same as described above. Figure 15B The processing is the same, therefore, after sound effect processing or sound field widening, we can obtain FR', FL', SL', SR', LB', RB', TFL', TFR', TRL' and TRR'.

[0260] In the multi-channel data management phase, with Figure 15B The difference is that, Figure 15C The input C to the center channel processing module is split into two sub-center channel data streams, denoted as C5 and C6. That is, sub-center channel data is only allocated to the two upper speakers, SPK1 and SPK2. For example, C5 is allocated to SPK1, C6 to SPK2, and no sub-center channel data is allocated to SPK3 and SPK4. It should be understood that the volume of these two center channel data streams, C5 and C6, can be equal or unequal.

[0261] For SPK1, in addition to allocating the center channel data C5, channel data (such as FL', TFL', and TRL') located near the upper left or upper left corner can also be allocated to the module used for SPK1 data management. Then, FL', TFL', TRL', and C5 can be combined to generate the audio data corresponding to speaker SPK1. The module for SPK1 data management can input the audio data corresponding to speaker SPK1 to PA1, so that PA1 can control speaker SPK1 to play the audio data.

[0262] For SPK2, in addition to allocating the center channel data C6, channel data located near the upper right or upper right corner (such as FR', TFR', and TRR') can be allocated to the module used for SPK1 data management. Then, FR', TFR', TRR', and C6 can be combined to generate the audio data corresponding to speaker SPK2. The module used for SPK2 data management can input the audio data corresponding to speaker SPK2 to PA2, so that PA2 can control speaker SPK2 to play the audio data.

[0263] For SPK3, only the channel data (e.g., SL' and LB') located near the lower left or lower left corner need to be assigned to the module used for SPK3 data management. Then, SL' and LB' are merged to generate the audio data corresponding to the speaker SPK3. The module for SPK3 data management can input the audio data corresponding to the speaker SPK3 to PA3, so that PA3 can control the speaker SPK3 to play the audio data.

[0264] For SPK4, only the channel data (e.g., SR' and RB') located near the lower right or lower right corner need to be assigned to the module used for SPK1 data management. Then, SR' and RB' are combined to generate the audio data corresponding to speaker SPK4. The module for SPK4 data management can input the audio data corresponding to speaker SPK4 to PA4, so that PA4 can control speaker SPK4 to play the audio data.

[0265] Taking the weighted merging of channel data allocated to speakers as an example again, combined with Figure 15C The following diagram illustrates the allocation and merging of audio channel data:

[0266] SPK1:FL'+0.707*C5+0.707*TFL'+0.707*TRL'

[0267] SPK2:FR'+0.707*C6+0.707*TFR'+0.707*TRR'

[0268] SPK3: SL'+BL'

[0269] SPK4: SR'+BR'

[0270] As mentioned above, 0.707 is the weight.

[0271] The inventors of this application discovered through careful research that electronic devices are typically rotated by the user during use, causing the device's orientation relative to the user to constantly change, which in turn leads to changes in the speaker's position relative to the user. If the audio data input to each speaker is determined solely based on the initial orientation of the electronic device relative to the user when playing the audio source, the audio data played by each speaker when the electronic device is rotated will be inaccurate.

[0272] Now combined Figure 15B and Figure 16 Describe the problems caused by changes in the orientation of electronic devices.

[0273] As can be seen from the preceding text, Figure 15B The electronic devices are positioned in the first position relative to the user. Figure 16 In is Figure 15B Based on the first placement posture, the second placement posture is achieved after rotating the electronic device 180°. From... Figure 15B As can be seen, the 4-channel audio data under the sky channel is input to speakers SPK1 and SPK2. Assuming the 4-channel audio data under the sky channel represents the sound of an airplane flying overhead, the user will hear the sound of an airplane flying from left to right. When the electronic device is rotated 180°, as... Figure 16As shown, SPK1 and SPK2 will be rotated from the top left and top right corners to the bottom left and bottom right corners. If the sound channel data of the aircraft flight is continued to be input into SPK1 and SPK2, a sound effect of the aircraft flying in the opposite direction along the ground will be produced, which is obviously an error.

[0274] Therefore, to avoid the aforementioned problems, the inventors of this application propose the following solution: The electronic device can detect the real-time placement posture of the electronic device relative to the user and obtain the real-time orientation of each speaker under the real-time placement posture, that is, obtain the real-time orientation of each speaker relative to the user. Furthermore, for each speaker, the electronic device can merge the third channel data, which matches the real-time orientation of the speaker, as the corresponding audio data for that speaker. In this way, even if the electronic device is rotated, it can still accurately achieve a three-dimensional sound field playback effect.

[0275] like Figure 16 As shown, using the solution of this application, after SPK1 and SPK2 are rotated from the upper left and upper right corners to the lower left and lower right corners, the four-channel data input of the sky channel is no longer continued to SPK1 and SPK2. Instead, based on the current placement of the electronic device, the four-channel data of the sky channel is accurately input to SPK4 in the upper left corner and SPK3 in the upper right corner in the current placement position. The sound effect of an airplane flying from left to right in the sky is still played.

[0276] For example, an electronic device can detect the real-time orientation of the electronic device relative to the user using one of the following: a gyroscope sensor, an accelerometer, a magnetometer, a touch sensor, or a pressure sensor.

[0277] Furthermore, the inventors of this application discovered through numerous experiments that concurrent playback scenarios may exist in certain situations. That is, while playing a first audio source, a second audio source may be played concurrently. For example, if a phone call or voice prompt is received while playing music, the phone call or voice prompt becomes the second audio source played concurrently. Another example is that devices supporting concurrent playback can play video or audio from two platforms simultaneously. This application does not limit the concurrent playback scenarios in its embodiments; any scenario where two audio sources are played simultaneously falls within the scope of the concurrent playback scenarios mentioned in this application.

[0278] In concurrent playback scenarios, when an electronic device plays audio through a speaker, it can simultaneously play audio information from a first audio source and audio information from a second audio source. Specifically, the electronic device needs to mix and merge the channel data of the first and second audio sources to obtain the audio data corresponding to each speaker, and then play the corresponding audio data through the speakers. In this way, the audio data played by the speakers contains audio information from multiple audio sources.

[0279] In some embodiments, in a concurrent playback scenario, the first audio source includes first channel data of a first number of channels. After multi-channel audio effect processing by a digital signal processor, third channel data of the first number of channels is obtained. The second audio source indicating playback includes fourth channel data of a second number of channels. Then, the electronic device can allocate fourth channel data to speakers whose orientation matches the channel orientation of the fourth channel data. For each speaker, the electronic device can combine the third and fourth channel data allocated to that speaker to obtain audio data corresponding to that speaker. As can be seen from the above, the third channel data allocated to the speaker refers to the third channel data whose channel orientation matches the orientation of that speaker.

[0280] The following example uses an electronic device with four speakers, where the first audio source is a 7.1.4 channel audio source (including FL, FR, C, SL, SR, LB, RB, SW, TFL, TFR, TRL, and TRR), and the second audio source is a stereo audio source (including left channel data L and right channel data R). Figure 15B and Figure 17 This section provides a more detailed explanation of how to handle concurrent playback scenarios.

[0281] Please see Figure 17 , Figure 17 The placement of electronic devices relative to the user's posture is as described above. Figure 15B Therefore, the orientation of each speaker is also consistent with the above. Figure 15B The above is consistent with the central theme, so I will not elaborate further.

[0282] Furthermore, the allocation of the first sound source is the same as... Figure 15B Based on the principle of orientation matching, the left channel data L of the second sound source is assigned to SPK1 and SPK3 located on the left, and the right channel data R of the second sound source is assigned to SPK2 and SPK4 located on the right.

[0283] The final channel allocation for the four speakers is as follows:

[0284] SPK1: FL', C1, TFL', TRL', L

[0285] SPK2: FR', C2, TFR', TRR', R

[0286] SPK3: SL', BL', L

[0287] SPK4: SR', BR', R

[0288] Taking the weighted merging of channel data allocated to speakers as an example again, combined with Figure 17 The following diagram illustrates the allocation and merging of audio channel data:

[0289] SPK1:FL'+0.707*C1+0.707*TFL'+0.707*TRL'+L

[0290] SPK2:FR'+0.707*C2+0.707*TFR'+0.707*TRR'+R

[0291] SPK3: SL'+BL'+L

[0292] SPK4: SR'+BR'+R

[0293] It should be understood that the above is only an example of how the left channel data L can be assigned to SPK1 and SPK3. In practice, a portion of the left channel data L can be assigned to SPK1 and another portion to SPK3, and it is not required that both speakers SPK1 and SPK3 be assigned the complete left channel data L.

[0294] It should be noted that when playing a second audio source, the electronic device identifies the number of second channels. For example, the electronic device identifies the second channel data through the application framework layer. Then, based on the number of second channels, it determines whether the second audio source is a multi-channel source, and subsequently processes it differently depending on whether it is a multi-channel source. Specifically, this is described in the following two cases:

[0295] Scenario 1: The second audio source is a multi-channel audio source.

[0296] Specifically, if the second audio source is identified as a multi-channel audio source based on the number of second channels, then all processing steps prior to multi-channel data management (i.e., before mixing into audio data for the speakers) can be performed on it. Then, in the multi-channel data management stage, the channel data corresponding to the first and second audio sources are mixed according to channel orientation to generate audio data corresponding to each speaker.

[0297] Scenario 2: The second audio source is not a multi-channel audio source.

[0298] Specifically, if the second audio source is not a multi-channel audio source, the application framework layer will also select a playback path corresponding to the number of second channels of the second audio source. However, the application framework layer does not need to perform preprocessing such as volume adjustment on the second audio source; it can directly pass the channel data of the second audio source to the hardware abstraction layer and notify the hardware abstraction layer of the selected playback path. The hardware abstraction layer can also create a playback path corresponding to the selected playback path and directly pass down the channel data of the second audio source, except that there can be one or two playback paths instead of multiple paths. Furthermore, the digital signal processor can also avoid performing multi-channel audio effect processing (such as three-dimensional sound field widening) on ​​the second audio source; instead, it can directly mix the channel data of the first audio source after audio effect processing (such as the third channel data) with the original channel data of the second audio source according to the channel orientation.

[0299] It should be understood that if the second audio source is not a multi-channel audio source, it is largely not the primary audio source, but rather a temporary audio source played concurrently. Therefore, its sound playback effect does not require a cinematic three-dimensional sound field. Thus, in the above solution, a simplified processing logic can be executed for the second audio source that is not a multi-channel audio source, streamlining steps such as volume preprocessing and sound effect processing. This effectively saves system resources without affecting or only slightly impacting the user experience.

[0300] The above example illustrates concurrent playback scenarios using stereo as the second audio source. It should be understood that concurrent playback scenarios with mono, stereo, 5.1 channel, 7.1 channel, 5.1.2 channel, and 7.1.4 channel second audio sources can also follow a similar mixing logic. Essentially, in the final multi-channel data management stage, the channel data of multiple audio sources are mixed according to channel orientation; there are no limitations on this.

[0301] This application also provides a chip system applied to an electronic device. The chip system includes one or more processors, which are used to invoke computer instructions to cause the electronic device to perform various functions or steps described in the above method embodiments.

[0302] This application also provides a computer-readable storage medium including computer instructions that, when executed on the electronic device, cause the electronic device to perform the various functions or steps described in the method embodiments.

[0303] This application also provides a computer program product that, when run on a computer, causes the computer to perform the functions or steps described in the above method embodiments.

[0304] Through the above description of the embodiments, those skilled in the art can clearly understand that, for the sake of convenience and brevity, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0305] In the several embodiments provided in this application, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. Furthermore, the mutual coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0306] The units described as separate components may or may not be physically separate. A component shown as a unit can be one or more physical units; that is, it can be located in one place or distributed in multiple different locations. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0307] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0308] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, essentially or in other words, the parts that contribute to the prior art, or all or part of the technical solutions, can be embodied in the form of a software product. This software product is stored in a storage medium and includes several instructions to cause a device (which may be a microcontroller, chip, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0309] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio playback method, characterized in that, The method is applied to an electronic device, wherein the electronic device is provided with more than two speakers and has a hardware interface supporting multi-channel playback; the method includes: Receive a playback command from a first audio source; the first audio source includes first channel data of a first number of channels, and the first number of channels is greater than or equal to 3; In response to the playback command, a playback path of the first number of channels is created; each playback path is used to transmit one target channel data; the target channel data is either the first channel data or the second channel data; the second channel data is obtained by preprocessing the first channel data; For each target channel data transmitted by each playback path, a corresponding third channel data is generated; wherein, the sound field width of each third channel data is not less than the sound field width of the corresponding target channel data, and at least one third channel data has a sound field width greater than the sound field width of the corresponding target channel data. Based on the third channel data, audio data corresponding to each of the speakers is generated, and each of the speakers is controlled to play the corresponding audio data through the hardware interface.

2. The method according to claim 1, characterized in that, The electronic device includes an application layer, an application framework layer, and a hardware abstraction layer; receiving the playback command from the first audio source includes: The application layer receives the playback command from the first audio source; The step of creating a playback path for the first number of channels in response to the playback command includes: The application framework layer selects a playback link that supports multi-channel playback based on the first number of channels; The hardware abstraction layer creates a playback path for the first number of channels based on the selected playback link.

3. The method according to claim 2, characterized in that, The application framework layer selects playback links that support multi-channel playback based on the first number of channels, including: Based on the first number of channels, the application framework layer selects either a native playback link that supports multi-channel playback or a custom playback link that supports multi-channel playback. The native playback link refers to the playback link built into the software system of the electronic device.

4. The method according to claim 1, characterized in that, The electronic device also includes a digital signal processor; The generation of corresponding third channel data for each target channel data transmitted from each playback path includes: The digital signal processor generates corresponding third channel data for each target channel data transmitted from each playback path; The step of generating audio data corresponding to each of the speakers based on the third channel data, and controlling each of the speakers to play the corresponding audio data through the hardware interface, includes: The digital signal processor generates audio data corresponding to each of the speakers based on the third channel data, and controls each speaker to play the corresponding audio data through the hardware interface.

5. The method according to any one of claims 1-4, characterized in that, The process of generating audio data corresponding to each speaker based on the third channel data includes: For each speaker, audio data corresponding to the speaker is generated based on the third channel data in the third channel data of the first number of channels, where the channel orientation matches the orientation of the speaker. The speaker's orientation refers to its current relative position to the user; the relative position of the same speaker to the user varies depending on the placement of the electronic device relative to the user.

6. The method according to claim 5, characterized in that, The third channel data of the first channel number includes center channel data and non-center channel data; the speakers provided by the electronic device include first type speakers and second type speakers; wherein, in the current placement posture of the electronic device relative to the user, the position of the first type speakers is higher than the position of the second type speakers; The method further includes: The center channel data is split into sub-center channel data that correspond one-to-one with each speaker in the first type of speaker; For each speaker, the audio data corresponding to the speaker is generated based on the third channel data in the third channel data of the first number of channels, where the channel orientation matches the orientation of the speaker. This includes: For each speaker in the first type of speaker, the sub-center channel data corresponding to the speaker and the non-center channel data whose channel orientation matches the orientation of the speaker are merged to obtain the audio data corresponding to the speaker. For each speaker in the second type of speaker, the non-center channel data whose channel orientation matches the speaker's orientation are merged to obtain the audio data corresponding to the speaker.

7. The method according to claim 6, characterized in that, The first audio source is a 7.1.4 channel audio source; the first type of speaker includes a first speaker and a second speaker, and the second type of speaker includes a third speaker and a fourth speaker; in the current placement posture of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left and upper right corners of the electronic device, respectively, and the third speaker and the fourth speaker are located at the lower left and lower right corners of the electronic device, respectively. For each speaker in the first type of speaker, the sub-center channel data corresponding to the speaker and the non-center channel data whose channel orientation matches the speaker's orientation are merged to obtain the audio data corresponding to the speaker, including: The front left channel data, top front left channel data, and top rear left channel data from the third channel data of the first channel quantity are merged with the sub-center channel data corresponding to the first speaker to obtain the audio data corresponding to the first speaker; the front right channel data, top front right channel data, and top rear right channel data from the third channel data of the first channel quantity are merged with the sub-center channel data corresponding to the second speaker to obtain the audio data corresponding to the second speaker. For each speaker in the second type of speaker, the non-center channel data whose channel orientation matches the speaker's orientation are merged to obtain the audio data corresponding to the speaker, including: The rear left channel data and rear left surround channel data from the third channel data of the first channel quantity are combined to obtain the audio data corresponding to the third speaker; the rear right channel data and rear right surround channel data from the third channel data of the first channel quantity are combined to obtain the audio data corresponding to the fourth speaker.

8. The method according to claim 5, characterized in that, Before generating audio data corresponding to each speaker by using third channel data whose channel orientation matches the orientation of the speaker in the third channel data based on the first number of channels, the method further includes: The position of each speaker is obtained relative to the real-time placement posture of the electronic device with respect to the user.

9. The method according to claim 5, characterized in that, The method further includes: Obtain the fourth channel data of the second audio source; the second audio source is an audio source played concurrently with the first audio source; For each speaker, the audio data corresponding to the speaker is generated based on the third channel data in the third channel data of the first number of channels, where the channel orientation matches the orientation of the speaker. This includes: For each speaker, the third channel data whose channel orientation matches the speaker orientation and the fourth channel data whose channel orientation matches the speaker orientation are combined to obtain the audio data corresponding to the speaker.

10. The method according to claim 9, characterized in that, The fourth channel data of the second audio source includes left channel data and right channel data; the electronic device includes a first speaker, a second speaker, a third speaker, and a fourth speaker; wherein, in the current placement posture of the electronic device relative to the user, the first speaker and the second speaker are located at the upper left and upper right corners of the electronic device, respectively, and the third speaker and the fourth speaker are located at the lower left and lower right corners of the electronic device, respectively; the left channel data is assigned to the first speaker and the third speaker; the right channel data is assigned to the second speaker and the fourth speaker.

11. The method according to any one of claims 1-4 or 6-10, characterized in that, Each loudspeaker has a one-to-one corresponding power amplifier PA; The step of controlling each of the speakers to play corresponding audio data through the hardware interface includes: For each speaker, the audio data corresponding to the speaker is transmitted to the PA corresponding to the speaker through the hardware interface, and then the audio data is input to the speaker for audio playback through the PA; When the speaker is playing audio, the current and voltage of the speaker are detected in real time, and the parameters of the PA are adjusted in real time according to the current and voltage to keep the speaker in the target working state.

12. An electronic device, characterized in that, The electronic device includes a memory, more than two speakers, and one or more processors; the memory, the more than two speakers, and the processors are coupled; wherein the more than two speakers are used for external audio playback, and the memory stores computer program code, the computer program code including computer instructions, which, when executed by the processor, cause the electronic device to perform the method as described in any one of claims 1-11.

13. A chip system, characterized in that, The chip system is applied to an electronic device, the chip system including one or more processors, the processors being configured to invoke computer instructions to cause the electronic device to perform the method as described in any one of claims 1-11.

14. A computer-readable storage medium, characterized in that, Includes computer instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-11.

15. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method as described in any one of claims 1-11.

Images