Audio playing method and electronic device

Abstract

The application provides an audio playing method and an electronic device, and relates to the field of terminals. The method comprises the following steps: starting a first application program, obtaining a first stereo audio signal in response to a first operation of a user, extracting a second stereo audio signal of a first audio object and a third stereo audio signal of a second audio object in the first stereo audio signal, performing azimuth estimation on the second stereo audio signal and the third stereo audio signal to determine a center azimuth signal and an eccentric azimuth signal, performing enhancement rendering on the center azimuth signal, performing spatial rendering on the eccentric azimuth signal, obtaining a target stereo audio signal through downmixing according to the rendered center azimuth signal and eccentric azimuth signal, and playing the target stereo audio signal. The sound played by the electronic device according to the target stereo audio signal is more layered and immersive, and the user can have a better listening experience.

Description

Technical Field

[0001] This application relates to the field of terminals, and more particularly to an audio playback method and an electronic device. Background Technology

[0002] With the continuous development of mobile portable devices, a superior audio-visual experience has become an essential attribute of high-end products. Spatial audio, as one of the core components for enhancing the audio-visual experience, can broaden the sound image, enhance the sense of sound layering and distance, and help users obtain the ultimate immersive audio-visual experience.

[0003] However, traditional external audio solutions suffer from problems such as a narrow and flat sound field, resulting in poor layering and immersion in external audio. Summary of the Invention

[0004] This application provides an audio playback method and an electronic device that can improve the layering and immersiveness of audio played by the electronic device, giving users a better listening experience.

[0005] To achieve the above objectives, this application adopts the following technical solution:

[0006] In a first aspect, an audio playback method is provided, which may include: launching a first application; in response to a first user operation, acquiring a first stereo audio signal; extracting a second stereo audio signal of a first audio object and a third stereo audio signal of the second audio object from the first stereo audio signal, wherein the audio tracks of the first audio object and the second audio object are different; performing azimuth estimation on the second stereo audio signal to determine a first center azimuth signal and a first off-center azimuth signal; performing azimuth estimation on the third stereo audio signal to determine a second center azimuth signal and a second off-center azimuth signal; enhancing and rendering the first center azimuth signal to obtain a third center azimuth signal; enhancing and rendering the second center azimuth signal to obtain a fourth center azimuth signal; performing spatial rendering on the first off-center azimuth signal to obtain a third off-center azimuth signal; performing spatial rendering on the second off-center azimuth signal to obtain a fourth off-center azimuth signal; downmixing the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal to obtain a target stereo audio signal; and playing the target stereo audio signal.

[0007] Understandably, electronic devices extract objects from the original stereo audio signal (such as the first stereo audio signal) to obtain stereo signals of different audio objects (such as the second stereo audio signal and the third stereo audio signal). Then, they estimate the orientation of the stereo signals of different audio objects to obtain center orientation signals (such as the first center orientation signal and the second center orientation signal) and off-center orientation signals (such as the first off-center orientation signal and the second off-center orientation signal). Enhancement rendering is performed on the center orientation signal to increase its volume and bring its sound field closer, enhancing its immersive feel. Spatial rendering is performed on the off-center orientation signals to render the different orientations, distances, and heights corresponding to different off-center orientation signals, and to widen the sound field width of the off-center orientation signals. This allows users to perceive a clear sense of orientation of different audio objects in the off-center orientations, a sense of layered heights, and a sense of depth in terms of distance. Based on the widened sound field, users can feel a sense of being surrounded by sound. The rendered stereo signals of each audio object are then remixed to obtain the target stereo audio signal. When the electronic device plays the sound based on the target stereo audio signal and the user listens, the user experiences the immersive feeling of the central sound field, as if the sound source corresponding to the central sound field is right in front of the user. The user also experiences the distinct directional sense of different audio objects from the off-center sound field, the depth of the sounds from different distances, the layering of the sounds from different pitches, and the sense of being surrounded by sound from the wider off-center sound field. Furthermore, by combining the above sensations from the central and off-center aspects of the target stereo audio signal, a better sense of spatial sound can be created, providing the user with a superior listening experience.

[0008] In one possible implementation, the first eccentric azimuth signal includes a first left channel eccentric azimuth signal and a first right channel eccentric azimuth signal, and the second eccentric azimuth signal includes a second left channel eccentric azimuth signal and a second right channel eccentric azimuth signal.

[0009] By separating the left and right channels of the second and third stereo audio signals, the left channel eccentricity signal and the right channel eccentricity signal can be obtained respectively, making the eccentricity signal obtained from the azimuth estimation more accurate and comprehensive.

[0010] In one possible implementation, the second stereo audio signal includes a first left channel audio signal and a first right channel audio signal. Azimuth estimation is performed on the second stereo audio signal to determine a first center azimuth signal and a first off-center azimuth signal. This includes: performing a short-time Fourier transform on the first left channel audio signal and the first right channel audio signal to obtain a first left channel time-frequency block and a first right channel time-frequency block, each time-frequency block including corresponding real and imaginary parts; obtaining a first center channel contribution factor of the second stereo audio signal based on the real and imaginary parts of the first left channel time-frequency block and the first right channel time-frequency block; obtaining a first center azimuth signal based on the first center channel contribution factor, the first left channel audio signal, and the first right channel audio signal; separating the first center azimuth signal from the first left channel audio signal to obtain a first left channel off-center azimuth signal; and separating the first center azimuth signal from the first right channel audio signal to obtain a first right channel off-center azimuth signal.

[0011] In this way, the stereo signal can be segmented and calculated based on the time-frequency block, reducing computational complexity and improving efficiency. Furthermore, the center channel contribution factor can be easily estimated based on the real and imaginary parts of the time-frequency block, thereby relatively accurately separating the center azimuth signal and the off-center azimuth signal.

[0012] In another possible implementation, spatial rendering of the first eccentric azimuth signal includes: performing distance control processing on the first eccentric azimuth signal; performing azimuth and height control processing on the first eccentric azimuth signal; and performing adaptive width control processing on the first eccentric azimuth signal.

[0013] In this way, by controlling the distance of the eccentric azimuth signal, the sense of depth and variation in the distance between different eccentric azimuth signals can be enhanced. By controlling the azimuth and height of the eccentric azimuth signal, the sense of precise direction and the sense of layering in height between different eccentric azimuth signals can be improved. By adaptively controlling the width of the eccentric azimuth signal, the sound field width of the eccentric azimuth signal can be expanded, allowing the user to feel a sense of being surrounded by sound.

[0014] In another possible implementation, adaptive width control processing is performed on the first eccentric azimuth signal, including: adaptively dividing the first eccentric azimuth signal to obtain a frequency range in which the energy of the first eccentric azimuth signal is concentrated at a preset percentage; calculating the adaptive spreading coefficient of the first eccentric azimuth signal within the frequency range; and adaptively eliminating crosstalk in the first eccentric azimuth signal based on the adaptive spreading coefficient.

[0015] In this way, the frequency range where energy is relatively concentrated can be determined through adaptive frequency division, so that adaptive width control can be performed within this frequency range, thereby avoiding distortion caused by excessive width expansion of off-center azimuth signals. Then, an adaptive expansion coefficient is calculated within this frequency range to adjust the delay calculated during adaptive crosstalk cancellation, thereby achieving targeted adaptive width control for different audio objects.

[0016] In another possible implementation, adaptive frequency division is performed on the first eccentric azimuth signal to obtain a frequency range in which the energy of the first eccentric azimuth signal is concentrated at a preset percentage, including: determining the total energy of the first eccentric azimuth signal; and determining the upper and lower bound frequencies of the frequency range based on the total energy and the preset percentage.

[0017] In this way, adaptive frequency division can determine the frequency range in which energy is concentrated in the eccentric azimuth signal, so that the eccentric azimuth signal can be adaptively crosstalk eliminated within this frequency range. This can reduce the width expansion of the eccentric azimuth signal and avoid distortion after rendering the eccentric azimuth signal.

[0018] In another possible implementation, the adaptive expansion coefficient of the first eccentric azimuth signal is calculated within the frequency range, including: within the frequency range, the adaptive expansion coefficient is calculated based on the energy of the first left channel eccentric azimuth signal, the energy of the first right channel eccentric azimuth signal, and the energy difference between the energy of the first left channel eccentric azimuth signal and the energy of the first right channel eccentric azimuth signal.

[0019] In this way, the energy difference between the left and right channels of the eccentric azimuth signal can be used to characterize whether the sound of the eccentric azimuth signal is biased to the left, right, or center. By combining the left, right, or center bias of the eccentric azimuth signal with the energy of the left and right channels, an adaptive expansion coefficient can be calculated. This adaptive expansion coefficient can adapt to different biases of the eccentric azimuth signal. Therefore, the calculated attenuation amount can be adjusted in subsequent adaptive crosstalk cancellation to match the bias of the eccentric azimuth signal. This achieves the effect of targeted adaptive crosstalk cancellation for eccentric azimuth signals corresponding to different audio objects.

[0020] In another possible implementation, adaptive crosstalk cancellation of the first eccentric azimuth signal is performed based on an adaptive spreading coefficient, including: determining the attenuation of the first eccentric azimuth signal when it is transmitted to the human ear based on the adaptive spreading coefficient; determining the delay of the first eccentric azimuth signal when it is transmitted to the human ear; and performing adaptive crosstalk cancellation of the first eccentric azimuth signal based on the attenuation and the delay.

[0021] In this way, it is possible to generate a signal with the same amplitude and opposite phase as the crosstalk sound of the corresponding channel by inverting, attenuating and delaying the eccentric azimuth signal, so as to superimpose it on the signal of the other channel, thereby canceling the crosstalk sound formed by the eccentric azimuth signal of the other channel to the human ear, and achieving the purpose of crosstalk elimination.

[0022] In another possible implementation, the first center azimuth signal is enhanced by rendering the following: energy enhancement processing of the first center azimuth signal; distance control processing of the first center azimuth signal; and de-reverberation processing of the first center azimuth signal.

[0023] Thus, the volume of the center azimuth signal can be increased by enhancing its energy. Distance control of the center azimuth signal can bring it closer to the sound field, enhancing its immersive feel. De-reverberation processing of the center azimuth signal removes reverberation, reduces reflections and delays in space, and improves its clarity.

[0024] In another possible implementation, extracting the second stereo audio signal of the first audio object and the third stereo audio signal of the second audio object from the first stereo audio signal includes: extracting audio objects from the first stereo audio signal according to an audio separation model to obtain the second stereo audio signal of the first audio object and the third stereo audio signal of the second audio object.

[0025] The audio separation model can employ relevant models from audio processing technologies, such as the Spleeter model, Demucs model, Audio Sep model, and MDX-net model, etc., without limitation. Using an audio separation model can relatively easily and accurately extract each audio object, improving the efficiency of audio object extraction.

[0026] In another possible implementation, the target stereo audio signal is obtained by downmixing the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal. This includes: performing sound effect processing on the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal; and downmixing the sound effect processed third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal to obtain the target stereo audio signal.

[0027] In this way, the sound field effect of the final target stereo audio signal can be further improved through appropriate sound effect processing, thereby enhancing the user's listening experience.

[0028] In another possible implementation, sound processing includes at least one of equalization, bass enhancement, dynamic range control, and limiting.

[0029] When using multiple audio effects, the specific processing order is not limited and can be configured according to actual needs. For example, equalization can adjust the loudness of the audio signal in different frequency ranges, gain or attenuate specific frequency bands, change the timbre, and make the audio playback more personalized. Bass enhancement can boost the low-frequency components in the audio signal to enhance the dynamism and depth of the music. Dynamic range control can map the dynamic range of the input audio signal to a specified dynamic range, compress or limit the dynamic range of the signal, making the sound softer or louder. Limiting can limit the maximum amplitude of the audio signal to prevent signal overload or clipping distortion.

[0030] In a second aspect, an electronic device is provided, comprising: a processor, a memory, and a communication interface. The memory and the communication interface are coupled to the processor. The memory stores computer program code, including computer instructions. When the processor executes the computer instructions, it causes the electronic device to perform the method as described in any one of the first aspects above.

[0031] Thirdly, a computer-readable storage medium is provided that stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device causes the electronic device to perform the method as described in any one of the first aspects above.

[0032] Fourthly, a computer program product comprising computer instructions is provided, which, when executed on an electronic device, causes the electronic device to perform the method as described in any one of the first aspects above.

[0033] Fifthly, an apparatus (e.g., a system-on-a-chip) is provided, comprising a processor for supporting an electronic device in implementing the methods described in the first aspect. In one possible design, the apparatus further comprises a memory for storing program instructions and data necessary for the electronic device. When the apparatus is a system-on-a-chip, it may be composed of chips or may include chips and other discrete devices.

[0034] The technical effects of any of the design methods in aspects two through five can be found in the technical effects of different design methods in aspect one, and will not be repeated here. Attached Figure Description

[0035] Figure 1 This is a schematic diagram illustrating an application scenario of external audio playback on a laptop computer, as provided in an embodiment of this application.

[0036] Figure 2 This is a hardware structure block diagram of an electronic device provided in an embodiment of this application;

[0037] Figure 3 This is a software system structure block diagram of the electronic device provided in the embodiments of this application;

[0038] Figure 4 This is a flowchart illustrating an audio playback method provided in an embodiment of this application;

[0039] Figure 5 This is a schematic diagram of a neural network for separating audio objects provided in an embodiment of this application;

[0040] Figure 6 This is a schematic diagram of another neural network for separating audio objects provided in an embodiment of this application;

[0041] Figure 7 This is a time-frequency schematic diagram of a separated center azimuth signal provided in an embodiment of this application;

[0042] Figure 8 This is a schematic diagram of a process for enhancing rendering of a center orientation signal according to an embodiment of this application;

[0043] Figure 9 This is a schematic diagram of a process for distance control of a center azimuth signal provided in an embodiment of this application;

[0044] Figure 10 This is a schematic diagram of a process for spatial rendering of an off-center orientation signal provided in an embodiment of this application;

[0045] Figure 11 This is a schematic flowchart of a distance control process for an eccentric azimuth signal provided in an embodiment of this application;

[0046] Figure 12 This is a schematic diagram of the head-related transfer function and human ear localization provided in the embodiments of this application;

[0047] Figure 13 This is a schematic flowchart of an embodiment of the present application for adaptive width control of an eccentric azimuth signal;

[0048] Figure 14 This is a schematic diagram of a scene where a person's head is relative to an external speaker, provided in an embodiment of this application.

[0049] Figure 15 This is a schematic diagram of an adaptive crosstalk cancellation provided in an embodiment of this application;

[0050] Figure 16This is a schematic diagram illustrating a scenario in which an audio playback method provided in this application embodiment is applied;

[0051] Figure 17 This is a schematic diagram illustrating a scenario in which another audio playback method provided in this application embodiment is applied. Detailed Implementation

[0052] The terminology used in the following embodiments of this application is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to include the plural expressions as well, unless the context clearly indicates otherwise. It should also be understood that “ / ” means “or,” for example, A / B can mean A or B; “and / or” in the text is merely a description of the relationship between related objects, indicating that three relationships can exist, for example, A and / or B can mean: A alone, A and B simultaneously, and B alone.

[0053] In this application, the reference to "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described in this application can be combined with other embodiments.

[0054] The terms "first" and "second" in the following embodiments of this application are for descriptive purposes only and should not be construed as implying relative importance or implicitly indicating the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature, and in the description of the embodiments of this application, unless otherwise stated, "multiple" means two or more.

[0055] With the continuous development of terminal devices, a superior audio-visual experience has become an essential attribute of high-end products. Spatial audio, as one of the core components for enhancing the audio-visual experience, can broaden the sound image, enhance the sense of sound layering and distance, and help users obtain the ultimate immersive audio-visual experience.

[0056] However, traditional external audio solutions suffer from problems such as a narrow and flat sound field, resulting in poor layering and immersion in external audio.

[0057] For example, taking a laptop computer as the terminal device, with a speaker on each side of the keyboard, refer to... Figure 1The diagram shown illustrates an application scenario for external audio playback on a laptop computer, as provided in an embodiment of this application. Figure 1 As shown, the user opens the laptop and places it in front of them, then uses the laptop's external audio output. In this case, the laptop uses a traditional external audio output solution, resulting in a relatively narrow and flat sound field (as shown by the filled-in texture in the image). For the user, listening to the laptop's audio will feel like the sound lacks depth, has a low pitch, and lacks immersion, leading to a poor listening experience.

[0058] This application provides an audio playback method applied to an electronic device. The electronic device launches a first application and, upon receiving a first user operation, acquires the original stereo audio signal (e.g., a first stereo audio signal) in response to the user's first operation. The electronic device then extracts audio objects from the original stereo audio signal, obtaining stereo signals for different audio objects (e.g., a second stereo audio signal for the first audio object, a third stereo audio signal for the second audio object, etc.). Then, it performs orientation estimation on the stereo signals of different audio objects to obtain a center orientation signal and an off-center orientation signal. For example, orientation estimation on the second stereo audio signal of the first audio object yields a first center orientation signal and a first off-center orientation signal; orientation estimation on the third stereo audio signal of the second audio object yields a second center orientation signal and a second off-center orientation signal. This allows for targeted rendering optimization for the center orientation signal and the off-center orientation signal. For example, enhanced rendering can be performed on the center orientation signal (e.g., enhanced rendering of the first center orientation signal yields a third center orientation signal, and enhanced rendering of the second center orientation signal yields a fourth center orientation signal). This increases the volume of the center azimuth signal and brings it closer to the sound field, enhancing its immersive feel. Spatial rendering can be performed on off-center azimuth signals (e.g., enhancing the first off-center azimuth signal to obtain the third off-center azimuth signal, and enhancing the second off-center azimuth signal to obtain the fourth off-center azimuth signal). This renders the different directions, distances, and heights corresponding to different off-center azimuth signals, widening the sound field width of the off-center azimuth signals. Users can then perceive a clear sense of location, varying heights, and varying distances of different audio objects in the off-center azimuth areas, creating a sense of being surrounded by sound. The widened sound field further enhances the feeling of being enveloped by sound. Finally, the rendered stereo signals of each audio object (such as the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal) can be remixed to obtain the target stereo audio signal. Thus, based on the aforementioned enhanced rendering of the central azimuth signal and the spatial rendering of the off-center azimuth signal, the rendered target stereo audio signal possesses a more immersive central azimuth sound field and a wider, more profound, layered, and directional off-center azimuth sound field. Consequently, when the electronic device plays the sound based on the target stereo audio signal and the user listens, the user can experience the immersive feeling of the central azimuth sound field, feeling as if the sound source corresponding to the central azimuth sound field is right in front of the user. Furthermore, the user can experience the distinct azimuth of different audio objects brought by the off-center azimuth sound field, the depth of the sounds of different audio objects varying in distance, the layering of the sounds of different audio objects varying in pitch, and the sense of being surrounded by sound brought by the wider sound of the off-center azimuth sound field.Furthermore, by combining the aforementioned sensations brought about by the target stereo audio signal in both the central and off-center positions, a better sense of spatial sound can be created, providing users with a better listening experience.

[0059] The audio playback method provided in this application can be applied to electronic devices that have speakers and can play audio externally. For example, the electronic device can be a tablet computer, a laptop computer, a mobile phone, an in-vehicle device, or a wearable device (e.g., a smart bracelet), etc. This application does not limit the specific type of electronic device.

[0060] Reference Figure 2 The diagram shown is a hardware structure block diagram of an electronic device provided in an embodiment of this application.

[0061] like Figure 2 As shown, the electronic device 200 may include components such as a radio frequency (RF) circuit 210, a memory 220, an input unit 230, a display unit 240, a sensor 250, an audio circuit 260, a wireless fidelity (WiFi) module 270, a processor 280, a power supply 290, and a Bluetooth module 2100. Those skilled in the art will understand that... Figure 2 The electronic device structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0062] The following is combined Figure 2 A detailed introduction to some of the components of electronic devices:

[0063] The memory 220 can be used to store software programs and modules. The processor 280 executes various functional applications and data processing of the electronic device by running the software programs and modules stored in the memory 220. The memory 220 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as audio playback function), a boot loader, etc.; the data storage area may store data created according to the use of the electronic device (such as audio data), etc. In addition, the memory 220 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device.

[0064] The input unit 230 can be used to receive input numerical or character information, and to generate key signal inputs related to user settings and function control of the electronic device. Specifically, the input unit 230 may include a touch panel 231 and other input devices 232. The touch panel 231 can collect touch operations performed by the user on or near it (such as operations performed by the user using a finger, stylus, or any suitable object or accessory on or near the touch panel 231), and drive the corresponding connected devices according to a pre-set program. Specifically, other input devices 232 may include, but are not limited to, one or more of the following: a physical keyboard, function keys (such as volume control buttons, power buttons, etc.), a trackball, a mouse, a joystick, etc.

[0065] The display unit 240 can be used to display information input by the user or information provided to the user, as well as various menus of the electronic device. The display unit 240 may include a display panel 241, which may optionally be configured as a liquid crystal display (LCD), an organic light-emitting diode (OLED), or the like.

[0066] The processor 280 is the control center of the electronic device. It connects various parts of the electronic device via various interfaces and lines. By running or executing software programs or modules stored in the memory 220, and by calling data stored in the memory 220, it performs various functions and processes data, thereby providing overall monitoring of the electronic device. Optionally, the processor 280 may include one or more processing units; preferably, the processor 280 may integrate an application processor and a modem processor. The application processor mainly handles the operating system, user interface, and applications, while the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 280.

[0067] Electronic devices can perform audio functions through audio circuit 260, and speakers 261, microphones 262, etc. connected to audio circuit 260. For example, music playback, recording, etc.

[0068] Audio circuit 260 is used to convert digital audio signals into analog audio signals for output, and also to convert analog audio input into digital audio signals. Audio circuit 260 can also be used for encoding and decoding audio signals. In some embodiments, audio circuit 260 can be located in processor 280, or some functional modules of audio circuit 260 can be located in processor 280. Speaker 261, also called a "loudspeaker," is used to convert audio electrical signals into sound signals. Microphone 262, also called a "microphone," is used to convert sound signals into electrical signals. In this embodiment, when a user wants to play first stereo audio aloud through an electronic device, the user can first operate the electronic device to launch a first application, and then perform the first operation of playing the first stereo audio. Thus, in response to the user's first operation, the electronic device can acquire the first stereo audio signal corresponding to the first stereo audio. Then, the electronic device can render the first stereo audio signal using the audio playback method provided in this application to obtain a target stereo audio signal. Furthermore, the electronic device can convert the target stereo audio signal into an analog audio signal through audio circuit 260, and output it as a sound signal through speaker 261 to achieve audio playback.

[0069] The speaker 261 can be configured in multiple ways as needed, and its position on the electronic device can also be configured accordingly. For example, taking a laptop as an example, there can be two speakers 261, located on both sides of the laptop keyboard; or, there can be four speakers 261, with two located on both sides of the laptop keyboard and the other two located on both sides of the laptop display panel 241, etc.

[0070] Of course, this is understandable. Figure 2 The illustration shown is merely an example when the electronic device is in the form of a laptop computer. If the electronic device is in the form of a tablet computer, handheld computer, mobile phone, PDA, smart home device (e.g., television), in-vehicle system (e.g., vehicle computer), game console, or other device forms, the structure of the electronic device may include more advanced technologies. Figure 2 The fewer structures shown can also include more than Figure 2 The structures shown are not limited here.

[0071] The methods described in the following embodiments can all be implemented in an electronic device having the above-described hardware structure. The embodiments of this application will be illustrated below with reference to the accompanying drawings.

[0072] It is understandable that the implementation of electronic device functions requires not only hardware support but also the cooperation of software architecture. In some examples, refer to... Figure 3 The diagram shown is a software system structure block diagram of the electronic device provided in an embodiment of this application.

[0073] In a layered architecture, the software system is divided into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. For example, the software system of an electronic device can be divided into three layers, from top to bottom: the application layer, the application framework layer, and the hardware abstraction layer (HAL).

[0074] The application layer can include a series of application packages. For example... Figure 3 As shown, the application package can include applications such as music, video, and recording. Of course, the application layer can also include other application packages, such as Bluetooth, calendar, shopping applications, chat applications, floating windows, gallery, email, messaging, maps, instant messaging applications, etc., which are not limited in this application.

[0075] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0076] For example, such as Figure 3 As shown, the application framework layer may include modules for providing audio processing services to applications within the application layer, enabling applications to call these modules to implement corresponding functions. For example, the audio playback method provided in this application can be deployed within this application framework layer. For instance, in this embodiment, the application framework layer may deploy modules such as audio object extraction, orientation estimation, energy enhancement, distance control, dereverberation, orientation and height rendering, sound effects processing, adaptive width control, and downmixing. Of course, the application framework layer may also include other modules such as system servers, system-level user interfaces, and window managers; this embodiment does not impose any limitations on these.

[0077] A hardware abstraction layer hides specific hardware details, providing a consistent interface for upper-layer software. For example, consider audio playback... Figure 3 The hardware abstraction layer shown can include an audio playback hardware abstraction layer (Audio Playback HAL). Different audio circuits or speakers may have different control methods and interfaces, but by using the Audio Playback HAL, a unified interface is provided for the upper-layer audio services. In this way, the upper-layer audio services only need to call these unified interface functions to control the speakers to play audio, without needing to know how the underlying drivers of the audio circuits or speakers are implemented.

[0078] Based on the aforementioned software architecture, when applications such as music, video, and recording in the application layer need to play audio externally, these applications can call the relevant modules in the application framework layer to process and render the original stereo audio signal to be played according to the audio playback method provided in this application, thereby obtaining the rendered target stereo audio signal. Finally, by calling the interface functions provided by the audio playback hardware abstraction layer in the hardware abstraction layer, the speaker of the electronic device is controlled to play the audio corresponding to the rendered audio signal.

[0079] For example, when a music application in the application layer is in the foreground of an electronic device, and the user performs the first operation in the music application to play stereo audio, the music application can call the audio object extraction module in the application framework layer to extract audio objects from the raw stereo audio signal output by the music application, so as to extract the stereo audio signals of different audio objects in the raw stereo audio signal. Then, the orientation estimation module performs orientation estimation on the stereo audio signals of the audio objects, thereby obtaining the center orientation signal and the off-center orientation signal in the stereo audio signal.

[0080] For the center azimuth signal, the energy enhancement module can be used to enhance the energy and increase the volume of the center azimuth signal. The distance control module can be used to control the distance and bring the sound field distance of the center azimuth signal closer, thus enhancing the sense of presence of the center azimuth signal. The dreverb module can be used to remove the reverb of the center azimuth signal, thereby reducing the reflection and delay of the center azimuth signal in space and improving the clarity of the center azimuth signal.

[0081] For off-center azimuth signals, distance control rendering can be performed sequentially through the distance control module to render different distances of different audio objects in the off-center azimuth, thus obtaining a sense of depth and distance between the sounds of different audio objects in the off-center azimuth. The azimuth and height rendering module can be used to render different azimuths and heights of different audio objects in the off-center azimuth, thus obtaining a clear sense of direction and a sense of layering in height between the sounds of different audio objects in the off-center azimuth. The adaptive width control module can be used to perform adaptive width control rendering to expand the sound field width of the off-center azimuth signal and enhance the sense of width of the sound in the off-center azimuth.

[0082] Finally, the audio effects processing module can further enhance the audio quality of the processed center and off-center signals by processing them. Then, the down-mixing module performs down-mixing on the center and off-center signals to obtain the target stereo audio signal. After obtaining the target stereo audio signal, the down-mixing module can control the electronic device's speakers to play the audio corresponding to the target stereo audio signal by calling the interface functions provided by the audio playback hardware abstraction layer.

[0083] Based on the above hardware and software architecture, the audio playback method provided in the embodiments of this application will be described below.

[0084] Reference Figure 4 The diagram shown is a flowchart illustrating an audio playback method provided in an embodiment of this application. This audio playback method can be applied to the aforementioned electronic device. After the electronic device launches a first application (e.g., an application for playing audio or capable of playing audio) and responds to a user's first operation (i.e., an operation to instruct the electronic device to play audio), it acquires the original stereo audio signal (e.g., a first stereo audio signal). Taking an example where the original stereo audio signal includes audio objects such as a first audio object (e.g., human voice) and a second audio object (e.g., drum sounds), the method proceeds as follows: Figure 4 As shown, the electronic device can first perform audio object extraction processing 4200 on the original stereo audio signal 4100 to obtain stereo audio signals corresponding to each audio object contained in the original stereo audio signal 4100, such as the human voice stereo signal 4300 (i.e., the second stereo audio signal of the first audio object) and the drum sound stereo signal 4400 (i.e., the third stereo audio signal of the second audio object). Different audio objects correspond to different audio tracks; for example, human voices and drum sounds are located on different tracks in the original stereo audio signal 4100. The human voice stereo signal 4300 and the drum sound stereo signal 4400 can be separated using a track separator or other audio separation model. Typically, the original stereo audio signal 4100 can include stereo signals of various different audio objects, with different audio objects corresponding to different audio tracks. For example, audio objects can include human voices, drum sounds, guitar sounds, piano sounds, etc. Therefore, each audio object in the original stereo audio signal 4100 can be extracted for separate rendering processing of the stereo signals of each different audio object.

[0085] After separating the stereo audio signals of different audio objects (such as human voice stereo signal 4300 and drum sound stereo signal 4400) from the original stereo audio signal 4100, the electronic device can perform orientation estimation processing on the stereo audio signals of different audio objects such as human voice stereo signal 4300 and drum sound stereo signal 4400, such as human voice orientation estimation processing 4310 and drum sound orientation estimation processing 4410, thereby separating the central orientation human voice 4311 (i.e., the first central orientation signal), the eccentric orientation human voice 4312 (i.e., the first eccentric orientation signal), and the central orientation drum sound 4411 (i.e., the second central orientation signal) and the eccentric orientation drum sound 4412 (i.e., the second eccentric orientation signal). By estimating the orientation of stereo audio signals from different audio objects, the center orientation signal and the off-center orientation signal can be separated. This makes it easier to perform corresponding rendering optimizations for the center orientation signal and the off-center orientation signal respectively, thereby improving the clarity of the center orientation signal and the spatial sense of the off-center orientation signal.

[0086] For example, the center-position vocal 4311 can be enhanced and rendered separately to obtain the third center-position signal. The off-center vocal 4312 can be spatially rendered to obtain the third off-center signal. The center-position drum sound 4411 can be enhanced and rendered separately to obtain the fourth center-position signal. The off-center drum sound 4412 can be spatially rendered separately to obtain the fourth off-center signal.

[0087] The enhancement rendering for the center-positioned vocals 4311 and the center-positioned drum sound 4411 can include energy enhancement processing 4511, distance control processing 4512, and de-reverb processing 4513. This is achieved by increasing the volume of the center-positioned vocals 4311 and drum sound 4411 through energy enhancement processing 4511, and by reducing the sound field distance of the center-positioned vocals 4311 and drum sound 4411 through distance control processing 4512. This enhances the sense of presence of the center-positioned vocals 4311 and drum sound 4411, allowing the user to feel the center-positioned vocals and drum sound are close to them when listening to audio played on an electronic device, creating a sense of presence and immersion as if the source of the vocals and drum sound is right in front of the user. By using the de-reverb processing 4513, the reverb of the center-position human voice 4311 and the center-position drum sound 4411 can be removed, reducing the reflection and delay of the center-position human voice 4311 and the center-position drum sound 4411 in space, thereby improving the clarity of the center-position human voice 4311 and the center-position drum sound 4411.

[0088] Spatial rendering for the off-center vocals 4312 and off-center drum sounds 4412 can include distance control processing 4521, orientation and height control processing 4522, and adaptive width control processing 4523. Distance control processing 4521 renders the distances of different audio objects, such as the off-center vocals 4312 and drum sounds 4412, in the off-center orientation, allowing users to perceive the different distances of different audio objects in the off-center orientation when listening to audio played on electronic devices, thus creating a sense of depth and variation in the distance of different audio objects in the off-center orientation. Orientation and height control processing 4522 renders the orientation and height of the off-center vocals 4312 and drum sounds 4412, allowing users to perceive the different orientations and heights of different audio objects in the off-center orientation when listening to audio played on electronic devices, thus creating a clear sense of direction and a sense of layering in the off-center orientation. By using adaptive width control processing 4523, the sound field width of off-center vocals 4312 and off-center drum sounds 4412 can be expanded respectively. This allows users to perceive a wider sound field from the off-center direction when listening to audio played by electronic devices, as if they are surrounded by the sound from that direction, resulting in a better sense of width. Combined with the sense of depth from distance control processing 4521, the sense of direction and height from orientation and height control processing 4522, and the sense of width from adaptive width control processing 4523, users can obtain a good sense of space when listening to audio played by electronic devices, clearly perceiving the spatial layers of different audio objects at different orientations, heights, and distances.

[0089] After rendering different audio objects (such as vocals and drum sounds) separately, each rendered signal can be further processed with audio effects 4600 and downmixed 4700 to obtain the left channel signal 4710 and right channel signal 4720 of the target stereo audio signal. This allows electronic devices to play stereo audio externally 4800 based on the target stereo audio signal. Audio effects processing can include one or more commonly used audio effects, such as equalization (e.g., EQ equalization) to adjust the loudness of the audio signal in different frequency ranges, gain or attenuate specific frequency bands, change the timbre, and make audio playback more personalized; bass enhancement to boost the low-frequency components in the audio signal, enhancing the dynamism and depth of the music; dynamic range control to map the dynamic range of the input audio signal to a specified dynamic range, compressing or limiting the dynamic range of the signal to make the sound softer or louder; and limiting to limit the maximum amplitude of the audio signal to prevent signal overload or clipping distortion. This is not limited here and can be configured according to actual needs. Through appropriate sound effect processing, the sound field effect of the final target stereo audio signal can be further improved, enhancing the user's listening experience.

[0090] For example, based on Figure 4 The method shown describes an electronic device that, when extracting audio objects from the original stereo audio signal 4100 to extract a second stereo audio signal (i.e., human voice stereo signal 4300) of a first audio object (such as a human voice) and a third stereo audio signal (i.e., drum sound stereo signal 4400) of a second audio object (such as drum sound), can use a pre-trained neural network for extracting audio objects to extract each audio object from the original stereo audio signal 4100. For example, as... Figure 5 As shown, the electronic device can input a first stereo audio signal into an encoder composed of a convolutional neural network or a recurrent neural network. The encoder converts the first stereo audio signal into multiple different features, and inputs these features into a separation network to form multiple matrices corresponding to the features. Further, the electronic device combines these matrices with the features and inputs them into a decoder for decoding. After decoding, other sounds such as human voices and drum sounds are separated. For example, as... Figure 6 As shown, the electronic device can perform time-frequency transformation on the first stereo audio signal, and then perform frequency division processing to split the first stereo audio signal into multiple frequency bands for separate processing of signals in different frequency bands. Thus, the frequency-divided signal can be input into N encoders (such as encoder 1, encoder 2, ..., encoder N) composed of semi-causal convolutional or causal convolutional networks. The encoders transform the signal into multiple different features, and these features are input into a recurrent convolutional network to form multiple matrices corresponding to the features. Further, the electronic device can input the multiple matrices and multiple features into N decoders (such as decoder 1, decoder 2, ..., decoder N) composed of semi-causal or causal convolutional networks for decoding. After decoding, the obtained signals are frequency band merged and then input into complex masks corresponding to different audio objects (such as human voices, drum sounds, etc.). Finally, based on the time-frequency transformation performed on the first stereo audio signal, each complex mask is subjected to inverse time-frequency transformation to obtain the stereo audio signal of each audio object.

[0091] The pre-trained neural network used to extract audio objects can be an audio separation model from related technologies, such as the Spleeter model, Demucs model, Audio Sep model, and MDX-net model, etc., without any restrictions.

[0092] When an electronic device performs orientation estimation on the stereo audio signals of the extracted first audio object (such as a human voice) and the second stereo audio signal (i.e., human voice stereo signal 4300) and the third stereo audio signal (i.e., drum sound stereo signal 4400) of the second audio object (such as a drum sound), for example, it can obtain the center channel contribution factor corresponding to the left and right channels of the stereo audio signal of the audio object based on the energy difference between the channels and the phase difference between the channels, and then separate the center orientation signal and the off-center orientation signal in the stereo audio signal of the audio object according to the center channel contribution factor.

[0093] For example, to Figure 4 Taking the stereo voice signal 4300 shown as an example for directional estimation, the left channel signal (i.e., the first left channel audio signal) and the right channel signal (i.e., the first right channel audio signal) of the stereo voice signal 4300 can be subjected to short-time Fourier transforms to obtain the time-frequency blocks corresponding to the left and right channel signals, respectively. The number of time-frequency blocks obtained from the left channel signal transformation (or the first left channel time-frequency block) is the same as the number of time-frequency blocks obtained from the right channel signal transformation (or the first right channel time-frequency block). The specific number of time-frequency blocks obtained is determined by the actual stereo voice signal 4300. For example, the left channel signal x_1(t) can be transformed by short-time Fourier transform to obtain 513 first left channel time-frequency blocks, and the right channel signal x_2(t) can be transformed by short-time Fourier transform to obtain 513 first right channel time-frequency blocks.

[0094] In this context, a time-frequency block can be viewed as the time-frequency information of a segmented time frame within the entire stereo voice signal 4300. Therefore, the first left-channel time-frequency block obtained from the left-channel signal transformation and the first right-channel time-frequency block obtained from the right-channel signal transformation correspond one-to-one according to the time frame. For example, the first left-channel time-frequency block corresponding to the first time frame obtained from the left-channel signal transformation corresponds to the first right-channel time-frequency block corresponding to the first time frame obtained from the right-channel signal transformation.

[0095] The time-frequency blocks obtained by performing short-time Fourier transforms on the left and right channel signals of the human voice stereo signal 4300 typically contain the real and imaginary parts corresponding to the time-frequency blocks.

[0096] After obtaining multiple time-frequency blocks corresponding to the left and right channel signals of the human voice stereo signal 4300, the center azimuth signal (i.e., the first center azimuth signal, also known as the center azimuth human voice 4311) and the eccentric azimuth signal (i.e., the first eccentric azimuth signal, also known as the eccentric azimuth human voice 4312) corresponding to each time frame of the human voice stereo signal 4300 can be calculated according to the time-frequency blocks of the left and right channel signals corresponding to each time frame.

[0097] For example, taking the first time frame as an example, the center channel contribution factor (i.e., the first center channel contribution factor) corresponding to the left and right channels can be calculated based on the real and imaginary parts of the first left channel time-frequency block of the left channel signal corresponding to the first time frame and the real and imaginary parts of the first right channel time-frequency block of the right channel signal corresponding to the first time frame. For example, it can be calculated using the following formula:

[0098]

[0099]

[0100] Where, α L α is the contribution factor of the center channel corresponding to the left channel signal. R L is the contribution factor of the center channel corresponding to the right channel signal. r L is the real part of the first left channel time-frequency block of the left channel signal. i R is the imaginary part of the first left channel time-frequency block of the left channel signal. r R is the real part of the first right channel time-frequency block of the right channel signal. i This represents the imaginary part of the first right channel time-frequency block of the right channel signal. The center channel contribution factor corresponding to the left channel signal indicates the degree of contribution of the left channel signal to the center channel (i.e., the center azimuth signal), and the center channel contribution factor corresponding to the right channel signal indicates the degree of contribution of the right channel signal to the center channel (i.e., the center azimuth signal). Therefore, the center azimuth signal can be separated based on the respective contributions of the left and right channel signals to the center azimuth signal using the center channel contribution factor.

[0101] For example, the center channel contribution factor α corresponding to the left and right channel signals can be used as a reference. L and α R This is used to calculate the first center azimuth signal corresponding to the first time frame of the stereo voice signal 4300, and to obtain the off-center azimuth signal by separating the center azimuth signals from the left and right channel signals. For example, it can be calculated using the following formula:

[0102] C = α L ×L in +α R ×R in

[0103] L = L in -C

[0104] R = R in -C

[0105] Wherein, C is the first center azimuth signal, L is the left channel eccentricity signal (i.e., the first left channel eccentricity signal), and R is the right channel eccentricity signal (i.e., the first right channel eccentricity signal). The first eccentricity signal includes the first left channel eccentricity signal and the first right channel eccentricity signal. in For the first time frame of the left channel signal, R in This is the first time frame of the right channel signal.

[0106] This allows for segmented calculation of stereo signals based on time-frequency blocks, reducing computational complexity and improving efficiency. Furthermore, it enables convenient estimation of the center channel contribution factor based on the real and imaginary parts of the time-frequency blocks, thus relatively accurately separating the center azimuth signal from the off-center azimuth signal. For example, as... Figure 7 As shown, the center azimuth signal obtained using other methods in related technologies still contains noise signals (such as the noise signal in the area shown by the dashed line in the figure). However, the center azimuth signal calculated based on the center channel contribution factor, as shown by the dashed line in the figure, shows good noise cancellation. Therefore, the center azimuth signal obtained using the method calculated based on the center channel contribution factor has less residual noise and is cleaner and more accurate compared to center azimuth signals obtained using other methods in related technologies.

[0107] It should be noted that the above example illustrates the process of estimating the orientation of a human voice stereo signal 4300 to obtain a first center orientation signal (i.e., center orientation human voice 4311) and a first off-center orientation signal (i.e., off-center orientation human voice 5312). The methods for estimating the orientation of stereo audio signals of other audio objects, such as drum sound stereo signal 4400, can refer to the above example and will not be elaborated upon here. For example, when estimating the azimuth of a drum sound stereo signal 4400 (i.e., the third stereo audio signal), a short-time Fourier transform can be performed on the second left channel audio signal and the second right channel audio signal of the drum sound stereo signal 4400 to obtain the second left channel time-frequency block and the second right channel time-frequency block. Then, based on the real and imaginary parts of the second left channel time-frequency block and the second right channel time-frequency block, the second center channel contribution factor is obtained. Thus, based on the second center channel contribution factor, the second center azimuth signal (i.e., the center azimuth drum sound 4411) is obtained. The second center azimuth signal in the second left channel audio signal is separated to obtain the second left channel eccentric azimuth signal, and the second center azimuth signal in the second right channel audio signal is separated to obtain the second right channel eccentric azimuth signal. The second eccentric azimuth signal (i.e., the eccentric azimuth drum sound 4412) includes the second left channel eccentric azimuth signal and the second right channel eccentric azimuth signal.

[0108] After separating the center azimuth signal (e.g., center azimuth vocal 4311, center azimuth drum 4411) and off-azimuth signal (e.g., off-azimuth vocal 4312, off-azimuth drum 4412) of the stereo audio signal of the audio object, enhancement rendering can be performed on each center azimuth signal. For example, enhancement rendering is performed on the center azimuth signal, such as... Figure 8 As shown, processing can include energy enhancement, distance control, and dereverberation. The order of these processes is not restricted. For example, the center azimuth signal can be enhanced first to increase its volume. Then, distance control can be applied to bring the center azimuth signal closer to the sound field, enhancing its immersive feel. Finally, dereverberation can be performed to remove reverberation, reduce reflections and delays in space, and improve clarity.

[0109] The energy enhancement, distance control, and dereverberation processing methods can be found in the implementation methods described in audio processing technologies.

[0110] For example, Figure 8 The distance control of the center azimuth signal shown can be processed based on the head-related transfer function. For example, using the... Figure 4 Taking the center-oriented human voice 4311 as an example for distance control, as shown... Figure 9 As shown, N subjects can be selected first, and the distance d between each of the N subjects and the center can be measured. n Near-field head correlation transfer function hrtf n and at the center of the distance d from the subject f far-field head correlation transfer function hrtf f Then, the corresponding head correlation transfer function is subjected to an FFT transform to obtain the frequency domain head correlation transfer function HRTF. n =FFT(hrtf) n ) and HRTF f =FFT(hrtf) f Then, the frequency domain head correlation transfer functions of the N subjects are averaged separately. Based on measurements of N subjects, the average near-field head correlation transfer function and average far-field head correlation transfer function at the center direction are obtained, applicable to most users. The average near-field head correlation transfer function reflects the sound transmission characteristics of a sound source close to the user at the center direction as it travels to the user's ears. The average far-field head correlation transfer function reflects the sound transmission characteristics of a sound source farther from the user at the center direction as it travels to the user's ears. A distance filter can be calculated based on these two average head correlation transfer functions. This distance filter can render the transmission effect of sound sources closer to the user based on the near-field frequency domain head correlation transfer function, and suppress the transmission effect of sound sources farther from the user based on the far-field frequency domain head correlation transfer function. This effectively brings the center-oriented human voice 4311 closer to the user's sound field, enhancing its immersive quality. For example, before processing the center-oriented human voice 4311 using the aforementioned distance filter, the distance filter can be smoothed to obtain the final distance filter DF = smooth(DF). This smoothed distance filter allows for distance control of the center-oriented human voice 4311, making the rendering of sound sources closer to the user and the suppression of sound sources farther from the user smoother, thus avoiding distortion of the center-oriented human voice 4311 after distance control. Of course, the above is only a specific example of distance control for the center-oriented human voice 4311. For the specific implementation of distance control for the center-oriented signals of stereo audio signals of other audio objects, such as the center-oriented drum sound 4411, the above example can also be referred to, and will not be repeated here.

[0111] After separating the center azimuth signal (e.g., center azimuth vocal 4311, center azimuth drum 4411) and the off-azimuth signal (e.g., off-azimuth vocal 4312, off-azimuth drum 4412) of the stereo audio signal of the audio object, spatial rendering can be performed on each off-azimuth signal. For example, spatial rendering is performed on the off-azimuth signal, such as... Figure 10 As shown, the processing can include distance control, azimuth and height control, and adaptive width control. The order in which these processes are executed is not restricted. For example, distance control can be applied to the off-center azimuth signal first to enhance its sense of depth. Then, azimuth and height control can be applied to the off-center azimuth signal to enhance its sense of layering and directionality. Finally, adaptive width control can be applied to the off-center azimuth signal to expand its sound field width and enhance its sense of width.

[0112] The methods for distance control and orientation and altitude control can all refer to the implementation methods in audio processing related technologies.

[0113] As an example, Figure 10 The distance control for the eccentric azimuth signal shown can be processed based on the loudspeaker array driving equations. For example, to... Figure 4 Taking the eccentric orientation of the human voice 4312 as an example for distance control, as shown... Figure 11 As shown, we can first obtain the target distance *r* and target orientation *θ* of the off-center voice 4312 to be rendered, as well as the early reflection order of the room size. The target distance *r* and target orientation *θ* can be the distance and orientation to which the off-center voice 4312 is to be rendered, so that after rendering, the user can perceive the orientation of the off-center voice 4312 as the target orientation *θ* and the distance between the user and the location of the off-center voice 4312 as the target distance *r*. The early reflection order of the room size reflects the number of times the sound source collides and reflects with the room walls or ceiling in a short period of time (usually between 50 and 80 milliseconds) after the sound source emits sound. Therefore, we can combine the head-related transfer function corresponding to the target distance *r* and target orientation *θ*. Based on the room size and early reflection order ω, the drive signals of each speaker are rendered to both ears according to the speaker array drive equation, generating binaural pulses: The generated binaural pulses may include the pulse I transmitted from the left channel speaker to the left ear. 1L The pulse I transmitted from the left channel speaker to the right ear 1R And the pulse I transmitted from the right channel speaker to the left ear. 2L The pulse I transmitted from the right channel speaker to the right ear 2R By generating binaural pulses, the transmission characteristics of sound emitted by the left and right channel speakers, located at target distance r and target azimuth θ, can be characterized based on the early reflection order of the room size and the target distance r and azimuth θ, respectively, as they are transmitted to the user's ears. Therefore, the distance-controlled left channel off-center azimuth human voice signal L can be finally output by convolving the binaural pulses. out =L in *I 1L +R in *I 2L And the right channel off-center vocal signal R out =R in *I 2R +L in *I 1RBy using convolutional binaural pulses, the transmission characteristics of the sound emitted by the left and right channel speakers, based on the early reflection order of the room size and located at the target distance r and target azimuth θ, are superimposed onto the rendered left-channel off-center azimuth human voice signal and the right-channel off-center azimuth human voice signal. This allows the rendered left-channel off-center azimuth human voice signal to simulate the sound emitted by the left channel speaker at the target distance r and target azimuth θ based on the early reflection order of the room size, and the rendered right-channel off-center azimuth human voice signal to simulate the sound emitted by the right channel speaker at the target distance r and target azimuth θ based on the early reflection order of the room size. This enables distance control processing of the off-center azimuth human voice 4312, allowing the user to perceive the off-center azimuth human voice 4312 as being located at the target distance r and target azimuth θ when listening to the sound played by the electronic device, thus enhancing the user's sense of depth when hearing the off-center azimuth human voice 4312. Of course, the above is only a specific example of distance control for off-center human voice 4312. For the specific implementation of distance control for off-center stereo audio signals of other audio objects, such as off-center drum sound 4412, the above example can also be referred to, and will not be repeated here.

[0114] As an example, Figure 10 The azimuth and altitude control of the eccentric azimuth signal shown can be processed based on head-related transfer function and human ear localization.

[0115] like Figure 12 As shown, for a sound source emitting a sound x(t), its transmission to a person's ears is affected by factors such as the time difference between the two ears, the difference in sound intensity, and the structure of the auricle, resulting in changes in the sound ultimately perceived by the human ear. These changes allow a person to locate the direction and altitude of the sound source. Therefore, the changes in sound transmission can be characterized by measuring the head correlation transfer function at corresponding directions and altitudes. Thus, the sound source emitting a sound x(t), after passing through the left ear head correlation transfer function h... L After processing (t), the actual sound x heard by the left ear can be obtained. L (t), the sound source emitted by the sound source x(t), through the right ear correlation transfer function h R (t) After processing, the actual sound x heard by the right ear can be obtained. R (t).

[0116] Therefore, in order to Figure 4Taking the off-center azimuth voice 4312 as an example for azimuth and height control, we can first select a suitable extended azimuth and height for the off-center azimuth voice 4312 based on the speaker layout, measure the head-related transfer function for the corresponding azimuth, and perform amplitude and phase correction to generate a height filter with azimuth (i.e., the head-related transfer function for the corresponding azimuth and height). Then, we use overlap-preserving to perform segmented convolution on the left and right channel signals.

[0117] x Lout =Conv(x L h L (t))

[0118] x Rout =Conv(x R h R (t))

[0119] Where x L The left channel off-center azimuth signal (i.e., the left channel off-center vocal signal), x R For the right channel off-center azimuth signal (i.e., right channel off-center vocals), h L (t) represents the height filter of the left channel (i.e., the left ear head correlation transfer function), h R (t) is the height filter of the right channel (i.e., the right ear head correlation transfer function). Then, x Lout For the processed left channel off-center vocal signal, x Rout This is the processed right channel off-center azimuth voice signal. Therefore, the resulting x... Lout It can simulate the sound heard by the left ear when a voice with an off-center left channel is emitted at a selected height and direction and transmitted to the left ear. Rout It can simulate the sound heard by the right ear when the right channel off-center voice is emitted at a selected height and direction and transmitted to the right ear. This allows the user to perceive the height and direction of the off-center voice 4312 when listening to it, thus enhancing the user's sense of location and the sense of layering in height when listening to the off-center voice 4312.

[0120] Of course, the above is only a specific example of controlling the orientation and height of the off-center human voice 4312. The specific implementation of controlling the orientation and height of the stereo audio signal of other audio objects, such as the off-center drum sound 4412, can also refer to the above example, and will not be repeated here.

[0121] For example, in Figure 10 When performing adaptive width control on the eccentric azimuth signal as shown, such as Figure 13As shown, for each frame of the stereo audio signal of the audio object, the corresponding eccentric azimuth signal can be adaptively divided to obtain the frequency range in which the energy is relatively concentrated. Then, the adaptive spreading coefficient is calculated within this frequency range, and the eccentric azimuth signal is adaptively crosstalk eliminated according to the adaptive spreading coefficient, thereby completing the adaptive width control.

[0122] For example, to Figure 4 Taking the processing of the eccentric azimuth signal (i.e., eccentric azimuth voice 4312, or the first eccentric azimuth signal) corresponding to the first time frame of the stereo voice signal 4300 shown as an example, the energy of the frame concentrated in a preset percentage of the frequency range can be calculated by using the energy of the left channel eccentric azimuth signal and the energy of the right channel eccentric azimuth signal in that frame, thereby completing adaptive frequency division. For example, the energy of different frequencies in that frame can be calculated separately using the following formula:

[0123] E1(m,k)=|x1(m,k)| 2

[0124] E2(m,k)=|x2(m,k)| 2

[0125] E(m,k)=E1(m,k)+E2(m,k)

[0126] Where m is the time domain subscript and k is the frequency domain subscript, x1(m,k) is the left channel eccentricity signal of the first time frame of the stereo signal (e.g., the left channel eccentricity signal after azimuth and height control processing in the aforementioned example), x2(m,k) is the right channel eccentricity signal of the first time frame of the stereo signal (e.g., the right channel eccentricity signal after azimuth and height control processing in the aforementioned example), E1(m,k) is the energy of frequency k in the left channel eccentricity signal in this frame, E2(m,k) is the energy of frequency k in the right channel eccentricity signal in this frame, and E(m,k) is the energy of frequency k in the eccentricity signal of the first time frame of the human voice stereo signal 4300.

[0127] Using the above formulas, the energy corresponding to different frequencies in the off-center azimuth signal of this frame of stereo human voice signal 4300 can be calculated. Therefore, the total energy of the off-center azimuth signal in this frame of stereo human voice signal 4300 can be calculated using the following formula:

[0128]

[0129] Where m is the time-domain subscript and k is the frequency-domain subscript, E total (m) represents the total energy of the eccentric azimuth signal of the first time frame of the human voice stereo signal 4300.

[0130] After calculating the total energy of the eccentric azimuth signal of the first time frame of the stereo human voice signal 4300, the frequency range in which the energy is concentrated can be calculated based on a preset percentage. For example, the upper and lower bound frequencies corresponding to the frequency range can be calculated based on the following formula:

[0131] E cumulative (m, k) lower )≥(1-α)·E total (m)

[0132] E cumulative (m, k) upper )≥α·E total (m)

[0133] Where, k lower k is the lower bound frequency of the frequency range. upper This represents the upper limit of the frequency range, where α is a preset percentage (e.g., 90%, 80%, or 85%, etc.; there are no restrictions here, and it can be set according to actual needs); E cumulative (m, k) lower ) and E cumulative (m, k) upper It can be calculated using the following formula:

[0134]

[0135] Based on the above example, the frequency range where energy is concentrated within a preset percentage in the first time frame of the stereo voice signal 4300 can be calculated. Therefore, adaptive crosstalk cancellation can be performed on the off-center azimuth signal of the first time frame within this frequency range. By adaptively dividing the frequency range where energy is concentrated in the stereo voice signal 4300, and then performing adaptive crosstalk cancellation on the off-center azimuth signal within this frequency range, excessive width expansion of the off-center azimuth signal can be reduced, avoiding distortion after rendering the off-center azimuth signal.

[0136] Ideally, the left channel signal reaches the left ear, and the right channel signal reaches the right ear. However, in external speaker mode, in addition to the above propagation paths, there are two crosstalk paths: the left channel signal from the left speaker to the right ear and the right channel signal from the right speaker to the left ear. Crosstalk paths cause spectral coloration and reduce the correlation between the signals received by the left and right ears, resulting in damage to the stereo imaging and deterioration of sound quality. Therefore, adaptive crosstalk cancellation can eliminate crosstalk signals between the left and right ears while maintaining the original sound quality. That is, it eliminates the transmission of the right channel signal from the right speaker to the left ear, and eliminates the transmission of the left channel signal from the left speaker to the right ear. Specifically, the right channel signal from the right speaker can be processed according to the attenuation and delay it incurs when it reaches the left ear to simulate the signal after it reaches the left ear. This signal is then inverted and superimposed on the left channel signal from the left speaker to cancel out the signal of the right channel signal from the right speaker after attenuation and delay, thereby eliminating left-ear crosstalk. Furthermore, crosstalk to the right ear can be eliminated by processing the left channel signal of the left speaker according to the attenuation and delay it generates when it is transmitted to the right ear, simulating the signal transmitted to the right ear, and then inverting the signal and superimposing it on the right channel signal of the right speaker.

[0137] Therefore, adaptive crosstalk cancellation can be performed by calculating the delay and attenuation of the left channel eccentricity signal and the right channel eccentricity signal, and then based on the delay and attenuation of the left channel eccentricity signal and the right channel eccentricity signal.

[0138] For example, before performing adaptive crosstalk cancellation on an off-center azimuth signal, an adaptive spreading coefficient corresponding to the off-center azimuth signal can be calculated. This allows the attenuation of the off-center azimuth signal during transmission to be adjusted based on the adaptive spreading coefficient during subsequent adaptive crosstalk cancellation. Since the stereo audio signals of different audio objects are different, the adaptive spreading coefficients calculated for their off-center azimuth signals will also differ. Therefore, when performing adaptive crosstalk cancellation, the adjustment of the attenuation amount through the adaptive spreading coefficient allows for targeted adaptive crosstalk cancellation for the off-center azimuth signals corresponding to different audio objects, improving the processing effect of adaptive crosstalk cancellation.

[0139] For example, the adaptive expansion coefficient can be calculated using the following formula:

[0140] Δ(m, k) = |x1(m, k)| 2 -|x2(m,k)| 2

[0141]

[0142]

[0143]

[0144] Where m is the time-domain subscript and k is the frequency-domain subscript, x1(m,k) is the left channel eccentricity signal of the first time frame of the human voice stereo signal 4300 (e.g., the left channel eccentricity signal within the frequency range obtained by adaptive frequency division in the aforementioned example), x2(m,k) is the right channel eccentricity signal of the first time frame of the human voice stereo signal 4300 (e.g., the right channel eccentricity signal within the frequency range obtained by adaptive frequency division in the aforementioned example), and Δ(m,k) is the energy difference between the left and right channels of the eccentricity signal, which can characterize whether the sound of the eccentricity signal is biased to the left, right, or center, thus allowing for the use of... This is used to indicate whether the eccentric azimuth signal's sound is biased to the left, right, or center. Based on the energy of the left and right channel eccentric azimuth signals, an initial expansion coefficient ψ(m, k) can be calculated. Then, by combining this with the eccentric azimuth signal's sound bias (left, right, or center), an adaptive expansion coefficient Ψ(m, k) can be calculated. This adaptive expansion coefficient can adapt to different biases in the eccentric azimuth signal's sound, thus adjusting the final calculated attenuation amount during subsequent adaptive crosstalk cancellation. This ensures that the adaptive crosstalk cancellation based on this attenuation amount matches the sound bias of the eccentric azimuth signal, achieving a targeted adaptive crosstalk cancellation effect for different audio objects with eccentric azimuth signals.

[0145] When performing adaptive crosstalk cancellation, the delay and attenuation of the left channel off-center azimuth signal, as well as the right channel off-center azimuth signal, can be calculated first. The attenuation calculation can incorporate the previously calculated adaptive spreading coefficient, thereby adjusting the attenuation based on the bias of the off-center azimuth signal. For example, such as... Figure 14 As shown, taking the distance D (e.g., 40cm, 30cm, etc.) between the center of the head and the external speaker, the angle θ between the center of the head and the origin of the external speaker, the distance K between the external speaker and the origin, and the radius R of the head as an example, the signal delay and attenuation can be calculated using the following formula:

[0146]

[0147]

[0148]

[0149]

[0150] Where δ1 is the average value of the adaptive expansion coefficients calculated in the previous example and mapped according to the mapping function f(x). For example, δ1 = f(∑ m，k Ψ(m,k) / mk), due to the aforementioned adaptive expansion coefficient, can adapt to different biases in the sound of the eccentric azimuth signal. Therefore, by introducing the average value of the expansion coefficient after mapping the adaptive expansion coefficient into the attenuation, the calculated attenuation can be adjusted according to the bias of the eccentric azimuth signal. This allows the adaptive crosstalk cancellation based on the attenuation to adapt to the bias of the eccentric azimuth signal, thus achieving targeted adaptive crosstalk cancellation for eccentric azimuth signals corresponding to different audio objects; Δt L and Δt R Δα represents the delay of the left channel eccentricity signal and the right channel eccentricity signal, respectively. L and Δα R These represent the attenuation amounts of the left channel off-center azimuth signal and the right channel off-center azimuth signal, respectively.

[0151] Finally, the left channel eccentricity signal and the right channel eccentricity signal of the first time frame of the adaptively crosstalk-cancelled stereo voice signal 4300 can be calculated based on the obtained delay and attenuation. For example, as Figure 15 As shown, the left and right channel signals (i.e., the left channel off-center signal L and the right channel off-center signal R) can be first divided into three frequency bands: high frequency, low frequency, and mid frequency. Secondly, crosstalk suppression is applied to the mid-frequency band, which is most sensitive to the human ear. Taking left-ear crosstalk cancellation as an example: the right channel off-center signal is inverted, attenuated, and delayed to generate an equal-amplitude, inverted signal of left-ear crosstalk, which is then superimposed on the left channel off-center signal. This cancels out the crosstalk caused by the right channel off-center signal to the left ear, achieving crosstalk cancellation (the operation for right-ear crosstalk cancellation is similar). Thirdly, a simple superposition is applied to the less perceptible low and high frequency bands to ensure bass quality and high-frequency timbre. Finally, the high-frequency and low-frequency components of the left and right channels, after time delay and gain adjustment, are superimposed onto the other channel respectively, enhancing the sound field widening effect and expanding the sound field width. These are then superimposed onto the crosstalk-cancelled left and right channel output signals, resulting in adaptive crosstalk-cancelled left and right channel off-center azimuth signals. This eliminates crosstalk between the left and right ears and widens the sound field width of the off-center azimuth signal. Specifically, this can be calculated using the following formula:

[0152]

[0153] Where μ is the attenuation, d is the delay, and β and γ are the low-frequency and high-frequency gains of the channel (preset parameters). L represents the number of iterations. lowL is the low-frequency signal of the left channel eccentricity signal in the first time frame of the human voice stereo signal (e.g., the left channel eccentricity signal within the frequency range obtained by adaptive frequency division in the aforementioned example). mid The intermediate frequency signal (IF) of the left channel off-center azimuth signal in the first time frame of a stereo voice signal, L high R is the high-frequency signal of the left channel off-center azimuth signal in the first time frame of the human voice stereo signal. low R is the low-frequency signal of the right channel eccentricity signal in the first time frame of the human voice stereo signal (e.g., the right channel eccentricity signal within the frequency range obtained by adaptive frequency division in the aforementioned example). mid R is the intermediate frequency signal of the right channel off-center azimuth signal in the first time frame of the human voice stereo signal. high It is the high-frequency signal of the right channel off-center orientation signal in the first time frame of the human voice stereo signal.

[0154] Thus, the final off-center azimuth signal of the first time frame of the stereo human voice signal 4300, after adaptive width control processing, is obtained. By adaptively dividing the frequency range where energy is relatively concentrated, and then performing adaptive crosstalk cancellation on the off-center azimuth signal of the stereo human voice signal 4300 within this frequency range, excessive width expansion of the off-center azimuth signal can be reduced, avoiding distortion after rendering. Before adaptive crosstalk cancellation, an adaptive expansion coefficient is calculated, which can be used to adjust the attenuation of the off-center azimuth signal during transmission during subsequent adaptive crosstalk cancellation. Since the stereo audio signals of different audio objects are different, the adaptive expansion coefficients calculated for their off-center azimuth signals are different. Therefore, by adjusting the attenuation through the adaptive expansion coefficient, targeted adaptive crosstalk cancellation can be performed on the off-center azimuth signals corresponding to different audio objects, improving the processing effect of adaptive crosstalk cancellation. When performing adaptive crosstalk cancellation, the delay and attenuation of the left channel off-center azimuth signal, as well as the right channel off-center azimuth signal, can be calculated first. The calculation of the attenuation can incorporate the previously calculated adaptive expansion coefficient, thereby adjusting the attenuation based on the bias of the off-center azimuth signal. This ensures that the adaptive crosstalk cancellation based on the attenuation matches the sound bias of the off-center azimuth signal, thus achieving targeted adaptive crosstalk cancellation for off-center azimuth signals corresponding to different audio objects. Through adaptive crosstalk cancellation, left and right ear crosstalk can be eliminated, and the sound field widening effect can be enhanced by the superposition of the high-frequency and low-frequency components of the left and right channels after time delay and gain, expanding the sound field width of the final processed off-center azimuth signal. Of course, the above is only a specific example of adaptive width control for off-center azimuth vocals 4312. The specific implementation methods for adaptive width control of off-center azimuth signals of stereo audio signals of other audio objects, such as off-center drum sounds 4412, can also refer to the above example, and will not be elaborated here.

[0155] In this embodiment of the application, based on the foregoing example, the eccentric azimuth signal of each time frame of the human voice stereo signal 4300 can be calculated separately, thereby performing spatial rendering on the eccentric azimuth signal of the entire human voice stereo signal 4300 to obtain the rendered eccentric azimuth signal.

[0156] After rendering the stereo audio signals corresponding to different audio objects, the rendered center and off-center azimuth signals of each audio object's stereo audio signal can be downmixed to obtain the target stereo audio signal, which can then be used by subsequent electronic devices to play audio. For example, based on... Figure 5As shown, the signals after enhanced rendering of the center-position human voice, the signals after spatial rendering of the off-center human voice, the signals after enhanced rendering of the center-position drum sound, and the signals after spatial rendering of the off-center drum sound can be downmixed. Through downmixing, the signals corresponding to each audio object can be remixed, and the center-position signal and the off-center signal can be reintegrated into the left and right channel signals of stereo audio, thereby obtaining the target stereo audio signal.

[0157] For example, before downmixing, other audio effects processing can be applied to the rendered center and off-center signals of each audio object, such as equalization (e.g., EQ), bass enhancement, dynamic range control, and limiting. There are no restrictions here; configuration is based on actual needs. When multiple audio effects are used, the specific processing order is not limited and can be configured according to actual requirements. Through appropriate audio effects processing, the sound field effect of the final target stereo audio signal can be further improved, enhancing the user's listening experience. For example, equalization can adjust the loudness of the audio signal in different frequency ranges, gaining or attenuating specific frequency bands, changing the timbre, and making audio playback more personalized. Bass enhancement can boost the low-frequency components of the audio signal to enhance the dynamism and depth of the music. Dynamic range control can map the dynamic range of the input audio signal to a specified dynamic range, compressing or limiting the signal's dynamic range to make the sound softer or louder. Limiting can limit the maximum amplitude of the audio signal to prevent signal overload or clipping distortion.

[0158] After the electronic device obtains the target stereo audio signal using the method described above, it can play the target stereo audio signal. For example, the electronic device can play the stereo audio through a speaker.

[0159] For example, taking a laptop computer as an example, which includes two speakers located on either side of the keyboard, the laptop can render the original stereo audio signal according to the audio playback method provided in this application to obtain a target stereo audio signal. The laptop can then play audio through the speakers based on this target stereo audio signal. In this case, based on the aforementioned example, the target stereo audio signal has a more immersive central directional sound field and a wider, more profound, layered, and directional off-center directional sound field compared to the original stereo audio signal. Therefore, as... Figure 16 As shown, when a user places the laptop in front of them to play sound, the laptop outputs stereo audio through speakers located on either side of the keyboard. Because the laptop's speakers are located on either side of the keyboard, the user is positioned to the side of the sound field (shown in shaded areas in the diagram). The sound played by the laptop is compared to... Figure 1 The sound shown, based on a traditional external audio solution, offers a wider and fuller sound field. The center of this sound field, enhanced by rendering, feels more immersive, allowing the user to perceive the sound as close to them, thus improving immersion. Furthermore, the off-center areas, after spatial rendering, are wider and possess greater depth, layering, and spatial awareness. Users can perceive distinct locations, varying distances, and pitches of different audio objects in these off-center areas, creating a strong sense of layering and a feeling of being surrounded by sound, resulting in a superior sense of space.

[0160] For example, taking a tablet computer as an example, which includes four speakers located at the four corners of the tablet, the tablet computer can render the original stereo audio signal according to the audio playback method provided in this application to obtain a target stereo audio signal. The tablet computer can then play audio through the speakers based on this target stereo audio signal. When the tablet computer plays the target stereo audio signal, depending on the display orientation of the tablet computer, the two speakers located on the left side of the display orientation can be used to play the left channel of the target stereo audio signal, and the two speakers located on the right side of the display orientation can be used to play the right channel of the target stereo audio signal. In this case, based on the aforementioned example, since the target stereo audio signal has a more immersive central directional sound field and a wider, more profound, layered, and directional off-center directional sound field compared to the original stereo audio signal, therefore... Figure 17 As shown, when a user faces a tablet computer and the tablet plays sound, the user is directly in front of the sound field (shaded in the image). Compared to sound from traditional external speakers, the tablet's sound field is wider and fuller. The center of the tablet's sound field, enhanced by rendering, feels more immersive, making the sound closer to the user and increasing immersion. Furthermore, the off-center areas, after spatial rendering, are wider and have more depth, layering, and spatial awareness. Users perceive distinct locations, varying distances, and pitches of different audio objects in these off-center areas, creating a strong sense of layering and a feeling of being surrounded by sound, thus creating a good sense of space. Compared to the aforementioned... Figure 16 The laptop shown is positioned to the side of the sound field when playing sound, while a user can experience a similar sound field effect when using a tablet positioned in front of the sound field.

[0161] Of course, in practical applications, the electronic device that uses the audio playback method provided in this application to play audio can also be other electronic devices capable of playing audio, besides the laptops and tablets mentioned above; there are no limitations here. The number of speakers provided by the electronic device for playing audio can also be any number, two or more; there are no limitations here.

[0162] According to the audio playback method provided in this application, different audio objects in the original stereo audio signal can be rendered separately, and the location estimation of the stereo signal can be added. It can render the center location signal and the off-center location signal separately. By enhancing the rendering of the center location signal, the volume of the center location signal can be increased, and the sound field distance of the center location signal can be shortened, enhancing the immersive feeling of the center location signal. By spatially rendering the off-center location signal, different locations, distances, and heights corresponding to different off-center location signals can be rendered, and the sound field width of the off-center location signal can be widened. This allows users to perceive a clear sense of location, a layered sense of height, and a sense of depth in terms of distance for different audio objects in the off-center location. Based on the widened sound field, users can feel a sense of width as if they are surrounded by sound. Therefore, the rendered target stereo audio signal has a more immersive center location sound field and a wider, more profound, layered, and directional off-center location sound field. In this way, when electronic devices play audio based on the target stereo audio signal, users can feel the immersive feeling brought by the central directional sound field, the immersive feeling that the sound source corresponding to the central directional sound field is as if it is in front of the user, and the clear sense of direction of the sound of different audio objects brought by the off-center directional sound field, the sense of depth of the sound of different audio objects being near and far, the sense of layering of the sound of different audio objects being high and low, and the sense of width of being surrounded by sound brought by the wider sound of the off-center directional sound field, thus obtaining a good sense of space and a better listening experience.

[0163] Based on the examples given above, those skilled in the art will obviously be able to make various equivalent modifications or changes, which also fall within the scope of the embodiments of this application.

[0164] It is understood that, in order to achieve the aforementioned functions, the electronic device includes corresponding hardware structures and / or software modules for performing each function. Those skilled in the art should readily recognize that, based on the units and algorithm steps of the examples described in conjunction with the embodiments disclosed herein, the embodiments of this application can be implemented in hardware or a combination of hardware and computer software. Whether a function is executed by hardware or by computer software driving hardware depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the embodiments of this application.

[0165] This application also provides an electronic device, which includes: a memory and one or more processors; the memory is coupled to the processors; wherein the memory stores computer program code, which includes computer instructions, and when the computer instructions are executed by the processor, the electronic device performs the method provided in the foregoing embodiments.

[0166] This application also provides a computer-readable storage medium including computer instructions that, when executed on an electronic device, cause the electronic device to perform the method provided in the foregoing embodiments.

[0167] This application also provides a computer program product containing executable instructions that, when run on an electronic device, cause the electronic device to perform the method provided in the foregoing embodiments.

[0168] This application also provides an apparatus (e.g., a system-on-a-chip) including a processor for supporting an electronic device in implementing the methods provided in the foregoing embodiments. In one possible design, the apparatus further includes a memory for storing necessary program instructions and data for the electronic device. When the apparatus is a system-on-a-chip, it can be composed of chips or may include chips and other discrete devices.

[0169] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions within the technical scope disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An audio playback method, characterized in that, The method includes: Launch the first application; In response to the user's first action, acquire the first stereo audio signal; Extract the second stereo audio signal of the first audio object and the third stereo audio signal of the second audio object from the first stereo audio signal. The audio tracks of the first audio object and the second audio object are different. The second stereo audio signal is subjected to orientation estimation to determine the first center orientation signal and the first off-center orientation signal; The third stereo audio signal is subjected to azimuth estimation to determine the second center azimuth signal and the second off-center azimuth signal; The first center azimuth signal is enhanced and rendered to obtain the third center azimuth signal; The second center azimuth signal is enhanced and rendered to obtain the fourth center azimuth signal; Spatial rendering is performed on the first eccentric azimuth signal to obtain the third eccentric azimuth signal; Spatial rendering is performed on the second eccentric azimuth signal to obtain the fourth eccentric azimuth signal; Based on the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal, the target stereo audio signal is obtained by downmixing. Play the target stereo audio signal.

2. The method according to claim 1, characterized in that, The first eccentric azimuth signal includes a first left channel eccentric azimuth signal and a first right channel eccentric azimuth signal, and the second eccentric azimuth signal includes a second left channel eccentric azimuth signal and a second right channel eccentric azimuth signal.

3. The method according to claim 2, characterized in that, The second stereo audio signal includes a first left channel audio signal and a first right channel audio signal. The step of performing azimuth estimation on the second stereo audio signal to determine a first center azimuth signal and a first off-center azimuth signal includes: Perform a short-time Fourier transform on the first left channel audio signal and the first right channel audio signal to obtain the first left channel time-frequency block and the first right channel time-frequency block, the time-frequency block including a real part and an imaginary part; The first center channel contribution factor of the second stereo audio signal is obtained based on the real and imaginary parts of the first left channel time-frequency block and the real and imaginary parts of the first right channel time-frequency block. The first center azimuth signal is obtained based on the first center channel contribution factor, the first left channel audio signal, and the first right channel audio signal. Separate the first center azimuth signal from the first left channel audio signal to obtain the first left channel off-center azimuth signal; The first center azimuth signal in the first right channel audio signal is separated to obtain the first right channel off-center azimuth signal.

4. The method according to claim 2, characterized in that, The spatial rendering of the first eccentric azimuth signal includes: The first eccentric azimuth signal is processed for distance control. The first eccentric azimuth signal is subjected to azimuth and altitude control processing. The first eccentric azimuth signal is subjected to adaptive width control processing.

5. The method according to claim 4, characterized in that, The adaptive width control processing of the first eccentric azimuth signal includes: Adaptive frequency division is performed on the first eccentric azimuth signal to obtain the frequency range in which the energy of the first eccentric azimuth signal is concentrated in a preset percentage. The adaptive spread coefficient of the first eccentric azimuth signal is calculated within the frequency range. Adaptive crosstalk cancellation is performed on the first eccentric azimuth signal based on the adaptive expansion coefficient.

6. The method according to claim 5, characterized in that, The step of adaptively dividing the first eccentric azimuth signal to obtain a frequency range in which the energy of the first eccentric azimuth signal is concentrated at a preset percentage includes: Determine the total energy of the first eccentric azimuth signal; The upper and lower bound frequencies of the frequency range are determined based on the total energy and the preset percentage.

7. The method according to claim 5, characterized in that, The adaptive spread coefficient of the first eccentric azimuth signal calculated within the frequency range includes: Within the specified frequency range, the adaptive expansion coefficient is calculated based on the energy of the first left channel eccentric azimuth signal, the energy of the first right channel eccentric azimuth signal, and the energy difference between the energy of the first left channel eccentric azimuth signal and the energy of the first right channel eccentric azimuth signal.

8. The method according to claim 5, characterized in that, The adaptive crosstalk cancellation of the first eccentric azimuth signal based on the adaptive expansion coefficient includes: Based on the adaptive expansion coefficient, the attenuation of the first eccentric azimuth signal when it is transmitted to the human ear is determined; Determine the delay when the first off-center orientation signal is transmitted to the human ear; Adaptive crosstalk cancellation is performed on the first off-center azimuth signal based on the attenuation amount and the delay amount.

9. The method according to claim 1 or 2, characterized in that, The enhancement rendering of the first center azimuth signal includes: The first center azimuth signal is subjected to energy enhancement processing; Perform distance control processing on the first center azimuth signal; The first center azimuth signal is subjected to de-reverberation processing.

10. The method according to claim 1 or 2, characterized in that, The step of extracting the second stereo audio signal of the first audio object and the third stereo audio signal of the second audio object from the first stereo audio signal includes: The first stereo audio signal is extracted using an audio separation model to obtain the second stereo audio signal of the first audio object and the third stereo audio signal of the second audio object.

11. The method according to claim 1 or 2, characterized in that, The step of downmixing the target stereo audio signal based on the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal includes: Sound effects processing is performed on the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal; The target stereo audio signal is obtained by downmixing the third center azimuth signal, the third off-center azimuth signal, the fourth center azimuth signal, and the fourth off-center azimuth signal after sound effect processing.

12. The method according to claim 11, characterized in that, The sound processing includes at least one of equalization, bass enhancement, dynamic range control, and limiting.

13. An electronic device, characterized in that, The electronic device includes: a processor, a memory, and a communication interface; the memory and the communication interface are coupled to the processor, the memory is used to store computer program code, the computer program code including computer instructions; wherein, when the processor executes the computer instructions, the electronic device performs the method as described in any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions, which, when executed by a processor, perform the method as described in any one of claims 1 to 12.

15. A chip system, characterized in that, The chip system includes a processor and an interface circuit, which are interconnected via a line; wherein the interface circuit is used to receive signals from an electronic device and send the signals to the processor, the signals including computer instructions; when the processor executes the computer instructions, the chip system performs the method as described in any one of claims 1 to 12.

Images