Audio processing method and related equipment

Abstract

The embodiment of the present application provides an audio processing method and related equipment. The method includes: at a first moment, an electronic device plays a first audio in a first application; the hardware layer of the electronic device has a first volume mark; while the electronic device continues to play the first audio, at a second moment, the electronic device receives a trigger operation for playing a second audio in a second application; in response to the trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio; the volume of the second audio is the first volume, and the hardware layer of the electronic device has a first volume mark; at a third moment, the electronic device ends playing the second audio, and continues to play the first audio in the first application at the first volume; the hardware layer of the electronic device has a first volume mark. In this way, the sound effects of the same audio played by the electronic device are the same, reducing the problem of differences in the sound effects of the same audio played twice by the electronic device in the interactive playback scenario.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to an audio processing method and related equipment. Background Art

[0002] An electronic device may be installed with multiple applications, such as social applications, music applications, video applications, and phone applications, etc. These applications may support the electronic device to play audio using a speaker, for example, the electronic device may play voice messages in a social application, the electronic device may play songs in a music application, and so on.

[0003] In some possible scenarios, the electronic device runs a music application, and the music application is in the state of playing music; the electronic device receives a WeChat voice message, and when a trigger operation for playing the voice message is received, the electronic device pauses playing the music and plays the voice message of the WeChat application; after the voice message is finished playing, the electronic device continues to play the music.

[0004] In the above scenario, the music sound effects heard by the user before listening to the voice message are different from the music sound effects heard after listening to the voice message, which affects the user's usage experience. Summary of the invention

[0005] The embodiments of the present application provide an audio processing method and related devices, which are applied to the field of terminal technology. The electronic device can process the audio through the application framework layer and the hardware layer in sequence to improve the sound effect of the audio. At the same time, in the interactive playback scenario, the electronic device plays the same audio twice with the same sound effect to improve the user's listening experience.

[0006] In a first aspect, an embodiment of the present application provides an audio processing method, which is applied to an electronic device. The method includes:

[0007] At a first moment, an electronic device plays a first audio in a first application; the volume of the first audio is the first volume, and the hardware layer of the electronic device has an identification of the first volume; while the electronic device continues to play the first audio, at a second moment, the electronic device receives a trigger operation for playing a second audio in a second application; wherein the audio stream type of the first audio and the second audio is the same; the first moment is earlier than the second moment; in response to the trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio; the volume of the second audio is the first volume, and the hardware layer of the electronic device has an identification of the first volume; at a third moment, the electronic device ends playing the second audio, and continues to play the first audio in the first application at the first volume; the hardware layer of the electronic device has an identification of the first volume; the second moment is earlier than the third moment.

[0008] In this way, when the electronic device plays music for the first time and when the electronic device continues to play music, the hardware layer of the electronic device includes an identifier of the first volume. Therefore, in the scenarios of playing music twice, the hardware layer of the electronic device can use the volume of the media sound to process the music, so that the sound effects of the same audio played by the electronic device are the same, reducing the problem of differences in sound effects when the electronic device plays the same audio twice in similar playback interaction scenarios.

[0009] In a possible implementation, in response to a trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio, including: in response to the trigger operation, the electronic device pauses playing the first audio, the electronic device detects that the abstract audio output device of the hardware abstraction layer has changed, and the second application is in the play thread, and the hardware abstraction layer sends the first volume identifier to the hardware layer; wherein the abstract audio output device is related to the value of the audio calibration database ACDB; the electronic device plays the second audio at the first volume. In this way, the hardware layer includes the identifier of the first volume, and the hardware layer of the electronic device can use the first volume to process the sound effect of the first audio, so that the electronic device plays the same audio with the same sound effect, thereby improving the user experience.

[0010] In a possible implementation, when the audio stream of the first application starts in the electronic device, the electronic device associates the abstract audio output device of the hardware abstraction layer with the first ACDB value of the first application; when the audio stream of the second application starts in the electronic device, the electronic device updates the abstract audio output device of the hardware abstraction layer to associate with the second ACDB value of the second application. In this way, the electronic device can play the corresponding audio through different channels.

[0011] In a possible implementation, when the electronic device ends playing the second audio and continues playing the first audio in the first application at the first volume, the electronic device keeps the abstract audio output device of the hardware abstraction layer associated with the second ACDB value. In this way, when the electronic device continues to play the first audio, the electronic device will not switch the channel, and the first volume in the subsequent ADSP module will not be reset to the default value.

[0012] In a possible implementation, the hardware abstraction layer sends the identifier of the first volume to the hardware layer, including: the hardware abstraction layer sends the identifier of the first volume to the hardware layer by calling the update_sound_effect_param() function. In this way, the hardware layer includes the identifier of the first volume, and the subsequent hardware layer can use the first volume to perform sound effect processing on the second audio.

[0013] In a possible implementation, before the hardware abstraction layer sends the identifier of the first volume to the hardware layer, it also includes: when the audio output device changes, the hardware layer resets the volume value in the hardware layer to the preset volume; the hardware layer sets the volume value in the hardware layer to the first volume according to the identifier of the first volume, including: when the hardware layer obtains the identifier of the first volume, the hardware layer sets the volume value in the hardware layer to the first volume by calling the set_current_volume_index() function. In this way, the hardware layer can set the volume value from the default value to the first volume, and the hardware layer can subsequently use the first volume to process the second audio with sound effects.

[0014] In a possible implementation, before the electronic device plays the first audio in the first application, it includes: the application framework layer of the electronic device processes the first audio in the first application to obtain the processed first audio; the hardware layer of the electronic device processes the processed first audio to obtain the re-processed first audio; the electronic device plays the first audio in the first application, including: the electronic device plays the re-processed first audio through a speaker. In this way, the first audio is the audio processed by the sound effect processing method in the application framework layer and the sound effect processing method in the hardware layer; the first audio played again is also the audio processed by the sound effect processing method in the application framework layer and the sound effect processing method in the hardware layer; so when the electronic device plays the same audio twice, the problem of obvious difference in the sound effect of the first audio caused by playing the second audio in the interactive scene is reduced, the stability of the audio effect is improved, and the user experience is improved.

[0015] In a possible implementation, the hardware layer includes an advanced digital signal processing ADSP module. In this way, the electronic device can perform a second sound effect processing on the audio based on the ADSP module to further improve the audio sound effect.

[0016] In a second aspect, an embodiment of the present application provides an electronic device, which may also be referred to as a terminal, a terminal device, a user equipment (UE), a mobile station (MS), a mobile terminal (MT), etc. The electronic device may be a mobile phone, a smart TV, a wearable device, a tablet computer (Pad), a computer with a wireless transceiver function, a virtual reality (VR) electronic device, an augmented reality (AR) electronic device, a wireless terminal in industrial control, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in transportation safety, a wireless terminal in a smart city, a wireless terminal in a smart home, etc.

[0017] The electronic device comprises: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the method as in the first aspect.

[0018] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the method of the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the method of the first aspect.

[0020] In a fifth aspect, an embodiment of the present application provides a chip, the chip including a processor, the processor being used to call a computer program in a memory to execute the method of the first aspect.

[0021] It should be understood that the second to fifth aspects of the present application correspond to the technical solutions of the first aspect of the present application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation methods are similar and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of an interface for playing interactive scenes provided in an embodiment of the present application;

[0023] Figure 2 A schematic diagram of the structure of an electronic device 100 provided in an embodiment of the present application;

[0024] Figure 3 A schematic diagram of the software structure of the electronic device 100 provided in an embodiment of the present application;

[0025] Figure 4 A schematic diagram of a process of playing a first audio provided in an embodiment of the present application;

[0026] Figure 5 A schematic diagram of an audio stream type provided in an embodiment of the present application;

[0027] Figure 6 A schematic diagram of a process of playing a second audio provided in an embodiment of the present application;

[0028] Figure 7 A schematic diagram of a process of continuing to play a first audio provided in an embodiment of the present application;

[0029] Figure 8 A schematic diagram of another process of playing a first audio provided in an embodiment of the present application;

[0030] Fig. 9 A schematic diagram of another process of playing a second audio provided in an embodiment of the present application;

[0031] Fig.10 A schematic diagram of another process of continuing to play a first audio provided in an embodiment of the present application;

[0032] Fig.11 A flowchart of an audio processing method provided in an embodiment of the present application;

[0033] Fig.12 A flowchart of another audio processing method provided in an embodiment of the present application;

[0034] Fig.13 A schematic diagram of internal interaction of an electronic device provided in an embodiment of the present application;

[0035] Fig.14 A schematic diagram of internal interaction of another electronic device provided in an embodiment of the present application;

[0036] Fig.15 This is a schematic diagram of internal interaction of another electronic device provided in an embodiment of the present application;

[0037] Fig.16 A schematic diagram of the structure of an audio processing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0038] In order to clearly describe the technical solutions of the embodiments of the present application, some terms and technologies involved in the embodiments of the present application are briefly introduced below:

[0039] 1. Sound effect processing method

[0040] 1) DTS sound effects

[0041] DTS sound is a high-quality digital audio encoding technology that can compress multi-channel audio data into a digital stream to provide audio performance that exceeds CD standards in movies, games and other media, thereby providing users with a better sound experience. DTS sound encoding technology can provide a higher dynamic range, a higher compression ratio, more channel conversions, etc., thereby providing users with a better sound experience.

[0042] 2) Dolby Audio

[0043] Dolby sound effect refers to Dolby directional logic surround sound, which can synthesize four-channel stereo into two channels through specific encoding means during recording, that is, the original four signals of left channel (L), right channel (R), center channel (C), and surround channel (S) are encoded and synthesized into LT, RT composite two-channel signals. During playback, the encoded two-channel composite signals LT and RT are restored through the decoder to four independent signals of left, right, center, and surround that do not interfere with each other. After amplification, they are input into the left speaker, right speaker, center speaker and surround speakers respectively.

[0044] 3) Histen sound effects

[0045] Histen audio effects use the latest and most advanced audio processing algorithms, combined with psychoacoustic models and auditory perception models, to maximize the performance of the audio systems of various mobile devices, restore the texture and clarity of high-quality sound, and expand the sense of space and direction of sound through three-dimensional (three dimensions) 3D sound field technology, providing users with a better sound quality experience.

[0046] The embodiments of the present application introduce three sound effect processing methods: DTS sound effect, Dolby sound effect and Histen sound effect, but the embodiments of the present application are not limited to these three sound effect processing methods.

[0047] 2. Cascading sound effects

[0048] Cascading audio refers to processing audio using at least two audio processing methods.

[0049] Exemplarily, the application framework layer of the electronic device includes a first sound effect processing method, and the advanced digital signal processor (ADSP) chip in the hardware layer of the electronic device is deployed with a second sound effect processing method, and the second sound effect processing method can make up for the defects of the first sound effect processing method. When the electronic device detects audio, the audio can be processed by the first sound effect processing method in the application framework layer to obtain processed audio, and the processed audio can be processed again by the second sound effect processing method in the hardware layer to obtain reprocessed audio. The first sound effect processing method and the second sound effect processing method can be any two of the above three sound effect processing methods.

[0050] 3. Other terms

[0051] In the embodiments of the present application, words such as "first" and "second" are used to distinguish the same items or similar items with substantially the same functions and effects. For example, the first time period and the second time period are only used to distinguish different time periods, and do not limit their order. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0052] It should be noted that in the embodiments of the present application, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0053] In the embodiments of the present application, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the previous and next associated objects are in an "or" relationship. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can represent: a, b, c, ab, a--c, bc, or abc, where a, b, c can be single or multiple.

[0054] It should be noted that the "at..." in the embodiment of the present application can be the instant when a certain situation occurs, or can be a period of time before or after a certain situation occurs, and the embodiment of the present application does not specifically limit this. In addition, the display interface provided in the embodiment of the present application is only an example, and the display interface can also include more or less content.

[0055] In a possible implementation, the electronic device can play audio from multiple applications through a speaker. In some scenarios, when the electronic device plays audio from the same application in sequence, the sound effects of the audio in the application may be different.

[0056] For example, Figure 1 A schematic diagram of a scenario in which a difference in audio sound effects occurs in a possible implementation is shown.

[0057] In scenario 1, the electronic device runs a music application and plays music. Figure 1 As shown in interface a in , the play button 101 is in the play state.

[0058] In scenario 2, when the electronic device is playing music, it receives a WeChat voice message. The electronic device can display the WeChat chat interface, such as Figure 1 As shown in interface b in FIG. 1 , the user can click on the WeChat voice message 102. When receiving the user's trigger operation on the WeChat voice message 102, the electronic device plays the audio of the WeChat voice message 102. In the process of the electronic device playing the WeChat voice message 102, the electronic device does not play the music in the music application.

[0059] In scenario three, when the electronic device finishes playing the WeChat voice message, the electronic device can continue to play music. For example, after the electronic device finishes playing the WeChat voice message 102, the electronic device can display Figure 1 In the c interface. Figure 1 In the interface shown in c, the electronic device displays that the play button 103 is in the play state, and the electronic device continues to play music.

[0060] In scenario one, the electronic device plays music; in scenario three, the electronic device continues to play music; but the sound effects of the music heard by the user are different. For example, when the electronic device continues to play music, the volume of the music heard by the user may become louder or quieter, resulting in the user needing to reset the volume value of the music based on the volume button in the setting application or the volume button on the side of the electronic device, affecting the user's experience.

[0061] In the above scenario, the reason why the electronic device has different sound effects between the music heard by the user twice is because:

[0062] In the embodiment of the present application, the electronic device is deployed with cascaded sound effects, for example, the application framework of the electronic device is deployed with sound effect processing method 1, and the ADSP module of the hardware layer of the electronic device is deployed with sound effect processing method 2. The sound effect processing method 1 deployed by the application framework can be understood as the audio processing method in the native process, and the sound effect processing method 2 deployed by the ADSP chip can be understood as: further processing the audio data in the audio processing method in the native process to improve the sound effect, sound quality, etc. of the audio.

[0063] When the electronic device plays music from a music application, the electronic device uses the first channel to play the audio; wherein the volume value set in the ADSP module is related to the volume value of the music, and the ADSP module processes the music audio based on the sound effect processing method 2 and the volume value of the music.

[0064] When an electronic device plays a WeChat voice message from the WeChat application, the electronic device uses the second channel to play the audio. The switching of the channel causes the ADSP module to reload the sound effect processing method 2, causing the volume value in the ADSP module to become the default value. After the electronic device resumes playing music, the electronic device continues to use the second channel to play the audio, and no channel switching occurs. The volume value in the ADSP module uses the default value, which has nothing to do with the volume value of the music. The ADSP module processes the music audio based on the sound effect processing method 2 and the default value. Therefore, the volume parameters used by the ADSP module when processing music audio are different, resulting in different music sound effects played by the electronic device.

[0065] It can be understood that the change in the volume value of the music heard by the above user can be understood as follows: when the music is played for the first time, the ADSP module uses the volume of the music to perform sound effects on the music audio; when the music is played for the second time, the ADSP module uses the default value (for example, the volume value is 0) to process the music audio, so the user perceives a change in the music sound effect that the playback sound may become louder or smaller.

[0066] In view of this, an embodiment of the present application provides an audio processing method, when an electronic device is playing audio, if the electronic device detects that a device (or channel) has been switched, the electronic device will resend the volume value of the audio to the ADSP side, so that the ADSP side can process the audio processed by audio processing method 1 based on the volume value of the audio and audio processing method 2. In this way, when the device that the electronic device uses to play audio is switched, the electronic device can still obtain the audio processed based on audio processing method 2 and the volume value of the audio, and the audio effect is relatively stable. For example, the volume of the music played by the electronic device heard by the user is the same as the volume of the music continued to be played by the electronic device, thereby improving the stability of the audio effect and improving the user experience.

[0067] In order to better understand the embodiments of the present application, the structure of the electronic device in the embodiments of the present application is introduced below:

[0068] Figure 2 1 shows a schematic diagram of the structure of the electronic device 100. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0069] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0070] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0071] The controller can generate an operation control signal according to the instruction operation code and the timing signal to complete the control of fetching and executing instructions. A memory can also be set in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory can save instructions or data that the processor 110 has just used or circulated. If the processor 110 needs to use the instruction or data again, it can be called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved.

[0072] In some embodiments, the processor 110 may be specifically an ADSP. The ADSP is used to process audio data, such as performing sound effect processing on received voice data and performing mixing processing on multiple voice data.

[0073] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuits sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0074] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0075] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. The antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0076] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0077] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0078] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0079] The internal memory 121 can be used to store computer executable program codes, and the executable program codes include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0080] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.

[0081] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0082] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.

[0083] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0084] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.

[0085] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.

[0086] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.

[0087] The software system of the electronic device 100 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture, etc. The embodiment of the present application takes the Android system of the layered architecture as an example to exemplify the software structure of the electronic device 100.

[0088] Figure 3It is a schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application. The layered architecture divides the software into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the Android system can be divided into four layers, namely, the application layer, the application framework layer, the hardware abstraction layer (HAL) and the kernel layer (kernel) from top to bottom. The hardware architecture of the electronic device also includes a hardware layer, which can exchange information with the software architecture layer of the electronic device through an interface.

[0089] The application layer can include a series of application packages. The application layer runs applications by calling the application programming interface (API) provided by the application framework layer. Figure 3 As shown, the application package may include applications such as music applications, social applications, video applications, settings applications, email applications, calendar applications, map applications, and phone applications. Among them, the electronic device can play audio based on applications such as music applications, social applications, video applications, and map applications. For example, the electronic device can play songs based on the music application, the electronic device can play voice messages and voice calls based on the social application, the electronic device can play videos and audio based on the video application, and the electronic device can play navigation routes based on the map application. The settings application can be used to set the volume of the electronic device according to the user operation. For example, a volume button is set on the side of the electronic device, and the user can adjust the volume of the electronic device by pressing the volume button.

[0090] The application framework layer provides an application programming interface (API) and a programming framework for the applications in the application layer. The application framework layer includes some predefined functions.

[0091] like Figure 3As shown, the application framework layer may include a java layer and a native layer. The java layer is used to provide a programming framework for the java language, and the native layer is used to provide a programming framework for the C language or C++. Among them, the java layer may include an audio acquisition module 1 (AudioTrack_1), an audio system module 1 (AudioSystem_1) and an audio management module (AudioManager). The native layer includes an audio acquisition module 2 (AudioTrack_2), an audio system module 2 (AudioSystem_2), an audio delivery module (Audioflinger) and an audio policy management module (Audiopolicymanager). The audio delivery module includes DTS sound effects. It should be noted that the audio delivery module including DTS sound effects is only an example, and the embodiment of the present application does not limit the specific sound effect processing method included in the audio delivery module.

[0092] The audio acquisition module 1 is used to acquire the audio of at least one application in the application layer and pass it to the audio acquisition module 2. The audio acquisition module 2 is used to obtain the audio from the audio acquisition module 1 and pass it to the audio delivery module. The audio delivery module is used to obtain the audio from the audio acquisition module 2 and process the audio through the DTS sound effect to obtain the processed audio.

[0093] The audio management module is used to obtain audio information, such as the audio stream type, when the user uses the electronic device to play audio. The audio management module is also used to pass the audio stream type to the audio system module 1. The audio system module 1 is used to obtain the audio stream type from the audio management module and pass the audio stream type to the audio system module 2. The audio system module 2 is used to obtain the audio stream type from the audio system module 1 and pass the audio stream type to the audio policy management module.

[0094] The purpose of the HAL layer is to abstract the hardware and provide a unified interface for querying hardware devices for upper-layer applications, or to provide data storage services for upper-layer applications. Figure 3 As shown, the HAL layer may include an audio output module (audio_stream_out) and an audio device module (audio_hw_device).

[0095] The audio policy management module is used to obtain the audio stream type from the audio system module 2 and transmit the audio stream type to the audio device module.

[0096] In the scenario where the electronic device plays audio, the audio device module can instruct the audio output module corresponding to the audio stream type to transmit the audio stream, and the music device module can also transmit the volume corresponding to the audio stream type to the corresponding audio output module. For example, when it is detected that the audio path (the path can be an internal path for audio stream transmission controlled by an audio calibration database (acdb)) changes, or the device corresponding to the path changes, the audio device module can send the volume corresponding to the audio stream type to the audio output module.

[0097] The kernel layer is the layer between hardware and software. Figure 3 As shown, the kernel layer may include one or more of the following: camera driver, audio driver, sensor driver, display driver, etc.

[0098] like Figure 3 As shown, the hardware layer may include: ADSP module, speaker, camera, etc. The ADSP module can be deployed in the ADSP chip, and the ADSP module includes XSITEN sound effect. XSITEN sound effect is the name of a sound effect processing method, which can represent Histen sound effect, but the embodiments of the present application are not limited to this. The ADSP module is used to obtain audio processed by DTS sound effect from the kernel layer, and is also used to obtain the volume value corresponding to the audio stream from the hardware abstraction layer when the speaker plays audio. The ADSP module is also used to re-process the audio processed by DTS sound effect through XSITEN sound effect to obtain the re-processed sound effect, and play the re-processed sound effect through the speaker. The speaker can be the above Figure 2 The speaker 170A is shown in the figure, but this embodiment of the present application is not limited to this.

[0099] Figure 3 A brief introduction is given to the software architecture of the electronic device in the embodiment of the present application. The interaction process between software modules will be described in detail in conjunction with the audio processing method provided by the implementation of the present application, which will not be repeated here.

[0100] In an embodiment of the present application, at a first moment, the electronic device can play the first audio of the first application, for example, the first application is a music application, and the first audio is the music in the music application; at a second moment, the electronic device can pause the first audio and play the second audio of the second application, for example, the second application can be a WeChat application, and the second audio is a WeChat voice message; at a third moment, the electronic device ends playing the second audio and continues playing the first audio. Among them, the audio stream type of the first audio is the same as the audio stream type of the second audio, the audio flow path of the first audio is different from the audio flow path of the second audio, and the first moment is earlier than the second moment, and the second moment is earlier than the third moment.

[0101] In the first possible implementation method: at a first moment, the electronic device can obtain the processed first audio through sound effect processing method 1, and obtain the first audio processed again through sound effect processing method 2 and the first volume; at a second moment, the electronic device can obtain the processed second audio through sound effect processing method 1, and obtain the second audio processed again through sound effect processing method 2 and the default volume value; at a third moment, the electronic device can obtain the processed first audio through sound effect processing method 1, and obtain the first audio processed again through sound effect processing method 2 and the default volume value.

[0102] In a second possible implementation method: at a first moment, the electronic device may obtain the processed first audio through sound effect processing method 1, and obtain the first audio processed again through sound effect processing method 2 and the first volume; at a second moment, the electronic device may obtain the processed second audio through sound effect processing method 1, and obtain the second audio processed again through sound effect processing method 2 and the first volume; at a third moment, the electronic device may obtain the processed first audio through sound effect processing method 1, and obtain the first audio processed again through sound effect processing method 2 and the first volume.

[0103] Combine the following Figure 4 , Figure 6 and Figure 7 The audio processing method in the first possible implementation manner in the embodiment of the present application is described.

[0104] Figure 4 Taking the playing of music by an electronic device as an example, a flowchart of an audio processing method in a first possible implementation manner at a first moment is shown. The method can be applied to Figure 1 In the scenario shown, this is not limited to this in the present application embodiment. Figure 4 As shown, the method may include the following steps:

[0105] S401: An electronic device receives a trigger operation for playing a first audio in a display interface of a first application.

[0106] The first application may correspond to a music application, and the first audio may correspond to the audio of the music application, wherein the audio stream type of the first audio may be media audio (AUDIO_STREAM_MUSIC, which may also be expressed as stream type stream=3).

[0107] S402: In response to a triggering operation for playing a first audio, an audio acquisition module of the application framework layer detects the first audio.

[0108] During the process of transmitting audio, the audio acquisition module may transmit the audio stream type stream=3 (AUDIO_STREAM_MUSIC).

[0109] Exemplarily, the audio acquisition module may detect the first audio and the audio stream type of the first audio through a start() interface.

[0110] S403: The audio acquisition module sends the first audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0111] Specifically, the audio acquisition module obtains the first audio and the audio stream type of the first audio, and transmits the first audio and the audio stream type of the first audio to the audio delivery module.

[0112] The audio delivery module is provided with an audio effect processing method 1 (for example, the audio effect processing method 1 may be a DTS audio effect). After obtaining the first audio based on the openoutput() interface, the audio delivery module may use the DTS audio effect to process the audio to obtain the processed first audio.

[0113] The audio output module can obtain the processed first audio from the audio delivery module through the adev_set_parameters() interface.

[0114] It should be noted that, according to the Google native mechanism, the first audio may correspond to the first channel. For example, the first audio is the audio of the music application, and the first channel may be acdb_id = 210. The channel may affect the output device of the audio stream. For example, the audio output module may include device 1, and the first channel may correspond to device 1. Device 1 of the audio output module may obtain the processed first audio.

[0115] S404: The audio output module may obtain a first volume of the first audio, and send the processed first audio and the first volume to the ADSP module.

[0116] The volume value may be a volume value set by the user in the settings application. The audio output module may obtain a volume value corresponding to the audio stream type.

[0117] For example, the audio stream type may include media sound, ringtone, and alarm, etc. The audio stream type and the corresponding volume value may be as follows: Figure 5 For example, the volume value corresponding to media sound 501 is 6, the volume value corresponding to ring tone 502 is 5, the volume value corresponding to alarm tone 503 is 6, etc. The embodiment of the present application only exemplarily describes the setting interface of the audio stream type, but the interface does not limit the solution of the embodiment of the present application.

[0118] The audio output module can obtain the corresponding first volume according to the audio stream type of the first audio. For example, the audio stream type of the first audio is 3, the audio stream type 3 is media audio, and the volume of the media audio is 6 (such as Figure 5 As shown), the first volume is 6.

[0119] It should be noted that when the electronic device plays the audio of the media sound for the first time, the HAL layer will record the volume value corresponding to the media sound. Later, when the electronic device plays the media sound again and the user does not readjust the volume value of the media sound in the settings application during this period, the electronic device can obtain the volume value of the previous media sound playback by calling the start_output_stream function.

[0120] The embodiment of the present application takes the case where the electronic device is not playing the media sound for the first time when playing music as an example to illustrate the process of the audio output module obtaining the first volume. In the embodiment of the present application, the electronic device may also play the media sound (music in the music application) for the first time at the first moment after adjusting the volume value of the media sound, and the electronic device reads the adjusted volume value in the setting application. The embodiment of the present application does not limit this.

[0121] S405, the ADSP module re-processes the processed first audio through the XSITEN sound effect and the first volume to obtain the re-processed first audio.

[0122] It should be noted that after the ADSP module obtains the first volume and the processed first audio, the ADSP module may process the first audio again to obtain the reprocessed first audio, wherein the sound effects of the processed first audio and the reprocessed first audio are different.

[0123] S406. The ADSP module plays the reprocessed first audio through a speaker.

[0124] Accordingly, the electronic device plays the audio in the music application.

[0125] The embodiment of the present application provides an audio processing method, which can perform secondary processing on audio processed by DTS sound effects through XSITEN sound effects to obtain better quality sound effects.

[0126] Figure 6 Taking the electronic device playing a WeChat voice message as an example, a flow chart of the audio processing method in the first possible implementation at the second moment is shown. Figure 6 As shown:

[0127] S601: An electronic device receives a trigger operation for playing a second audio in a display interface of a second application.

[0128] The second application may correspond to the WeChat application, and the second audio may correspond to the audio of the WeChat application, wherein the audio stream type of the second audio may be media audio (AUDIO_STREAM_MUSIC, which may also be expressed as stream type stream=3).

[0129] S602: In response to a trigger operation for playing a second audio, an audio acquisition module of the application framework layer detects the second audio.

[0130] The electronic device stops playing the first audio, and the audio acquisition module detects the second audio.

[0131] During the process of transmitting audio, the audio acquisition module may transmit the audio stream type stream=3 (AUDIO_STREAM_MUSIC).

[0132] Specifically, the audio acquisition module 1 can detect the second audio and the audio stream type through the start() interface. And the audio acquisition module 2 can obtain the second audio and the audio stream type from the audio acquisition module 1 through the start() interface. Among them, the audio stream type can be stream=3.

[0133] S603: The audio acquisition module sends the second audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0134] Specifically, the audio acquisition module obtains the second audio and the audio stream type of the second audio, and transfers the second audio and the audio stream type of the second audio to the audio delivery module.

[0135] After obtaining the second audio based on the openoutput() interface, the audio delivery module may use the DTS sound effect to process the audio to obtain the processed second audio.

[0136] The audio output module can obtain the processed second audio from the audio delivery module through the adev_set_parameters() interface.

[0137] It should be noted that, according to the Google native mechanism, the second audio may correspond to the second channel. For example, the second audio is the audio of the WeChat application, and the second channel may be acdb_id = 214. The channel may affect the output device of the audio stream. For example, the audio output module also includes device 2, and the second channel may correspond to device 2. Device 2 of the audio output module may obtain the processed second audio.

[0138] In the embodiment of the present application, the path for the electronic device to play the audio stream type stream=3 is switched from device 1 to device 2.

[0139] S604: The audio output module sends the processed second audio to the ADSP module of the hardware layer.

[0140] It is understandable that between the first moment and the second moment, the electronic device does not receive a trigger operation for adjusting the volume value, and the audio stream type of the first audio and the second audio is the same, so the second volume corresponding to the second audio is the same as the first volume. The electronic device may not repeatedly obtain the volume value. Therefore, the processed second audio is sent to the ADSP here, and the first volume or the second volume is not sent.

[0141] S605, the ADSP module reloads the XSITEN sound effect and obtains the processed second audio.

[0142] The ADSP module obtains the processed first audio from device 1. After stopping receiving the first audio, the ADSP module obtains the processed second audio from device 2. Since the device transmitting the audio stream is switched from device 1 to device 2, the device is switched and the ADSP module needs to reload the XSITEN sound effect. During the process of reloading the XSITEN sound effect, the volume value in ADSP is reset to the default value, which may be 0.

[0143] S606. The ADSP module processes the processed second audio again through the XSITEN sound effect and default values ​​to obtain the reprocessed second audio.

[0144] Since the volume value in the XSITEN sound effect is the default value, not the first volume, there is a difference between the sound effect of the second audio after reprocessing and the sound effect of the first audio after reprocessing.

[0145] It is understandable that since the first audio and the second audio are audio from different applications, although the sound effects of the first audio and the second audio are different, the user may not perceive it obviously. For example, a user uses an electronic device to play music, and during the music playback, the user uses the electronic device to play a WeChat voice message. The volume of the WeChat voice message heard by the user is related to factors such as the volume of the voice message sender and the media volume of the electronic device; therefore, the contrast between the first audio and the second audio is weak, and the user perceives that the difference in sound effects is not obvious.

[0146] S607. The ADSP module plays the processed second audio through a speaker.

[0147] Correspondingly, the electronic device plays the WeChat voice message in the WeChat application.

[0148] It should be noted that the embodiment of the present application uses a default value of 0 to illustrate the audio processing method in the embodiment of the present application. The default value may also be other values, and the embodiment of the present application does not limit this.

[0149] Figure 7Taking the electronic device continuing to play music as an example, a flow chart of the audio processing method in the first possible implementation at the third moment is shown. Figure 7 As shown:

[0150] S701: The electronic device stops playing the second audio, and the audio acquisition module obtains the first audio.

[0151] The first audio may correspond to the music audio in the first application at the third moment. The first audio may be the same audio as the first audio, or may be a different audio. The audio stream type of the first audio is stream=3.

[0152] Specifically, the audio acquisition module can detect the first audio and the audio stream type through the start() interface.

[0153] Correspondingly, when the electronic device finishes playing the WeChat voice message, the electronic device can continue to play the music. The process of the electronic device continuing to play the music can be shown in steps S702-S705:

[0154] S702: The audio acquisition module sends the first audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0155] Specifically, the audio acquisition module obtains the first audio and the audio stream type of the first audio, and transmits the first audio and the audio stream type of the first audio to the audio delivery module.

[0156] After obtaining the first audio based on the openoutput() interface, the audio delivery module may use the DTS sound effect to process the audio to obtain the processed first audio.

[0157] The audio output module can obtain the processed first audio from the audio delivery module through the adev_set_parameters() interface.

[0158] It should be noted that, in the native mechanism, the electronic device uses the channel with acdb_id=210 to play music, and the electronic device uses the channel with acdb_id=214 to play WeChat voice messages. After finishing playing the WeChat voice message, the electronic device still uses the channel with acdb_id=214 to play music, so when playing music again, the electronic device does not switch the channel, and the first audio continues to use the second channel of the second audio (for example, the channel of the first audio is acdb_id=214), so in step S702, the audio output module is still device 2.

[0159] S703: The audio output module sends the processed first audio to the ADSP module of the hardware layer.

[0160] S704, the ADSP module re-processes the processed first audio through the XSITEN sound effect and default values ​​to obtain the re-processed first audio.

[0161] It can be understood that since the device transmitting the second audio is device 2 and the device transmitting the first audio is also device 2, no device switching occurs in the process, so the ADSP module does not need to reload the XSITEN sound effect, and the volume value in the ADSP module remains the default value.

[0162] S705. The ADSP module plays the reprocessed first audio through a speaker.

[0163] Correspondingly, the electronic device plays the music in the music application again.

[0164] It is understandable that when playing music for the first time, the electronic device processes the processed first audio again through the XSITEN sound effect, and the volume parameter in the XSITEN sound effect is the first volume; and when playing music for the second time, the electronic device processes the processed first audio again through the XSITEN sound effect, and the volume parameter in the XSITEN sound effect is the default value; the change in the volume parameter causes the sound effects of the electronic device playing music before and after to be different.

[0165] Combine the following Figure 8 , Fig. 9 and Fig.10 The audio processing method in the second possible implementation manner in the embodiment of the present application is described.

[0166] Figure 8 Taking the playing of music by an electronic device as an example, a flowchart of an audio processing method in a second possible implementation manner at a first moment is shown. The method can be applied to Figure 1 In the scenario shown, this is not limited to the embodiments of the present application.

[0167] like Figure 8 As shown, the method may include the following steps:

[0168] S801: An electronic device receives a trigger operation for playing a first audio in a display interface of a first application.

[0169] The first application may correspond to a music application, and the first audio may correspond to the audio of the music application, wherein the audio stream type of the first audio may be media audio (AUDIO_STREAM_MUSIC, which may also be represented as stream type stream=3). The audio stream type may also include ringtones (AUDIO_STREAM_RING, stream=2) and alarms (AUDIO_STREAM_ALARM, stream=4).

[0170] S802: In response to a trigger operation for playing a first audio, an audio acquisition module of the application framework layer detects the first audio.

[0171] During the process of transmitting audio, the audio acquisition module may transmit the audio stream type stream=3 (AUDIO_STREAM_MUSIC).

[0172] Specifically, the audio acquisition module 1 can detect the first audio and the audio stream type through the start() interface. And the audio acquisition module 2 can obtain the first audio and the audio stream type from the audio acquisition module 1 through the start() interface. Among them, the audio stream type can be stream=3.

[0173] Among them, the audio acquisition module 1 can be a java file of the java layer, and the audio acquisition module 2 can be a CPP file of the native layer. The java layer and the native layer can communicate through the java-native interface (java native interface, JNI), and JNI can integrate the java file and the CPP file together.

[0174] S803: The audio acquisition module sends the first audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0175] Specifically, the audio acquisition module obtains the first audio and the audio stream type of the first audio, and transfers the first audio and the audio stream type of the first audio to the audio delivery module.

[0176] The audio delivery module is provided with an audio effect processing method 1 (for example, the audio effect processing method 1 may be a DTS audio effect). After obtaining the first audio based on the openoutput() interface, the audio delivery module may use the DTS audio effect to process the audio to obtain the processed first audio.

[0177] The audio output module can obtain the processed first audio from the audio delivery module through the adev_set_parameters() interface.

[0178] It should be noted that, according to the Google native mechanism, the first audio may correspond to the first channel. For example, the first audio is the audio of the music application, and the first channel may be acdb_id = 210. The channel may affect the output device of the audio stream. For example, the audio output module may include device 1, and the first channel may correspond to device 1. Device 1 of the audio output module may obtain the processed first audio.

[0179] When the audio stream of the first application is started in the electronic device, the abstract audio output device of the hardware abstraction layer is associated with the first ACDB value of the first application. After the electronic device calls the start_output() function to obtain the first audio, the electronic device can load the ACDB file corresponding to the first audio to obtain the first ACDB value, wherein the hardware abstraction layer can be provided with an ACDB file. For example, acdb_id=210 is associated with device 1 of the audio output module.

[0180] S804: The audio output module may obtain a first volume of the first audio, and send the processed first audio and the first volume to the ADSP module.

[0181] The volume value may be a volume value set by the user in the settings application. The audio output module may obtain a volume value corresponding to the audio stream type.

[0182] For example, the audio stream type may include media sound, ringtone, and alarm, etc. The audio stream type and the corresponding volume value may be as follows: Figure 5 For example, the volume value corresponding to the media sound 501 is 6, the volume value corresponding to the ring tone 502 is 5, the volume value corresponding to the alarm tone 503 is 6, and so on.

[0183] The audio output module can obtain the corresponding first volume according to the audio stream type of the first audio. For example, the audio stream type of the first audio is 3, the audio stream type 3 is media audio, and the volume of the media audio is 6 (such as Figure 5 As shown), the first volume is 6.

[0184] It should be noted that when the electronic device plays the audio of the media sound for the first time, the HAL layer will record the volume value corresponding to the media sound. Later, when the electronic device plays the media sound again and the user does not readjust the volume value of the media sound in the settings application during this period, the electronic device can obtain the volume value of the previous media sound playback by calling the start_output_stream function.

[0185] The embodiment of the present application takes the case where the electronic device is not playing the media sound for the first time when playing music as an example to illustrate the process of the audio output module obtaining the first volume. In the embodiment of the present application, the electronic device may also play the media sound (music in the music application) for the first time at the first moment after adjusting the volume value of the media sound, and the electronic device reads the adjusted volume value in the setting application. The embodiment of the present application does not limit this.

[0186] S805, the ADSP module re-processes the processed first audio through the XSITEN sound effect and the first volume to obtain the re-processed first audio.

[0187] The volume value in the ADSP module is the first volume, for example, the value of xsiten_process_volume_index is 6.

[0188] It should be noted that after the ADSP module obtains the first volume and the processed first audio, the ADSP module may process the first audio again to obtain the reprocessed first audio, wherein the sound effects of the processed first audio and the reprocessed first audio are different.

[0189] S806. The ADSP module plays the reprocessed first audio through a speaker.

[0190] Accordingly, the electronic device plays the audio in the music application.

[0191] The embodiment of the present application provides an audio processing method, which can perform secondary processing on audio processed by DTS sound effects through XSITEN sound effects to obtain better quality sound effects.

[0192] Fig. 9 Taking the electronic device playing a WeChat voice message as an example, a flow chart of the audio processing method in the second possible implementation at the second moment is shown. Fig. 9 As shown:

[0193] S901. An electronic device receives a trigger operation for playing a second audio in a display interface of a second application.

[0194] The second application may correspond to the WeChat application, and the second audio may correspond to the audio of the WeChat application, wherein the audio stream type of the second audio may be media audio (AUDIO_STREAM_MUSIC, which may also be expressed as stream type stream=3).

[0195] S902: In response to a trigger operation for playing a second audio, an audio acquisition module of the application framework layer detects the second audio.

[0196] The electronic device stops playing the first audio, and the audio acquisition module detects the second audio.

[0197] During the process of transmitting audio, the audio acquisition module may transmit the audio stream type stream=3 (AUDIO_STREAM_MUSIC).

[0198] Specifically, the audio acquisition module 1 can detect the second audio and the audio stream type through the start() interface. And the audio acquisition module 2 can obtain the second audio and the audio stream type from the audio acquisition module 1 through the start() interface. Among them, the audio stream type can be stream=3.

[0199] S903: The audio acquisition module sends the second audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0200] Specifically, the audio acquisition module obtains the second audio and the audio stream type of the second audio, and transfers the second audio and the audio stream type of the second audio to the audio delivery module.

[0201] After obtaining the second audio based on the openoutput() interface, the audio delivery module may use the DTS sound effect to process the audio to obtain the processed second audio.

[0202] The audio output module can obtain the processed second audio from the audio delivery module through the adev_set_parameters() interface.

[0203] It should be noted that, according to the Google native mechanism, the second audio may correspond to the second channel. For example, the second audio is the audio of the WeChat application, and the second channel may be acdb_id = 214. The channel may affect the output device of the audio stream. For example, the audio output module also includes device 2, and the second channel may correspond to device 2. Device 2 of the audio output module may obtain the processed second audio.

[0204] When the audio stream of the second application is started in the electronic device, the abstract audio output device of the hardware abstraction layer is associated with the second ACDB value of the second application. After the electronic device calls the start_output() function to obtain the second audio, the electronic device can load the ACDB file corresponding to the second audio to obtain the second ACDB value, wherein the hardware abstraction layer can be provided with an ACDB file. For example, acdb_id=214 is associated with device 2 of the audio output module.

[0205] In the embodiment of the present application, the path for the electronic device to play the audio stream type stream=3 is switched from device 1 to device 2.

[0206] S904: When the audio device module detects that the device is switched and the electronic device is running in a play thread, the audio device module obtains a first volume.

[0207] Device switching can be understood as the audio output module switching from device 1 to device 2 in step S903. In the embodiment of the present application, the audio device module of the electronic device can determine that the device is switched by calling the adev_create_audio_patch() function.

[0208] The playback thread can be used to represent that the current application is in the state of playing audio; when the application program running in the electronic device, the functional component or system program built into the electronic device system is running, a corresponding program thread will be generated in the background of the electronic device. The types of program threads are different, such as: the thread of the player program, the thread of the browser program, or the thread of the core file required for the system operation, so it is necessary to determine the thread related to the playback.

[0209] Among them, threads related to playback include: threads generated when an APP with an audio playback function running in an electronic device or a player built into the system of an e-book device (such as an Android system) is running. For example, when an electronic device plays a WeChat voice message, the second application runs in the playback thread; for another example, when an electronic device plays music, the first application runs in the playback thread. For example, when an electronic device detects the patch_playback thread, it determines that the electronic device is in the playback thread.

[0210] The audio device module may obtain a first volume, and the first volume may be a volume value sent from the previous audio output module to the ADSP module.

[0211] It should be noted that in the embodiment of the present application, the audio device module may issue an instruction when it detects that the second application occupies the playback thread. The music device module may not determine the specific application currently running, but issue an instruction when it detects that the playback thread is enabled and the device is switched. The embodiment of the present application does not limit this.

[0212] S905: The audio device module transmits the first volume to the audio output module.

[0213] The audio device module transmits the first volume to the device 2 of the audio output module. Specifically, the audio output module can obtain the first volume when calling the start_output_stream function.

[0214] S906. The ADSP module reloads the XSITEN sound effect, and obtains the processed second audio and the first volume.

[0215] Specifically, 1. ADSP obtains the processed second audio from the audio output module. The audio output module may be device 2, and the audio output module may send the first volume to the ADSP module by calling the update_sound_effect_param() function. During this process, the device transmitting the audio stream is switched, and the ADSP module needs to reload the XSITEN sound effect.

[0216] 2. The ADSP module reloads the XSITEN sound effect and sets the volume value to the default value. Since the device receiving the audio stream is switched from device 1 to device 2, the ADSP module needs to reload the XSITEN sound effect. During the reloading of the XSITEN sound effect, the volume value in the ADSP is reset to the default value, which can be 0.

[0217] It should be noted that the ADSP module can obtain information about device changes from the hardware abstraction layer. For example, when the audio device module detects that the device has changed, it sends the information about the device change to the ADSP module, so that the ADSP module resets the XSITEN sound effect based on the information.

[0218] 3. The ADSP module obtains the first volume sent by the audio output module and sets the volume value to the first volume. The audio output module may be device 2, and the ADSP module may obtain the first volume from the audio output module. The volume value in the ADSP module is the default value. After obtaining the first volume, the ADSP module may set the volume value to the first volume by calling the set_current_volume_index() interface.

[0219] The volume value in the ADSP module is the first volume, for example, the value of xsiten_process_volume_index is 6.

[0220] S907, the ADSP module re-processes the processed second audio through the XSITEN sound effect and the first volume to obtain the re-processed second audio.

[0221] At the second moment, the volume value in the XSITEN sound effect is the first volume, and the XSITEN sound effect processes the processed second audio again based on the first volume.

[0222] S908. The ADSP module plays the second processed audio through a speaker.

[0223] Correspondingly, the electronic device plays the WeChat voice message in the WeChat application.

[0224] In an embodiment of the present application, the electronic device can, in the above-mentioned interactive scenario, perform secondary processing on the second audio processed by the DTS sound effect through the XSITEN sound effect to obtain a better quality sound effect.

[0225] Optionally, in step S904, when the audio device module detects device switching and the electronic device is running in the play thread, before the audio device module obtains the first volume, the method further includes:

[0226] S909: In response to a triggering operation for playing the second audio, the audio management module obtains properties of the second audio.

[0227] The attribute of the second audio may be an output device (device) corresponding to the second channel and a volume value (index) of an audio stream type corresponding to the second audio, etc. For example, the output device corresponding to the second channel is device 2.

[0228] S910: The audio management module transmits the attribute of the second audio to the audio system module.

[0229] Specifically, after obtaining the attributes of the second audio, the audio management module transmits the attributes of the second audio to the audio system module 1 .

[0230] After obtaining the attributes of the second audio, the audio system module 1 transmits the attributes of the second audio to the audio system module 2.

[0231] The audio system module 1 may be a java file of the java layer, and the audio system module 2 may be a CPP file of the native layer. The java layer and the native layer may communicate via JNI, and JNI may integrate the java file and the CPP file together.

[0232] S911. The audio system module transmits the attribute of the second audio to the audio policy service module.

[0233] S912: The audio policy service module sends the attributes of the second audio to the audio device module.

[0234] It should be noted that steps S901-S909 can be a process for the electronic device to transmit the audio stream of the second audio from the upper layer of the software architecture so that the underlying hardware plays the second audio after processing again. Modules such as the audio management module, the audio system module, and the audio policy service module can be used to manage the control process of the audio stream. For example, after the second application processes the process related to the trigger operation for playing the second audio, the audio management module obtains the properties of the second audio, the audio system module and the audio policy service module play a connection function, and after the audio policy service module obtains the properties of the second audio, it sends the properties of the second audio to the audio device module of the hardware abstraction layer.

[0235] S913: The audio device module instructs the audio output module to transmit the second audio device according to the attribute of the second audio.

[0236] The audio device module may be a class for managing audio devices, and the audio device module may instruct the audio output module to use the corresponding device to transmit the second audio. For example, the audio device module instructs the audio output module to use device 2 to transmit the second audio. In addition, the audio device module may instruct the audio output module (device 2) to open the corresponding audio stream through the open_out_put_stream interface, so that the audio output module establishes a stream object, and encapsulates the audio stream of the second audio of the audio delivery module.

[0237] Fig.10 Taking the electronic device continuing to play music as an example, a flow chart of the audio processing method in the second possible implementation at the third moment is shown. Fig.10 As shown:

[0238] S1001: The electronic device stops playing the second audio, and the audio acquisition module obtains the first audio.

[0239] The first audio may correspond to the music audio in the first application at the third moment. The first audio may be the same audio as the first audio, or may be a different audio. The audio stream type of the first audio is stream=3.

[0240] Specifically, the audio acquisition module 1 can detect the first audio and the audio stream type through the start() interface. And the audio acquisition module 2 can obtain the first audio and the audio stream type from the audio acquisition module 1 through the start() interface. Among them, the audio stream type can be stream=3.

[0241] Correspondingly, when the electronic device finishes playing the WeChat voice message, the electronic device can continue to play the music. The electronic device can continue to play the music as shown in steps S1002-S1005:

[0242] S1002: The audio acquisition module sends the first audio to the audio output module of the hardware abstraction layer via the audio delivery module.

[0243] Specifically, the audio acquisition module obtains the first audio and the audio stream type of the first audio, and transmits the first audio and the audio stream type of the first audio to the audio delivery module.

[0244] After obtaining the first audio based on the openoutput() interface, the audio delivery module may use the DTS sound effect to process the audio to obtain the processed first audio.

[0245] The audio output module can obtain the processed first audio from the audio delivery module through the adev_set_parameters() interface.

[0246] It should be noted that, in the native mechanism, the electronic device uses the channel with acdb_id=210 to play music, and the electronic device uses the channel with acdb_id=214 to play WeChat voice messages. After finishing playing the WeChat voice message, the electronic device still uses the channel with acdb_id=214 to play music, so when playing music again, the electronic device does not switch the channel, and the first audio continues to use the second channel of the second audio (for example, the channel of the first audio is acdb_id=214), so in step S702, the audio output module is still device 2.

[0247] S1003: The audio output module sends the processed first audio to the ADSP module of the hardware layer.

[0248] S1004, the ADSP module re-processes the processed first audio through the XSITEN sound effect and the first volume to obtain the re-processed first audio.

[0249] It can be understood that since the device transmitting the second audio is device 2 and the device transmitting the first audio is also device 2, no device switching occurs in the process. Therefore, the ADSP module does not need to reload the XSITEN sound effect, and the volume value in the ADSP module is still the first volume. For example, the value of xsiten_process_volume_index is 6.

[0250] S1005. The ADSP module plays the reprocessed first audio through a speaker.

[0251] Correspondingly, the electronic device plays the application in the music application again, wherein the volume of the reprocessed first audio heard by the user is the same as the volume of the reprocessed second audio.

[0252] It is understandable that since the first audio after two reprocessings (the first music playback and the continued music playback) has been processed by two sound effect processing methods (once by the application framework layer and once by the ADSP module of the hardware layer), and the volume parameters are the same, the sound effects of the music heard by the user for the first time and the second time are the same. In this way, when switching the audio streams of different applications, the sound effects of the audio played by the electronic device before and after the same application are the same, and the user will not perceive the change in the sound effects, thereby improving the stability of the audio sound effects.

[0253] The above embodiments are combined Figure 4 , Figure 6 and Figure 7 The internal interaction of the software modules in the first possible implementation manner in the embodiment of the present application is described below. Fig.11 The process in the first possible implementation manner is described.

[0254] Fig.11 FIG. 1 is a flow chart of an audio processing method provided by an embodiment of the present application. Fig.11 As shown:

[0255] S1101: At a first moment, an electronic device plays a first audio in a first application, wherein the ADSP module has a first volume indicator.

[0256] Correspondingly, at the first moment, the electronic device plays the music in the music application, wherein the audio stream type of the first audio is stream=3, and the first channel corresponding to the first audio is acdb_id=210.

[0257] There is an identifier for the first volume in the ADSP module, which can be understood as follows: the volume value in the ADSP module is the first volume, and the first volume can be a volume value corresponding to the audio stream type, for example, the first volume can be a volume value of the media sound.

[0258] Step S1101 can refer to Figure 4 The relevant description in will not be repeated here.

[0259] S1102: The electronic device continues to play the first audio. At a second moment, the electronic device receives a trigger operation for playing the second audio.

[0260] Correspondingly, while the electronic device is playing music, the electronic device receives a trigger operation for playing a WeChat voice message.

[0261] Among them, the audio stream type of the second audio is stream=3, and the second channel corresponding to the second audio is acdb_id=214.

[0262] S1103: In response to the trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio; wherein there is no mark of the first volume in the ADSP module.

[0263] Correspondingly, the electronic device pauses playing music and plays WeChat voice messages. Among them, the ADSP module detects changes in the device, and the ADSP module reloads the XSITEN sound effect. The volume value of the ADSP module is the default value, not the first volume.

[0264] Steps S1102-S1103 can be referred to Figure 6 The relevant description in will not be repeated here.

[0265] S1104. At a third moment, the electronic device stops playing the second audio and continues playing the first audio in the first application; wherein there is no mark of the first volume in the ADSP module.

[0266] Correspondingly, the electronic device stops playing the WeChat voice message and continues playing music; wherein, the device has not changed, so the volume value in the ADSP module continues to use the volume value in step S1103, and the volume value in the ADSP module remains the default value.

[0267] exist Figure 8 , Fig. 9 and Fig.10 Based on the embodiment shown, the present application embodiment provides an audio processing method. Exemplarily, Fig.12 A flowchart of an audio processing method provided in an embodiment of the present application.

[0268] like Fig.12 As shown, the audio processing method may include the following steps:

[0269] S1201: At a first moment, an electronic device plays a first audio in a first application; the volume of the first audio is a first volume, and a hardware layer of the electronic device has an identifier of the first volume.

[0270] The first application can be as described above Figure 1 The music application shown in the a interface in the figure, the first audio may correspond to the music in the music application, but the embodiments of the present application are not limited to this. The volume of the first audio is the first volume, which can indicate that the electronic device plays the sound in the first application at the first volume. For example, the first volume is related to the audio stream type of the first audio. The audio stream type of the first audio can be media audio (stream=3), and the first volume is 6 (such as Figure 5 The hardware layer of the electronic device has a first volume identifier, which indicates that the volume value in the hardware layer is the first volume.

[0271] The electronic device plays the first audio in the first application, and the specific implementation method can be as described above Figure 8 S801 to S806 in the above description will not be repeated here.

[0272] S1202: While the electronic device is continuously playing the first audio, at a second moment, the electronic device receives a trigger operation for playing the second audio in a second application; wherein the audio stream type of the first audio and the second audio are the same; and the first moment is earlier than the second moment.

[0273] The second application can be as described above Figure 1The WeChat application shown in the b interface, the second audio may correspond to the WeChat voice message in the WeChat application, but the embodiments of the present application are not limited to this; the audio stream types include: phone tone (voice_call, stream=0), system tone (system, stream=1), ringtone (ring, stream=2), media tone (music, stream=3), alarm (alarm, stream=4), etc.

[0274] The trigger operation for playing the second audio in the second application may correspond to Figure 1 The trigger operation for the WeChat voice message 102 in the interface shown in b in FIG. The audio stream type of the first audio and the second audio is the same, for example, the audio stream type of the second audio is media audio (stream=3).

[0275] S1203: In response to the triggering operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio; the hardware layer of the electronic device has an identifier of the first volume.

[0276] The hardware layer of the electronic device has an identifier of the first volume, which may indicate that the volume value in the hardware layer is the first volume.

[0277] The electronic device plays the second audio in the second application, and the specific implementation method can be as described above Fig. 9 S901 to S913 in the above description will not be repeated here.

[0278] S1204: At a third moment, the electronic device ends playing the second audio and continues playing the first audio in the first application; the hardware layer of the electronic device has an identifier of the first volume; the second moment is earlier than the third moment.

[0279] The hardware layer of the electronic device has an identifier of the first volume, which may indicate that the volume value in the hardware layer is the first volume.

[0280] The electronic device continues to play the first audio in the first application. The specific implementation method can be as described above. Fig.10 S1001 to S1005 in the above description will not be repeated here.

[0281] In an embodiment of the present application, when the electronic device plays music for the first time and when the electronic device continues to play music, the hardware layer of the electronic device includes an identifier of the first volume. Therefore, in the scenarios of playing music twice, the hardware layer of the electronic device can use the volume of the media sound to process the music, so that the sound effects of the same audio played by the electronic device are the same, thereby reducing the problem of differences in the sound effects of the same audio played twice by the electronic device in similar playback interaction scenarios.

[0282] Optionally, in response to a trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio, including: in response to the trigger operation, the electronic device pauses playing the first audio, the electronic device monitors that the abstract audio output device of the hardware abstraction layer changes, and the second application is in the playback thread, and the hardware abstraction layer sends a first volume identifier to the hardware layer; wherein the abstract audio output device is related to the value of the audio calibration database ACDB; the electronic device plays the second audio at the first volume.

[0283] The abstract audio output device may correspond to device 1 and device 2 in the audio output device. The abstract audio output device may be associated with acdb_id. For example, when the electronic device receives music for playing music applications and starts streaming, acdb_id=210, and the abstract audio output device may be device 1; when the electronic device receives a WeChat voice message for WeChat applications and starts streaming, acdb_id=214, and the abstract audio output device may be device 2. The change of the abstract audio output device may be understood as the switching of device 1 to device 2. The electronic device may monitor the change of the abstract audio output device of the hardware abstraction layer by calling the adev_create_audio_patch() function. The playback thread may be the patch_playback thread.

[0284] This step can refer to the relevant description in step S904, which will not be repeated here. In this way, the hardware layer includes the identifier of the first volume, and the hardware layer of the electronic device can use the first volume to process the first audio, so that the electronic device plays the same audio with the same sound effect, thereby improving the user experience.

[0285] Optionally, when the audio stream of a first application is started in the electronic device, the electronic device associates the abstract audio output device of the hardware abstraction layer with the first ACDB value of the first application; when the audio stream of a second application is started in the electronic device, the electronic device updates the abstract audio output device of the hardware abstraction layer to associate with the second ACDB value of the second application.

[0286] Among them, when the audio stream of the first application is started in the electronic device, the abstract audio output device of the hardware abstraction layer is associated with the first ACDB value of the first application, which can be understood as: when the electronic device plays the first audio of the first application, the hardware abstraction layer can obtain the corresponding abstract audio output device according to the first ACDB value of the first audio, for example, the acdb_id of the music in the music application is 210, and the abstract audio output device can be device 1; starting the audio stream of the first application in the electronic device can be understood as: the electronic device calls the shart_output() function.

[0287] When the audio stream of the second application is started in the electronic device, the electronic device updates the abstract audio output device of the hardware abstraction layer and associates it with the second ACDB value of the second application. This can be understood as: when the electronic device plays the second audio of the second application, the hardware abstraction layer can obtain the corresponding abstract audio output device according to the second ACDB value of the second audio. For example, in the WeChat voice message in the WeChat application, the acdb_id=214, and the abstract audio output device can be device 2. Starting the audio stream of the second application in the electronic device can be understood as: the electronic device calls the shart_output() function.

[0288] For this step, reference may be made to the relevant descriptions in steps S803 and S903, which will not be repeated here.

[0289] In this way, the electronic device can play the corresponding audio through different channels.

[0290] Optionally, when the electronic device finishes playing the second audio and continues to play the first audio in the first application at the first volume, the electronic device keeps the abstract audio output device of the hardware abstraction layer associated with the second ACDB value.

[0291] The electronic device maintains the association between the abstract audio output device of the hardware abstraction layer and the second ACDB value. It can be understood that when the electronic device continues to play the first audio of the first application, the hardware abstraction layer can obtain the corresponding abstract audio output device according to the second ACDB value of the second audio. For example, the acdb_id of the WeChat voice message in the WeChat application is 214, and the abstract audio output device can be device 2.

[0292] For this step, please refer to the relevant description in step S1002, which will not be repeated here.

[0293] In this way, when the electronic device continues to play the first audio, the electronic device will not switch the path, and the first volume in the subsequent ADSP module will not be reset to the default value.

[0294] Optionally, the hardware abstraction layer sends the identifier of the first volume to the hardware layer, including: the hardware abstraction layer sends the identifier of the first volume to the hardware layer by calling an update_sound_effect_param() function.

[0295] The process of the hardware abstraction layer sending the first volume can refer to the relevant description in step S906, which will not be repeated here.

[0296] In this way, the hardware layer includes the identifier of the first volume, and the subsequent hardware layer can use the first volume to perform sound effect processing on the second audio.

[0297] Optionally, before the hardware abstraction layer sends the identifier of the first volume to the hardware layer, it also includes: when the audio output device changes, the hardware layer resets the volume value in the hardware layer to a preset volume; the hardware layer sets the volume value in the hardware layer to the first volume according to the identifier of the first volume, including: when the hardware layer obtains the identifier of the first volume, the hardware layer sets the volume value in the hardware layer to the first volume by calling the set_current_volume_index() function.

[0298] Among them, the preset volume may correspond to the default value after the volume value in the ADSP module is reset. When the hardware layer obtains information from the hardware abstraction layer that the audio output device has changed, the ADSP module will reload the XSITEN sound effect. At this time, the volume value in the ADSP module is the default value.

[0299] For this step, please refer to the relevant description in step S906, which will not be repeated here.

[0300] In this way, the hardware layer can set the volume value from the default value to the first volume, and the subsequent hardware layer can use the first volume to perform sound effect processing on the second audio.

[0301] Optionally, before the electronic device plays the first audio in the first application, it includes: the application framework layer of the electronic device processes the first audio in the first application to obtain the processed first audio; the hardware layer of the electronic device processes the processed first audio to obtain the re-processed first audio; the electronic device plays the first audio in the first application, including: the electronic device plays the re-processed first audio through a speaker.

[0302] This step can refer to Figure 8 or Fig.10 The relevant description in the illustrated embodiment will not be repeated here.

[0303] It is understandable that the process in which the electronic device continues to play the first audio in the first application at the first volume is similar to this process and will not be repeated here.

[0304] In this way, the first audio is audio processed by the sound effect processing method in the application framework layer and the sound effect processing method in the hardware layer; the first audio played again is also audio processed by the sound effect processing method in the application framework layer and the sound effect processing method in the hardware layer; therefore, when the electronic device plays the same audio twice, the problem of obvious difference in the sound effect of the first audio caused by playing the second audio in the interactive scene is reduced, thereby improving the stability of the audio sound effect and thus improving the user experience.

[0305] Optionally, the hardware layer includes an advanced digital signal processing ADSP module, so that the electronic device can perform a second audio processing on the audio based on the ADSP module to further improve the audio sound effect.

[0306] Optional, combine the following Fig.13 and Figure 3 The relevant software modules in the electronic device illustrate the internal interaction process of the electronic device executing the audio processing method in the embodiment of the present application. Fig.13 A schematic diagram of an internal interaction of an electronic device provided in an embodiment of the present application is shown, which may correspond to Figure 8 The scenario of the embodiment shown is as follows Fig.13 As shown:

[0307] At a first moment, the electronic device plays a first audio in a first application; the volume of the first audio is a first volume, and the hardware layer of the electronic device has an identifier of the first volume.

[0308] The first application may be a music application.

[0309] Specifically, when the electronic device receives a trigger operation for playing the first audio, the music application of the application layer sends the first audio to the audio acquisition module 1 of the Java layer of the application framework layer; after the audio acquisition module 1 obtains the first audio, it passes the first audio to the audio acquisition module 2 of the native layer through the JNI interface; after the audio acquisition module 2 obtains the first audio, it passes the first audio to the audio delivery module.

[0310] The audio delivery module includes at least one audio effect processing method, such as DTS audio effect. The audio delivery module processes the first audio using DTS audio effect to obtain the processed first audio, and sends the processed first audio to the audio output module of the hardware abstraction layer.

[0311] After the audio output module obtains the processed first audio, it sends the processed first audio to the ADSP module of the hardware layer. The ADSP module includes at least one sound effect processing method, for example, XSITEN sound effect, and the volume value in the XSITEN sound effect is the first volume. The ADSP module processes the processed first audio using the XSITEN sound effect to obtain the reprocessed first audio. The ADSP module transmits the reprocessed first audio to the speaker, and the speaker plays the reprocessed first audio.

[0312] Optional, combine the following Fig.14 and Figure 3 The relevant software modules in the electronic device illustrate the internal interaction process of the electronic device executing the audio processing method in the embodiment of the present application. Fig.14A schematic diagram of an internal interaction of an electronic device provided in an embodiment of the present application is shown, which may correspond to Fig. 9 The scenario of the embodiment shown is as follows Fig.14 As shown:

[0313] During the process of the electronic device continuously playing the first audio, at the second moment, the electronic device receives a trigger operation for playing the second audio in the second application; wherein the audio stream type of the first audio and the second audio is the same; the first moment is earlier than the second moment; in response to the trigger operation, the electronic device pauses playing the first audio, and the electronic device plays the second audio; the volume of the second audio is the first volume, and the hardware layer of the electronic device has an identifier of the first volume. wherein the second application can be a WeChat application.

[0314] Specifically, when the electronic device receives a trigger operation for playing the second audio during the playing of the first audio, the electronic device pauses playing the first audio, and the WeChat application of the application layer sends the second audio to the audio acquisition module 1 of the Java layer of the application framework layer; after the audio acquisition module 1 obtains the second audio, it passes the second audio to the audio acquisition module 2 of the native layer through the JNI interface; after the audio acquisition module 2 obtains the second audio, it passes the second audio to the audio delivery module. The audio delivery module includes at least one sound effect processing method, for example, DTS sound effect. The audio delivery module uses DTS sound effect to process the second audio, obtains the processed second audio, and sends the processed second audio to the audio output module of the hardware abstraction layer.

[0315] In addition, in response to the trigger operation for playing the second audio, the audio management module of the Java layer of the application framework layer obtains the properties of the second audio from the WeChat application of the application layer, and the properties of the second audio include the audio stream type and the output device, etc. The audio management module passes the properties of the second audio to the audio system module 1, and the audio system module 1 passes the properties of the second audio to the audio system module 2 through the JNI interface; after the audio system module 2 obtains the properties of the second audio, it passes the properties of the second audio to the audio policy service module; after the audio policy service module obtains the properties of the second audio, it sends the properties of the second audio to the audio device module of the hardware abstraction layer. The audio device module can monitor device switching and playback threads. Exemplarily, when the audio device module detects that the output device is switched and the electronic device is in the playback thread, the audio device module passes the first volume to the audio output module.

[0316] After obtaining the processed second audio and the first volume, the audio output module sends the processed second audio and the first volume to the ADSP module of the hardware layer. The ADSP module includes at least one sound effect processing method, for example, XSITEN sound effect, in which the volume value is updated from the default value to the first volume, and the ADSP module processes the processed second audio using the XSITEN sound effect to obtain a re-processed second audio. The ADSP module transmits the re-processed second audio to the speaker, and the speaker plays the re-processed second audio.

[0317] Optional, combine the following Fig.15 and Figure 3 The relevant software modules in the electronic device illustrate the internal interaction process of the electronic device executing the audio processing method in the embodiment of the present application. Fig.15 A schematic diagram of an internal interaction of an electronic device provided in an embodiment of the present application is shown, which may correspond to Fig.10 The scenario of the embodiment shown is as follows Fig.15 As shown:

[0318] At the third moment, the electronic device ends playing the second audio and continues playing the first audio in the first application at the first volume; the hardware layer of the electronic device has an identification of the first volume; the second moment is earlier than the third moment.

[0319] Specifically, when the electronic device finishes playing the second audio and continues to play the first audio, the music application of the application layer sends the first audio to the audio acquisition module 1 of the Java layer of the application framework layer; after the audio acquisition module 1 obtains the first audio, it passes the first audio to the audio acquisition module 2 of the native layer through the JNI interface; after the audio acquisition module 2 obtains the first audio, it passes the first audio to the audio delivery module.

[0320] The audio delivery module includes at least one audio effect processing method, such as DTS audio effect. The audio delivery module processes the first audio using DTS audio effect to obtain the processed first audio, and sends the processed first audio to the audio output module of the hardware abstraction layer.

[0321] After the audio output module obtains the processed first audio, it sends the processed first audio to the ADSP module of the hardware layer. The ADSP module includes at least one sound effect processing method, for example, XSITEN sound effect, wherein the XSITEN sound effect is not reloaded, and the volume value in the XSITEN sound effect is still the first volume. The ADSP module processes the processed first audio using the XSITEN sound effect to obtain the reprocessed first audio. The ADSP module transmits the reprocessed first audio to the speaker, and the speaker plays the reprocessed first audio.

[0322] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0323] The audio processing method of the embodiment of the present application has been described above, and the device for executing the above audio processing method provided by the embodiment of the present application is described below. Those skilled in the art can understand that the method and the device can be combined and referenced with each other, and the relevant device provided by the embodiment of the present application can execute the steps in the above audio processing method.

[0324] like Fig.16 As shown, the audio processing device 1600 can be used in a communication device, a circuit, a hardware component or a chip, and the audio processing device includes: a display unit 1601 and a processing unit 1602. The display unit 1601 is used to support the display step performed by the audio processing device 1600; the processing unit 1602 is used to support the audio processing device 1600 to perform the information processing step.

[0325] In a possible implementation, the audio processing device 1600 may also include a communication unit 1603. Specifically, the communication unit is used to support the audio processing device 1600 to perform the steps of sending and receiving data. The communication unit 1603 may be an input or output interface, a pin or a circuit.

[0326] In a possible embodiment, the audio processing device may further include: a storage unit 1604. The processing unit 1602 and the storage unit 1604 are connected via a line. The storage unit 1604 may include one or more memories, and the memory may be a device used to store programs or data in one or more devices or circuits. The storage unit 1604 may exist independently and be connected to the processing unit 1602 of the audio processing device via a communication line. The storage unit 1604 may also be integrated with the processing unit 1602.

[0327] The storage unit 1604 may store computer-executable instructions of the method in the terminal device so that the processing unit 1602 executes the method in the above embodiment. The storage unit 1604 may be a register, a cache, or a RAM, etc. The storage unit 1604 may be integrated with the processing unit 1602. The storage unit 1604 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, and the storage unit 1604 may be independent of the processing unit 1602.

[0328] The audio processing method provided in the embodiment of the present application can be applied to electronic devices with communication functions. The electronic device includes an electronic device, and the specific device form of the electronic device can refer to the above related description, which will not be repeated here.

[0329] An embodiment of the present application provides an electronic device, which includes: a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, so that the electronic device executes the above method.

[0330] The embodiment of the present application provides a chip. The chip includes a processor, and the processor is used to call a computer program in a memory to execute the technical solution in the above embodiment. Its implementation principle and technical effect are similar to those of the above related embodiments, and will not be repeated here.

[0331] The embodiments of the present application also provide a computer-readable storage medium. The computer-readable storage medium stores a computer program. The above method is implemented when the computer program is executed by the processor. The method described in the above embodiment can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. If implemented in software, the function can be stored as one or more instructions or codes on a computer-readable medium or transmitted on a computer-readable medium. Computer-readable media can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium that can be accessed by a computer.

[0332] In one possible implementation, a computer-readable medium may include RAM, ROM, compact disc read-only memory (CD-ROM) or other optical disk storage, disk storage or other magnetic storage devices, or any other medium that is intended to carry or store the required program code in the form of instructions or data structures and can be accessed by a computer. Moreover, any connection is appropriately referred to as a computer-readable medium. For example, if a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL) or wireless technology (such as infrared, radio and microwave) is used to transmit software from a website, server or other remote source, the coaxial cable, fiber optic cable, twisted pair, DSL or wireless technology such as infrared, radio and microwave are included in the definition of medium. Disks and optical disks as used herein include optical disks, laser disks, optical disks, digital versatile disks (DVD), floppy disks and Blu-ray disks, where disks usually reproduce data magnetically, while optical disks reproduce data optically using lasers. Combinations of the above should also be included in the scope of computer-readable media.

[0333] An embodiment of the present application provides a computer program product, which includes a computer program. When the computer program is executed, the computer executes the above method.

[0334] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable device to generate a machine, so that the instructions executed by the processing unit of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0335] The above specific implementation methods further illustrate the purpose, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above are only specific implementation methods of the present invention and are not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present invention should be included in the scope of protection of the present invention.

Claims

1. An audio processing method, It is characterized in that Used in electronic equipment, including: At a first moment, the electronic device plays a first audio in a first application; the volume of the first audio is a first volume, and the hardware layer of the electronic device has an identifier of the first volume; During the process in which the electronic device continues to play the first audio, at a second moment, the electronic device receives a trigger operation for playing a second audio in a second application; wherein the first audio and the second audio have the same audio stream type; and the first moment is earlier than the second moment; In response to the trigger operation, the electronic device pauses playing the first audio, the electronic device detects that the abstract audio output device of the hardware abstraction layer changes, and the second application is in the playback thread, and the hardware abstraction layer sends the first volume identifier to the hardware layer; wherein the abstract audio output device is related to the value of the audio calibration database ACDB; The electronic device plays the second audio at the first volume; At a third moment, the electronic device ends playing the second audio and continues playing the first audio in the first application at the first volume; the hardware layer of the electronic device has an identification of the first volume; the second moment is earlier than the third moment.

2. The method according to claim 1, It is characterized in that When the audio stream of the first application is started in the electronic device, the electronic device associates the abstract audio output device of the hardware abstraction layer with the first ACDB value of the first application; When the audio stream of the second application starts in the electronic device, the electronic device updates the association between the abstract audio output device of the hardware abstraction layer and the second ACDB value of the second application.

3. The method according to claim 2, It is characterized in that When the electronic device finishes playing the second audio and continues to play the first audio in the first application at the first volume, the electronic device keeps the abstract audio output device of the hardware abstraction layer associated with the second ACDB value.

4. The method according to claim 1, It is characterized in that The hardware abstraction layer sends the identifier of the first volume to the hardware layer, including: The hardware abstraction layer sends the identifier of the first volume to the hardware layer by calling the update_sound_effect_param() function.

5. The method according to claim 1, It is characterized in that Before the hardware abstraction layer sends the identifier of the first volume to the hardware layer, the method further includes: When the audio output device changes, the hardware layer resets the volume value in the hardware layer to a preset volume; The hardware layer sets the volume value in the hardware layer to the first volume according to the identifier of the first volume, including: when the hardware layer obtains the identifier of the first volume, the hardware layer sets the volume value in the hardware layer to the first volume by calling a set_current_volume_index() function.

6. The method according to any one of claims 1 to 5, It is characterized in that Before the electronic device plays the first audio in the first application, the method includes: The application framework layer of the electronic device processes the first audio in the first application to obtain a processed first audio; The hardware layer of the electronic device processes the processed first audio to obtain a re-processed first audio; The electronic device plays the first audio in the first application, including: the electronic device plays the re-processed first audio through a speaker.

7. The method according to any one of claims 1 to 5, It is characterized in that The hardware layer includes advanced digital signal processing ADSP modules.

8. An electronic device, It is characterized in that include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the electronic device performs the method according to any one of claims 1 to 7.

9. A computer-readable storage medium storing a computer program. It is characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

10. A computer program product, It is characterized in that The method comprises a computer program, which, when being executed, enables a computer to execute the method according to any one of claims 1 to 7.

Images