Audio processing method and electronic device

Abstract

The present application discloses an audio processing method and an electronic device, wherein the method comprises: when a first audio and a second audio are to be played simultaneously, the first audio is processed based on a first sound effect parameter corresponding to the first audio to obtain a first audio to be mixed, and the second audio is processed based on a second sound effect parameter corresponding to the second audio to obtain a second audio to be mixed; the first sound effect parameter is different from the second sound effect parameter; and the audio is played based on the first audio to be mixed and the second audio to be mixed. By implementing the present application, the first audio and the second audio can be processed using different sound effect parameters, thereby ensuring the difference in sound effects corresponding to the first audio and the second audio when the audios are played concurrently, so that the user can normally distinguish multiple concurrently played audios.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of terminal technology, and in particular to an audio processing method and an electronic device. Background Art

[0002] At present, when using some audio and video applications in electronic devices such as tablets and mobile phones, there may be audio concurrency scenarios. For example, when a user is using video software to browse a video, the electronic device will play the audio corresponding to the video. If the alarm set by the user rings during this process, the electronic device may play the audio corresponding to the video and the audio corresponding to the alarm at the same time.

[0003] In this audio concurrency scenario, the electronic device will use the same set of sound effect parameters to process multiple concurrent audios. For example, the sound effect parameters corresponding to the alarm clock are used to process the audio corresponding to the video and the audio corresponding to the alarm clock; or, first, a set of relatively average sound effect parameters is obtained based on the sound effect parameters corresponding to the alarm clock and the sound effect parameters corresponding to the video, and then this set of sound effect parameters is used to process the audio corresponding to the video and the audio corresponding to the alarm clock. This sound effect processing method causes the difference in tone, sound, etc. of multiple concurrent audios when they are played, and users cannot normally distinguish between multiple concurrently played audios. How to ensure the difference in the concurrent playback of multiple audios in audio concurrency scenarios has become one of the problems that need to be solved urgently. Summary of the invention

[0004] The present application provides an audio processing method and an electronic device, which processes different audios using different sound effect parameters, thereby ensuring the differences when different audios are played concurrently.

[0005] In a first aspect, the present application provides an audio processing method, the method comprising:

[0006] When a first audio and a second audio are to be played simultaneously, the first audio is processed based on a first sound effect parameter corresponding to the first audio to obtain a first audio to be mixed, and the second audio is processed based on a second sound effect parameter corresponding to the second audio to obtain a second audio to be mixed; the first sound effect parameter is different from the second sound effect parameter; and audio playback is performed based on the first audio to be mixed and the second audio to be mixed.

[0007] By implementing the method described in the first aspect, the first audio and the second audio can be processed using different sound effect parameters, thereby ensuring the difference in sound effects corresponding to the first audio and the second audio when the audios are played concurrently, so that the user can normally distinguish multiple concurrently played audios.

[0008] In a possible implementation, the above-mentioned method of processing the first audio based on the first sound effect parameters corresponding to the first audio to obtain the first audio to be mixed, and processing the second audio based on the second sound effect parameters corresponding to the second audio to obtain the second audio to be mixed, specifically includes: in a first thread, processing the first audio based on the first sound effect parameters corresponding to the first audio to obtain the first audio to be mixed; the first thread is a thread created for performing sound effect processing on the first audio; in a second thread, processing the second audio based on the second sound effect parameters corresponding to the second audio to obtain the second audio to be mixed, the second thread is a thread created for performing sound effect processing on the second audio, and the second thread is a parallel thread of the first thread.

[0009] It can be seen that by creating two parallel threads, it is possible to provide feasibility for differentiated sound effect processing of the first audio and the second audio.

[0010] In a possible implementation, the audio playback based on the first audio to be mixed and the second audio to be mixed includes: determining a predicted mixing peak value based on the first audio to be mixed and the second audio to be mixed; if the predicted mixing peak value is less than or equal to a first threshold, mixing the first audio to be mixed and the second audio to be mixed to obtain a first mixed audio; and playing the first mixed audio.

[0011] It can be seen that when the predicted mixing peak value is less than or equal to the first threshold, the first audio to be mixed and the second audio to be mixed are mixed, so that the first mixed audio can not only reflect the respective sound effects of the first audio to be mixed and the second audio to be mixed when playing, but also ensure that no clipping distortion occurs.

[0012] In a possible implementation, the above-mentioned determining the predicted mixing peak value based on the first audio to be mixed and the second audio to be mixed includes: performing peak detection on the signal amplitude corresponding to the first audio to be mixed to obtain a first peak value; performing peak detection on the signal amplitude corresponding to the second audio to be mixed to obtain a second peak value; and determining the predicted mixing peak value based on a first mixing ratio corresponding to the first audio to be mixed, a second mixing ratio corresponding to the second audio to be mixed, the first peak value, and the second peak value.

[0013] It can be seen that before mixing, peak detection can be performed on the first audio to be mixed and the second audio to be mixed obtained after the sound effect processing, and the mixing peak of the mixing process can be predicted according to the peak detection result. Compared with the method of performing mixing first and then obtaining the mixing peak, this method of predicting the mixing peak before performing mixing can improve the processing efficiency while ensuring the quality of the mixed audio.

[0014] In a possible implementation, the method further includes: if the predicted mixing peak value is greater than a first threshold value, gain controlling the first audio to be mixed and the second audio to be mixed based on the first threshold value and the predicted mixing peak value; performing mixing processing based on the gain-controlled first audio to be mixed and the gain-controlled second audio to be mixed to obtain a second mixed audio; the mixing peak value corresponding to the second mixed audio is equal to the first threshold value; and playing the second mixed audio.

[0015] It can be seen that when the predicted mixing peak value is greater than the first threshold, the first audio to be mixed and the second audio to be mixed can be gain controlled before mixing, so that the obtained second mixed audio will not have clipping distortion during playback.

[0016] In a possible implementation, the mixing peak value corresponding to the second mixed audio is equal to the first threshold value, and the gain control of the first audio to be mixed and the second audio to be mixed based on the first threshold value and the predicted mixing peak value includes: determining a difference between the predicted mixing peak value and the first threshold value; based on the difference and a first mixing ratio corresponding to the first audio to be mixed, gain controlling the signal amplitude corresponding to the first audio to be mixed; based on the difference and a second mixing ratio corresponding to the second audio to be mixed, gain controlling the signal amplitude corresponding to the second audio to be mixed.

[0017] It can be seen that the first audio to be mixed and the second audio to be mixed can be gain controlled according to the difference between the predicted mixing peak and the first threshold, thereby reducing the peak value of the signal amplitude corresponding to the first audio to be mixed and the peak value of the signal amplitude corresponding to the second audio to be mixed.

[0018] In one possible implementation, the first sound effect parameters include fixed sound effect parameters of the first audio and one or more adjustable sound effect parameters of the first audio; the second sound effect parameters include fixed sound effect parameters of the second audio and one or more adjustable sound effect parameters of the second audio.

[0019] It can be seen that when supporting the use of different sound effect parameters for sound effect processing of the first audio and the second audio, it not only supports the use of different fixed sound effect parameters of the audio type for sound effect processing, but also supports users to manually set different sound effect parameters for different audios for sound effect processing.

[0020] In a second aspect, the present application provides an electronic device, comprising one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes the audio processing method in the first aspect and any possible implementation thereof.

[0021] In a third aspect, the present application provides an audio processing device, which includes a function / unit for executing the audio processing method in the above-mentioned first aspect and any possible implementation manner thereof.

[0022] In a fourth aspect, the present application provides a chip system, which is applied to an electronic device. The chip system includes at least one processor and an interface, and the interface is used to receive computer instructions and transmit them to at least one processor; at least one processor runs the computer instructions so that the electronic device executes the audio processing method in the above-mentioned first aspect and any possible implementation method thereof.

[0023] In a fifth aspect, the present application provides a computer-readable storage medium, which stores computer instructions. When the computer instructions are executed on an electronic device, the electronic device executes the audio processing method in the above-mentioned first aspect and any possible implementation method thereof.

[0024] In a sixth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the audio processing method in the first aspect and any possible implementation thereof.

[0025] It can be understood that the beneficial effects that can be achieved by the electronic device, audio processing device, chip system, computer-readable storage medium, and computer program product provided above can refer to the beneficial effects in the first aspect and any possible implementation method thereof, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] Figure 1 It is a sound effect schematic diagram provided in an embodiment of the present application;

[0027] Figure 2 It is a schematic diagram of a single audio playback scenario and a multiple audio concurrent playback scenario provided in an embodiment of the present application;

[0028] Figure 3 It is a schematic diagram of the software structure of an electronic device provided in an embodiment of the present application;

[0029] Figure 4 This is a schematic diagram of the software interaction process in a single audio playback scenario;

[0030] Figure 5 It is a schematic diagram of the software interaction process in multiple audio playback scenarios;

[0031] Figure 6 It is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;

[0032] Figure 7It is a flowchart of an audio processing method provided in an embodiment of the present application;

[0033] Figure 8 It is a software interaction flow diagram of an audio processing method provided in an embodiment of the present application;

[0034] Fig. 9 It is a comparative schematic diagram of frequency response in an audio concurrency scenario provided by an embodiment of the present application;

[0035] Fig.10 It is a structural schematic diagram of a chip system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0036] The technical solutions in the embodiments of the present application will be described clearly and in detail below in conjunction with the accompanying drawings. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in the text is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0037] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as suggesting or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features, and in the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.

[0038] The term "user interface (UI)" in the following embodiments of the present application refers to a medium interface for interaction and information exchange between an application or operating system and a user, which realizes the conversion between the internal form of information and the form acceptable to the user. The user interface is a source code written in a specific computer language such as Java and extensible markup language (XML). The interface source code is parsed and rendered on an electronic device, and finally presented as content that the user can recognize. The commonly used form of user interface is a graphical user interface (GUI), which refers to a user interface related to computer operations that is displayed in a graphical manner. It can be a visual interface element such as time, date, text, icon, button, menu, tab, text box, dialog box, status bar, navigation bar, widget, etc. displayed on the display screen of an electronic device.

[0039] In order to facilitate understanding of the solution provided by the embodiment of the present application, the following introduces the relevant concepts involved in the embodiment of the present application:

[0040] 1. Playback mode: There are four audio playback modes in the Android audio framework, namely deep buffer playback mode, low latency playback mode, offload playback mode and multi-channel playback mode. Among them, audio with low playback latency requirements can be played in deep buffer playback mode or offload playback mode, such as music, audio in videos, call ringtones, etc. The processing flow of audio in deep buffer playback mode and offload playback mode is different; audio with high playback latency requirements can be played in low latency playback mode, such as alarm clocks, game sounds, prompt sounds, etc.; for multi-channel audio, multi-channel playback mode can be used for playback. Different types of audio have corresponding playback modes. The corresponding playback modes of different types of audio are pre-defined playback modes. Different playback modes can be used to indicate the processing flow of audio.

[0041] 2. Sound effect parameters: Sound effect parameters refer to the parameters used when processing audio. The processed audio can have certain sound effects when playing. Different types of audio need to achieve different sound effects when playing, so the sound effect parameters used when processing different types of audio are different. For example, Figure 1Sound effects such as "Sound Effects-Music", "Sound Effects-Video", "Sound Effects-Movie", "Sound Effects-Game", "Sound Effects-Prompt Sound", "Sound Effects-Ringtone", and "Sound Effects-Alarm Clock" are shown. Among them, "Sound Effects-Music" is the sound effect corresponding to music; "Sound Effects-Video" is the sound effect corresponding to a small video or a short video; "Sound Effects-Movie" is the sound effect corresponding to a long video or a movie; "Sound Effects-Game" is the sound effect corresponding to a game; "Sound Effects-Prompt Sound" is the sound effect corresponding to a prompt sound; "Sound Effects-Ringtone" is the sound effect corresponding to a ringtone; and "Sound Effects-Alarm Clock" is the sound effect corresponding to an alarm clock. The sound effect parameters corresponding to the various sound effects listed above are not exactly the same, and the sound effect parameters may have a corresponding relationship with the playback mode. Therefore, the playback mode corresponding to the audio may be determined according to the type of audio, and then the sound effect parameters corresponding to the audio may be determined according to the playback mode corresponding to the audio. Finally, the audio may be processed according to the determined sound effect parameters and the processing flow corresponding to the playback mode. Optionally, the sound effect parameters corresponding to the audio can also be determined in combination with the audio type and the playback mode. For example, if the playback mode corresponding to the audio is the Deep buffer playback mode, and the Deep buffer playback mode corresponds to "Sound Effect-Music", "Sound Effect-Video", "Sound Effect-Movie", "Sound Effect-Game" and so on, then it is also necessary to determine which of these multiple sound effects the sound effect corresponding to the audio is based on the audio type. For example, the sound effect corresponding to music is "Sound Effect-Music", and accordingly, the sound effect parameters corresponding to music are the parameters corresponding to "Sound Effect-Music".

[0042] The following is an introduction to the audio playback scenario:

[0043] The audio playback scenarios include the scenario of a single audio playback and the scenario of multiple audios playing concurrently.

[0044] For example, Figure 2 As shown, Figure 2 The left side of shows a scenario of single audio playback, wherein when the electronic device plays the video "Take a Brave Step", the electronic device can play the audio in the video. Figure 2 The right side of shows a scenario where multiple audios are played concurrently. If the alarm goes off when the electronic device is playing the video "Take a Brave Step", the audio in the video and the audio corresponding to the alarm will be played at the same time.

[0045] The following describes the audio processing flow in these two scenarios in combination with the software architecture of the electronic device:

[0046] Among them, an operating system, such as iOS, Android and other operating systems, runs in the electronic device. The operating system of the electronic device can adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture or a cloud architecture. The embodiment of the present application takes the Android system of the layered architecture as an example to illustrate the software structure of the electronic device. It should be noted that although the embodiment of the present application is described by taking the Android system as an example, its basic principles are also applicable to electronic devices of other operating systems.

[0047] Figure 3 It is a software structure block diagram of an electronic device provided in an embodiment of the present application. The software structure adopts a layered architecture, which divides the software into several layers, and each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In an embodiment of the present application, the operating system (taking the Android system, the Android system running on the AP as an example) can be divided into five layers, from top to bottom, respectively, the application layer (application layer, APP layer), the application framework layer (framework layer, FWK layer), the hardware abstraction layer (hardware abstraction layer, HAL) (or called the hardware layer), the kernel layer (kernel layer) and the hardware layer.

[0048] ①. The application layer can include a series of application packages. Figure 3 As shown, the application package includes various audio applications. Audio applications refer to applications that can trigger the playback of audio. For example, music APP, video APP, alarm clock APP, game APP, etc. can receive playback requests respectively, thereby triggering the playback of music, audio corresponding to the video (including dubbing and soundtrack in the video, etc.), ringing of the alarm clock, and audio corresponding to the game (such as game sound). After any audio application in the application layer receives the user's playback operation, it can respond to the playback operation, first determine the type of audio to be played, and then determine the playback mode corresponding to the type of audio to be played, and send the determined playback mode and audio to be played to the application framework layer for processing.

[0049] ② The application framework layer provides application developers with an application programming interface (API) framework and various services and management tools for accessing core functions, including interface management, data access, application layer messaging, application package management, phone management, location management, etc. The application framework layer includes some predefined functions and algorithms. Figure 3As shown, the application framework layer may include an audio processing module, etc. When the audio processing module receives the audio to be played and the playback mode, it can first determine the sound effect parameters corresponding to the audio to be played according to the playback mode and the type of the audio to be played, perform sound effect processing on the audio to be played according to the sound effect parameters and the playback mode, and then the audio processing module sends the processed audio to the hardware abstraction layer.

[0050] ③. The hardware abstraction layer is an interface layer between the operating system kernel and the hardware circuit. Its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and can provide a virtual hardware platform for the operating system. Figure 3 As shown, the hardware abstraction layer may include an audio hardware abstraction HAL, etc., which is used to carry the processed audio sent by the audio processing module and send it to the kernel layer for further processing.

[0051] ④. The kernel layer is the layer between hardware and software, including audio drivers. Audio drivers include advanced digital signal processors (ADSP) and Codec registers.

[0052] Among them, after receiving the processed audio carried by the hardware abstraction layer, the audio driver can process the processed audio according to ADSP, which includes an analog front end (AFE), an audio device manager (ADM) and an audio stream manager (ASM), etc. ADSP can be used to further process the processed audio according to the platform parameter file, and the platform parameter file can still include a part of the sound effect parameters corresponding to the audio to be played. In other words, the sound effect processing of the audio to be processed includes processing based on the audio processing module at the application framework layer, and processing based on ADSP at the kernel layer. Optionally, the sound effect processing of the audio to be processed can be completed based on the audio processing module only in the application framework layer; or the sound effect processing of the audio to be processed can be completed based on ADSP only in the kernel layer. Which layer of the software architecture performs the sound effect processing of the audio to be processed can be determined by the playback mode corresponding to the audio to be processed, and this application does not limit this.

[0053] Furthermore, the ADSP sends the further processed audio to the Codec register, and the Codec register performs operations such as digital-to-analog conversion on the further processed audio and then sends it to the hardware layer.

[0054] ⑤. The hardware layer includes hardware such as speakers. After the speakers receive the data obtained by digital-to-analog conversion, they can play the audio to be played after the sound effect processing based on the data obtained by digital-to-analog conversion.

[0055] It is understandable that the above software architecture is only an example. In specific implementation, the software structure of the electronic device may also include more functional modules in the above layers, which will not be elaborated in this application.

[0056] Based on the above software architecture, when there is a single audio to be played, taking the audio to be played in the video as an example, it can be based on Figure 4 The process shown in the figure is used to play a single audio. When the video APP in the APP layer receives a video play request, the audio to be played in the video can be determined according to the video play request ( Figure 4 The audio to be played in the video is the audio in the video corresponding to the video playback request) and the playback mode corresponding to the audio to be played in the video is the Deep Buffer playback mode. Then, the video APP sends the audio to be played in the video and the Deep Buffer playback mode to the FWK layer. The FWK layer processes the audio to be played in the video based on a part of the sound effect parameters of "Sound Effect-Video" corresponding to the Deep Buffer playback mode, and sends the processed audio to the HAL layer. The HAL layer moves the processed audio to the ADSP of the Kernel layer. The ADSP further processes the processed audio based on the platform parameter file and sends it to the Codec. The platform parameter file includes another part of the sound effect parameters of "Sound Effect-Video"; the Codec performs digital-to-analog conversion on the further processed audio and sends it to the speaker for playback.

[0057] When there are multiple audios to be played at the same time (also called multiple audios being played concurrently, or audio concurrent scenarios, etc.), taking the audio in the video and the alarm clock to be played at the same time as an example, based on Figure 5The process shown is used to play multiple audios. Among them, when the alarm clock APP in the APP layer receives the alarm ringing request during the video playback, it can determine the audio to be played (such as an alarm) of the alarm clock and the corresponding playback mode of the audio to be played of the alarm clock as a low-latency playback mode according to the alarm ringing request. Then, because the video has not been played yet when the alarm ringing request is received, and the audio processing is performed in a single thread, the APP layer will send the audio to be played in the video, the audio to be played of the alarm clock, and the low-latency playback mode to the FWK layer. The FWK layer processes the audio to be played in the video and the audio to be played of the alarm clock based on part of the sound effect parameters of "sound effect-alarm clock" corresponding to the low-latency playback mode, and sends the processed concurrent audio to the HAL layer. The HAL layer moves the processed concurrent audio to the ADSP of the Kernel layer. ADSP further processes the processed audio based on the platform parameter file, which includes another part of the sound effect parameters of "sound effect-alarm clock"; Codec converts the concurrent audio obtained by further processing into digital-to-analog format and sends it to the speaker for playback.

[0058] Depend on Figure 4 and Figure 5 As shown in the processing flow, when there is a single audio to be played, the sound effect parameters corresponding to the single audio can be used to process the single audio and then play it; however, when there are multiple audios to be played at the same time, the sound effect parameters corresponding to the newly received request (such as the alarm ringing request) will be used to process the multiple audios and play them concurrently, resulting in the multiple audios having little difference after the sound effect processing, and will bring the user a poor sense of hearing. For example, in order to achieve a better reminder effect, the required sound effect of the alarm clock requires a higher pitch and a louder sound. If the tone and sound of the sound effect corresponding to the video are moderate, when the alarm clock and the video are played concurrently, the tone and sound of the video will suddenly increase, thereby affecting the user's sense of hearing. At present, the sound effect parameters corresponding to the multiple audios can be debugged as close as possible, so as to avoid the sudden increase of the tone and sound of the video in the scene where the alarm clock and the video are played concurrently. However, this solution still makes the multiple audios have little difference after the sound effect processing, and the user cannot normally distinguish the multiple concurrently played audios.

[0059] In order to ensure the difference in sound effects of multiple different audios in the audio concurrency scenario, this application proposes an audio processing method and an electronic device, which can be a terminal device, such as a smart phone, a tablet computer, a laptop computer, a desktop computer, a smart speaker, a smart watch, a smart car, etc., but not limited to these. The software architecture of the electronic device can be referred to Figure 3 The structural schematic diagram shown is not repeated here.

[0060] The hardware architecture of the electronic device can be found in the following Figure 6 The hardware structure diagram shown is:

[0061] like Figure 6 As shown, the electronic device may include: a processor 110 , a memory 120 , an audio module 170 , and a display screen 194 .

[0062] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the hardware structure of the electronic device. In other embodiments of the present application, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0063] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0064] The controller can be the nerve center and command center of the electronic device. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0065] A memory may also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. Repeated access is avoided, the waiting time of the processor 110 is reduced, and the efficiency of the system is improved. The processor 110 calls the instructions or data stored in the memory, so that the electronic device executes the audio processing method executed by the electronic device in the following method embodiment.

[0066] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0067] The electronic device can implement an audio playback function through an application processor and an audio module 170 , wherein the audio module 170 includes a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, and the like.

[0068] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.

[0069] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device can listen to a video, music, an alarm clock, or a call through the speaker 170A.

[0070] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.

[0071] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can make a sound by putting his mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device can be provided with at least one microphone 170C. In other embodiments, the electronic device can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device can also be provided with three, four or more microphones 170C to realize the collection of sound signals, noise reduction, identification of sound sources, and directional recording functions, etc.

[0072] The earphone jack 170D is used to connect a wired earphone.

[0073] The electronic device implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0074] The display screen 194 is used to display images, videos, etc. The display screen 194 includes the display panel in the above-mentioned embodiment. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicrOLED, Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0075] The audio processing method provided in the embodiment of the present application is further described in detail below:

[0076] like Figure 7 As shown, Figure 7 It is a flowchart of an audio processing method provided in an embodiment of the present application. Figure 7 The subject executing the method shown may be an electronic device, or the subject may be a chip in the electronic device. Figure 7The electronic device is used as an example to illustrate the execution subject of the method. The execution subjects of the methods shown in other figures of the embodiments of the present application are similar, and will not be repeated in the following text. The hardware and software structure of the electronic device can refer to the above Figure 3 and Figure 6 The structure shown. Figure 7 The audio processing method shown includes steps 701 to 702. Among them:

[0077] 701. When a first audio and a second audio are to be played simultaneously, the electronic device processes the first audio based on a first sound effect parameter corresponding to the first audio to obtain a first audio to be mixed, and processes the second audio based on a second sound effect parameter corresponding to the second audio to obtain a second audio to be mixed.

[0078] In the embodiment of the present application, the first audio or the second audio may be automatically triggered to be played, or may be triggered to be played by a user, etc. Furthermore, the first audio and the second audio may be triggered to be played at the same time, or the first audio and the second audio may be triggered to be played one after another. The present application does not limit the triggering method and the triggering sequence of the first audio and the second audio to be played.

[0079] For example, the first audio is the audio in the video that the user clicks to play in the video APP, and the second audio is the audio of the alarm set in advance by the user. In this case, the first audio is triggered by the user to be played, and the second audio is automatically triggered to be played. If the moment when the user clicks to play the video is the same as the ringing time of the alarm set in advance by the user, the first audio and the second audio are triggered to be played at the same time. If the video that the user clicks to play has already started playing, and a certain moment in the process of video playback (i.e., the first audio is played) is the ringing time of the alarm, then the first audio is triggered to be played before the second audio, and a part of the first audio has been played.

[0080] In an embodiment of the present application, when the electronic device determines that the first audio and the second audio are to be played simultaneously, it can first determine the playback modes corresponding to the first audio and the second audio, and then determine the first sound effect parameters according to the playback mode corresponding to the first audio and the type of the first audio, and determine the second sound effect parameters according to the playback mode corresponding to the second audio and the type of the second audio. The first sound effect parameters corresponding to the first audio are different from the second sound effect parameters corresponding to the second audio.

[0081] Optionally, when the type of the first audio and the type of the second audio are different, the first sound effect parameter is different from the second sound effect parameter. For example, the type of audio can be divided into application scenarios of the audio, and the application scenarios can be music playback scenarios, short video playback scenarios, long video playback scenarios, game scenarios, alarm clock scenarios, etc. The requirements for audio sound effects in each application scenario are different. Therefore, different types of audio in different application scenarios correspond to different sound effect parameters. For example, in the music playback scenario, the music is required to be fidelity, flat and with low noise; in the video playback scenario, the human voice in the audio is required to be fidelity and highlight the human voice; in the game scenario, the audio is required to enhance the low frequency so that the user can hear the background sound and footsteps in the game; in the alarm clock scenario, in order to achieve the reminder effect, the audio pitch and loudness are required to be high.

[0082] Optionally, the first sound effect parameters may include fixed sound effect parameters of the first audio and adjustable sound effect parameters of the first audio; the second sound effect parameters may include fixed sound effect parameters of the second audio and adjustable sound effect parameters of the second audio.

[0083] Among them, the fixed sound effect parameters are fixed sound effect parameters pre-set before leaving the factory. The fixed sound effect parameters of the first audio and the fixed sound effect parameters of the second audio can be set before leaving the factory according to the type of the first audio and the type of the second audio. For example, music sound effect parameters are pre-set for music playback scenes, and game sound effect parameters are pre-set for game scenes. This type of sound effect parameters is fixedly stored in the electronic device and cannot be adjusted by the user. Optionally, the electronic device can update this type of fixed sound effect parameters according to the update instruction received from the server.

[0084] Adjustable sound effect parameters refer to sound effect parameters that can be adjusted by receiving user operations. For example, the adjustable sound effect parameters can be volume, including media volume, ringtone volume, alarm volume, and notification volume, etc. The user can adjust the media volume, ringtone volume, alarm volume, and notification volume to different values. For example, the audio application corresponding to the first audio or the second audio, such as a music APP, supports users to select different sound effect parameters or users to adjust them adaptively, then these user-selectable or adaptively adjustable sound effect parameters are adjustable sound effect parameters.

[0085] Furthermore, the electronic device may process the first audio based on the first sound effect parameter determined above to obtain the first audio to be mixed, and process the second audio based on the second sound effect parameter to obtain the second audio to be mixed.

[0086] In one possible implementation, the electronic device may process the first audio based on the first sound effect parameters corresponding to the first audio in a first thread to obtain the first audio to be mixed; and in a second thread, process the second audio based on the second sound effect parameters corresponding to the second audio to obtain the second audio to be mixed.

[0087] The first thread is a thread created for processing the first audio with sound effects, and the second thread is a thread created for processing the second audio with sound effects, and the second thread is a parallel thread of the first thread. By creating two parallel threads, it is possible to provide feasibility for processing the first audio and the second audio with different sound effect parameters.

[0088] Optionally, the first thread is a thread that is triggered and created when the first audio is triggered to be played; the second thread is a thread that is triggered and created when the second audio is triggered to be played; the first thread and the second thread can be created simultaneously or successively. For example, when the first audio and the second audio are triggered to be played simultaneously, the first thread and the second thread can be triggered and created simultaneously. For another example, when the first audio and the second audio are triggered to be played successively, such as first triggering the video to play, and then triggering the alarm to ring during the video playing process, the thread corresponding to the video is created before the thread corresponding to the alarm.

[0089] It should be noted that, in the case where the first audio and the second audio are triggered to be played one after another, if the first audio is triggered to be played before the second audio, when the sound effect processing is performed on the first audio, the sound effect processing is performed on the unplayed part of the first audio.

[0090] 702. The electronic device plays audio based on the first audio to be mixed and the second audio to be mixed.

[0091] In a possible implementation, the electronic device may first obtain a predicted mixing peak value based on the first audio to be mixed and the second audio to be mixed, and the predicted mixing peak value indicates the peak value of the mixed audio obtained by mixing directly based on the first audio to be mixed and the second audio to be mixed. Then, the electronic device may determine whether to mix directly based on the first audio to be mixed and the second audio to be mixed according to the predicted mixing peak value, and after obtaining the mixed audio, the electronic device may play the audio based on the mixed audio.

[0092] The following first introduces how to get the predicted mixing peak:

[0093] In a possible implementation, the electronic device first performs peak detection on the signal amplitude corresponding to the first audio to be mixed to obtain a first peak value; performs peak detection on the signal amplitude corresponding to the second audio to be mixed to obtain a second peak value; and then determines a predicted mixing peak value based on a first mixing ratio corresponding to the first audio to be mixed, a second mixing ratio corresponding to the second audio to be mixed, the first peak value, and the second peak value.

[0094] The first mixing ratio corresponding to the first audio to be mixed and the second mixing ratio corresponding to the second audio to be mixed are pre-set. Optionally, the first mixing ratio and the second mixing ratio can be set according to the type of the first audio and the type of the second audio.

[0095] For example, if the first peak value is x1, the second peak value is x2, the first mixing ratio is a1, and the second mixing ratio is a2, then the predicted mixing peak value y1 can be obtained by the following formula 1:

[0096] y1=x1*a1+x2*a2 (Formula 1)

[0097] Furthermore, after the predicted mixing peak value is obtained, the following process may be performed:

[0098] In a first possible implementation manner, if the predicted mixing peak value is less than or equal to the first threshold, the electronic device mixes the first audio to be mixed and the second audio to be mixed to obtain a first mixed audio; and the electronic device plays the first mixed audio.

[0099] In the embodiment of the present application, if the predicted mixing peak value is less than or equal to the first threshold value, it means that the mixed audio directly obtained by mixing the first audio to be mixed and the second audio to be mixed can not only reflect the sound effects of the first audio to be mixed and the second audio to be mixed, but also ensure that no clipping distortion occurs. Therefore, the electronic device can mix the first audio and the second audio based on the first mixing ratio and the second mixing ratio to obtain the first mixed audio.

[0100] For example, since waveform clipping may occur when the amplitude of the audio signal is greater than 0 dB, resulting in distortion and affecting the user's hearing experience, the first threshold may be pre-set to any value less than or equal to 0 dB.

[0101] In a second possible implementation, if the predicted mixing peak value is greater than the first threshold value, gain control is performed on the first audio to be mixed and the second audio to be mixed based on the first threshold value and the predicted mixing peak value; mixing processing is performed based on the first audio to be mixed after gain control and the second audio to be mixed after gain control to obtain a second mixed audio; and the second mixed audio is played.

[0102] Specifically, the electronic device may first determine the difference between the predicted mixing peak and the first threshold; based on the difference and the first mixing ratio corresponding to the first audio to be mixed, perform gain control on the signal amplitude corresponding to the first audio to be mixed; based on the difference and the second mixing ratio corresponding to the second audio to be mixed, perform gain control on the signal amplitude corresponding to the second audio to be mixed, and then, based on the first mixing ratio and the second mixing ratio, mix the gain-controlled first audio to be mixed and the gain-controlled first audio to be mixed to obtain the second mixed audio.

[0103] Among them, the mixing peak value corresponding to the second mixed audio is less than or equal to the first threshold value. If the first predicted mixing peak value is greater than the first threshold value, it means that the mixed audio obtained by mixing directly based on the first audio to be mixed and the second audio to be mixed will have clipping distortion. Therefore, it is necessary to first determine the difference between the predicted mixing peak value and the first threshold value, and then perform gain processing on the first audio to be mixed and the second audio to be mixed according to the difference, so as to reduce the signal amplitude corresponding to the first audio to be mixed and the signal amplitude corresponding to the second audio to be mixed.

[0104] Optionally, when the first threshold is pre-set, it can also be increased or decreased according to different tuning requirements. For example, when the first threshold is larger, the loudness of the mixed audio obtained after mixing is louder, and it has a better reminder effect, but the shortcomings in the sound quality after mixing will be more obvious when the loudness is higher. The smaller the first threshold, the better the sound quality of the mixed audio obtained after mixing, but the loudness is smaller and the reminder effect is weaker. Therefore, when different types of audio are played concurrently, different first thresholds can be set. For example, for concurrent scenarios of alarm clocks and videos, in order to ensure the reminder effect, the first threshold can be set higher in advance. For concurrent scenarios of music and games, in order to ensure the sound quality effect, the first threshold can be set lower in advance.

[0105] It should be noted that the first mixing ratio, the second mixing ratio, and the numerical values ​​involved in the gain control during the above mixing are only examples. In the specific implementation, there may be other situations, and this application does not limit this. In addition, the above content cites the processing flow in the concurrent scenario of two audios. In the specific implementation, the concurrent scenario of two or more audios can still be processed with reference to the above processing flow. For example, when there are three audios concurrently, the sound effect parameters corresponding to these three audios can be used in three threads to perform sound effect processing on these three audios, and then mixing processing and peak detection can be performed to obtain the mixed audio and play it.

[0106] After obtaining the first mixed audio or the second mixed audio, the electronic device can play the first mixed audio or the second mixed audio. Since the played first mixed audio or the second mixed audio includes a part processed based on the first sound effect parameter and a part processed based on the second sound effect parameter, the first mixed audio or the second mixed audio can reflect the difference in sound effects between the first audio and the second audio.

[0107] Optionally, the electronic device may perform the above-mentioned peak detection, mixing processing and gain control operations in the first thread or the second thread, or the electronic device may recreate a parallel thread and perform the above-mentioned peak detection, mixing processing and gain control operations in the newly created thread, and the present application does not limit this. For example, the electronic device obtains the first audio to be mixed in the first thread, and after obtaining the second audio to be mixed in the second thread, the second audio to be mixed is sent to the first thread for peak detection, mixing processing and gain control. For another example, the electronic device creates a third thread for peak detection, mixing processing and gain control, and the electronic device sends the first audio to be mixed or the second audio to be mixed to the third thread for processing.

[0108] It should be noted that if there is still unplayed audio in the first audio or the second audio after the first audio to be mixed or the second audio to be mixed is played, such as the video has not been played after the video and the alarm are played concurrently, the unplayed audio can be played after the sound effect processing without the need to perform peak detection, mixing processing and gain control. Figure 4 If there are multiple audios to be played at the same time, you can follow the process shown in Figure 7 The described content is played after sound effect processing, peak detection, mixing processing and gain control.

[0109] based on Figure 7 The described embodiment can process the first audio and the second audio using different sound effect parameters, thereby ensuring the difference in sound effects corresponding to the first audio and the second audio when the audio is played, so that the user can normally distinguish multiple concurrently played audios.

[0110] Combine the following Figure 3 The software structure shown is Figure 7 The software processing flow involved is introduced as follows:

[0111] Take the above content where the first audio is the audio in the video and the second audio is the audio corresponding to the alarm clock as an example:

[0112] like Figure 8 As shown, Figure 8: is a flowchart of software interaction in an audio processing method provided in an embodiment of the present application, wherein:

[0113] APP layer: The video APP in the APP layer first obtains the first audio, and then determines that the playback mode corresponding to the first audio is the Deep Buffer playback mode according to the type of the first audio (such as the type corresponding to the small video scene). The alarm APP in the APP layer first obtains the second audio, and then determines that the playback mode corresponding to the second audio is the low-latency playback mode according to the type of the second audio (such as the type corresponding to the alarm scene). Then, the video APP sends the first audio and the Deep Buffer playback mode to the FWK layer, and the alarm APP sends the second audio and the low-latency playback mode to the FWK layer.

[0114] FWK layer: For the first audio received and the Deep Buffer playback mode, the FWK layer (such as the audio processing module in the FWK layer) first determines that the first sound effect parameter of the first audio is the parameter corresponding to "Sound Effect-Video" according to the type of the first audio and the Deep Buffer playback mode, and then the FWK layer uses a part of the parameters corresponding to "Sound Effect-Video" to process the first audio, and sends the processed audio of the first audio to the HAL layer. For the second audio received and the low-latency playback mode, the FWK layer (such as the audio processing module in the FWK layer) first determines that the second sound effect parameter of the second audio is the parameter corresponding to "Sound Effect-Alarm" according to the type of the second audio and the low-latency playback mode, and then the FWK layer uses a part of the parameters corresponding to "Sound Effect-Alarm" to process the second audio, and sends the processed audio of the second audio to the HAL layer.

[0115] HAL layer: The HAL layer transfers the received audio data after the first audio is processed and the audio data after the second audio is processed to the ADSP in the Kernel layer.

[0116] ADSP in the Kernel layer: ADSP further processes the received audio processed by the first audio based on the first platform parameter file to obtain the first audio to be mixed; ADSP further processes the received audio processed by the second audio based on the second platform parameter file to obtain the second audio to be mixed. The first platform parameter file includes another part of parameters corresponding to "sound effect-Video", and the second platform parameter file includes another part of parameters corresponding to "sound effect-alarm clock". Then, ADSP performs peak detection based on the first audio to be mixed and the second audio to be mixed to obtain the predicted mixing peak value, and ADSP determines whether the predicted mixing peak value is greater than the first threshold. If the predicted mixing peak value is less than or equal to the first threshold, ADSP performs mixing processing based on the first audio to be mixed and the second audio to be mixed. If the predicted mixing peak value is greater than the first threshold, ADSP first performs gain control on the first audio to be mixed and the second audio to be mixed, and then performs mixing processing. ADSP sends the result of the mixing processing to Codec.

[0117] Codec in the Kernel layer: The Codec performs digital-to-analog conversion on the mixing result and sends it to the hardware layer.

[0118] Speakers in the hardware layer: The speakers play data according to the digital-to-analog converted data.

[0119] Optional, complete audio processing for the first or second audio, except Figure 8 It is shown that it needs to be executed in the FWK layer and ADSP, and it can also be executed only in the FWK layer, or only in the ADSP, etc. This application does not limit this.

[0120] The following is an example of the effect of this application:

[0121] Take the example of the first audio being music and the second audio being the audio corresponding to the alarm clock:

[0122] like Fig. 9 As shown, Fig. 9 (a) is a frequency response diagram of multiple audios played concurrently when the same set of sound effect parameters are used to process multiple audios. Among them, if a target sound effect parameter close to the sound effect parameters corresponding to the music and the sound effect parameters corresponding to the alarm is debugged first, and then the target sound effect parameter is used to process the music and the audio corresponding to the alarm and then play them, the solid line in (a) represents the frequency response of the music at different frequencies, and the dotted line represents the frequency response of the alarm at different frequencies. It can be seen from (a) that the frequency response difference between music and alarm at different frequencies is small. In this case, the reminder effect of the alarm is reduced. If you want to improve the reminder effect of the alarm, you may need to actively increase the volume of the alarm, so that the volume of the music and the audio corresponding to the alarm is higher after the sound effect is processed at a higher volume.

[0123] Fig. 9 (b) is a frequency response diagram of multiple audios played concurrently when multiple audios are processed separately using different sound effect parameters. Among them, if the sound effect parameters corresponding to the alarm clock are used to process the audio corresponding to the alarm clock, and the sound effect parameters corresponding to the music are used to process the music, and the two audios processed by the sound effect are mixed after peak detection, then the solid line in (b) represents the frequency response of the music in the mixed audio at different frequencies, and the dotted line represents the frequency response of the alarm clock in the mixed audio at different frequencies. By comparing the frequency response of the alarm clock in (a), it can be seen that when the frequency is 4KHz~6KHz, the frequency response of the alarm clock in (b) is improved, and the improvement is about 2dB-4dB, so that the brightness and clarity of the alarm clock when the mixed audio is played are improved. In addition, by comparing the frequency response of the alarm clock and the frequency response of the music in (a) and (b), it can be seen that the difference in the frequency response of the music and the alarm clock at different frequencies is also more obvious.

[0124] The embodiment of the present application further provides an electronic device, which may include: one or more processors and one or more memories. The one or more memories are coupled to the one or more processors, and the one or more memories are used to store computer program codes, and the computer program codes include computer instructions. When the one or more processors execute the computer instructions, the electronic device executes each function or step performed by the electronic device in the above method embodiment.

[0125] An embodiment of the present application further provides an audio processing device, which includes a unit for executing the functions / units in the electronic device in the above embodiment.

[0126] The present application also provides a chip system, such as Fig.10 As shown, the chip system includes at least one processor 1001 and at least one interface circuit 1002. The processor 1001 and the interface circuit 1002 can be interconnected via lines. For example, the interface circuit 1002 can be used to receive signals from other devices (such as a memory of an electronic device). For another example, the interface circuit 1002 can be used to send signals to other devices (such as processor 1001). Exemplarily, the interface circuit 1002 can read instructions stored in the memory and send the instructions to the processor 1001. When the instructions are executed by the processor 1001, the electronic device can execute the various steps in the above embodiments. Of course, the chip system can also include other discrete devices, which are not specifically limited in the embodiments of the present application.

[0127] This embodiment further provides a computer-readable storage medium, in which computer instructions are stored. When the computer instructions are executed on an electronic device, the electronic device executes each function or step executed in the above method embodiment.

[0128] This embodiment also provides a computer program product. When the computer program product is run on a computer, the computer executes each function or step executed by the electronic device in the above method embodiment.

[0129] In addition, an embodiment of the present application also provides a device, which can specifically be a chip, component or module, and the device may include a connected processor and memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor can execute the computer-executable instructions stored in the memory so that the chip performs the various functions or steps performed by the electronic device in the above method embodiment.

[0130] Among them, the electronic device, audio processing device, computer-readable storage medium, computer program product or chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above and will not be repeated here.

[0131] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0132] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0133] The unit described as a separate component may or may not be physically separated, and the component shown as a unit may be one physical unit or multiple physical units, that is, it may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiment.

[0134] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0135] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium, including several instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of each embodiment method of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (Read-Only Memory, ROM), random access memory (Random Access Memory, RAM), disk or optical disk and other media that can store program code.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present application and are not intended to limit it. Although the present application has been described in detail with reference to the preferred embodiments, a person of ordinary skill in the art should understand that the technical solution of the present application may be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of the present application.

Claims

1. An audio processing method, characterized in that: The method comprises: In the case where a first audio and a second audio are to be played simultaneously, the first audio is processed based on a first sound effect parameter corresponding to the first audio to obtain a first audio to be mixed, and the second audio is processed based on a second sound effect parameter corresponding to the second audio to obtain a second audio to be mixed; the first sound effect parameter is different from the second sound effect parameter; Performing peak detection on the signal amplitude corresponding to the first audio to be mixed to obtain a first peak value; Performing peak detection on the signal amplitude corresponding to the second audio to be mixed to obtain a second peak value; Determining a predicted mixing peak value based on a first mixing ratio corresponding to the first audio to be mixed, a second mixing ratio corresponding to the second audio to be mixed, the first peak value, and the second peak value, wherein the first mixing ratio and the second mixing ratio are preset; If the predicted mixing peak value is less than or equal to a first threshold, mixing the first audio to be mixed and the second audio to be mixed to obtain a first mixed audio; Play the first mixed audio.

2. The method according to claim 1, characterized in that The processing of the first audio based on the first sound effect parameter corresponding to the first audio to obtain the first audio to be mixed, and the processing of the second audio based on the second sound effect parameter corresponding to the second audio to obtain the second audio to be mixed, comprises: In a first thread, the first audio is processed based on a first sound effect parameter corresponding to the first audio to obtain a first audio to be mixed; the first thread is a thread created for performing sound effect processing on the first audio; In the second thread, the second audio is processed based on the second sound effect parameters corresponding to the second audio to obtain the second audio to be mixed. The second thread is a thread created for performing sound effect processing on the second audio. The second thread is a parallel thread of the first thread.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: If the predicted mixing peak value is greater than the first threshold, performing gain control on the first audio to be mixed and the second audio to be mixed based on the first threshold and the predicted mixing peak value; Performing mixing processing based on the first audio to be mixed after gain control and the second audio to be mixed after gain control to obtain a second mixed audio; the mixing peak value corresponding to the second mixed audio is less than or equal to the first threshold; The second mixed audio is played.

4. The method according to claim 3, characterized in that The mixing peak value corresponding to the second mixed audio is equal to the first threshold, and the gain control of the first audio to be mixed and the second audio to be mixed based on the first threshold and the predicted mixing peak value includes: Determining a difference between the predicted mix peak value and the first threshold; Based on the difference and a first mixing ratio corresponding to the first audio to be mixed, performing gain control on a signal amplitude corresponding to the first audio to be mixed; Based on the difference and a second mixing ratio corresponding to the second audio to be mixed, gain control is performed on a signal amplitude corresponding to the second audio to be mixed.

5. The method according to claim 1 or 2, characterized in that: The first sound effect parameter includes one or more of a fixed sound effect parameter of the first audio and an adjustable sound effect parameter of the first audio; The second sound effect parameters include one or more of fixed sound effect parameters of the second audio and adjustable sound effect parameters of the second audio.

6. An electronic device, characterized in that: include: One or more processors, one or more memories; wherein the one or more memories are coupled to the one or more processors, the one or more memories are used to store computer program codes, the computer program codes include computer instructions, and when the one or more processors execute the computer instructions, the electronic device executes the method as described in any one of claims 1-5.

7. A chip system, applied to electronic equipment, characterized in that: The chip system includes at least one processor and an interface, wherein the interface is used to receive computer instructions and transmit them to the at least one processor; the at least one processor executes the computer instructions so that the electronic device executes the method described in any one of claims 1-5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the computer instructions are executed on an electronic device, the electronic device executes the method according to any one of claims 1 to 5.

Images