Audio information processing method, electronic equipment and medium

Abstract

The invention discloses an audio information processing method, electronic equipment and a medium. The audio information processing method comprises the following steps: displaying a first interface of a first application; in the process of displaying the first interface, playing first audio data of the first application; in a process of playing the first audio data, in response to a first operation, setting a process of the first application to be mute, and in response to the first operation, closing an audio stream of the first application; and at a set time after the audio stream of the first application is closed, releasing the mute of the process of the first application. According to the embodiment of the invention, the user experience of audio playing of the electronic equipment can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio information processing method, electronic equipment and medium. Background Art

[0002] With the development of computer technology, the use of computer products has become more and more diverse. Computer products such as computers and mobile phones are used in combination with software or applications to achieve a variety of different functions. For example, audio and video playback functions can be achieved through audio and video playback software or applications. In the process of playing audio and video or other data streams through software or applications, the data stream to be played is continuously output to the playback module of the computer product, and the user's operation on the application generally occurs and ends in a short period of time. In order to improve the playback effect of the data stream, the output of the data stream needs to be adapted to the operation performed by the user in a short period of time. Summary of the invention

[0003] The present application provides an audio information processing method, electronic device and medium, which can improve the playback effect of audio information.

[0004] The technical solution is as follows:

[0005] In a first aspect, a method for processing audio information is provided, including: while running a first application, an electronic device displays a first interface of a first application; while displaying the first interface, first audio data of the first application is played; while playing the first audio data, in response to a first operation, a process of the first application is muted, and, in response to the first operation, an audio stream of the first application is closed; and at a set time after closing the audio stream of the first application, the process of the first application is unmuted.

[0006] The first application may be an application that can play a data stream when running in the foreground, and stops outputting the data stream when the first application is configured to be in the background. The data stream played by the first application may include: an audio stream and / or a video stream. In addition, the audio stream of the first application may also be included in other data to form a composite data stream.

[0007] The audio stream of the first application may include the first audio data of the first application.

[0008] Setting the process of the first application to be muted may also be referred to as setting the process of the first application to be muted, or muting the process of the first application. Unmuting the process of the first application may also be referred to as canceling the muted state of the process of the first application, or setting the process of the first application to be unmuted.

[0009] When the process of the first application is set to be muted, the audio stream of the first application may be played at a volume of 0 or almost 0.

[0010] If the process of the first application is not muted when the first application is switched to the background running state, since it takes a certain amount of time to transmit the audio stream from the first application to the hardware playback module of the electronic device, when the output of the audio stream of the first application is stopped in response to the first operation, there is still audio stream in the transmission process that has not yet been transmitted to the hardware playback module. This part of the audio stream has not yet been played. As this part of the audio stream that has not yet been played continues to be transmitted to the hardware playback module, the hardware playback module will also play the audio stream, resulting in the user's perceived effect that the audio stream is still being broadcast in a short period of time after the first operation is implemented, resulting in a bad auditory experience.

[0011] By using the above method provided by the embodiment of the present application, when the electronic device runs the first application in the foreground, it can change the running state of the first application in response to the first operation of the user during the process of playing the audio stream, and as the running state switches, the first application stops outputting the audio stream to be played. Before the first application stops outputting the audio stream, all the output audio streams are still played through the hardware playback module of the electronic device. When the first application changes the running state in response to the first operation, the process of the first application can be first set to mute. After being muted, the playback of the audio stream of the first application cannot be heard by the user, and the user can feel that the audio playback of the first application is stopped almost at the same time as the first operation is implemented, thereby improving the user's auditory experience.

[0012] In one embodiment, the audio information processing method also includes: during the process of playing the first audio data of the first application, in response to a second operation, during the process of playing the first audio data, the electronic device sets the process of the first application to mute; after the second operation, the electronic device stops the activity of the first application; after stopping the activity of the first application, in response to a third operation, the electronic device resumes the activity of the first application; based on resuming the activity of the first application, the process of the first application is unmuted.

[0013] The above-mentioned activity of resuming the first application may be used to switch the first application from a background running state to a foreground running state.

[0014] Through the above method, when the user's running state of the first application changes, and the process of the first application has been muted but has not yet been unmuted, based on the third operation, the activity of the first application is restored, and the process of the first application is unmuted, so that the process of the first application can play the audio stream normally after restoration.

[0015] In one embodiment, the audio information processing method also includes: after the second operation and before stopping the activity of the first application, displaying the second interface of the first application; in response to the second operation, muting the process of the first application; after the second operation, stopping the activity of the first application; after stopping the activity of the first application, in response to a third operation, resuming the activity of the first application; and unmuting the process of the first application based on resuming the activity of the first application.

[0016] Through the above method, when or after the activity of the first application is stopped in response to the second operation, the second interface of the first application is canceled. After the activity of the first application is resumed, the interface of the first application is redisplayed, and the process of the first application is unmuted, so that the unmuting operation and the resumption of the display of the interface of the first application are almost performed simultaneously, and the user can feel that the audio stream of the first application starts to play at the same time as the interface of the first application is resumed, so that the user's visual experience and auditory experience are consistent.

[0017] In one embodiment, unmuting the process of the first application based on resuming the activity of the first application includes: unmuting the process of the first application based on resuming the activity of the first application and the process of the first application is muted.

[0018] Through the above method, as long as the process of the first application is set to mute when the activity of the first application is resumed, the process of the first application is unmuted, thereby ensuring that the process of the first application plays audio normally when running in the foreground in various scenarios.

[0019] In one embodiment, the audio information processing method further includes: in the process of playing the audio data of the first application, in response to a fourth operation, setting the process of the first application to mute; after the fourth operation, playing the first motion effect.

[0020] The first animation effect may be an animation effect when the first application changes its running state, for example, the first animation effect may be an animation effect when the first application switches from a foreground running state to a background running state. The fourth operation may be used to change the running state of the first application, for example, the fourth operation may be used to switch the first application from a foreground running state to a background running state. When the running state of the first application is changed, the corresponding animation effect is played, thereby improving the diversity of the interaction methods between the electronic device and the user.

[0021] In one embodiment, the audio information processing method further includes: before the first motion effect is finished playing, in response to a fifth operation, playing a second motion effect; after starting to play the second motion effect, unmuting the process of the first application.

[0022] Before the first animation effect is finished playing, a new animation effect can be replayed according to the fifth operation performed by the user, and the process of the first application can be unmuted, so that the muted state of the process of the first application does not affect the user's normal use of the first application.

[0023] In one embodiment, the audio information processing method further includes: the electronic device stops playing the first motion effect in response to the fifth operation.

[0024] Through the above method, when playing the second motion effect, the electronic device can stop playing the first motion effect, so that the playing of the two different motion effects will not conflict.

[0025] In one implementation, after the first operation, the first application is set to be run in the background.

[0026] Setting the first application to be run in the background can also be expressed as: setting the first application (process) to a background running state, setting the first application (process) to run in the background, or running the first application (process) in the background.

[0027] In one implementation, after the second operation, the first application is set to be run in the background.

[0028] In one implementation, after the third operation, the first application is set to be executed in the foreground.

[0029] Setting the first application to be run in the foreground can also be expressed as: setting the first application (process) to the foreground running state, setting the first application (process) to be run in the foreground, or running the first application (process) in the foreground.

[0030] In one implementation, after the fourth operation, the first application is set to be run in the background.

[0031] In one implementation, after the fifth operation, the first application is set to be executed in the foreground.

[0032] In one embodiment, unmuting the process of the first application at a set time after closing the audio stream of the first application includes: unmuting the process of the first application at a first set time after receiving callback information from the first application; the callback information is used to indicate that the audio stream output by the first application has been played before the moment when the callback information is generated; and unmuting the process of the first application at a second set time if the callback information of the first application is not received.

[0033] Through the above method, the audio stream that has not been played can be played before the process of the first application is unmuted, so that when the process of the first application is unmuted, there will be no abnormal playback of the audio stream.

[0034] In one embodiment, after setting the process of the first application to mute, the method includes: sending a callback information registration request to an audio management service; the callback information registration request includes information of the first application; and based on the callback information registration request, receiving callback information of the first application sent by the audio management service.

[0035] Through the above method, callback information can be requested from the audio management service, so that the electronic device can determine according to the callback information that when the process of the first application stops outputting the audio stream, the audio stream that has not been played has been played silently.

[0036] In one implementation, before unmuting the process of the first application, the method further includes: switching the process of the first application from a foreground running state to a background running state.

[0037] On the second aspect, the embodiments of the present application also provide an audio information processing device that can implement the audio information processing method provided by any embodiment of the present application.

[0038] On the third aspect, the embodiments of the present application also provide an audio playback device that can implement the audio information processing method provided by any embodiment of the present application during the audio playback process.

[0039] In a fourth aspect, an embodiment of the present application provides an electronic device, the electronic device comprising: a processor and a memory;

[0040] The memory is used to store a program for the electronic device to execute the method provided in any embodiment of the present application, and to store data involved in implementing the method provided in any embodiment of the present application;

[0041] The processor is configured to execute the program stored in the memory.

[0042] Optionally, there are one or more processors and one or more memories.

[0043] Optionally, the memory may be integrated with the processor, or the memory may be provided separately from the processor.

[0044] The processing device in the fourth aspect mentioned above can be a chip. The processor can be implemented by hardware or by software. When implemented by hardware, the processor can be a logic circuit, an integrated circuit, etc.; when implemented by software, the processor can be a general-purpose processor, which is implemented by reading the software code stored in the memory. The memory can be integrated in the processor or can be located outside the processor and exist independently.

[0045] In the specific implementation process, the memory can be a non-transitory memory, such as a read-only memory (ROM), which can be integrated with the processor on the same chip or can be set on different chips. This application does not limit the type of memory and the setting method of the memory and the processor.

[0046] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which instructions are stored. When the instructions stored in the storage medium are executed on a computer, the computer can execute the method of the first aspect mentioned above.

[0047] In a sixth aspect, an embodiment of the present application provides a computer program product, which, when executed on a computer, enables the computer to execute any possible implementation method of the first aspect.

[0048] In a seventh aspect, an embodiment of the present application further provides a processor, comprising: an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive a signal through the input circuit and transmit a signal through the output circuit, so that the processor executes the method in any one of the embodiments of the first aspect above.

[0049] In the specific implementation process, the above-mentioned processor can be a chip, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a trigger, and various logic circuits. The input signal received by the input circuit can be, for example, but not limited to, received and input by a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter, and the input circuit and the output circuit can be the same circuit, which is used as an input circuit and an output circuit at different times. This application does not limit the specific implementation of the processor and various circuits.

[0050] The technical effects obtained by the above-mentioned second, third, fourth, fifth, sixth and seventh aspects are similar to the technical effects obtained by the corresponding technical means in the above-mentioned first aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1It is a structural schematic diagram of an electronic device provided in an embodiment of the present application;

[0052] Figure 2 This is a schematic diagram of an application scenario of an embodiment of the present application;

[0053] Figure 3 is another application scenario schematic diagram of an embodiment of the present application;

[0054] Figure 4 This is another application scenario schematic diagram of the embodiment of the present application;

[0055] Figure 5 This is another application scenario schematic diagram of the embodiment of the present application;

[0056] Figure 6 is a method flow chart of an embodiment of the present application;

[0057] Figure 7 is a flowchart of an example method of the present application;

[0058] Figure 8 This is a schematic diagram of an example of the dynamic effect of this application;

[0059] Fig. 9 It is a schematic diagram of a method flow chart of an example of the present application;

[0060] Fig.10 It is a schematic diagram of a method flow chart of an example of the present application;

[0061] Fig.11 is a schematic diagram of a method flow chart of another example of the present application;

[0062] Fig.12 It is a schematic diagram of another exemplary method flow of the present application;

[0063] Fig.13 It is a schematic diagram of another exemplary method flow of the present application;

[0064] Fig.14 It is a schematic diagram of a method flow chart of an example of the present application;

[0065] Fig.15 It is a schematic diagram of an exemplary device structure of the present application. DETAILED DESCRIPTION

[0066] In the following description, specific details such as specific system structures and technologies are provided for illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application can also be implemented in other embodiments without these specific details.

[0067] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of the described features, integrals, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or their collections. The terms "comprising", "including", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0068] It should be understood that the "one or more" mentioned in this application refers to one, two or more than two, and the "multiple" mentioned in this application refers to two or more. In the description of this application, unless otherwise specified, " / " means or, for example, A / B can mean A or B. The "and / or" in this article is just a way to describe the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.

[0069] In addition, in order to clearly describe the technical solution of the present application, the words "first", "second", etc. are used to distinguish the same or similar items with basically the same functions and effects. Those skilled in the art can understand that the words "first", "second", etc. do not limit the quantity and execution order, and the words "first", "second", etc. do not necessarily limit the difference.

[0070] The phrases such as "one embodiment" or "some embodiments" described in the present application mean that the specific features, structures or characteristics described in the embodiment are included in one or more embodiments of the present application. Therefore, the phrases such as "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments" etc. that appear in different places in the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways.

[0071] The display method provided in the embodiment of the present application can be applied to electronic devices. The electronic device can be a mobile phone, a tablet computer, a wearable device, a digital camera, a vehicle-mounted device, an augmented reality (AR) device, a virtual reality (VR) device, a notebook computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a laptop computer, etc., and the embodiment of the present application does not limit this.

[0072] First, the possible structure of the electronic device in the embodiments of the present application is introduced.

[0073] Figure 1 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application.

[0074] See also Figure 1 The electronic device 1000 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0075] It should be noted that Fig.10 The structure shown does not constitute a specific limitation on the electronic device 1000. In other embodiments of the present application, the electronic device 1000 may include Fig.10 The electronic device 1000 may include more or fewer components than those shown. Fig.10 A combination of some of the components shown, or the electronic device 1000 may include Fig.10 Subcomponents of some of the components shown. For example, Fig.10 The illustrated proximity light sensor 180G may be optional. Fig.10 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0076] The processor 110 may include one or more processing units. For example, the processor 110 may include at least one of the following processing units: an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and a neural-network processing unit (NPU). Different processing units may be independent devices or integrated devices.

[0077] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0078] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0079] Figure 1 The connection relationship between the modules shown is only a schematic illustration and does not constitute a limitation on the connection relationship between the modules of the electronic device 1000. Optionally, the modules of the electronic device 1000 may also adopt a combination of multiple connection modes in the above embodiments.

[0080] The electronic device 1000 can realize the display function through the GPU, the display screen 194 and the application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0081] The display screen 194 may be used to display images or videos. In some embodiments, the electronic device 1000 may include 1 or N display screens 194 , where N is a positive integer greater than 1.

[0082] The electronic device 1000 can realize the shooting function through the ISP, the camera 193, the video codec, the GPU, the display screen 194 and the application processor.

[0083] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can perform algorithm optimization on the noise, brightness and color of the image. The ISP can also optimize the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0084] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 1000 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.

[0085] Video codecs are used to compress or decompress digital videos. The electronic device 1000 may support one or more video codecs. Thus, the electronic device 1000 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, and MPEG4.

[0086] The electronic device 1000 can implement audio functions, such as music playing and recording, through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone interface 170D, and the application processor.

[0087] The distance sensor 180F is used to measure the distance. The electronic device 1000 can measure the distance by infrared or laser. In some embodiments, for example, in a shooting scene, the electronic device 1000 can use the distance sensor 180F to measure the distance to achieve fast focusing.

[0088] The buttons 190 include a power button and a volume button. The motor 191 can generate vibrations when the electronic device receives information. The motor 191 can be used to prompt information reception.

[0089] exist Figure 1 Based on the electronic device shown, the user can switch the running state of the application through the key 190. For example, the electronic device can switch the application from the foreground running state to the background running state based on the operation of the user pressing the lock screen key.

[0090] The audio information processing method provided in the embodiment of the present application can be applied to the scenario where an electronic device plays audio, video, or audio and video data streams. Next, examples are given to introduce the scenarios to which the embodiment of the present application can be applied.

[0091] Figure 2Schematic diagram of an application scenario of an embodiment of the present application. Figure 2 As shown in (a) in the figure, the electronic device can be configured with multiple third-party applications, which can include game applications, video playback applications, audio playback applications, web browser applications, etc. At the same time, the electronic device can also be configured with multiple system applications, which can include desktop launchers, photo album applications, voice memo applications, etc. On the desktop interface of the electronic device, the startup icons of various third-party applications and system applications can be displayed. In response to the user clicking the startup icon, the corresponding application is started and run.

[0092] For example, in response to a user clicking on a game application, the game application is started and run, and the running interface is as follows: Figure 2 As shown in (b) in FIG. 1 . During the running of the game application, the application layer of the electronic device runs at least one process of the game application. The running state of the game application may include a foreground running state and a background running state. Figure 2 The running state shown in (b) is the foreground running state of the game application.

[0093] Among them, the foreground running state may refer to the running of the application satisfying any one of the following: (1) the display interface of the application on the display screen of the electronic device; (2) the user can directly interact with the application through the display interface of the application; (3) the user can view the display interface of the application; (4) create an activity; (5) obtain the focus; (6) occupy the input module (keyboard and / or mouse, etc.) of the electronic device. The above-mentioned activity may be one of the components of the system of the electronic device, and the activity may be used to provide a window for the user to interact with the application. The above-mentioned focus may refer to the element in the user interface that the user is currently interacting with. In other words, the focus may be the view on the screen of the electronic device that can currently receive input. For example, when the user is editing text, the focus will remain on the text that the user is editing.

[0094] The background running state may mean that the application is running and satisfies at least one of the following conditions: (1) the display screen of the electronic device does not display the display interface of the application; (2) the user does not directly operate the display interface of the application; (3) the activity of the application loses focus; (4) the process of the application is not closed; (5) the application occupies the resources of the electronic device; (6) the display interface of the application is hidden or minimized.

[0095] exist Figure 2 In the running state shown in (b), in response to the user's sliding operation from the bottom of the mobile phone screen, as shown in FIG. Figure 2 As shown in (c) in FIG. 1 , the electronic device can display Figure 2The electronic device can switch the process of the game application from the foreground running state to the background running state. Figure 2 In the state shown in (b) of FIG. 1 , the game application may play audio and video data synchronized with the game progress during operation. The audio and video data may include at least one of audio data and / or video data. If the game application enables the playback function in the background running state, the game application will Figure 2 The running status shown in (b) returns Figure 2 When the desktop shown in (a) is displayed, the game application still outputs audio and video data, and the electronic device plays the audio and video data output by the game application in real time. If the game application does not enable the playback function in the background running state, then the game application Figure 2 The running status shown in (b) returns Figure 2 When the desktop shown in (a) is displayed, the game application can stop outputting audio and video data, and the electronic device stops playing the audio and video data of the game application as the game application enters the background running state. Figure 2 The running status shown in (b) returns Figure 2 During the process of viewing the desktop shown in (a), the screen of the electronic device can play a preset animation effect, such as Figure 2 As shown in (d) in .

[0096] In addition to responding Figure 2 In addition to the upward sliding operation corresponding to (c) in the figure, the electronic device can switch the process of the game application from the foreground running state to the background running state, and the electronic device can also respond Figure 3 The side sliding operation shown in (a) in Figure 2 The running status shown in (b) returns Figure 2 Alternatively, the electronic device may also respond to Figure 3 (b) The click operation of the floating ball 31 displays as follows Figure 3 The expanded floating ball 32 includes a desktop control 33. The electronic device can respond to a click operation on the desktop control 33 and return to the following. Figure 2 The desktop shown in (a).

[0097] In other possible implementations, the electronic device may respond to other operations so that the game application is Figure 2 The running state shown in (b) in the figure is switched to the background running state. Figure 4 As shown in (a) of FIG. 1 , in response to the upward sliding operation of the setting gesture, the electronic device may display the following Figure 4 The switching interface shown in (b) in FIG. In the switching interface, screenshots of the display interfaces of all currently running applications are displayed. At the same time, Figure 4The screenshot shown in (b) may be in response to Figure 4 When the upward sliding operation of the setting gesture shown in (a) is performed, the electronic device generates the latest received information corresponding to each application. The latest received information may refer to the electronic device in Figure 4 Information received by the electronic device at or before the swipe-up operation of the set gesture shown in (a).

[0098] Another situation where the electronic device responds to other operations and switches the game application from the foreground running state to the background running state is as follows Figure 4 As shown in (c) in the figure. The electronic device displays Figure 2 In the case of the display interface shown in (b) (it can also be expressed as: when the electronic device displays Figure 2 In the display interface shown in (b) in FIG. 1 , in response to a sliding operation from the upper edge of the screen, such as Figure 4 As shown in (c) in Figure 4 The system drop-down menu shown in (d) in FIG. Figure 4 When the system drop-down menu shown in (d) is displayed, the process of the game application loses focus, and the process of the game application changes from the foreground running state to the background running state.

[0099] Electronic devices can also respond to side sliding operations, such as Figure 5 As shown in (a) in FIG. 1 , the electronic device displays Figure 2 In the case of the display interface shown in (b) (it can also be expressed as: when the electronic device displays Figure 2 When the display interface shown in (b) is displayed, the following is displayed: Figure 5 The side-sliding navigation bar shown in (b) enables the game application to Figure 2 The foreground running state shown in (b) is switched to the background running state.

[0100] Figures 2 to 5The scene shown is still applicable to third-party applications such as video playback applications, audio playback applications, web browser applications, and system applications. Most applications have a data stream playback function, and the data stream may include a video stream, an audio stream, an audio and video stream, etc. When the application of the electronic device switches (or changes) from the foreground running state to the background running state, the data stream of the application will still be output to the playback module of the electronic device in a short time after the application is switched, so that the electronic device will continue to play in a short time. If the data stream includes audio data, the electronic device will still play the sound of the application (also called audio) in a short time after the running state of the application is switched to the background running state, affecting the user experience. At the same time, due to the switching of the running state, when the electronic device plays the audio of the application in a short time after the switching, abnormal sounds may be generated, and abnormal sounds may include pause sounds, current sounds, blasting (pop) sounds, etc.

[0101] Therefore, the embodiment of the present application provides an audio information processing method, which can prevent the data stream from being continuously output for a short period of time and affecting the user experience when the application switches from the foreground running state to the background running state. Figure 6 As shown, an audio information processing method according to an embodiment of the present application includes the following steps.

[0102] Step S61: the electronic device starts and runs the first application in response to a click operation on the start icon of the first application.

[0103] In an embodiment of the present application, the first application may be a system application of the electronic device or a third-party application of the electronic device. Further, the first application may be an application that can play audio data, audio-video data, or video data during operation. In other words, the first application may be an application that outputs a data stream to a playback module of the electronic device during operation, and the data stream may include audio data, video data, or audio-video data. The electronic device may display the video in a manner such as Figure 2 When the desktop is shown in (a), in response to a click operation on the start icon of the first application, the first application is started and runs in the foreground of the electronic device.

[0104] In a possible implementation, the first application may be at least one application. The first application may be pre-configured to: when in the background running state, the first application stops outputting the data stream to the playback module of the electronic device, or in other words, when in the background running state, the first application stops outputting the data stream to the playback module of the electronic device.

[0105] In other possible implementations, the click operation on the launch icon of the first application may include a single click operation, a long press operation, or a double click operation, etc. The electronic device may also pre-set a gesture operation, and in response to the pre-set gesture operation, the first application may be launched and run.

[0106] Step S62: While the first application is running, the electronic device displays the first interface of the first application.

[0107] In a possible implementation, the electronic device displays the first interface of the first application, which may mean that the first application is in the foreground running state on the electronic device. When the first application is started, at least one process of the first application is started, and the at least one process may include the main process of the first application. Further, the first application is in the foreground running state on the electronic device, which may include: the main process of the first application is in the foreground running state. The above-mentioned main process of the first application may refer to a process used to implement the main functions of the first application.

[0108] During the running of the first application, the first interface may be as follows Figure 2 As shown in (b), the activity of the first application obtains the focus, and the user can interact with the first application through the first interface. It can also be said that the focus of the electronic device is in the first interface. In a possible implementation, the first interface of the first application may include a full-screen display interface and / or a non-full-screen display interface. The first interface may include one area, or may include multiple areas, and the multiple areas are partially covered or the multiple areas are relatively independent.

[0109] In a possible implementation, the first interface may include at least one frame. For example, a frame in a video playback application during video playback may be the first interface. In other words, the first interface may be used to display at least one frame. For example, an audio playback application displays an interface corresponding to the audio during audio playback.

[0110] Step S63: During the process of displaying the first interface, the electronic device plays the first audio data of the first application.

[0111] In a possible implementation, during the process of displaying the first interface, the electronic device plays the first audio data, which may include: the electronic device simultaneously displays the first interface and the first audio data corresponding to the first interface. It can also be said that during the operation of the first application, the first interface is displayed by the electronic device while the first audio data is played. The first audio data may be audio data output by the first application when displaying the first interface.

[0112] In step S63, the electronic device can play the first audio data in a non-silent manner, that is, the first audio data played by the electronic device can be heard by human ears. The first audio data can be generated by the main process of the first application or by a non-main process of the first application.

[0113] In step S63, the audio data of the first application played during the process of the electronic device displaying the first interface of the first application is recorded as the first audio data. In other possible implementations, at the next continuous moment when the electronic device displays the first interface, the audio stream of the first application played by the electronic device may also include a part of the first audio data, or, at the next continuous moment when the electronic device displays the first interface, the audio stream played by the electronic device may be called the second audio stream.

[0114] Step S64: During the playing of the first audio data, the electronic device sets the process of the first application to mute in response to a first operation.

[0115] In a possible implementation, the first application continuously outputs a data stream containing the first audio data to the electronic device within a certain period of time, and the electronic device continuously plays the first audio data within a corresponding period of time. The first operation can be used to switch the first application from one running state to another running state. For example, the first operation can be used to switch the first application from a foreground running state to a background running state. Or it can be said that the first operation can be used to switch the first application from a foreground running state to a non-foreground running state, wherein the non-foreground running state can include a background running state.

[0116] Setting the process of the first application to be silent may include: silently playing the audio data output by the first application after the first operation occurs. It can also be said that in response to the first operation, the electronic device changes from a state of non-silently playing the first audio data to silently playing the second audio data, and the first application continuously outputs the second audio data after outputting the first audio data. The first audio data and the second audio data may be audio data continuously output by the first application. The process of the first application set to be silent may include the main process of the first application, or may include a process that outputs the first audio stream. That is, during the running of the first application, in addition to the main process of the first application, when the electronic device also runs a non-main process of the first application (it can also be expressed as: when the electronic device also runs a non-main process of the first application), the electronic device can set at least one process of the first application to be silent in response to the first operation.

[0117] Step S65: During the playing of the first audio data, the electronic device closes the audio stream of the first application in response to the first operation.

[0118] In a possible implementation, step S65 and step S66 can be implemented in any order or simultaneously. Closing the audio stream of the first application can also be expressed as: stopping outputting the audio stream of the first application. Among the audio streams output before closing the audio stream of the first application, there is still a part of the audio stream that has not been played when the audio stream of the first application is closed. This part of the audio stream that has not been played has been sent by the first application when the electronic device closes the audio stream of the first application. After closing the audio stream of the first application, it will still be transmitted between multiple modules inside the electronic device, and finally transmitted to the playback module of the electronic device. At the same time, this part of the audio stream that has not been played will continue to be played by the electronic device after the electronic device closes the audio stream of the first application, and will be played by the electronic device in a silent manner according to the setting of step S64.

[0119] In an embodiment of the present application, the audio stream output by the first application includes the first audio data. Further, the audio stream output by the first application may include all audio outputs output during the operation of the first application. The electronic device turns off the audio stream of the first application, which can also be said to be: the electronic device controls the first application so that the first application stops outputting the audio stream.

[0120] In the embodiment of the present application, the audio stream may include multiple audio data. The audio data of the first application mentioned in the embodiment of the present application may refer to the audio data included in the audio stream of the first audio.

[0121] Step S66: A set time after closing the audio stream of the first application, the electronic device unmutes the process of the first application.

[0122] In a possible implementation, the set time may correspond to the play time of the audio stream that has not been played when the electronic device responds to the first operation. The audio stream that has not been played can be estimated in advance to obtain the time required to play the audio stream that has not been played, and the set time is determined based on this time. The set time may be different for different electronic devices. Exemplarily, after closing the audio stream of the first application, the callback information of the first application is received, and within the set time of receiving the callback information, the electronic device unmute the process of the first application. In this case, the set time may be 0-1000 milliseconds (millisecond, ms), further, the set time may be 100-500ms, and further, the set time may be 200ms. Alternatively, after closing the audio stream of the first application, the callback information of the first application is not received, and within the set time, the electronic device unmute the process of the first application. In this case, the set time may be 0-10 seconds, and further, the set time may be 1-5 seconds. The above callback information is used to indicate that the audio stream output by the first application has been played before the callback information is generated.

[0123] The electronic device unmuting the process of the first application may include: the electronic device unmuting the process of the first application set to be muted in step S64. After the electronic device unmutes the process of the first application, the electronic device may play the audio stream of the first application at the volume configured by the electronic device when receiving the audio stream of the first application (which can also be expressed as: when the electronic device receives the audio stream of the first application).

[0124] In the case where the application sets the background audio playback (it can also be expressed as: when the application sets the background audio playback), the application can also play audio in the background running state. Inside the electronic device, when the audio stream of the application is played, the audio stream needs to be transmitted, and the audio stream is transmitted from the application to the hardware playback module of the electronic device. This transmission process takes a certain amount of time. In the case where the application does not set the background audio playback (it can also be expressed as: when the application does not set the background audio playback), if the application is in the background running state, the application no longer outputs the audio stream, and the electronic device no longer plays the audio stream of the first application. However, since it takes a certain amount of time for the audio stream to be transmitted from the application to the hardware playback module of the electronic device, at the moment when the application no longer outputs the audio stream, there is still a part of the audio stream in the previously output audio stream that has not been played, and this part of the audio stream has been generated and sent by the first application. The audio stream that has not been played will still be transmitted inside the electronic device to complete the transmission process from the first application to the hardware playback module. At the same time, after the hardware playback module receives the audio stream transmitted by the first application, it will play the audio stream, which will cause the electronic device to still play the audio stream of the first application in a short period of time after the first application switches from the foreground running state to the background running state. This situation will cause the user to perceive the bad experience of the audio staying slowly. Through the method provided in the embodiment of the present application, the electronic device can control the first application to stop outputting the audio stream according to the first operation when the first application outputs the first audio data, and keep the process of the first application silent within the set time, so that when the electronic device responds to the first operation, a part of the audio stream that has not been played will not be heard by the user, and the user will not perceive the phenomenon that the audio playback stays slowly when the audio stream of the first application stops outputting, thereby improving the user experience.

[0125] In the embodiment of the present application, when the electronic device responds to the first operation, it may refer to the moment when the electronic device generates response information according to the first operation and executes the response information. The moment when the electronic device executes the response information may include, for example, any at least one of the following: (1) the moment when the electronic device controls the first application to stop outputting the audio stream when and after the first operation occurs; (2) the moment when the electronic device stops displaying the display interface of the first application when and after the first operation occurs; (3) the moment when the electronic device releases the focus of the process of the first application when and after the first operation occurs.

[0126] At the same time, since the first application is pre-configured to stop playing audio data in the background running state, when the running state of the first application switches from the foreground running state to the background running state, the audio data of the first application changes from playing to stopping playing, causing the audio data played by the hardware playback module of the electronic device to change significantly in a short period of time, resulting in plosive sounds in the audio data played by the hardware playback module of the electronic device. If the method provided in the embodiment of the present application is not applied, when the first application stops outputting the audio stream, the process of the first application is set to mute, so that when the audio stream output state of the first application changes significantly, the generated plosive sound will not be heard by the user, thereby avoiding the plosive sound causing a bad auditory experience for the user.

[0127] In one embodiment, the audio information processing method also includes: in the process of playing the first audio data of the first application, in response to a second operation, in the process of playing the first audio data, muting the process of the first application; after the second operation, stopping the activity of the first application; after stopping the activity of the first application, in response to a third operation, resuming the activity of the first application; based on resuming the activity of the first application, unmuting the process of the first application.

[0128] In one embodiment, the audio information processing method also includes: after the second operation and before stopping the activity of the first application, displaying the interface of the first application; in response to the second operation, muting the process of the first application; after the second operation, stopping the activity of the first application; after stopping the activity of the first application, in response to a third operation, resuming the activity of the first application; and unmuting the process of the first application based on resuming the activity of the first application.

[0129] In the above embodiment, the interface of the first application displayed may include Figure 6 The first interface in the embodiment shown.

[0130] In one embodiment, unmuting the process of the first application based on resuming the activity of the first application includes: unmuting the first application based on resuming the activity of the first application and the process of the first application is muted.

[0131] In one embodiment, the audio information processing method further includes: in the process of playing the audio data of the first application, in response to a fourth operation, setting the process of the first application to mute; after the fourth operation, playing the first motion effect.

[0132] In one embodiment, the audio information processing method further includes: before the first motion effect stops playing, in response to a fifth operation, playing a second motion effect; after starting to play the second motion effect, unmuting the process of the first application.

[0133] In one embodiment, unmuting the process of the first application at a set time after closing the audio stream of the first application includes: unmuting the process of the first application at a first set time after receiving callback information from the first application; the callback information is used to indicate that the audio stream output by the first application has been played before the moment when the callback information is generated; and unmuting the process of the first application at a second set time when the callback information of the first application is not received.

[0134] The above-mentioned case where the callback information of the first application is not received may also be expressed as: when the callback information of the first application is not received.

[0135] In one embodiment, after setting the process of the first application to mute, the method includes: receiving a callback information registration request sent by an audio management service; the callback information registration request includes information of the first application; and based on the callback information registration request, receiving callback information of the first application sent by the audio management service.

[0136] Next, examples involved in the embodiments of the present application are introduced. When the first application is running in the foreground of the electronic device (it can also be expressed as: when the electronic device is running the first application in the foreground), the electronic device may respond to a first operation to switch the first application from the foreground running state to the background running state. Within a short period of time after the first application is switched to the background running state, when the electronic device has not yet unmuted the process of the first application, the electronic device may respond to a second operation to switch the first application from the background running state to the foreground running state again. Alternatively, when switching the first application from the background running state to the foreground running state, there may also be multiple consecutive operations, and the electronic device responds to multiple consecutive operations and only implements one switch of the first application from the background running state to the foreground running state. For a series of operations in different situations, the electronic device can perform the following operations: Figure 7 Steps shown.

[0137] Step S71: The desktop application of the electronic device sends an activity change registration request to the activity management service of the electronic device.

[0138] The activity change registration request is used to request the activity management service to send information about the touch interaction event to the desktop application (launcher) when a touch interaction event occurs. Exemplarily, a touch interaction event may also be referred to as a touch event, and a touch interaction event may include an event of pressing a view, an event of sliding a view, an event of ending this event due to reasons other than user operation, and an event of lifting a view. Among them, an event of pressing a view may be referred to as a down event, an event of sliding a view may be referred to as a move event, an event of ending this event due to reasons other than user operation may be referred to as a cancel event, and an event of lifting a view may be referred to as an up event.

[0139] After the electronic device is started, the desktop application of the electronic device may be an application started first. After the desktop application is started, an activity change registration request may be sent to the activity management service.

[0140] Step S72: When the electronic device displays the first interface of the first application and plays the first audio data corresponding to the first interface, the input module of the electronic device monitors the touch interaction event in real time.

[0141] For example, the touch interaction event can be encapsulated as a touch event object. Figure 2 As shown in (b) in .

[0142] When the input module monitors touch interaction events in real time, the area where the touch interaction events occur can be determined first. The display screen of the electronic device can be pre-divided into at least one hot zone, and at least one hot zone can include: a touch hot zone at the bottom of the display screen, a touch hot zone on the side of the display screen, and / or a touch hot zone at the top of the display screen. When a touch interaction event occurs in each hot zone (it can also be said that when a touch interaction event occurs in each hot zone), the electronic device can determine how to respond to the touch interaction event based on the corresponding hot zone and the touch interaction event.

[0143] Step S73: The desktop application receives information about the press event.

[0144] When a touch interaction event occurs, the input module generally first monitors a press event, and then may monitor other types of touch interaction events. The information of the press event may include the area where the press event occurs, the coordinate position where the press event occurs, etc. The above coordinate position may include a relative coordinate position and / or an absolute coordinate position.

[0145] Step S74: The desktop application generates and sends tracking effect information to the surface compositor (surfaceflinger) based on the information of the press event and the sliding event before the lift event, so that the surface compositor generates a tracking effect based on the tracking effect information.

[0146] The information of the hand-following effect can also be called the information of the hand-following animation effect. The above hand-following effect information is used to generate the hand-following effect. The hand-following effect can include the animation effect generated by the position of the touch point when the user touches the display screen. Or it can be said that the hand-following effect can also include the animation effect generated by the desktop application according to the position of the sliding event after each press event and before the lift event occurs.

[0147] For example, a desktop application can use the transaction.apply() function to send hand-tracking information to surfaceflinger.

[0148] For example, the effect of manual effect can be referred to Figure 8 As shown in FIG. 1 , in response to the sliding event, the first interface of the first application may change from Figure 2 The full screen display state shown in (b) is transformed into Figure 8 In the non-full screen display state shown in (a), as the position of the sliding event changes, the display interface of the non-full screen display state will also change position, such as Figure 8 In the process of position change of the first interface in the non-full screen display state, the desktop application can generate information of the hand-following effect, so that the surfaceflinger of the electronic device draws the hand-following effect in real time, such as Figure 8 As shown in (b) in .

[0149] Step S75: The desktop application receives the information of the lift event, and in response to user operation 1, controls the first application to stop outputting the audio stream and starts playing the animation corresponding to user operation 1.

[0150] When the desktop application receives the information of the lift event, it determines that user operation 1 is monitored according to the position of the lift event, the position of the sliding time period, and the position of the pressing time. User operation 1 is used to instruct the first application to exit the foreground running state. Accordingly, the animation effect corresponding to user operation 1 can be the animation effect when the first application exits the foreground running state, which can be equivalent to the first animation effect in the aforementioned embodiment.

[0151] The desktop application can respond to the information of the lift event and stop generating and sending the information of the follow-up hand effect before or when starting to play the motion effect corresponding to the user operation 1.

[0152] Exemplarily, the desktop application controlling the first application to stop outputting the audio stream may include: the desktop application sending notification information to the first application, where the notification information is used to notify the first application to stop outputting the audio stream.

[0153] When the desktop application starts playing the animation corresponding to user operation 1, it can send the animation start information to the activity management service through the animation start function (onanimationstart()).

[0154] Step S76: When the desktop application starts playing the animation corresponding to the user operation 1, an instruction to mute the process of the first application is sent to the audio management service.

[0155] Exemplarily, the instruction to mute the process of the first application may include: mutestream instruction. Further, the instruction to mute the process of the first application may include: mutestream(pid, 0). Wherein, pid is process identity, which indicates the process ID (identity, ID), which may be the ID of the process of the first application. The number 0 indicates that a mute operation is performed.

[0156] The desktop application can mute the process of the first application by sending an instruction to mute the process of the first application to the audio management service.

[0157] Step S77: When or after sending the instruction to mute the first application process to the audio management service, the desktop application sends a callback information registration request to the audio management service.

[0158] In step S77, the desktop application sends a callback information registration request to the audio management service, or in other words, the desktop application registers callback information with the audio management service. The callback information registration request can be sent by registering an audio playback callback function (register audioplaybackcallback).

[0159] The callback information registration request can also be called the audio playback status callback information registration request, which is used to request the audio playback service to send a callback message to the desktop application after the audio stream of the process of the first application is played. At the same time, since there is still a part of audio data that has not been played in the audio stream output by the first application when the response to the first operation begins in step S75, and this part of the audio stream that has not been played still needs to be transmitted to the hardware playback of the electronic device when the desktop application notifies the first application to stop outputting the audio stream. At the same time, when the hardware playback module receives the audio stream that has not been played, it will still perform the playback operation. It takes a certain amount of time to complete the transmission and playback of the audio data that has not been played inside the electronic device. During this time, the desktop application can continue to play the animation corresponding to the user operation 1 in real time, and can execute the following steps S781 to S782 multiple times.

[0160] Step S781: surfaceflinger sends a vertical synchronization (vsync) signal to the desktop application.

[0161] Step S782: The desktop application sends information about the animation corresponding to user operation 1 to surfaceflinger based on the vertical synchronization signal.

[0162] Exemplarily, the desktop application can send the information of the dynamic effect corresponding to the user operation 1 to the surfaceflinger through the composition function (compose()). After the surfaceflinger completes the drawing of the information of a frame of dynamic effect, the vsync signal is regenerated and sent to the desktop application. That is to say, after the surfaceflinger completes the drawing of the information of a frame of dynamic effect, it returns to step S781 and repeats step S781 to step S782.

[0163] After step S781, the desktop application can generate information about the animation corresponding to the user operation 1 based on the vertical synchronization signal.

[0164] Step S79: After playing the animation corresponding to the user operation 1, the desktop application sends the animation completion information to the activity management module.

[0165] Step S79 can also be expressed as: after playing the animation corresponding to user operation 1, the desktop application sends the animation completion information to the activity management service.

[0166] For example, the desktop application may use the animation completion function (onanimationfinished()) to send animation completion information to the activity management module.

[0167] In other possible examples, according to pre-settings, when the first application switches to the background running state, there may be no corresponding animation. In this case, the animation corresponding to the user operation 1 in step S75 and steps S781 to S79 ​​may not be executed.

[0168] Step S710: the activity management service calls a pause function (onpause()) on the first application.

[0169] Step S711: The activity management service calls a stop function (onstop()) on the first application.

[0170] Step S712: the first application responds to the information sent by the activity management service through the stop function, and notifies the audio management service to close the audio stream of the first application through the media stream function (mediasession).

[0171] In step S712, the audio management service releases the audio focus of the first application. Exemplarily, the first application may send a media session close message to the activity management service to notify the audio management service to close the audio stream of the first application.

[0172] Step S713: The desktop application receives the callback information sent by the audio management service.

[0173] The callback information is sent according to the callback information registration request sent by the desktop application to the audio management service in step S77. The callback information can be sent through a callback information function (onplaybackconfigchange).

[0174] Step S714: When the callback information is received, the process of the first application is unmuted after a first set time.

[0175] Step S714 may also be expressed as: when the callback information is received, the first application is unmuted after the first set time. Unmuting the process of the first application may include: sending an instruction to unmute the process of the first application to the audio management service. The instruction to unmute the process of the first application may include: a mutestream instruction. Further, the instruction to unmute the process of the first application may include: mutestream(pid, 1). The number 1 indicates that an unmute operation is performed.

[0176] Step S715: When no callback information is received, the process of the first application is unmuted after a second set time has passed since the first application was muted.

[0177] Step S715 may also be expressed as: if no callback information is received, the first application is unmuted at a second set time after the first application is muted. The process of unmuting the process of the first application in step S715 may be similar to that in step S714.

[0178] After muting the first application in step S76 and before unmuting the first application, the audio stream of the first application that has entered the background running state is played without sound. In this process, there may be a scenario in which the application background does not complete the stop process and is immediately started. When the first application is started again (or, when the first application is started again), the first application is switched from the background running state to the foreground running state. The desktop application can monitor the changes in the application life cycle, and after the first application is started again, check if the mute process has been initiated (or, when the process of the first application is muted), then trigger the active unmute process, and execute the following steps S716 to S718 to avoid silence after the first application returns to the foreground running state.

[0179] Step S716: In response to user operation 2, the activity management service starts the activity of the first application and sends resumption information to the first application.

[0180] The user operation 2 may include at least one of the following: (1) clicking on the notification information of the first application. In response to clicking on the notification information, the activity management service may restart the first application before passing step S714 or step S715. (2) triggering a third-party jump information operation. In response to the third-party jump information, the activity management service may restart the first application.

[0181] Exemplarily, the resume information may be sent via a resume function (onresume()).

[0182] Step S717: The activity management service sends a recovery notification to the desktop application.

[0183] Exemplarily, the resume notification may be sent via a notifyactivityresume() function.

[0184] Step S718: The desktop application sends an instruction to the audio management service to unmute the process of the first application.

[0185] The instruction for unmuting the process of the first application in step S718 may be similar to that in step S714.

[0186] In step S75, after the animation corresponding to user operation 1 starts to play, before step S79, there may be a situation where the first application is started quickly and continuously, and the process of the first application has not realized the process of switching from the foreground running state to the background running state, and the first application has not undergone a life cycle change (it can also be said that the activity management service has not generated information or recovery information of the stop function of the first application). In this case, the desktop application can play the startup animation of the first application (also known as the startup animation effect). When the startup animation is initiated, the audio of the first application may still be playing, and the audio focus does not switch, and there is a problem that the mute cannot be unmuted. The desktop application can check whether the process of the first application is muted. When the desktop application has initiated the mute process of the process of the first application and the startup animation of the first application starts, the desktop application can unmute the first application and execute the following steps S719 to S720.

[0187] In the example of this application, the startup animation of the first application may refer to the animation when the first application switches from the background running state to the foreground running state. Alternatively, the startup animation of the first application may refer to the animation when the first application switches from any running state (for example, an intermediate running state between two different running states) to the foreground running state.

[0188] Step S719: the input module sends information about the touch interaction event to the desktop application.

[0189] Exemplarily, the information of the touch interaction event in step S719 may be generated when user operation 3 occurs.

[0190] Step S720: The desktop application starts the animation corresponding to user operation 3.

[0191] Step S721: When the desktop application determines that the animation corresponding to the user operation 3 is started and the process of the first application is muted, the process of the first application is unmuted.

[0192] Step S721 can also be expressed as: when the desktop application determines that the animation corresponding to the user operation 3 is started and the process of the first application is muted, the process of the first application is unmuted.

[0193] In step S721, unmuting the process of the first application may include: sending an instruction to unmuting the process of the first application to the audio management service.

[0194] In a possible example of the present application, steps S716 to S718 can be performed after steps S71 to S715 are performed, and the activity management module switches the first application from the background running state to the foreground running state in response to user operation 4, and then restarts steps S72 to S76, and then is performed in a relatively short period of time based on user operation 2. Alternatively, steps S716 to S718 can be performed based on user operation 2 after steps S71 to S76 are performed.

[0195] Similarly, in a possible example, after steps S71 to S715 are implemented, the activity management module switches the first application from the background running state to the foreground running state in response to user operation 4, and then restarts the implementation of steps S72 to S75, and then implements them based on user operation 3 in a relatively short time. Alternatively, steps S719 to S721 may be implemented in a relatively short time after steps S71 to S75 are implemented, based on user operation 4. Alternatively, after steps S716 to S718 are implemented, the activity management module switches the first application from the background running state to the foreground running state in response to user operation 4, and then restarts the implementation of steps S72 to S75, and then implements them based on user operation 3 in a relatively short time.

[0196] Figure 7 Only an implementation example of an audio information processing method provided in an embodiment of the present application is shown, wherein each step and each module are not absolutely one-to-one corresponding, that is, each step does not necessarily use Figure 7 In other possible implementations, other functional modules may be used to implement the audio information processing method provided in the embodiment of the present application.

[0197] In a possible example, when the first application switches from the foreground running state to the background running state, or switches from the foreground running state to the background running state, and then switches from the background running state to the foreground running state in a short period of time, the electronic device can mute the process of the first application or unmute the process of the first application in a short period of time, thereby solving the problem that the audio stream of the first application is still audible in a short period of time after the first application switches to the background running state, affecting the user experience, and at the same time ensuring that muting the process of the first application will not affect the operation of the first application when it returns to the foreground running state. Next, the sequence of each step implemented by the electronic device in a relatively short period of time is introduced.

[0198] In one possible example, at a first moment, the electronic device displays a first interface of a first application, and plays audio data of the first application while displaying the first interface; in response to a first operation, at a second moment, the process of the first application is muted; after the first operation, at a third moment, the audio stream of the first application is closed; after closing the audio stream of the first application, at a fourth moment, the process of the first application is unmuted.

[0199] Furthermore, after the above-mentioned fourth moment, the electronic device displays the first interface of the first application, and plays the audio data of the first application during the display of the first interface; in response to the second operation, at the fifth moment, the electronic device may mute the process of the first application; after the second operation, at the sixth moment, the electronic device may stop the activity of the first application; after stopping the activity of the first application, in response to the third operation, resume the activity of the first application; based on resuming the activity of the first application, unmute the process of the first application.

[0200] Furthermore, at the above-mentioned fifth moment, the electronic device also displays the first interface of the first application, and plays the audio data of the first application during the display of the first interface; in response to the second operation, at the sixth moment, the electronic device may mute the process of the first application; after the second operation, at the seventh moment, the electronic device may stop the activity of the first application; after stopping the activity of the first application, in response to the third operation, the electronic device may resume the activity of the first application; after resuming the activity of the first application, the electronic device may unmute the process of the first application.

[0201] Further, based on the activity of the first application being resumed and the process of the first application being muted, the electronic device can unmute the process of the first application. That is to say, in any case (or at any time), when the activity of the first application is resumed, if the first application is muted according to the method provided in the embodiment of the present application, the electronic device unmute the process of the first application.

[0202] Further, at the eighth moment, the electronic device can display the first interface of the first application, and play the audio data of the first application during the display of the first interface; in response to the fourth operation, at the ninth moment, the electronic device can mute the process of the first application; after the fourth operation, the electronic device can play the first animation effect; before the first animation effect stops playing, in response to the fifth operation, the second animation effect starts at the tenth moment; after the second animation effect starts, the electronic device can unmute the process of the first application.

[0203] The above-mentioned first moment to the tenth moment may each include multiple time points.

[0204] The above-mentioned first moment to the tenth moment can be determined in order according to numbers, that is, the first moment occurs before the second moment, the second moment occurs before the third moment, and so on.

[0205] Based on starting the second animation and the process of the first application being muted, the electronic device can unmute the process of the first application.

[0206] After the second moment, the electronic device may set the first application to be executed in the background.

[0207] The electronic device involved in the embodiments of the present application may be configured with a certain software architecture. Fig. 9 Schematic diagram of an architecture (including software system and some hardware) applied in the embodiment of the present application. Fig. 9 As shown, the architecture of the electronic device is divided into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the application architecture may include: a hardware abstraction layer (HAL), an application framework layer (FWK), and an application layer.

[0208] like Fig. 9 As shown, the application layer can also be called the application program layer. The application program layer can include multiple applications, and the applications can be further divided into system applications and third-party applications. System applications can be applications configured when the electronic device is produced, such as Fig. 9 The desktop launcher shown in . Third-party applications can refer to applications provided by third parties, such as Fig. 9 The video applications, game applications, and audio applications shown in . Among them, the video application and the audio application can be used to play audio and video data, and the game application can also play audio data or audio and video data during operation. The desktop launcher can be used to display installed applications, which can be the last application started during the system startup process of the electronic device, or the first application started among all applications of the electronic device. The above-mentioned desktop launcher can be equivalent to the desktop application in the aforementioned embodiment of the present application.

[0209] The application layer can also include some configuration information required when the application is running, such as Fig. 9 In addition to the desktop launcher, the application layer can also include other system applications. At the same time, the application layer can also include functional modules for collaboration with third-party applications or system applications such as desktop launchers, such as Fig. 9The scene recognition module shown in . The scene recognition module is used to determine whether it meets the scene set by the application listed in the whitelist based on the information generated by the application layer and the application framework layer. And when it meets the scene set by the application listed in the whitelist (it can also be expressed as: when it meets the scene set by the application listed in the whitelist), it generates information for controlling the process of the corresponding application.

[0210] The whitelist may be determined based on information pushed by the cloud, and the whitelist may be updated based on the latest information pushed by the cloud. In some examples of the present application, the whitelist may be used to list at least one application that needs to be processed by the audio information processing method provided by the embodiment of the present application. Updates to the whitelist may include adding new applications, deleting applications, and modifying application names.

[0211] like Fig. 9 As shown, the application framework layer includes audio track, input module, playback monitor, audio manager, audio service and activity manager service (AMS). The audio service can also be called audio manager service.

[0212] Among them, the audio track, audio manager, and audio service can be part of the audio framework (audio) of the electronic device. In the software architecture of the electronic device, the audio framework can be used for the collection and output of audio data, the control of audio streams, the management of audio devices, the adjustment of volume, etc.

[0213] The input module is used to obtain operation information of the application from the bottom layer and provide the operation information to the desktop launcher. The operation information may include information about the upward sliding operation on the application interface, information about the side sliding operation on the application interface, etc. Based on the operation information obtained by the input module, the desktop launcher can control the application and change the audio output activity of the application accordingly. Thus, the information about the change of the audio output activity of the application is transmitted to the scene recognition module, which determines the corresponding scene in combination with the change of the application activity detected by the AMS.

[0214] The corresponding scene determined by the scene recognition module according to the user operation can be transmitted to the desktop launcher, and the desktop launcher can mute the application process according to the corresponding scene, or unmute the application process. Alternatively, the scene recognition module can determine the corresponding scene according to the user operation, and then the scene recognition module can mute the application process according to the corresponding scene, or unmute the application process.

[0215] In a possible implementation of the embodiment of the present application, the scene recognition module can be combined with a desktop launcher, and the desktop launcher implements the function of the scene recognition module, for example Figure 7 Alternatively, the scene recognition module can be a separate module running in the application layer or application framework layer.

[0216] exist Fig. 9 The hardware abstraction layer shown, including the audio hardware abstraction layer (audio HAL), can be used to abstract the functions of the underlying audio hardware into a set of unified interfaces for use by upper-layer applications.

[0217] It should be understood that Fig. 9 The software architecture shown only shows part of the software modules of the electronic device. The electronic device may also include other software modules. Fig. 9 Not shown in .

[0218] In a possible implementation of the present application, the electronic device may, after starting the desktop application, implement the following operation based on the user's operation on the first application: Figure 7 The process of steps S71 to S715 shown in FIG. Fig. 9 In one example, the audio information processing method may be as follows: Fig.10 shown.

[0219] When the first application is in the foreground running state (it can also be expressed as when the first application is in the foreground running state), the first application can play the audio stream non-silently. When the first application is in the foreground running and the first application plays the audio stream non-silently, the user slides up on the display interface of the first application, and the desktop application executes the animation effect when exiting the foreground running state. At the same time, the scene recognition module recognizes that the first application exits the foreground playing state based on the notification of stopping the media stream sent by the audio management service, and then notifies the audio management service to mute the process of the first application. After the scene recognition module notifies the audio management service to unmute the first application, the scene recognition module sends a callback information registration request to the audio management service, which is used to request the audio management service to send callback information to the scene recognition module after the audio stream of the process of the first application is played. After the audio management service receives the callback information registration request, the callback information registration request can be recorded, and the recorded callback information registration request can be used to monitor the audio stream of the process of the first application.

[0220] Still refer to Fig.10After the scene recognition module notifies the audio management service to mute the process of the first application, the audio management service sets the mute setting for the process of the first application, and the audio stream of the process of the first application changes from the non-muted playback state to the muted playback state. After detecting that a swipe-up operation is performed on the display interface of the first application, the first application starts to implement the operation state switching operation. The swipe-up operation is equivalent to Figure 6 The first operation in the embodiment shown. The activity management service successively notifies the first application to stop the process of the first application running in the foreground through a pause instruction and a stop instruction, and notifies the process of the first application to enter the background running state (through the onpause() function). Among them, the pause instruction is sent through the pause function, and the stop instruction is sent through the stop function. Based on the stop instruction, the process of the first application stops outputting the audio stream. After stopping the output of the audio stream, the audio focus of the process of the first application is released, and the audio focus release information (abandonaudiofocus) is sent to the audio management service. Before the process of the first application stops outputting the audio stream, there is still a small amount of audio data that has been output by the process of the first application, but has not yet been transmitted to the hardware playback module of the electronic device for playback. In this case, the audio management service continues to monitor the playback of the audio data of the process of the first application, and when or after all the audio data before the process of the first application stops outputting the audio stream is sent to the hardware playback module, the audio management service sends a callback information to the scene recognition module according to the recorded callback information registration request. The scene recognition module unmute the process of the first application at a set time after receiving the callback information. The desktop application can complete the playing of the animation effect after the first application completes the switching from the foreground running state to the background running state.

[0221] Apart from Fig.10 In addition to the examples shown, Figure 7 In the example shown, after the electronic device implements steps S71 to S76, it can also continuously implement the process of steps S716 to S718 in a short period of time. Fig. 9 The software architecture shown in FIG. 1 is, in one example, Fig.11 shown.

[0222] exist Fig.11 In the example shown, the activity management service notifies the first application to stop the process of the first application running in the foreground through a pause instruction and a stop instruction, and the steps performed by the electronic device before this are the same as those Fig.10 same.

[0223] based on Fig.11As shown, after the activity management service sends a stop instruction, before the process of the first application stops outputting the audio stream based on the stop instruction, the first application detects an associated start operation on the first application or a start operation on the first application. The activity management service sends a resume instruction, and accordingly, the first application receives the resume instruction, which is used to instruct the first application to stop entering the background running state and return to the foreground running state. Among them, returning to the foreground running state can also be called resuming the foreground running state.

[0224] When or after the first application receives the resume instruction, the first application may send a notification to the scene recognition module to resume the foreground running state, and the scene recognition module may unmute the process of the first application based on the notification audio management service of resuming the foreground running state. After the first application completes the switch from the foreground running state to the background running state, the desktop application may complete the playback of the animation effect.

[0225] Apart from Fig.10 , Fig.11 In addition to the example shown, after the desktop application starts playing the animation effect, within a short period of time, the user receives the start operation of the first application again. In this case, the operation performed by the electronic device is as follows Fig.12 Before the scene recognition module notifies the audio management module to mute the process of the first application, Fig.12 The steps performed in the example shown are the same as Fig.10 Similarly, after the scene recognition module notifies the audio management module to mute the process of the first application, the scene recognition module can record the muted state of the process of the first application. In response to the user's start operation on the first application, the desktop application stops the animation effect that has already started playing, and then restarts the animation effect. At the same time, a notification message of starting to play the animation effect is sent to the scene recognition module. After the scene recognition module receives the notification of starting to play the animation effect, based on the notification and the recorded background sound state of the process of the first application, it recognizes that there are multiple consecutive start operations, and notifies the audio management service to unmute the process of the first application.

[0226] Apart from Figures 10 to 12 In addition to the example shown, if the first application is configured to play the audio stream in the background running state, the first application will be muted but cannot be muted after entering the background running state. To avoid the problem of unrecoverable background audio caused by abnormal scenarios, a timeout anti-stuck mechanism can be added, such as Fig.13 As shown, a timeout detection with a set time is added. After the set time has passed, the scene recognition module recognizes that the process of the first application has released the audio focus, and notifies the audio management module to unmute the process of the first application. Fig.13 The corresponding set time may be the second set time in the aforementioned embodiment.

[0227] Another example of an audio information processing method in this application can refer to Fig.14 As shown, the following steps are included. Fig.14 The steps shown can be performed by a setting module of the electronic device. The setting module can include one or more modules.

[0228] Step S141: When the electronic device displays the interface of the first application, in response to a swipe-up operation, an animation effect of the first application switching from a foreground running state to a background running state begins to be played.

[0229] Step S141 can also be expressed as: when the electronic device displays the interface of the first application, in response to a swipe up operation, starting to play the animation effect of the first application switching from the foreground running state to the background running state.

[0230] Step S142: when the first application exists in the preset whitelist, the process of the first application is muted and the audio focus is monitored.

[0231] In the case where the first application exists in the preset whitelist, it may also be expressed as: when the first application exists in the preset whitelist.

[0232] In the electronic device, when the audio of the first application is played using the hardware playback module of the electronic device, the process of the first application obtains the audio focus. In step S142, the audio focus can be monitored by sending a callback information registration request to the audio management service.

[0233] Step S143: After the mute is set, the process of the first application is muted.

[0234] When the process of the first application is muted, the audio stream output by the process of the first application cannot be heard by the user, that is, when the audio stream output by the process of the first application is played, the volume may be 0 or close to 0.

[0235] Step S144: the animation effect of the first application switching from the foreground running state to the background running state is completed.

[0236] Step S145: the first application switches from a foreground running state to a background running state.

[0237] Step S145 includes a pause operation performed by the first application according to a pause instruction of the onpause() function and a stop operation performed by the first application according to an instruction of the onstop() function.

[0238] Step S146: The process of the first application stops outputting the audio stream.

[0239] Step S147: The first application releases the audio focus.

[0240] Step S148: receiving the callback information sent by the audio management service.

[0241] Step S149: Determine whether the process of the first application is muted, if so, proceed to step S1410, otherwise end.

[0242] Step S1410: At a set time, unmute the process of the first application.

[0243] Step S1410 includes unmuting the process of the first application at a first set time when the callback information is received, or unmuting the process of the first application at a second set time when the callback information is not received.

[0244] The audio information processing device provided in the embodiment of the present application can be used to perform the following Figure 6 And the steps of the method provided by any related embodiment. An audio information processing device provided by an embodiment of the present application includes: Fig.15 The display module, play module, mute module, close module and release module are shown.

[0245] The display module is used to display a first interface of a first application.

[0246] The playing module is used to play the first audio data of the first application during the display of the first interface.

[0247] The mute module is used to mute the process of the first application in response to a first operation during the playing of the first audio data.

[0248] The closing module is used to close the audio stream of the first application in response to the first operation.

[0249] The release module is used to unmute the process of the first application at a set time after the audio stream of the first application is closed.

[0250] The audio information processing device provided in the embodiments of the present application can implement the audio information processing method provided in any one of the embodiments of the present application.

[0251] The embodiments of the present application also provide an audio playback device, which can implement the audio information processing method provided in any one of the embodiments of the present application when playing audio.

[0252] In the embodiment of the present application, "under the circumstances of..." may also be expressed as "when...", and "based on..." may also be expressed as "according to...". The embodiment of the present application takes audio playback as an example for explanation. In other possible implementations, the embodiment of the present application may also be applicable to playback of other types of data, for example, may also be applicable to playback of video.

[0253] An embodiment of the present application also provides an electronic device having the audio information processing device provided by any embodiment of the present application.

[0254] An embodiment of the present application also provides an electronic device having the audio playback device provided by any embodiment of the present application.

[0255] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0256] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0257] The embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the steps in the above-mentioned method embodiments can be implemented.

[0258] The embodiments of the present application also provide a computer program product. When the computer program product is executed on an electronic device, the electronic device can implement the steps in the above-mentioned various method embodiments.

[0259] The embodiment of the present application also provides a chip system, the chip system includes a processor, the processor is coupled to a memory, and the processor executes a computer program stored in the memory to implement the steps of any method embodiment of the present application. The chip system can be a single chip or a chip module composed of multiple chips.

[0260] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (such as: coaxial cable, optical fiber, data subscriber line (Digital Subscriber Line, DSL)) or wireless (such as: infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access, or a data storage device such as a server or data center that includes one or more available media integrations. The available medium may be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a digital versatile disc (DVD)), or a semiconductor medium (such as a solid state disk (SSD)).

[0261] The above are optional embodiments provided for the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the technical scope disclosed in the present application shall be included in the protection scope of the present application.

Claims

1. An audio information processing method, characterized in that: include: Displaying a first interface of a first application; During displaying the first interface, playing first audio data of the first application; During the playing of the first audio data, in response to a first operation, muting the process of the first application, and, in response to the first operation, closing the audio stream of the first application; After a set time has passed since the audio stream of the first application was closed, the process of the first application is unmuted.

2. The method according to claim 1, characterized in that The method further comprises: In response to a second operation, during the process of playing the first audio data of the first application, setting the process of the first application to be muted during the process of playing the first audio data; After the second operation, stopping the activity of the first application; After stopping the activity of the first application, in response to a third operation, resuming the activity of the first application; Based on resuming the activity of the first application, the process of the first application is unmuted.

3. The method according to claim 2, characterized in that The method further comprises: after the second operation and before stopping the activity of the first application, displaying the second interface of the first application; In response to the second operation, setting the process of the first application to be muted; After the second operation, stopping the activity of the first application; After stopping the activity of the first application, in response to a third operation, resuming the activity of the first application; Based on resuming the activity of the first application, the process of the first application is unmuted.

4. The method according to claim 1, characterized in that: The step of unmuting the process of the first application based on resuming the activity of the first application includes: Based on resuming the activity of the first application and the process of the first application being muted, the process of the first application is unmuted.

5. The method according to claim 1, characterized in that The method further comprises: During the playing of the audio data of the first application, in response to a fourth operation, setting the process of the first application to be muted; After the fourth operation, the first motion effect is played.

6. The method according to claim 5, characterized in that The method further comprises: Before the first motion effect is finished playing, in response to a fifth operation, playing a second motion effect; After the second animation effect starts playing, the process of the first application is unmuted.

7. The method according to claim 6, characterized in that The method further comprises: In response to the fifth operation, the first motion effect is stopped from playing.

8. The method according to claim 6 or 7, characterized in that: The method further comprises: In response to the fifth operation, the interface of the first application is displayed.

9. The method according to any one of claims 1 to 8, characterized in that: After the first operation, the first application is set to be run in the background.

10. The method according to any one of claims 1 to 9, characterized in that: The step of unmuting the process of the first application at a set time after closing the audio stream of the first application includes: Unmuting the process of the first application at a first set time after receiving callback information from the first application; the callback information is used to indicate that the audio stream output by the first application has been played before the time when the callback information is generated; When the callback information of the first application is not received, the process of the first application is unmuted at a second set time.

11. The method according to claim 10, characterized in that After the process of the first application is set to be muted, the method further includes: Sending a callback information registration request to an audio management service; the callback information registration request includes information of the first application; Based on the callback information registration request, callback information of the first application sent by the audio management service is received.

12. The method according to any one of claims 1 to 11, characterized in that: Before unmuting the process of the first application, the method further includes: Switch the process of the first application from a foreground running state to a background running state.

13. An electronic device, characterized in that: The electronic device comprises: a processor and a memory; The memory is used to store a program for the electronic device to execute the method according to any one of claims 1 to 12, and to store data involved in implementing the method according to any one of claims 1 to 12; The processor is configured to execute the program stored in the memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method according to any one of claims 1 to 12.

Images